Have you bedded the action? Those Ruger M77's have a tiny recoil lug, and Ruger uses angry beavers to chew the wood out of the action area. The wood to metal fit was, and last I saw, poor. Lots of round for the action to slide around. The angled front action screw makes for great advertising, but it won't hold the action in place. If you have not completed bedded the action, then the thing is sliding around inside the stock, each shot. Which, depending on how much clearance you have between barrel and barrel channel, the barrel may be touching the barrel channel, and differently, between shots.
The 70's M77's, the barrels were the best barrels $15.00 could buy. In my opinion, they were crap. Ruger now makes their own barrels, and their hammer forged barrels are good. So, depending on the vintage, the barrel quality is not very good.
The final quess, is that a 338 Win Mag is a monster, and maybe a little flinching has developed?