You can hang out at Benchrest Central for a while and you'll figure out what affects accuracy. On a bolt gun, BR shooters take a factory receiver and square up the bolt face to the bore, the lugs to the notches in the receiver, tighten the chamber, etc.
For example, if only one lug is contacting (not unknown on a factory rifle), the action twists and binds during firing. Suppose the chamber is oversized. In effect, the round can lay with the tip pointing down.
There are (or used to be) a tons of threads about Savage vs Remington accuracy. I suspect it has a lot more to do with when the rifle was made and how good the tooling was versus brand. If your barrel was the first one of a production run, it's probably real good. If it's number 1,000 out of 1,000, it's probably not so good because the tooling is not as sharp.
As for action type, I have an AR that is accurate beyond belief. If I had to win a shooting contest, I'd take it over my 22-250 Rem 700, and it's plenty accurate for most anything.