Bore size for the 22 Mag is 0.224" while the bore size for the 22LR is 0.223". Yes, that's not much, but remember that the chamber size and more importantly, the relation of the chamber mouth to the bore is extremely critical. I've never been a fan of any 'early' Ruger barrel and found them the poorest of any of the large arms makers, however they do seem to have stepped up their game in the barrel area as of late (I feel due to the pressure of Savage coming on strong) as I have a RPRimfire that shoots lights out.
So yes, it could be combination of ... being a Ruger, with a 'just OK' mfg'd barrel of one bore size for 2 calibers, along with sloppy chambers that made the inherent accuracy of one or either caliber to be well, ... sub-par. All I know is that any friend who had the Ruger S-S convertible always complained about accuracy of one round or the other, maybe both.
With that said, there are many other instances out there of other Manufacturers shipping out one barrel size for 2 calibers. A recent notorious example was RIA sending out some 9mm 1911s with a 38 Super barrel. With jacketed bullets, any such bore difference is even worse; as the 1911 I got in 9mm wouldn't hold one minute of barn door! Seriously, 1-2' groups @ 50 feet! Yet after complaining and complaining and finally getting a new/correct '9mm' barrel from them, it would then print 2" groups or better.