To me the biggest issue with recoil is during practice, not the comparatively rare shot on a deer. In practice, the reason recoil is undesirable is not because I'm a wimp, but because I want to be able to spot myself. I definitely expect more precision at longer range than what is ordinarily practical with iron sites. I try to shoot my hunting rifle at 1 MOA out to 300 yards. Even though I do use a scope, it's one with a wide field of view (1.5-5X). If it was a 7 pound rifle in .30-06, the only way I'd know where the POI was is if I could find it in the spotting scope a few minutes after the fact. I'm convinced deer will fall very reliably with rounds fired generating much less recoil and that what's much more important is skill to consistently hit vitals with accuracy. With that said, I've already mentioned the possibility to handload nearly any sufficiently large centerfire rifle cartridge with the appropriate powder and bullet to levels that are both suitable for anyone starting out (no need for a crappy .22LR), to levels that allow for minimum recoil while still effectively taking game, and on up.
3.5" 12 gauge in a light shotgun like my Mossberg 535 ain't real nice, either, but those slugs take the cake.
