The limit on bore size, or gauge, is 10ga for WATERFOWL only. It's a Federal regulation. To say it a little differently, is bore size, is how many lead balls that are the same size as the bore, equal a pound.
Shotguns don't have as much recoil as a rifle because the load is going down a smooth barrel, not one with rifling that would cause resistance and more recoil. Yes there's going to be a lot of recoil, but not as much as a rifle. The British use to hunt lions from the backs of elephants with 4 bore rifles. But the rifling was about 1 turn in 32 feet . Just enough to stabilize the big ball, not not so much to give a lot of resistance and cause too much recoil.
Years back I made a 8ga O/U muzzleloading shotgun. For some reason I settled on 4oz of shot and 400grs of 1F.. Sometimes 30@ of 32 cal RBs. With the bird shot it would knock down geese from 65 yards. If shot straight up at overhead geese, it would push me flat on my back. Sadly, I sold it by mistake at a gun show. Oh well.