The shotgun has massive recoil, and very limited ammunition capacity. They are much heavier, more awkward to use, and harder to mount a light to. True they can be devastating at close range, but they just have too many limitations for me. Have one... rarely use it.
My handgun is the FIRST thing I'd grab. However, this is only because of it's small size. (Same reason I carry it every day.) It sits on my night stand, so I can grab it while I'm still in bed if need be...
Once I'm sitting up, I have an SBR (11.3" .300 blackout AR) right there in the corner to grab. This is actually my home defense gun. The AR has more ammo (30 rounds) of a more potent caliber. (110 gr VorTx .300 Blackout) It still mounts a light, and is small enough to easily move around with inside the house. It is much more stable with good solid two hand operation, and a third, and fourth points of contact. (my shoulder, and cheek) It has an Aimpoint that is ALWAYS on, and back up sights with tritium inserts. I also have a lifetime of training, and experience (28 years in the U. S. Army) with the AR platform that make me much better with this weapon system, than I am with my handgun.
Now, I live in a small town, but even if I lived in a large population center.. in fact especially if I lived in a large population center, (higher crime rates) I would make the same choice.
I (personally) will never used overpenetration as a factor in choosing a defensive weapon. It's a non-factor in my book. If my life, and/or the lives of my loved ones are in danger, my one, and only concern is to stop the threat... now. To do that, I want the most effective weapon I can get. To all those who would choose an AR, I would say not to use a pistol caliber, or if a .300 Blackout, not to use subsonic rounds. You want a rifle round. One that will be devastating to the target.
Small caliber rounds moving fast will break up in most cases when passing through barriers, so minimal risk to others, though not optimal, is preferable to not stopping the threat to me, and mine. The velocity when they hit the targets makes all the difference in the world though. Keep in mind that the more capable firearm (and the more capable you are with it) the less rounds you will need to fire to stop any threat. This, combined with the proper selection of ammo for it will reduce the danger downrange, and help those you mean to protect.
I'll always pick a rifle over a shotgun, or handgun as my primary defensive gun, given the chance to.