Not exactly. Take the Model 12 Winchester as an example; a 12 gauge M12 is larger and heavier than a 16 gauge version. There's enough difference that you can feel it. Same with finer double guns. They're built on scaled frames and are smaller and lighter than their larger bored counterparts. Of course, these days gunmakers build everything on the same sized frame and you can't tell the difference in them.
As far as the 20 gauge goes, I own one and I rarely use it. Just haven't had any luck with it. On the flip side I have 3 16's. There is no comparison between it and the 20 as far as knocking critters out of trees or the air. Sure, the 20 doesn't kick hard but neither does the 16 in a scaled gun. For all round shotgun work, give me a 16 any day of the week.
Mac