I've always worn what I call a "pistol belt", I guess the same thing. No, one does not want to/should not hang stuff on one's pant's belt. The trick to a pistol/hunting (?) belt is to wear it a tiny bit below your pant's belt or belt line.
I used to carry quite a bit on the pistol belt, canteen, pistol, pistol ammo, knife, rifle ammo, etc. I've since trimmed it down to just rifle ammo, pistol and knife. Almost all my pistols have ammo "self contained" with/in the holster. If I do have a separate pouch for the pistol ammo, it's small and only carries 12 rounds. I carry my water in a over-shoulder canteen, boda-bag, or some in my pack. The wild country I wander has lots of water so I don't have to carry much. Close to home on the State Lands I wouldn't drink the water, so I have to carry quite a bit.
Suspenders with belt such as the GI/LBE system works well, I've used that quite a bit, and a pack can go right over it. A leather strap that goes across the chest and to the belt on the pistol side also gives some suspension and if adjusted right is comfortable and spreads out the load.
As I hunt/hike/trekk/explore and play map-and-compass in some very remote and wild country, I always go "full" gear. So a pistol belt and pack are important. Also, if I take off my pack for any reason, or was separated from it, (not sure how that would happen) I want to still have fire, sidearm, and big knife on me.