I wear loose fitting dress pants, Dockers, etc. with big pockets and choose my wardrobe with carry in mind. I carry a J-frame S&W M&P 360 in a soft pocket holster with masking object in the front pocket. Lightweight, compact, rounded and masked well, it just looks like a slight rounded bulge in my pocket.
I carry things in my other pockets as well such as a pocket pc phone, small wallet, keys, leatherman in a sheath and speed loaders in a nylon folding knife pouch in the back pocket beside my wallet. Everything balances out with no sagging.
As long as it's legal i don't worry but the antis would certainly make a stink about it if they knew.