I wear cloth/leather type gloves while shooting/loading my guns because I don't like handling lead, and I usually keep a light coat of oil on my guns and don't like getting that on my hands either.
I wear latex gloves while reloading ammo or cleaning the bore of a gun, once again, because I don't like handling lead, and cleaning solvents are nasty.
I wear cloth type gloves when wiping my guns down after cleaning the bore before they go in the safe, because latex gloves leave a powdery residue on them.
The only gun I don't wear gloves to shoot is my carry gun, and that's because it's "germ free". No one else shoots it, and I don't oil it because it rides in my pocket and I don't want remoil in there either.
No one else I know shoots/loads/cleans with gloves though, and admittedly, I'm OCD about germs/dirt/nastiness and prefer to prevent any of that stuff from touching my skin.
Yeah, I'm a weirdo.