It will not be any less safe for you to carry. As you said, the safety is between your ears (with some contribution from the longer heavier DA pull.)
I carry Smith & Wesson "traditional" autos, which also have a slide-mounted safety, this way. This kind of safety is awkward to disengage, especially under time pressure, if used as a safety. I think of them as merely a decocker and use them as such. In other words, when I wish to cease a string of fire, I thumb down the safety to decock, then immediately push it back up so the gun is ready to go. This is not normally done under as much time pressure and is a little more natural motion.
One thing to be aware of is that, if you should be disarmed in a struggle, the safety being off will make it easier for the attacker to shoot you with it. However, this applies to any pistol without a manual safety, as well as revolvers, for that matter.