I do not "panic" or get even slightly nervous because I am confident in my ability to make sure the gun is clear.
I dry fire only with my competition gun, which is also the only gun of its type I own, and it is never kept loaded, and none of its mags are ever kept loaded either, so that helps. I won't be accidentally picking up a carry gun and thinking I'm going to be dryfiring it.
When I'm ready to dryfire, put my gear on, re-verify gun is unloaded, verify mags are unloaded (load with maroon colored snap caps if I need them for what I'm doing that day) and proceed with the session. If I got "scared" at this point, I would reevaluate my gun-clearing ability and correct whatever part of it was deficient and making me nervous. Personally I don't stick my fingers into locked open autoloader actions, but if that is your thing, have at it. Stick a rod down the barrel till you see it pop out in the breach if you have to. Whatever you do, make sure it is RELIABLE, REPEATABLE, and most importantly, makes YOU confident that the gun is unloaded.
Get ammo out of the room if that is your thing; but personally I have ammo in a lot of places and don't feel the need to check every cranny of the room for ammo. I also have targets in several spots and often are moving between rooms anyway. Since I'm starting with unloaded mags and have no intention of sticking any ammo into them during the session, I'm not worried about it. Again, the mags I'm using to dryfire and the mags I'm using in my carry gun are not interchangable and the former are never loaded in the house so there is no chance of picking up a loaded mag for my dryfire gun by accident. Therefore I'm not worried, even if I was dryfiring while sitting on top of a pallet of ammo.
Assuming you have rationally functioning brain, if you are nervous, that is your subconsious expressing less than complete confidence in your process. Address this confidence problem by correcting whatever places exist for mistakes to occur, and you should be fine after that.