Always after shooting them, or taking them out. Like hunting, I may be out in the woods for 6-8 hours and not fire a shot. That doesn't mean they aren't covered is sweat, water, and all the other woods debris that comes with pushing through the thick stuff. In that case, they will get cleaned thoroughly.
If I just walk out into the backyard and shoot maybe 5 rounds, they will get a good wiping off and a pass or two with a bore snake. So really it depends but at the very least it will be wiped off, oiled and get a pass with a bore snake if it has been shot, or just wiped off if handled.