The sights should always be used except for extremely close shots where missing does not pose lethal threat to others, but there is a difference between looking at the front sight and looking through or over the sights.
If you get familiar with a firearm you can point it where you are looking and be pretty close, so the sights are only necessary to add a slight adjustment or track randomly moving targets.
For a test see if you can take your handgun in your hand by your side, look where you want to aim at some small target, close your eyes, quickly bring your gun up aligned with where your sight picture will be and aimed at the target, then open your eyes and if you find your pistol aimed at what you wanted to hit and lined up with your eye when you look through the sights without making any adjustment then you don't really need to focus your vision on the front sight to use that pistol. Try it on a target in front of you, to your sides, and make sure it is a fairly small target.
Be honest with yourself, if you need to bring the sight into focus to shoot well then do so.
I can do it quite well with a two handed grip, and some guns are off with a one handed grip.
Most self defense situations are within 7 yards, many much closer, and while your adrenaline and being shot at will make you a worse shot, humans are large targets and not particularly difficult to hit.
I know I can clearly see the front sight while looking at the target within 5 yards, and never need to actually focus my vision on the front sight. I can see the sight slightly better, and the target and peripheral objects worse if I look at the sight.
Within 3 yards or 10 feet the typical range a bad guy will make himself known in public aiming at specific spots without ever focusing on the sights is not difficult.
So actually putting my front sight into focus to hit anything larger than an orange at that range is a waste of vision.
Now if I used a firearm I was not familiar with or don't handle often I would need to check the sights to insure they were aligned right and may find myself bringing them into focus when doing so.
More precision shots at distance, and typical rifle shooting with iron sights make a lot more precision use of the sights to make slight adjustments that make a bigger difference. But close range with a pistol bringing the sights into clear focus would be mainly to compete or shoot small groups to try and impress people, not hit something as large as a head or center of a torso at the expense of seeing the target and possibly their accomplice clearly.
But everyone is different, and you need to be accountable for where you put your rounds.