That sounds normal, especially if you're pointing more than aiming. Pointing is a more effective way to engage a threat in a defensive situation, but it takes more practice.
A fundamental difference between the two techniques is that, when you point, your eyes focus on the target, whereas when you aim, your eyes focus on the front sight. Often, when you aim, you use only one eye. Aiming is more fatiguing and limits your peripheral vision, but it does provide the tiniest groups (not the priority when your life is in danger). It seems that few people practice pointing. It's good that you do.
If you keep both eyes open and focused on the target at a distance, the gun will be blurred and you will get a double image. This is as true of a pencil held at arm's length as it is of a gun.