tkcomer, if you are using the milspec sights that came on the rifle, you ought to know that the rear sights are not windage adjustable, so then you have to use the front sights to drift left or right as needed.
The Rule: The rear sight, if adjustable for windage, needs to be moved in the direction that you want the bullet to go.
The front sight, to adjust for windage, needs to be moved in the opposite direction that you want the bullet to go.
Now that you have that indelibly etched in your mind, use a brass punch to drift that front sight in the correct direction. A little at a time.
Good luck with your reloads.
NCsmitty