What you are seeing is the accuracy of the ammunition (or lack thereof).
Any average to match grade commercial ammo or reloads using quality components are likely to shoot FAR better.
My M&A-parts Heavy AR upper, shot 1/2" to occasionaly better with ammo it liked. Often 2" or worse with ammo such as you described. However, that accuracy level was deemed acceptable for the ammo when it was loaded.
You probably won't see peak accuracy from your barrel until you have over 500rds through it, so keep shooting it. But, by all means keep cleaning it to prevent copper alloy fouling.
Try some of the various plastic tipped ammo with bullets of 55gr or heavier. In my experience the bullets are premium grade intended for serious varminting and are usually match grade. Several heavy bbl AR's I've loaded for shot them VERY VERY well.
RE: 77-80gr bullets.
Those labeled as LTB=length tolerant bullet; seated to 2.226" OAL or LESS, will feed through the magazines. If you want to shoot the longer VLD's=very low drag, you'll need to single feed them. These are often seated to beyond 2.30"oal and will not feed through a magazine.
There are replacement followers that make the magazine a single shot magazine where you only drop the round on the follower and depress the bolt release. However, you can "finger feed" the rounds into the chamber and then release the bolt with conventional magazines. The single shot adaptors are intended for shooting under match conditions where time is of essence (time-fire, or even slow fire stages are still "timed").
My adapter was the Satern model. Sinclair also offers one.
The two bullets you described are designed to be normally seated, hence are LTB's. The Hornady 75gr A-max, and the Berger VLD's are intended for single feeding loaded to "over length" diminsions.
Of the two, the 69gr will be better suited for 1/9" twist barrels for which it was designed. The 77gr will be better with 1/8 or 1/7 twist barrels such as the long range match barrels. Which is better depends on the twist rate of YOUR barrel. It is normally stamped on the barrel in front of the gas block. Only a few 1/9 twist barrels will give acceptable results with the >75gr bullets. Even the ones that will shoot the heavier ones accurately are less accurate at ranges under 300yds. It takes the longer ranges to allow the heavier bullet's characteristics to prevail. (less long range drop and wind drift). A characteristic know as "nutation" or wobble is what decreases short range accuracy. This is NOT to say the heavier bullets are inaccurate at short range, but are slightly less accurate until they reach their intended range of use (300yds+).