This isn't at all unusual with what to expect with a group, and with a 5 shot group this is even more of a factor. A lot changes through the course of a 5 shot group with high pressure bottle neck cartridges.
Barrel harmonics change as it warms up, and as it expands. This is one of the reasons why I always shoot one or two cold shots, to foul the barrel, and to bring it up in temperature some.
And thermal conduction from the chamber effects powder too, the warmer the chamber gets, the quicker the cartridge warms up. As the powder temp increases, variations in the pressure curve and peak pressures is effected.
And the barrel continues to foul following each round fired, this will increase friction from one round to the next, which in turn will impact the consistency of the group.
To acquire the most consistent groups possible, I would recommend taking one or two shots to foul the barrel, and to bring it up in temperature. Wait 5 minutes or so, then begin shooting your 5 round group, allowing enough time between shots for the barrel to cool some. But don't let it get too cold, or you'll basically be shooting a cold bore on every round fired. And don't leave a round in the chamber during each cool down, this allows the powder to warm up, as does letting the ammunition sit in the sun.
Is your brass all trimmed to the same lengths?
Do you sort your brass by head stamp & weight?
Are you weighing the charges on a scale, or using a powder measure?
Uniforming primer pockets?
Have you run the round across a chrony to see how much velocity spread is taking place.
As for the variations between powders, one may be more thermally sensitive than the other. I usually load with Varget for the .223, so I can't really address the powders you are using or how they might vary in performance as you've said. I also don't shoot an expensive .223 either, just a plain jane rig, but it shoots pretty well for me.
But actually, I think one flier in 5 is a pretty good group considering that the one flier is only increasing the group by .5" - .75". That doesn't sound to bad for an AL-ing action, at least not to me. If all else fails, you might consider shooting 3 shot groups v.s. 5 shots. Also try to see if you are changing your hold, this can have a pretty significant impact on your groups.
GS