Usually when I get the velocity I want with a particular powder and bullet combination, but still not getting quite the groups I want I will start to play with the seating depth a bit. I will load a couple of boxes up a bit long, then take one of my small single stage presses out to the range and start off as long as will function in the magazine. I will shoot 3-5 rounds at that length, then seat the next 5 in around .005 deeper. I will continue to do this until I run out of shells or see a marked improvement in my groups compared to what the original length was.
If I get down to the original length and still haven't seen any improvement, or I run out of shells, I will come back with another couple or so boxes and start where I left off and go a bit more. Usually I find at least two groups which will be smaller either on the long or short end. Similarly, while working up with the powder charge there is usually one load on the low end and one on the upper end of the data which show better accuracy than anything else.
If none of the above works I might also switch to a little less primer, as I normally use the Win-WLR for 98% of my loads. They are a solid hot primer and I have for the most part not had issues with. However I have a few loads which threw groups like you speak of that dropped right into tightly clustered balls when I simply switched to a CCI or Fed. Everything else being the same, sometimes it is simply a matter of ignition which makes the difference.
This all said, I do have a couple of VERY particular rifles which took me a long time to find "THE" load in. Sometimes it is simply a matter of persistence and trying a few combinations which might not be what you thought they should, no matter how many other people get great results from them. I have a Rem BDL in .270 which will shoot 46.5grs of IMR-3031 and the 130gr Nosler BT into literally one hole groups if I do my part out to 200yds. It is simply the most accurate load I have ever developed for the rifle. I worked it up for my mom when she was still hunting in her late 60's and early 70's. Recoil is almost non existent in the 7.5# rifle and it has dropped more deer than any other load I have shot through it. Bottom line is it simply works, but you won't ever see anyone recommending it as something good over any of the 4350's or 4831's. Which in mentioning none of them has ever shot a decent group from that rifle with anything less than a 150gr bullet. When my daughter as only 11 she dropped a feral hog with it at a lasered 365yds with one perfect shot. It just works.
As for your groups, would I hunt with them, heck yes, and not think twice about it. There are plenty of folks out there that consider pie plate accuracy at 100yds a decent load. I am not amongst them, but it don't matter. If you are confident with the load, pick your shots reasonably, and can put the bullet through the boiler works your going to have meat on the ground. Me personally, I like those little multi-shot ragged edged holes at what ever range I have my rifle sighted in at. I like to know that I have done all I can do for the rifle and load, and the rest is on my. That way if and when something does go wrong, I am not second guessing my equipment the rest of the weekend. I also know up front that I am usually the weakest link in the equation, and come crunch time, I don't want to have in the back of my mind, "is this shot going to be the one which flies 3" out of the group or will it be the next one?"
I have a little Ruger Compact chambered in .308 which has never shot much if any better than what you have, even with handloads. In fact it actually shoots the cheap Remington CL's better than anything. Still in all, it is the most used rifle I have in my safe, and has taken enough deer and hogs to fill several folks chest freezers many times over.
Like I said if your confident in the combo, go hunt with it, and be happy. Work on the tweaking after the season.