I use the M4 stock, A2 grip with the bump sanded off, the aluminum Magpul trigger guard that predates MOE, and the MOE handguard.
The MOE HG is a lot tougher than the A2 rifle HGs I've used in the past. My only complaint is the slots don't run the length of it as the do with the midlength and carbine, I mount my sling a little forward of the bbl nut.
It replaced a quad rail that had all kinds of slots to chew up my hand when I didn't have covers, be as ergonomic as a 2x4 with low profile covers, or be to big for my hand when I had covers. It also made the rifle front heavy, while providing a mount for my VFG, light, and sling loop... I really like the smooth, streamlined, light, and tough MOE HGs.
If you want an ol' skool look with a KISS approach, check out CNCGuns.com and see if Justin has any A1 carry handles.
I would not get rid of the PSA for an Olympic. I have never held a PSA. I own an Olympic .45 upper that needed a few fixes. Ironically, I put it on a billet lower...