To the OP,
I would choose the Colt, and in fact have followed my own advice on this. I own one Colt, an M4A1 roll marked SOCOM II. While this was a limited run, and you won't find one like it the points I will make in Colt's favor on my rifle will largely apply to even the most basic LE6920, or LE6720, or better yet the LE6960 CCU.
I am also not going to denigrate the S&W, I don't have any direct experience with the S&W line of AR's they might be good rifles but I haven't inspected one personally to tell you.
When you buy a Colt you will get a carbine built as closely to the US military technical data package as possible while still being civilian legal. The benefit here is mostly in material specifications, testing and certification of the critical parts, and assembly steps and techniques. Without any further ado this is what we know the Colt offers, and what you should frankly consider the minimum acceptable quality in an M4 pattern carbine for serious use:
1.) Chrome Moly Vanadium 4150 barrel steel (abbreviated CMV), the barrel is also high pressure tested (HP) and magnetic particle inspected (MP). In the case of the Colt it will also have a chrome lined bore and chamber for increased life and corrosion resistance. This barrel will have a true 5.56X45mm NATO chamber, so you can safely run hotter 5.56mm NATO ammo with no pressure issues, it will also have a gas port appropriately sized for 5.56mm NATO. Barrel twist will be military standard 1:7" which will stabilize the heaviest bullets you can still load in the magazine. The gas block and front sight post will be taper pinned in place, this is a strong all steel part and will never come loose. The barrel will be manganese phosphate finished on the exterior, and the finish will include all the area under the gas block/front sight post which many el-cheapo's skip.
Some other makers offer 4150 CMV barrels with all the same tests that are salt bath nitrocarburized and that is also a good durable case hardening and corrosion resistant process too.
Colt also individually tests each barrel, they are not batch tested, so your barrel on that carbine will have been tested.
2.) Carpenter 158 bolt, shot peened, HP tested, and MP tested individually. You definitely want this, and should accept no less in a bolt. Additionally the Colt will include a tool steel extractor with the current M4 spec extractor spring.
3.) Bolt carrier will be a full auto profile carrier, chrome lined inner bore where the bolt lives. Properly staked gas key bolts, this is critical and a lot of el-cheapo outfits skimp on this. If those bolts loosen you will have gas leaks and short stroking, failures to feed, and no hold open on an empty magazine. Gas leaks are bad m'kay...
4.) Upper and lower receivers made from 7075-T6 forgings, type 3 hard coat anodized. Additionally you'll get a tougher Mil-Spec 7075-T6 buffer tube, and Colt will properly stake the living crap out of the castle nut that holds the buffer tube in place so it doesn't rotate on you under hard use. For some reason (OK no mystery to save $$$) el-cheapo guns often use a weaker 6061-T6 buffer tube, and more often than not don't bother staking the castle nut.
5.) Heavy weight buffer, or H2 buffer. A lot of el-cheapo brands use standard carbine buffers with steel weights because they are cheap. Colt uses either H or H2 buffers with tungsten weights because they moderate recoil better, delay bolt opening a bit to increase extractor life, and generally make for a smoother shooting carbine. Tungsten is expensive though.
There are probably a few things I missed but the heart of the rifle is the barrel, bolt, carrier, and trigger. I didn't address the trigger because GI spec triggers generally suck, and are the first thing I replace with a Geissele Automatics 2 stage.
Generally speaking the above features are what you should be looking for at a minimum for a good M4 type carbine if it is going to be a potential bet your life on it, has to work, fighting gun. I'm not going to tell you that it has to be a Colt, there are plenty of other reputable manufacturers out there building guns to this spec and even higher.
If it were me, and I was buying my first one, I'd save a bit more coin if you can, and buy a Colt LE6960, otherwise known as the Colt Combat Unit Carbine. It's mid length with all the above features plus a more logical barrel profile, a great M-LOK free floating fore end, and good Magpul furniture. Just needs iron sights of your choice and a good combat optic that meets your needs, plus a good sling. I'd happily take one out of the box, run a bore snake down the barrel with a bit of CLP, lube it all up, put sights on it, slap my Nightforce NXS 1-4X24 on top or my old Aimpoint, mount my Vickers sling, load a bunch of mags; and go take a high round count class with total confidence it will run like a raped ape. Probably buy a good light for it too. Then you're done, general purpose carbine achieved. Add a trigger that doesn't suck when you can.