so, what’s the deal with the 165’s?
Some of it is fad. Some of it is performance. Some of it is preference. Some of it is splitting the difference.
Most 40S&W handguns have a sight height that is set for 180gr bullets at approx 980fps. Some handguns shoot great with 180's, while others will shoot closer to the sights with 155's. It depends on whether it is set up for a center hold or driving the dot. Most reloaders typically adjust the powder charge so that the POA is the POI at a certain distance. (more than 7 yards - haha).
For some, the POI shift for 165's is not as drastic as 155's, when a handgun is set up for 180's.
Additionally, every bullet has a velocity window. If you're below it, there's little or no expansion. Then, there's the optimal range. Then, over a certain velocity there could be overexpansion, however, this isn't a 40S&W issue, but is possible with some of the same bullets in 10mm. The notable exception would be Remington Golden Saber losing their jackets, but that is typical for their design.
Most 155 JHP's get great expansion due to the additional velocity. Some 180gr JHP's don't have good expansion due to their lower velocity. For some, 165's were seen as a happy medium between having enough velocity for expansion without the stout recoil of 155's, with the POI being close enough to the sights.
Also, ten years ago, manufacturers were promoting the lightweight 135gr .400" bullets to make their 40S&W handguns into 9mm-like performance for some shooters. It is rather ironic that today's return to 9mm has brought interest in .355" 147gr JHP bullets, which usually chrono under 1000fps.
Excessive penetration seemed to be issue with the 180’s, not having enough velocity to expand from the compact/sub compact short barrels...
If you've been napping under a rock for 10 years, many today are quick to note the improvements to 9mm JHP's. This technology is not lost on 40S&W. Without listing them all and leaving out someone's favorite JHP, there are a lot of excellent choices. Well known brands make 12-18" of penetration and expand into diameters over 0.70", even in short barreled firearms. If you peruse this source:
https://www.luckygunner.com/labs/self-defense-ammo-ballistic-tests/#40SW it is easy to see that bullet weight alone does not directly correlate to over penetration or lack of expansion. There are more winners than losers across 155/165/180 when it comes to 12-18" penetration and full expansion.