I have a few sub-$100 guns but they aren't that thrilling. If pressed for cheapest few, and avoiding air/airsoft, I have to go with my HK 26.5 flare pistol. Yes, I use it occasionally, and it's much fun.
Some other favorites are probably preposterous deals, but were also so much in trade value that it's hard to pin a price on it
But almost surely my best cost/benefit is the Remington 700 PSS. Right around $600 at the time which may seem expensive compared to most things in this thread, but when I do my part, is reliably close to a half-moa gun with factory ammo. I have upgraded and updated, but not the barrel or action. It is both lovely to shoot, and you can spend a lot of happy time behind a precision rifle per round fired, so it's a cheap and quiet range day. And when you look at the shocking price of the precision rifles I see most people seriously shooting, I am happier yet with it.