I've got a few firearms in calibers that are generally considered obsolete, although I can still find new-manufactured ammo fairly easily...
7.65x53 Argentine. This was basically the first smokeless cartridge made for Mauser designs, I think the Belgians and maybe Turks also used it initially? But I think before WW2 it was basically just Argentina using it. It has an interesting context relevant in this country, because the small ring Argentine rifles were made before 1898, and thus fall under "antique" description, and in most parts of the country bypass the need of a FFL to transfer. And yet, you get a bolt-action rifle with a 5 rd magazine, in a caliber roughly equal to any other .30 cal rifle. Mauser did make safety improvements (re gassing issues) in later designs, but I've never come across any internet tales detailing serious disasters with them. People still shoot them, and PPU makes runs of ammo.
9x23 Largo. This is a subset of 9mm pistol ammo that the Spanish (and I think Austrians with their Steyrs) pursued, and Spain stuck with when most of the world settled on 9x19. I have bought legit Largo made by Steinel, and 9 Steyr is made in runs by Fiocchi (I have some). This is comparable to 9mm Luger in ballistics, and the Star pistols I own also can use barrels that run 9mm Luger.
7.62 x 25 Tokarev. Almost completely removed from arsenals, but was the Russian pistol and submachine gun caliber of WW2, and for maybe a decade afterwards. Again, I have a pistol with that, that also has a barrel to run 9mm Luger.
9x18 Makarov, arguably obsolete, although I think still in use some places in official manner. Some pretty nice smaller pistols, in roughly equivalent to .380acp purpose. Can find a lot of that being made and sold.
The 7.65 Argentine gives you a good solid rifle cartridge for a rifle you can have totally under the radar. The Largo and Tokarev ammo can replace 9mm for usage, and was available during the panic when 9mm was sold out. The Mak replaces .380, and is cheaper per rd.