In Kentucky, even resident fees have gone up. I buy a sportsman's license every year; this year it was $95, and I've bagged one deer, nothing else all year. For many years I bought one, and ended up with no place to fish or hunt. Next year (although not early enough in the year to be practical), I'll be eligible for a senior license, and I plan to take advantage of that....$12. We also have a resident senior lifetime sportsman's license, which is $180; I doubt I'd live long enough, or be spry enough for all those years, to make that one worthwhile.
If you want an example of non-resident fees that are truly worth complaining about, look at the Canadian provinces. Most of them gouge American hunters outrageously with their fees.
As for hunter numbers dropping, I believe that in my state, at least, there is no shortage of hunters, or potential hunters, even though license sales are indeed down. Desire to hunt among the populace is as great as I've ever seen it. The problem is a dire shortage of hunting opportunities near the population centers, especially near Lexington. A lot of potential hunters get discouraged, and just don't keep buying licenses.
The property I currently hunt is very small, but because of its habitat mix, it's very attractive to deer. The landowner is constantly pestered by people asking permission to hunt, some of whom offer him crazy lease fees, and by people wanting to buy his land for hunting purposes, even though it's only 18 acres!! One of the neighboring landowners is wealthy, and has repeatedly offered him twice what he paid for the property only 4 years ago, just to add it to his land so he could use it for 2 weeks out of the year. That same scenario is played out regularly all over this part of the state, and I don't see that trend slowing down in my lifetime.
Edited to add: I think that nonresident hunters are actually increasing in our state; Kentucky is now the trendy spot to pursue a B&C buck. What happened to Kansas a few years back is what's happening here now.