The .243 would definitely be my first choice. It's a dandy caliber on deer when using the right bullet. There are plenty of options for "compact" or "youth" model guns in .243. Ammo is plentiful and relatively inexpensive.
You might also want to look into .257 Roberts, though you will have to find a used rifle as I don't think there are any factory models offered in that caliber right now.
If you reload and are willing to look into slightly "overlooked" calibers, you can get into some really cool calibers that are great for young shooters. Most of those "older" calibers were very mild recoiling.