I always say a child's first birthday is for the parent's ego, not for the child. I guess buying a rifle in this situation proves it.
Don't take it personal, please. Of course buy him a rifle if you want to.
(I would recommend a Savage Cub Mini Youth as a first rifle, but I also recommend...