Yes, I think that is the main reason, for the weight. However, the barrel manufacturers, and the barrel smiths seem to all stick to the modern standards of barrel size in relation to caliber. I think that's too bad, most barrels are a bit overly heavy for the caliber, if not way too heavy. I can appreciate wanting a nice slim light barrel on a rifle, instead of an obese hunk of steel suitable for massive over-loads or prying rocks out of the ground.
When I decided on caliber for my Jeager, I went with .62 for the simple reason that would be the lightest barrel I could get in that barrel size. A .54 would have been much heavier. I already had lots of molds and "stuff" for .58", and love that caliber, but again the .62 would be slightly lighter. (and I now love the .62") (no regrets)
So, it is a problem, the barrel makers won't make "slim" barrels, and the barrel smiths usually won't bore a barrel over a certain, self imposed limit. Won't be easy to get a smith to bore a 13/16's out to .54", but again I totally appreciate what you are trying to do.