I would carry your 357mag. When I backpack, I try to keep the weight down, and the firearm is part of that. My main concern are two-legged predators. In the Rockies, I would add a good can of bear spray.
I have carried a 44mag on a backpack trip, and found it to be a pain in the a**. Carried a 38 snub many times, and it was just about right. My favorite now is a Tokarev pistol. The pistol itself is fairly light and compact, was cheap to purchase, so I don't worry about trail abuse, and is mil-spec tough. 7.62x25 has plenty of penetration in FMJ, and is flat-shooting out to 100yds. Plus, can be had in JHP.
As you have found in this thread, most responses are from people who have never carried a full pack over rough terrain, and have no clue what adding a 4lb revolver plus ammo and carry rig will do, let alone wearing an extra t-shirt to conceal it.