One factor that sometimes isn't taken into consideration is that ignition is slower in a cap & ball revolver, and the powder burns slower. Combine that with a long, 7 1/2 to 8 inch barrel, and it becomes clear that careful holding of the sights (which are crude at best) and follow-through after the trigger is pulled is critical.
I have used charges as light as 15 grains of 3Fg + wads under the ball for indoor gallery shooting at 50 feet. Be sure the ball is firmly seated and the powder slightly compressed. If there is an air space between the ball and powder you can rupture the chamber wall.
It was well said by some champions in my club that if one learned to shoot black powder handguns well, anything else would be easy.