I've never hunted deer with a cap and ball revolver but I do know something about the accuracy and power of them out to 40 yards and more.
I would not use the 1860 Colt or Colt Walker or Dragoons for deer hunting.
Why?
Because their sights are atrocious: just a notch in the hammer and a brass front sight that catches light and becomes an indistinct glowing orb in sunlight.
Such sights are not adjustable. Typically, these revolvers place their balls or bullets higher than the point of aim. Nor is it unusual to find one that shoots to the right or left of point of aim. Again, this can't easily be compensated for.
Hunting requires accuracy for proper shot placement. It's as simple as that. Accuracy is more important than power. Look at how many poachers have taken deer with .22 Long Rifle bullets; they depend upon accuracy, not power.
The Ruger Old Army, target-grade Remington 1858 and any others
with adjustable sights would be best, to ensure accuracy.
Next-best would be the Remington with its fixed sights, since such sights can be altered to shoot to the point of aim.
Just yesterday I was out with my Uberti-made `58 Remington .44 caliber. At 20 yards, from a benchrest, it put six Lee 200 gr. conical bullets into a group 1-1/2 inches high and 3/4 inch wide.
This is the best group I've recorded with these particular bullets; generally I find round balls to be more accurate than conical bullets.
This would make a good, close-range deer load.
The load particulars are:
Six Lee bullets into 1-1/2 X 3/4-inch group at 20 yards
Uberti-made replica of Remington 1858, fixed sights, Ser. No. 419XXX
20 yards, benchrest.
Weather sunny, 40s and Sun 90 degrees to my left
Lee 200 gr. conical bullet (catalog no. 450-200-IR) cast of soft lead
Bullet lubricated with Gatofeo No. 1 lube (a mix of mutton tallow, canning paraffin and beeswax)
26.4 grs. Goex FFFG powder, thrown by spout on flask.
Remington No. 11 cap, pinched to cling to the nipple.
No wad between bullet and powder
Unknown velocity ( I didn't bring my chronograph) but muzzle velocity estimated at 800 feet per second.
Point of aim: 1-1/2 inches below center of group or, expressed another way, most bullets hit 1-1/2 inches above where I aimed.
This load would certainly kill a deer, if the bullet were accurately placed.
Some of you may wonder why I shot at 20 yards, instead of the more standard 25 yards. Well, the local gravel pit was a mudhole and 20 yards was the longest I could shoot without having to tramp through thick, gooey mud to change targets!
Of course, if my groups had been atrocious, I would have blamed the mud as well:
"It's the mud. It's clingy and it's sucking down the bullets and throwing them all over the place. Yep ... mud all right ... darn it."