Of the two, .32 & .36, the .36 caliber offers more accuracy, and is easier to load. Unless you have a stainless steel range rod, the .32 caliber can be a little tricky to load with a wood ramrod, since its diameter must be smaller than the bore size. A 1/4" diameter wood ramrod is fragile if one is not careful.
The .36 caliber is the caliber of choice among national champion competitors. It has the optimum ballistics for delivery out to 100 yards, and easier on the shoulder than a .45 or .50 caliber. The late Bob Watts of Stone Mtn, GA, who built my flint .40 caliber longrifle, won the NMLRA National Flint Offhand Championship with his .36 caliber longrifle in 1976.
Between 1860 & 1890, the most popular caliber for accurate target shooting, bench or offhand, was .38 caliber.
The .45 caliber is, IMHO, also great for target work, but the .36 caliber shoots flatter, and less recoil.