An aquaintance of mine was a Colorado Wildlife techician/area manager. He on several occasions won the NRA PPC National Championship "Shotgun" match. He used a Remington M870 with a 30" full choke (fixed) with 3" "BB's" for coyotes. I've even got a box of 10 Activ brand 3" nickle plated BB shot he swapped me for some 9pellet-OO he needed to shoot the shotgun match in 1990.
He said that he frequently made shots on coyotes to in excess of 80yds. He was the first person I ever heard of using a shotgun on coyotes. He stated he preferred plated BB's, but used #4Buck when he couldn't get BB's. He didn't reload shotgun...
My personal choice in coyote "shotgun" loads is a hand load of 22 #4Buck (cast from Lee gang-mould) in my 20ga pump. I shoot it through a TrueGlo "extra-full" Turkey choke. It measures 0.615" IRRC.
The pattern it shoots at 25yds is amazing... approx. 10" spread, though at 40yds, it opens to around 25" due to some of the mis-shapen shot "flying" out of the pattern. Typically a small dense cluster with 2-4 flyers... But, my shot is much, much harder than typical production buck shot.
I've not connected with a coyote with it, but two shots and two deer (~100lb does) at 30-40yds convinces me it'll work on a 40-50lb coyote.
A 12ga 3" magnum #4buck with 41pellets from a mod to extra-full choke is awsome medicine for coyotes under 50yds... where most of them are shot in my neck of the woods.
My personal longest shot on a coyote is ~110yds. Got him with a .22wmr w/30gr Remington V-max in my back yard.
Most devastating load I've shot a coyote with is a 115gr Nosler B.T. from my .257wbymag. @ ~3,350fps. At 20yds, it blew "her" in half... except for a strand of hide along the belly holding the two halves together. btw; I don't hunt deer with that load anymore.....!
FWIW; in our part of the country, the hides are never worth the trouble of skinning them out. (Georgia; doesn't get cold enough to prime the hides...).
I hunted deer extensively back in the early 80's with dogs and shotguns in E.Alabama. Patterns with a shotgun and buckshot are much like shooting groups with a rifle. Each firearm/choke tube/load is a rule unto it'self. I've seen very tight patterns with OO-buck and improved cylinders and a 30"full choke Rem. 870 that shot abysmal "dough-nut" patterns with OO-buck, but watched owner drop a doe deer at a measured 88yds with 3" load of #4buck. We counted 8 fatal pellet hits in head/neck/chest area. One hit an eye, another the brain...
Thats why we "pattern" our shotguns, and try different "loads"....