It happens when the brass gets hard enough the case cannot expand to fit the chamber without splitting.
It can happen as soon as the first firing.
Some cases are annealed a number of times during the forming process as well as after. With milspec ammunition, they don’t even allow them to remove the evidence of the last anneal but it’s common for the factory stuff to be shiny.
So what causes the brass to get hard? It work hardens, so taking a case that “blows out” or expands to the chamber when fired then putting it into a sizing die that squeezes it down, then dragging an expander back through the neck are all going to make it harder. How much harder depends on how much your moving it.
Take a benchrest gun, make some brass that fits the chamber, we call this “fire forming” and use a bushing that gives you .002” neck tension and you’ll get more than 20 firings without annealing them. The key is they are not moving around as much.
If none of that is an option, annealing is, I’d suggest having batches of both and good records so you’ll know if you are negatively effecting your shooting in the name of longevity.