The Allure and Shortcomings of Statistics and Actual Data

Status
Not open for further replies.

Kleanbore

Moderator
Staff member
Joined
Aug 13, 2008
Messages
17,480
In the defensive use of firearms, very few things can reliably predicted before they happen.

When we wonder what our real risks are, or ponder which handgun might be the most effective, there is a natural tendency to ask about what actual event data might tell us.

Unfortunately, it would be a mistake to expect to be able to conclude much from actual data.

The internet age and the advent of search engines have given many people an expectation that a lot of useful information from past events should be available to frame their thought processes. What we find, however, is (1) that there have been too few incidents involving shots fired, when compared to the number of factors that influence the results, to draw much in the way of useful conclusions, and (2) the details of the information recorded about those incidents does not really tell us much.

In addition, most people do not have a sufficient understanding of statistical analysis to appreciate the shortfalls inherent in the usefulness of statistical data.

Some of the staff do have training and professional experience in this area. We have made an attempt to put together a note to help share some of our understanding with those who have not struggled through it themselves. We have strived for a balance of simplicity and substance.

The bottom line is that statistical analysis of actual event data will not help very much when it comes to making informed decisions. For that reason, we have included other information pertaining to defensive shooting.

Much of the following has appeared in various other threads over a period of several years. The efforts of many other THR members and staff are embodied in the following.

*********

We'll start with a discussion of statistical analysis.

A Math-Free Discussion of the Pitfalls of Statistics

Averages Just Don't Mean Very Much

One of the claims most often heard in discussions about defensive shootings, and no one can validate it, goes something like "the average gunfight is over in N rounds", where N may be 3, 4, or 5. That is often used to justify the belief that a handgun containing only a few rounds would be all that a defender should need for primary carry.

One problem with that thought process is that there have been many incidents--perhaps half of the total--in which the average number, whatever it may be, would not have enabled success. That's why averages are averages.

On would need to know more than the average.

If we had the data--and we do not--we would have to consider the extreme values that have occurred, and the frequency distribution around the average.

Even then, because of the small number of actual defensive shooting incidents, we would not be wise to place much confidence in the actual data.

No, actual data do not provide a good way to make assessments of how many shots will likely suffice in a defensive encounter.

**********

Another thing that we are likely to try to come to grips with is the question of whether we might be advised to carry different defensive firearms in different circumstances.

Confusing Likelihood and Consequence in Risk Assessment

It is very natural to fall in the trap of thinking something like "there is little reason to expect trouble while I'm on this little errand, so I'll just drop a small pistol into my pocket...."

Let's get one thing out of the way first: if we really think that heading out somewhere might put us in any real danger, we should not go.

But back to the subject: It may well be that one would be less likely to be mugged or car-jacked outside a flower-shop or salon in a good neighborhood than at a filling station near a travelled throughway. BUT should an attack occur, there is no reason whatsoever to think that what we would need to defend ourselves would be any less.

One would want the same capability in any defensive situation.

No, trying to assess probability does not help us with that question, either.

**********
When we discuss the likelihood of needing to defend ourselves, we often hear things like "one's chances of being attacked by a criminal are one in _________". That may be true, for an "average" person, on an annual basis. But that's not something we can take to the bank.

The Impact of the Period of Exposure

The likelihood that a civilian not actively engaged in the apprehension of dangerous fugitives will be involved in violence on any one day is very, very low. In risk management terms, it is characterized as "much less than remote".

Over a period of a year, the likelihood is much higher, but still very low.

The statistics we have are annual. There are national and city data that show the number of crimes committed in the year. From those data, we compute annual per capita numbers.

By the way, those official data are for completed crimes. They do not include the number of times that the presence of a firearm have prevented.

The annual data still appear very low, but we rarely make carry decisions on the basis of how likely we are to need a firearm at least once during this calendar year. We expect to live longer than that.

Over the long haul, the likelihood is much higher. Statistics are funny things. The per capita likelihood of being victimized at least once between the ages of twelve and forty is high enough to cause anyone to be concerned.

To calculate that takes some number crunching.

We'll try to illustrate that phenomenon this way.

If we cast a die, the likelihood of coming up with a six is one out of six. It's the same for every throw.

Now, consider the odds of never coming with up a six, not even one time, over a number of throws of the die.

We would calculate those odds though what is known as the binomial expansion. But we shouldn't really have to run the numbers to realize that it would be very, very unusual indeed to not roll a six at least once, in twenty-five rolls of the die.

The odds of coming up with a six at least once is 100% minus the chance of not coming up with a in any throw. It is a huge number.

That's precisely the way we would calculate the odds of being victimized by crime at least once in a several year period. The annual per capita odds are very low, but the likelihood over time builds up more quickly than one might expect.

The Impact of Important Variables

Those annual crime statistics are made up of broad populations of different kinds of people. They include people who do not go to bars and people who do; people who work in quick-shops, and people who don't; people who have to make deliveries in high crime areas, and people who don't; and so on.

Thus, we really cannot conclude very much about our risks from published crime data.

********

We can now turn to the question of what statistical data may mean, or not mean, when it comes to the evaluation of the effectiveness of different handguns for self defense.

The question of handgun calibers might serve a defender best is one that cannot be addressed by analyzing data from actual events.

There have been a few studies that have attempted to answer the question of which calibers are the most effective for defense. Unfortunately, because the data are devoid of information regarding the may different things that determine how quickly a person struck by bullets is stopped, they do not provide meaningful answers.

The Impact of Important Causal Correlations on Handgun Effectiveness

One main problem is that there are many, many factors that go into the making of an immediate physical stop. Stopping a human aggressor with bullets is not at all like knocking over a steel plate or blasting a water jug. The human body is not a homogenous mass. It is a very complex collection of interrelated cardiovascular systems, respiratory organs, central and distributed nervous systems, tendons, bones, and other things,

Immediately stopping a violent criminal aggressor is a function of what specific tissue within the body is destroyed by how many bullets, and with a moving target that's largely a matter of luck. Other factors include the predisposition of a person to fall, chemical influences (drugs and adrenalin), size and fitness, etc.

That depends upon a lot more than terminal ballistics.

Post-mortem forensic evaluation will describe body damage, but there are no databases of detailed information for stops effected in violent attacks.

Limitations Imposed by Sample Size

Even if we had insight into what has taken place in actual shooting incidents, we would not be able to make any sense of it. Special Agent Urey Patrick of the FBI Training Academy at Quantico, VA, who is an expert in the field of handgun wounding mechanics, tells us that, due to the complexity or the human body and what it takes to disable it, data from even a hundred shootings would neither provide much in the way of information regarding causal correlations or give us a basis for prediction.

Special Agent Patrick goes on to illustrate the uncertainties of statistical data by discussing the simplest of all possible examples.

We all know that when we flip a coin, the probability of it landing heads up is 50%--on average.

But there is a lot of variation around that average, Patrick describes having flipped some coins.

In one twenty-flip trial, the coin came up heads five times. In another, the coin came up heads eight times

Was there something wrong? No. That's just the nature of statistical uncertainty.

Again, that's the simplest possible example. There's only one variable. When we consider all of the variables involved in human wounding, we see that trying to use actual data to draw meaningful conclusions would be folly.

No, we cannot rely on actual statistics to assess handgun effectiveness in the real world.

**********

That leaves us with the question of how we can make decisions reasonably.

Having realized that statistics from actual incidents won't help much, the concealed carrier is faced with the question of how to make informed decisions on firearm selection.

We'll try to suggest some pointers.

Regarding Caliber

Many people tend to really overthink this subject. In reality, if a cartridge is good enough, that's what really counts. Experts in the field of handgun wounding mechanics have concluded that an additional few thousandths of an inch in bullet diameter, or a few extra foot-pounds of kinetic energy, will not make any meaningful difference. Bullets do not "knock down" people. The only things that really count when it comes to terminal ballistics are penetration and a reasonable expanded diameter.

Realistically, any of the standard service calibers--.38 Special, 9MM, .40 or .45, will, when loaded with premium defensive ammunition, "do the job" if the bullets hit in the right places timely.

As a matter of fact, cartridge effectiveness is among the least important factors in self defense.

Choose one and don't worry about it.

******

Regarding the Size and Weight of the Gun


This is an individual issue.

Small guns are easier to conceal, but they are harder to shoot. Before choosing one for primary defensive carry, one should take it from the practice range to a defensive shooting class and try using it in some of the realistic drills, and/or try it in competition.

As a rule, most people can shoot a handgun with an adequate grip, a sight radius that is reasonable, and enough weight to dampen the recoil more effectively than a very small, light, ultra compact firearm.

However, a handgun that is too large and heavy is not only more difficult to conceal for most people, it may be uncomfortable and not lend itself to all-day carry.

The choice will involve compromise. The decision cannot be made by anyone but the carrier.

**********

Regarding Ammunition Capacity

This obviously enters into the question of size and weight.

It will largely be a matter of judgment--informed judgment.

Five shot revolvers have been among the most popular of the concealable revolvers since the demise of the original six-shot Colt Cobra and Detective Special. In "revolver days", that additional round was considered by many law enforcement to provide an important advantage, and six shot revolvers for concealed carry are returning to the market-place.

Most semiautomatics provide still more capacity. Some pistols with single column magazines contain eight or ten rounds.

How many rounds will one need, in the grave extreme of a violent criminal attack? There is no way to know that in advance.

The most important thing to remember is that a bullet that strikes an assailant but that does not destroy critical tissue may have little immediate effect.

More shots on target will increase the probability of a timely physical stop. That's why many of the drills in defensive pistol training call for the firing of several very rapid shots, and why defenders are trained to fire until the attacker drops.

On the subject of reloads: they are a good thing, but when one considers that a defender will be hard put to draw and fire timely in a Tueller Drill scenario, the difficulty of trying to add a reload to the drill becomes evident.

It may be useful to watch some videos of some actual LEO encounters. That should help dispel both the myth of the "one shot stop" and the oft-heard accusation of "spray and pray" shooting.

Watching re-runs of The Best Defense TV can also be helpful.

Good training will help a lot. We cannot over-emphasize the recommendation for training. Competition would also give an appreciation of having adequate ammunition capacity

The carrier must make the ultimate decision, hopefully on an informed basis.

Most people would prefer to end up with few rounds left over rather than running out prematurely . There is, of course, practical upper limit, defied by how long a defensive shooting incident can reasonably be expected to continue.

In any event, looking at averages from actual incidents would not be helpful. In fact, averages, by their very nature, are rarely useful for much of anything that has to do with the needs of an individual.

One other thing: people may, from their practice sessions shooting at a single stationary target, start thinking in terms of defending against a single assailant. However, while that is a possibility, it is not a prudent assumption. Instructor Tom Givens emphasizes that multiple assailants--two or more--are becoming more and more common.

Persons wanting to consider what the probabilities of rounds expended may be under different assumptions may want to search for posts by JohnKSa in this forum.

**********

We hope this proves helpful.

Here are some take-away points:

  • Don't spend time looking for the magic cartridge, and to not believe that more power will provide additional "knock-down" capability.
  • Do not expect one shot stops of the kind we see on television.
  • Understand that immediate physical stops depend upon what is struck inside the body, and that since that is largely matter of luck, more shots fired rapidly will give a better chance of success.
  • Do not conclude that, in circumstances in which the likelihood of experiencing a violent criminal attack may be relatively low, what would be required to respond to such an attack should it occur would be any different than in any other place or time. One should think not in terms of likelihood, but in terms of what kind of incident one wants to be able to handle.
  • Don't under-estimate the importance of good training.
And..,do not try to make decisions on the basis of statistical averages. Averages, whether we are speaking of arithmetic means, modes, or medians, are only measures of central tendency, and they are not predictors of likely, best-case, or worse outcomes.

For anyone interested, the likelihood of a risk occurring over an extended period of exposure, illustrated above in the example of casing a die, can be calculated by using the Binomial Probability Calculator.


Binomial Probability Calculator
Use the free, online Binomial Calculator to compute individual and cumulative binomial probability. For help in ...
 
Not to mention that most of the statistics cited by people to justify their choices are bogus anyway.
Well, the oft-repeated "3 shots in 3 seconds at 3 yards" and its variations are certainly highly suspect.
 
I don't do statistics, psychic ability, magic 8 ball.
My carry philosophy:
Of the concealable guns one owns, carry what would be preferred in hand if you had to defend yourself, regardless of location or light.
There is nowhere I'd prefer to defend myself with a 38 snub / LCP 380 than a Glock 19/23 and I carry accordingly.

If one prefers minimal carry, its on them.

44il4l.jpg
 
I've experienced too many things which statistically were unlikely to happen to pay attention to statistics. They include walking into an armed robbery in front of my house in my "safe" Chicago suburb shortly after we moved here, four occurrences of "safe" medical procedures which turned out badly for myself, my ex twice and my dad, who died from it and a friend getting sick while drunk and choking to death. Statistically the odds of any of these things happenening were minimal. I understand that most of these are not gun related, but my point is that IMHO relying on statistics to make decisions is not wise.
 
Last edited:
Instructor Tom Givens emphasizes that multiple assailants--two or more--are becoming more and more common.
John Correia, of Active Self Protection, made the same observation, based on reviewing hundreds (thousands?) of videos of actual self defense shootings for his ASP channel on YouTube. John spoke of this in one of his presentations at the NRA Self Defense Expo in Sept 2019 in Fort Worth. That was a major factor in my decision to upgrade my EDC from a 5 shot S&W 642 .38 snubby to a double stack 17+1 M&P 9mm.
 
I'll dip my toe into this one. Statistically , what is most likely to happen in a sd situation? You will most likely only deal with one or two idiots with one or two shots being fired. I've watched a lot of ASP videos also. If you look at individual robberies ( vs store ) the most important things are paying attention and being ready. I carry my 642 regularly. I can have a full grip on it and no one would be the wiser. If I take just a second I can hit a Can at 30 yrds. At spitting distance it's point and shoot. Nothing wrong with a g19 but I feel adequately armed with my 642.
 
I don't do statistics, psychic ability, magic 8 ball.
My carry philosophy:
Of the concealable guns one owns, carry what would be preferred in hand if you had to defend yourself, regardless of location or light.
There is nowhere I'd prefer to defend myself with a 38 snub / LCP 380 than a Glock 19/23 and I carry accordingly.

If one prefers minimal carry, its on them.

View attachment 922918

I carry pocket guns and I carry a Snub 38 and 9mm. And I can use them well and very proficient with them, and train often and diligently with them. . And have been carrying them for years. I know many that carry a Glock 19 etc that cannot shoot them worth a hoot.
Some statistics say I should NEVER leave my house, While other Statistics say I am most likely be injured IN my house.
 
Last edited:
You will most likely only deal with one or two idiots with one or two shots being fired.
Could happen, but what would that mean?

One point: trained defenders are taught to shoot until the attacker goes down. That has to do with things anatomical. It would be most unlikely for a trained defender to shoot one or two shots in a violent criminal attack.

I feel adequately armed with my 642.
How one may "feel" will not materially affect the outcome.

I do not expect tp be victimized. But should it happen, I think that my being "adequately armed" will be a mall factor in determining wether I will come out of it unscathed.

I have no reason to believe it highly likely that I would.


.
 
Status
Not open for further replies.
Back
Top