Sampling error, cherry-picking and story telling – Rory Miller


Some background first.  There are a lot of experts on criminal behavior out there.  There are also a lot of ‘experts’.

‘Experts’ first.
The biggest problem is that bad guys lie.  A lot of the so-called experts were clearly snowed.  They were manipulated from the very beginning to tell the story that the bad guy wanted told, and they were either not smart enough to realize it or, more likely, too arrogant to admit even to themselves that they were conned.  They then pass on information tailor-made by the criminal to citizens and other experts.

The second problem with ‘experts’ is agendas. Not just they agenda they may want to advance, but the agenda that they want to believe.  For most people, violence is big and scary and a lot of people are driven to do something and so they latch on to something they think might work, hope will work, and either insist that it will or create, or cherry-pick research to show that it will.  And this becomes pseudoscience really damn fast.  (Point of Order, boys and girls: Science, by it’s nature cannot prove anything. ‘Proof’ as such only exists in mathematics and logic.  The scientific method is the most powerful tool ever devised for disproving things.  It is the ultimate bullshit detector, not a truth detector.)

Experts, the genuine thing, are imperfect as well.  For bad ‘experts’ the problems are usually cherry-picking data in order to create a narrative to support their agenda.  In real experts, the problem is usually sampling error and the inability to do real research.

Real research first.  All the anecdotes and computer models and surveys would just be background research in real science.  They would be the data you looked at to design the experiment.  The nature of the beast is that these kinds of research, even optimally designed, can show correlation but not causation.  The classic case is the statistic that the more churches there are in a given area, the more violent crime there is.  Correlation is not causation.  You can jump on the statistic saying it proves religion causes crime or crime causes religion or religious people are hypocrites… but that is all just talking about your internal workings.  The simple fact is the more people there are in an area, the more crime there will be and the more churches (and grocery stores and schools and everything else) there will be.  Correlation is not causation.

In order to show causation, you must design an experiment.  Take a hundred cities of the same size and build  four extra churches in fifty randomly chosen cities from the hundred and see if the crime rate changes.  If crime rate remains unchanged, you have disproven the hypothesis.  If it changes, you have shown it might be the independent variable.  Might.  Nature of the Scientific Method.  Rules things out, not in.

Because of the nature of violence and society, no real experiment in violence and criminality will ever pass a university ethics board.  Which means that what experts we have are basing things on background research of often dubious value and their own experiences, which vary widely.

Case in point, and this is where it gets to the question about why so few criminals will kill over a property crime- I recently read an article by an extremely experienced super-max prison psychologist who stated that hardened criminals will reflexively kill to keep from being caught.  And implying that every robbery should be treated as a deadly force encounter.

Sampling error.  This author (who seemed a great observer with a ton of insight) dealt with a fraction of a fraction of the criminal population.  The ones who had done serious violent crimes.  Got caught. Couldn’t bargain it down either out of stupidity, history or stubbornness.  Couldn’t follow the rules on violence even under the scrutiny of the prison system (you don’t get to super-max by singing “Kumbaya” too loudly).  Within that population?  Hell, yeah.  The majority will kill for any reason or no real reason at all.  But that’s not normal.

We booked about 40,000 people a year at my old agency.  A handful of serial killers, killers, robbers…and lots and lots of petty thieves and druggies and drunk drivers and Domestic Violence cases.  The majority of those 40k didn’t stay very long.  Prisons are so crowded that judges, recog and other programs were looking for any excuse to keep people on the streets.  So most of the people who stayed for any length of time were violent and/or multiple repeat offenders.

Most of the robbers didn’t hurt anyone.  Because it didn’t suit their purposes.  The goal is to get money (usually for drugs) and not get caught (cause withdrawals are a bitch) and not get hurt (because it makes it harder to do crimes tomorrow.)  If they showed a weapon and you handed over your wallet, you’d likely report the robbery to the police, it would get a little attention, but an arrest would be unlikely.  Unless they found the gun exactly as you described it and something of yours like a credit card still on the bad guy, a conviction would be iffy.  Even with that, it would likely be bargained down.  This criminal will do a lot of crimes, but he’ll never see the inside of a super-max, and never make it onto that particular expert’s radar.

On the other hand, if the guy shoots his victims, he becomes a very high priority for enforcement and their is a ton more forensic evidence which makes a conviction more likely.  And long sentences. And, if he is also stupid inside prison, he will get to super-max.

That’s just practicality, but there is a social side to it as well.  DO NOT count on this dynamic in places where no crime will be investigated or where no one cares or there is no law.  This relative lack of violence is a practical adaptation to this environment.  It is not because the robbers I dealt with were ‘nicer’.

So, bring this back to you.
There are lots of experts out there as well as ‘experts’ and not one of us knows it all.  We all have our experience or our research or research that we have borrowed.  No one has definitive answers and we all have blind spots.  Listen to as many different voices as you have time for.  Try to pick out the agenda.  If a self-defense instructor’s answer to all problems is lethal force, he will be cherry picking sources to make that sound reasonable.  Don’t sweat it.  Recognize the agenda, absorb the non kool-aid parts and move on.

And, this is huge:  Examine your agenda occasionally. What you want to believe will always get in the way of what you know.

Leave a Reply