The list won’t be exhaustive, and I wish all of the things on here were simple and had simple solutions. Most martial arts training isn’t effective for SD purposes. Some of it is on the teaching end, and some of that is on the learning end, and some is in the culture.
That last sentence might take a long time to sink in, so examples: Teaching by rote wires the reactions to the wrong part of the brain for use in a fight. Bad teaching. The most exciting aspect of training feels the most realistic, even if it requires the most artificiality to maintain safety. The student will take away the wrong lesson. Bad learning. And a hierarchical cult of obedience is exactly what an exploitive predator dreams about. Sometimes the culture farms victims.
The dueling paradigm. Other than for fun or sport or balancing things within a social group, people don’t square off. Because it’s dumb. If you had to take out the biggest, scariest martial athlete you can imagine, how would you do it? Exactly. From behind with a weapon. And maybe friends.
This has a lot of implications for MA/SD. The paradigm sets you up to expect distance, time and warning, none of which will exist unless you are monkey dancing. People who are successful at dueling or sparring believe (sometimes, I hope rarely) that the skills will transfer to ambush survival…and they don’t.
Unarmed. With the exceptions of corrections, hospital security, and secure mental facilities, almost no profession goes hands-on unarmed. Because it is stupid. If you know things are going to go bad, you get a weapon. And friends. And intel. And surprise. Crime, by the way, is another profession that uses weapons. And this goes back to the dueling/sparring paradigm. To get good at unarmed dueling is to develop skill at a very bad strategy, a strategy which has the sole purpose of stroking your ego. Don’t quit playing. I love to play. But don’t make it something it’s not. If someone was trying to kill someone you loved would you tap them on the shoulder and step back so that they could face you at the appropriate distance? Or would you hit them in the back of the neck with the best tool you could find? Your choice, but one choice is stupid and that choice is the one you have likely practiced most.
Inbreeding. You train together and you get used to dealing with each other. When I taught at a dojo, my class were infighters. They were really good at doing all the things that infighters do. But nature of a class setting, they were spending all of their time practicing against other infighters, which is a pretty rare category. Frankly, this is a slight problem for infighters and grapplers. It takes very little to close range, especially at ambush distance. But it can be a huge problem for strikers.
Bad metrics. How do you measure if something works? The military has a “Lessons Learned” program. My team, and Search and Rescue and even the Reception crew when I was sergeant there used After-Action Debriefing protocols. This will get you better continuously– provided you have actions to debrief. Without those actions, it is much harder. I wonder what percentage of students of an SD instructor are attacked on average, how often… but I feel the numbers are too low.
When people don’t have a reality check they have this really stupid tendency to make up a reality check. ‘Make up’ and ‘reality’ rarely belong in the same thought. I almost always pick on karate for this. When I look at their kata and kihon, they have possibly the best body mechanics for infighting that I’ve seen… then they choose to test it at sparring range, where it sucks. Or, worse, point contact range where it sucks AND it screws up everybody’s sense of distance and time.
Scenarios can be solid gold to test some things, but only if the scenarios are incredibly realistic (ideally based on real events) and the role players are superb actors and the facilitator really knows his stuff, especially the debrief. Without that it can ingrain incredibly bad habits. Doing the instruct’s fantasy at high speed is still doing fantasy.
The Safety/Effectiveness scale. Fighting, especially recovery from ambush, is a very dangerous thing. One of the biggest challenges is training people to fight without injuring them. Straight up, if neither you nor your opponent are scared or need medical attention, it’s not a fight. It has nothing to do with fighting. Trying to approximate the skills without the injuries is a very fine line. Weapons arts have the advantage in that they can make the weapons safe. Much harder to do with throws and neck twists. MA tend to make the techniques safe…and more safe the higher speed the training. And so the safety artifacts ingrain right along with the techniques.
Modality. Related to metrics. The measure of effectiveness is how much damage something does. Do bones break? Does the guy go down? Again, less of a problem in grappling arts, but a hellish issue in striking arts. When you can’t actually do what you are supposed to do (collapse tracheas or cause concussions, say) the instructor’s default seems to be whether it looked right. Fighting is about touch, not about looks. Pretty, crisp, geometrically clean can be seen. Power and structure need to be felt.