How is it possible that forty-two officers responding to the same training scenario would show almost no consistent planning or execution? That’s what researchers from Force Science, East Carolina University, and Montclair State University found as they analyzed data from their 2023 police response study.1
Readers familiar with Force Science News will remember that the live-acted 2023 study was designed to explore how experience and training might influence the visual scan patterns and performance of police officers during a critical incident. In the study, active-duty police officers were recorded as they responded to a simulated automobile crash that devolved into a shooting. Participants wore heart rate monitors, body cameras, and eye trackers to document arousal levels and visual scan patterns (i.e., where officers looked, when they looked, and for how long). The data provided important insights into the relationship between visual scan patterns, arousal, and overall performance during critical incidents.2
From Vision to Variability
While the initial focus of the 2023 study was on visual scan patterns and threat recognition, researchers made an interesting observation as they continued to analyze the data. Although they designed the research so that each pair of officers would respond to the same live-acted scenario, researchers noticed a high degree of variability in the officers’ decision-making and performance.
In this latest peer-reviewed study, researchers detailed how they measured the response variability and offered theories that might help explain the officers’ inconsistent performance. This article summarizes the response variability and describes the researchers’ novel approach to assessing the effectiveness of police training and experience.
Measuring Variability
One goal of the police-response study was to assess officers’ responses to the same scenario. As such, a critical component of the study was the consistency in which the live-acted scenario was executed. Dispatchers and actors were disciplined in their performance, and the information provided to the officers did not deviate from the script.
From there, researchers examined response variability in two broad areas and provided the following summary of observations.
Spatial Variability described where officers parked their cars, their subsequent movement patterns, and their final positions when the suspect discharged his weapon.
Researchers broadly noted,
- “The parking arrangement of the paired officers was never the same twice, considering both the vehicles’ orientation and their distance from the scene. Similarly, the movements of each pair of officers were unique. This implies no consistent tactical planning and execution across the 42 officers.”
- “The extensive dispersion in the officers’ movements was also reflected in very high [variability] for the officers’ final positions when the assailant fired.”
- “Participants using a more refined visual search were less variable in their final position when the assailant fired than those who had more scattered scan paths.”
Temporal Variability included the timing of the officers’ weapon draw, aiming, and firing in relation to the assailant’s weapon discharge.
Here, researchers provided the following key insights,
- “Weapon responses also showed great variability between officers.”
- “While several officers drew their weapons on exiting their vehicles, 36% of the officers did not unholster their weapons until after the assailant fired his weapon.”
- “The most variable weapon response was the timing of when the officers aimed their weapons at the assailant.”
- “Very few officers aimed their weapons at the assailant before he fired his weapon…”
A critical insight for use-of-force investigators was observed in how long officers took to return fire,
“While 81% of officers returned fire, this was, on average, almost 2 s [after the suspect fired]. This is much slower than those times reported in past research. For example, Lewinski and Hudson (2003) report .31 s for simple reaction times (one stimulus; one response) and 0.56 s for a decision-based reaction time. Similarly, Blair et al. (2011) found times of 0.39 s for a shoot/don’t shoot task. Our data show that in complex environments, it may not be reasonable to expect such rapid responses, and that time to react is powerfully mediated by the officers’ tactical positioning [emphasis added].”
Theory: Shared Cognition (“Team Knowledge”)
To help explain the officers’ response variability, researchers considered the potential role of “shared cognition.” Shared cognition, sometimes referred to as “team knowledge,” includes the ability of individuals to interpret and act consistently with members of their group based on a shared understanding of the operating environment. This shared cognition is reflected in the use of common procedural and tactical knowledge that results from shared training and experience.
Drawing on the robust literature exploring shared cognition, researchers theorized that the effectiveness of police training and experience may be inferred from the type and degree of response variability to the same scenario. Researchers explained, “This is, to date, an untapped method for assessing police training effectiveness, yet it has theoretical roots in the knowledge and shared cognition literature.” Explaining their theory, “If officers facing the same scenario respond in similar ways, this points to their use of in-common procedural and tactical knowledge accumulated through training and experience.”
Dr. Robert Horn, Associate Professor in motor behavior at Montclair State University and researcher on the current study, noted, “When individuals within groups receive sufficient training and experience, over time, we expect them to develop collective knowledge. That means we expect them to share a common body of task-relevant information, team-relevant information, and shared mental models. With shared mental models, we would expect to see consistency in how group members, in this case police officers, would interpret similar information. While we can always expect some variability, with shared cognition we would also expect to see evidence of adaptive situational awareness and situational understanding.”
Dr. Horn continued, “If police training and experience within a single department is effective at instilling shared mental models for use in critical incidents, we would expect shared cognition to reduce variability in officers’ performance. In contrast, as we pointed out in the study, highly variable responses to the same scenario suggest the absence of shared cognition and may point to the need for more influential training. In that case, we might recommend the addition of high-volume training that combines a variety of situational awareness and decision-making tactical exercises with timely and effective feedback.”
Where to Next?
Force Science will continue to explore the application of shared cognition research to policing, and the degree to which shared cognition might result in more consistent situational understanding and reduced response variability. Ultimately, we hope to understand better when assessment and response variability may be the product of poor training—and the absence of shared knowledge—and when it is simply the inevitable consequence of officers engaged in individual and otherwise reasonable perception, interpretation, and decision-making in dynamic and complex operating environments.
Related Force Science News
- Gaze Control and Tactical Decision-Making Under Stress in Active-Duty Police Officers During a Live Use-of-Force Response and published in the peer-reviewed Journal of Motor Behavior, 6, 2023. [↩]
- Robert R. Horn, William J. Lewinski, Gustavo Sandri Heidner, Joshua Lawton, Craig Allen, Michael W. Albin & Nicholas P. Murray (30 Nov 2023): Assessing between-officer variability in responses to a live-acted deadly force encounter as a window to the effectiveness of training and experience, Ergonomics, DOI: 10.1080/00140139.2023.2278416 [↩]
As a retired police officer with experience in patrol units and as a training officer, I don’t find any of the findings here to be shocking. In most of the police agencies I am familiar with, patrol officers do very little training as a group due to manpower and scheduling issues. You really can’t pull an entire shift away for a day or more without creating massive scheduling nightmares. As a result, officers report to training one or two at a time off a shift. Even with the training being scripted and trainers attempting to control the activity as similar as possible, officers are many times training with officers they may not train with on a regular basis or, depending on the size of the agency, they may not even know the other officer. Tactics and training by specialized units, especially tactical units, may have a more standardized and uniformed response. It would be great if manpower and scheduling would allow entire patrol shifts to train together on a regular basis. Not knowing who the test subject officers were or how their agency functioned, this may answer some of those “unexpected” results from this study.
My company is the only training company I know of that has addressed the issue of officers having no doctrinal tactical response to high threat situations. For that reason we developed a series of approach tactics and immediate action tactical drills, similar to what the military would call battle drills, to deal with very common to highly dangerous situations with branching options allowing everything from tactical withdrawal, de-escalation, or overwhelming force and hostage rescue response. Departments always teach felony traffic stops and then neglect the equally common felony response to threats on foot or in other situations. Most of these responses can be trained in a single day, implemented into other subject training material easily, and as long as they are enforced on the street, can give officers a huge tactical edge. In dealing with dangerous contacts. Training officers should consider similarly implementing immediate action drills.
Sam H. after reading the article I was thinking the same way as you. I am a retired police officer after 35 years of service and I was a firearms instructor and in the use of force and shooting situations. I always heard from management,” Hurry up get it done,don’t take to long overtime is through the roof.” I would be willing to bet with everything that is going on with law enforcement, through out the country, it must be a lot harder to get people properly trained. Oh and one other thing, officer don’t make a mistake in the use of force, if you do, you will suffer the consequences. Sam H. Excellent response.
congrats on your retirement.
Again, as above, retired police officer, patrol and FTO, SWAT, Detectives office with 30+ years in three departments both Small city, small county and medium city at retirement. Then 10 years with a federal DHS contractor. Many responses are variable, even with standardized training as officers were taught to THINK, and not embrace a lowest common denominator in training. Also permanent or very familiar partners react differently from two haphazardly assigned pairs with little prior coordination. Not all officers are comfortable making decisions, especially in a very structured or ridged command structure. Many aren’t comfortable in the use of force, especially in the current hyper political climate of second guessing their actions.
Another consideration, how long was the planning process, joint training and total simulated runs conducted by the SEALS for Operation Neptune Spear (Killing of Osama Bin Laden)? I’ll assume it was considerable with the sole intent of getting a team who has previously trained and worked together to perform at top levels and respond as a focused unit. Asking a typical American LE agency to be able to have the funding, staffing, training locations and qualified instructors to get even 1% of that performance is unattainable unless we are talking about individual SWAT teams. With the considerable differences in experience, age, bias, training focus (and interest shown at training), fitness levels of a typical patrol shift in America, the fact you saw these results should have been predictable.
In addition to some of the issues already discussed involving manpower, administration focus on training, and the like, there is also the lack of uniformity in training.
Many officers these days are seeking training outside of the department, which on its face is a good thing, but when said outside training teaches conflicting tactics it can greatly contribute to the issue you brought up. Especially as I’ve seen a trend of disregarding department training for various reasons including it not being in line with their favorite youtube instructor or “special ops” instructor training class they went to.
Also, as Kirkpatrick’s model shows us, training alone only accounts for about a 15 percent application of the training if it is not followed up with holding people accountable for actually applying the training, increasing to over 80% in situations where the administration has rewards and penalties for applying or not applying the training. You can have the best training in the world but if no one is held to actually applying what is learned you simply will have people who see no benefit to applying or retaining it.
I understand the idea that officers from the same department with the same training would have shared cognition, but I think that the articles authors may want to focus more on post training experience. While all officers from one department may have been trained to approach a scenario in the same manner, since training those officers have applied the training differently to real life scenarios, each taking what they felt the valuable parts of training were and magnifying them, while at the same time reducing parts of training the individual felt was not important. Those experiences, which become schema, altering how the officers respond, essentially making the training theirs.
The bottom line is this: POST Training and then any institutional training is and has been insufficient for tactical and shared cognitive responses for decades. Most LE are members of Public Employee Unions which care about officers being better employees but don’t actually care about officer safety or the public at large: There’s no money in it.
Retired now but after 30 years with a large agency in a large metro area with assignments from patrol to investigations to gang enforcement to narcotics to OIS investigations I’m not at all surprised at the variability of results for a few reasons.
First and foremost, I’d guess a majority of the officers in the experiment not only weren’t familiar with the officer they were paired with but very possibly had very limited experience with working two man cars. Most small to mid sized agencies employ one man cars except to train new recruits or as tactical units.
Next, training varies not just between agencies but over time in an agency. What may have been considered “good tactics” just a few years ago may not be strictly forbidden. When I began my career it was perfectly acceptable and even desirable for an officer to have a hand on his holstered firearm when approaching a suspicious vehicle on a traffic stop. As of about 10 years ago my former agency’s written policy not only discourages that but forbids officers to unholster their firearm in almost any situation.
Finally, a rise of anti-police sentiment in social media has made line officers very hesitant to use force even when force is both justified and needed. Look at the many videos available online of multiple officers wrestling with a combative arrestee. Just one punch would ended many of those situations much more quickly and without serious injury to either the arrestee or any officer.
It all goes back to time and funding. The new “training” in Law Enforcement is often just watching the on line video that training sends out to just check the box. And watch that video between runs And get it done so the Lt. Doesn’t get a call from upstairs. If your real lucky you might get hands on training once or twice a year. And now you are a Professional. Tiger Woods and Magic Johnson should have been so lucky,,,,it would have saved them so much time. Real in person scenario based training is the only way to go. It has to have reps, stress and time to do it. You can’t play sports once or twice a year and be any good. But the powers above want police to be perfect doing so. And don’t get me started on the prosecution of officers who can’t.
Based on the comments above it seems that some are thinking that variability in the officer’s response was automatically a bad thing. I did not pick up that idea from the article itself, but maybe I missed it. Either way, I do think it is a fair question to ask “Is all this variability wrong?”
Anyone who works in Law Enforcement will tell you that there a hundred ways to do our job. Some are great, some are bad and a large amount of them are acceptable. Suboptimal sure, but not what could be labeled a failure.
Here is my point, was the study looking to find how officers successfully deal with a dynamic incident or was it looking for the one “right” way to deal with this specific incident? Hopefully this type of research can help flesh out the overarching principles of violent encounters, like priority of life, threat recognition, using cover, etc.
For instance, a gun fight in many ways is similar to a physical fight, in that there is no one right way to win. If this study instead was having 42 officers fight the same guy, one at a time in a fist fight (lets just pretend the role player never got tired and threw the first punch the same way every time), no one would be scoffing at the fact that the officers all reacted differently. Some would be successful and some wouldn’t. Those that were successful would probably not use the same means to be successful. Some may have better boxing skills and punched, others used wrestling techniques to get a takedown while others maybe used a judo throw. Either way, everyone who was successful was either lucky or they all followed some over-arch principles of fighting (distance management, staying mobile, getting inside position, etc.).
But to take the results of this study as “Since all these cops did this differently , a bunch of them must have done it wrong” I think is short-sighted. Rather, “what principles and concepts did all the successful cops do? We should train more of those.”
There is an issue here no one has mentioned: responding to an accident is not a tactical event. It was an accident response, per the narrative given, that turned into a shooting. If that was not the case, and the shooting occurred before officer’s arrival, different story. That needs some clarification for me.
Because if officers pulled their guns getting out at an accident scene, they need to be corrected. That is not appropriate. Could be an issue of the training priming them to do so.
Then this is an ambush situation. It is no surprise that there would be a tremendous amount of variability in response to that.
Various organizations are teaching pre-incident planning and decision making. PERF’s ICAT does so. The Washington State Patrol Tactics program discusses it.
We can do all the studies, we can talk to all the retired consultants and opinion makers, and we can end up right back where we started. There is zero accountability for law enforcement failures in the bigger picture, and rarely for the individual officer. Increasingly the latter faces jail time, for essentially “not doing what his agency failed to train him to do.” If we want to reduce variability in tactical response, then increase agency accountability for failures to train their officers, and then, for failing to train their officers regularly in relevant and validated subject matter.
This is well known to everyone paying attention. Yet here we are.
I’ve enjoyed reading the article and the responses from everyone so far. I will say this, I do understand where Mr. Leblanc’s mindset is and I do agree to a point. I’ve been in our profession for almost 30 years now and I can definitely state that we have all seen numerous case law as it reflects to the failure of a department and its supervisors to train their officers. We know we have the obligation but often we fall short for various reasons. We know we must do better in this area and I do believe we are trying.
I did like the comment in regard to seeking outside training and I do feel that this can be both good and bad. The good part is exposing officers to other ways to do things, the bad is simply that the outside training is a nice thing to have but typically it is not adopted by the department as a whole, which defeats the concept of shared cognitive/team knowledge.
If leadership sends officers to outside training or brings in outside training, then leadership must decide, once they have had after action review, whether they want to use those concepts learned as department policy. If they do and adopt that training, then they can accomplish the shared cognitive/team knowledge. I will say this, it is extremely frustrating for an officer to go get some great training and then leadership does even want to hear about that training and how it can help the department. I have personally experienced this and it made me feel like I wasted my time.
In closing, I do believe we are trying, for instance (go, bad, or indifferent), ALEERT training. When a department says this is the standard they are going to use for Active Shooter, and train the officers to that standard, they develop that shared cognitive/team knowledge they need to succeed. We still have a long road to travel, but we can never give up….. Stay safe brothers and sisters, we are not done yet.
One of the aspects of this study that stuck to me is that there is no way to control all of the factors that would influence an officer’s perception and response. Human behavior in complex systems is often self-organizing, meaning it naturally evolves in response to changing constraints without explicit external control. This can lead to diverse outcomes as each officer self-organizes their behavior in response to the unique combination of individual, task, and environmental constraints they experience. Even Graham v. Connor considers that an office’s actions don’t have to be “right”, for example matching a believed shared cognition ideal. The officer’s actions have to be reasonable. Keep in mind that two reasonable people can have different views and responses. From an ecological dynamic perspective, you can control the environment and the task, as they did in this study, but you cannot control all of what a participant brings and how they will self-organize. Many factors play into this: height of the officer, experience, familiarity with the area, the last call they took similar to this, and confidence in skill set. These all could change the response. A taller officer with a better view may approach differently than a shorter officer who may maximize cover (6’6” vs 5’2”).
Each officer has unique physical, psychological, and experiential characteristics. These constraints include physical fitness, cognitive abilities, past experiences, training levels, and personal beliefs. These individual traits heavily influence the way each officer perceives and responds to a situation, leading to varied responses to the same stimuli. This concept describes how individuals perceive their environment and how this perception guides their actions. Officers continuously interact with their environment, with their perceptions and actions influencing each other cyclically. This leads to emergent behavior, where similar external conditions can result in various actions depending on how each officer pairs perception with action.
One of the aspects of this study that stuck to me is that there is no way to control all of the factors that would influence an officer’s perception and response. Human behavior in complex systems is often self-organizing, meaning that it naturally evolves in response to changing constraints without explicit external control. This can lead to diverse outcomes as each officer self-organizes their behavior in response to the unique combination of individual, task, and environmental constraints they experience. Even Graham v. Connor considers that an office’s actions don’t have to be “right”, for example matching a believed shared cognition ideal. The officer’s actions have to be reasonable. Keep in mind that two reasonable people can have different views and responses. From an ecological dynamic perspective, you can control the environment and the task, as they did in this study, but you cannot control all of what a participant brings and how they will self-organize. Many factors play into this: height of the officer, experience, familiarity with the area, the last call they took similar to this, and confidence in skill set. These all could change the response. A taller officer with a better view may approach differently than a shorter officer who may maximize cover (6’6” vs 5’2”).
Each officer has unique physical, psychological, and experiential characteristics. These constraints include physical fitness, cognitive abilities, past experiences, training levels, and personal beliefs. These individual traits heavily influence the way each officer perceives and responds to a situation, leading to varied responses to the same stimuli. This concept describes how individuals perceive their environment and how this perception guides their actions. Officers continuously interact with their environment, with their perceptions and actions influencing each other cyclically. This leads to emergent behavior, where similar external conditions can result in various actions depending on how each officer pairs perception with action.