May 2021
Early intervention systems (EIS) are data-driven tools used by law enforcement agencies to identify officers who may be at risk of misconduct, public complaints, or other negative outcomes. Such intervention systems are designed to be preventive rather than punitive, reducing the chances of an adverse event by providing identified officers with supervisory support and services, such as structured supervisor-officer conversations, therapy sessions, and trainings. In addition, the use of an EIS is intended to enable police departments to prioritize officer support resources by zeroing in on the officers who are most likely to benefit from preventive intervention.
Summary Assessment
- Early Intervention Systems (EIS) may be effective in reducing police violence and other forms of misconduct. However, there is scant research finding direct, causal effects of such systems on officer performance outcomes.
- Systems that employ statistical prediction models – as opposed to rule-based threshold systems – more accurately identify officers at high risk of an adverse event.
- To effectively reduce the risk of police misconduct, interventions for officers identified as a high risk must be tailored to the individual’s specific risk factors and needs.
- Interventions that rely on supervisory coaching could benefit from the review and discussion of body-worn camera footage from relevant incidents.
- Systems that rely on administrative data may perpetuate structural biases that lead to disparate identification of officers based on race, gender, and other personal identity characteristics, unless they are properly vetted for equity and fairness considerations.
Current Practice and Research
Law enforcement EISs were first implemented in the 1980s in response to a recommendation from the U.S. Commission on Civil Rights that all police departments use such tools to better hold officers accountable for their behaviors. They gained prominence in 1997 when the U.S. Department of Justice (DOJ) required the Pittsburgh Police Department to implement an EIS as part of a consent decree resulting from DOJ’s first “pattern-or-practice” investigation. Since that time, use of the tools has grown, but overall remains relatively rare, with only 12.4% of law enforcement agencies (including sheriff’s offices and state police forces) having such a system in place by 2016. As with many system-analytic technologies, EISs are more likely to be used in larger police departments. For example, in 2016, 69.3% of the approximately 540 local, county, or regional police departments in the U.S. with 100 or more officers had an EIS, and 80.5% of the approximately 87 agencies with 500 or more officers had such a system in place. These statistics do not provide insights on how regularly or consistently an agency uses these systems, nor is it known which interventions are applied when an officer is identified. For those agencies that use an EIS, the most common performance indicators examined are use-of-force incidents, officer-involved shooting incidents, community-initiated complaints, community- or department-initiated commendations or awards, departmental disciplinary actions, reassignment history, officer-involved civil suits or administrative claims, possible or reported officer misconduct, and vehicle pursuits and crashes. Agencies use one of two types of EIS tools: a rule-based tool, which flags officers whose indicators exceed a pre-determined threshold of events, and prediction models, which employ predictive analytics that enable the use of multiple risk factors through identification of statistical relationships. Rule-based EIS tools are more commonly used than prediction-based ones.
Research Evidence
The effectiveness of EISs depends on two factors. The first is the predictive accuracy of the system—whether the EIS can accurately identify officers who are at high risk for a future adverse event. The second is whether and how the agency acts on the results of that prediction—for example, whether a supervisor or other unit intervenes with the officer (and the degree to which that intervention is effective).
Assessing EIS Predictive Power
The most common EISs use a rule-based thresholding which produces an alert when an officer’s indicators exceed a pre-determined threshold value. Such alerts notify the department or supervisor that the identified officer is in need of corrective intervention of some kind. The thresholds can be based on the raw counts of an indicator (e.g., a number of complaints within a certain time period), the ratio of two indicators (e.g., use-of-force reports to arrests), or a higher volume of a given indicator in comparison to other officers in similar positions and settings. While rule-based EISs have been in use for several decades, their predictive ability is poor in comparison to statistically-based models. However, rule-based systems may still perform better at reducing misconduct or other negative outcomes than the practice of randomly assigning officers to interventions or relying solely on programs for which participation is voluntary.
While rule-based EISs have been in use for several decades, their predictive ability is poor in comparison to statistically based models.
Newer statistical methods for EIS models that rely on predictive analytics are more accurate in identifying officers who will have a future adverse event. Prediction EIS models first identify indicators associated with adverse outcomes based on historical data and then integrate that information with multiple other risk factors to identify high-risk officers. These systems perform relatively well. For example, researchers conducted a case study in a metropolitan police department and found that high-risk officers identified by prediction models experienced adverse events at a rate 1.7 times higher than those identified by a rule-based EIS. Another study found that predictive-model based systems increased the accurate identification of high-risk officers by 75% while reducing false identifications by 20%.
In addition, research has found that the use of measures of misconduct that occur relatively frequently, such as public complaints, is easier to predict. However, departments tend to focus more on predicting adverse events that are more serious but less frequent, such as excessive use of physical force. This tradeoff should be considered during development of an EIS predictive model. Agencies and researchers alike also need to be able to study problematic officers and their backgrounds in order to further refine the accuracy of EIS tools. But the lack of transparency around officers who have disciplinary histories makes it difficult to further develop this knowledge base.
While prediction EIS models have garnered considerable attention within the academic research community, they have not yet been widely adopted in policing operations. Additional research is needed to evaluate the implementation and impact of such tools, employing different indicators and simulating a variety of predictive outcomes. For example, there is reason to believe that accuracy of prediction increases with an officer’s tenure, and that more recent events are better predictors than more distant ones. EISs are not without their critics. Some experts have noted the possibility that officers could refrain from engaging in certain activities and interactions with community members for fear of being flagged for intervention. Others caution that officers could feel unfairly persecuted as a result of being flagged, leading to increased job-related stress. Both of these unintended outcomes could result in higher rates of adverse events. In addition, predictive analytics such as those employed in a statistically-based EIS have come under increased scrutiny in recent years for their potential to introduce or perpetuate biases. While no research exists specifically on the topic of biases associated with the use of an EIS for police officers, it is conceivable that structural biases associated with the data used for predictive purposes could lead to disparate identification of officers based on age, tenure, race, gender, and other personal characteristics. For example, supervisors may be biased when choosing who to discipline based on the officer’s race or gender, generating data that feeds into the EIS in a manner that bakes in these biases.
Preventing Adverse Events
Given that the main purpose of an EIS is to prevent an adverse event, it is crucial that the system not only accurately predict high-risk officers, but also that departments and supervisors employ effective responses in engaging with those officers. A wide array of responses could be employed with high-risk officers, ranging from coaching or training on respectful and procedurally just interactions with community members, to the provision of mental health, behavioral (e.g., anger management), or substance abuse services. Most models begin with a meeting between the high-risk officer and his or her supervisor to review prior incidents, discuss plans for handling similar situations in the future, develop clear remediation plans to hold officers accountable to these interventions, and determine if additional services are needed. As such, successful EISs rely on the agency’s existing accountability infrastructure, in particular the fairness, consistency, transparency, and organizational justice that shape internal disciplinary systems.
A wide array of responses could be employed with high-risk officers, ranging from coaching or training on respectful and procedurally just interactions with community members, to the provision of mental health, behavioral (e.g., anger management), or substance abuse services.
Studies on the impact of EISs on public safety and officer performance outcomes have yielded mixed findings, with very few identifying direct causal effects. A study using an experimental design examined the differences in a number of public complaints between 118 EIS-flagged officers who had to participate in an officer-community member interaction training program and a matched control group of officers. The number of formal complaints significantly decreased during the study for both treatment and control groups, with no significant differences between the two groups in the number of complaints. Other studies have employed less rigorous pre- and post-intervention designs. The earliest evaluation of a rule-based EIS implemented in Minneapolis and New Orleans found that the use of EIS and accompanying interventions reduced citizen complaints for flagged officers by 67% and 62%, respectively. Another study observed a similar 71.1% reduction in formal complaints among 44 Australian officers, and research with a larger sample of 526 New Zealand officers found a 72.5% reduction in complaints. An examination of the organizational-level impacts of EISs on formal complaints of police use of force yielded different findings. Drawing from a sample of 94 agencies that were identified as having implemented an EIS between and 2003 and 2007, the author found no statistically significant difference between the mean rates of public complaints of use of force by the agencies from pre- to post-EIS implementation. While three of these four studies suggest EISs are highly effective, they are hindered by weak designs because they did not control for intervening historical and other factors that could also have influenced outcomes of interest. Given such deficiencies, it is impossible to draw any cause-and-effect conclusions about the impact of the early intervention tools.
While EISs have been available to law enforcement agencies for decades and used routinely in many of the largest police departments, the empirical evidence to support their effectiveness has not yet caught up with practice.
Another study employing a rigorous randomized controlled trial design examined the impact of supervisory coaching of officers with a high predicted risk of use of force, injury, or public complaint. In this intervention, involving the Seattle Police Department, officers were not identified via a formal EIS risk tool but rather by a generated risk score based on the level of crime on the officer’s beat and the officer’s number of sick days, secondary jobs, and overtime hours in the most recent month. The intervention consisted of meetings with supervisors to review a recent “priority one” incident or one for which a report had been filed or an arrest had been made. As part of the process, the supervisors then walked officers through their thought processes in a respectful manner that invited personal reflection. While no differences were detected in complaints filed against officers or in measures of community engagement between treatment and comparison groups, the evaluation found that treated officers made 25% fewer arrests per incident immediately following the supervisory sessions, which declined to a 12% reduction several weeks later. Reductions in use of force were also observed, ranging from 15% to 50% over time. These findings offer some insights on the potential effectiveness of an intervention that could be employed following EIS identification. While EISs have been available to law enforcement agencies for decades and used routinely in many of the largest police departments, the empirical evidence to support their effectiveness has not yet caught up with practice. Few studies have applied experimental research designs that can make causal claims about the benefits EISs may have on policing outcomes, and very little research has been conducted on how EISs and associated interventions differ across officer demographics, jurisdictions, and types of law enforcement agencies.
Critical Policy Elements
- Because the impact of an EIS will depend on a variety of decisions about their measures and identification criteria, agencies should approach the use of such systems carefully and holistically, ensuring fidelity to both the officer identification and the supervisory intervention components of the system.
- Prior to implementing an EIS, police agencies should ensure that high-quality data exists to enable the prediction of adverse outcomes that are both high-priority and predictable. This can include common outcome measures such as citizen complaints and use-of-force events; officer activity measures, such as citations and arrests; and officer history characteristics, such as positions, assignments, and tenure. Smaller agencies with less data infrastructure may find it difficult to systematically collect and employ these data.
- Agencies should recognize that the data used in EISs may be influenced by the biases and stereotypes of supervisors, which may lead to inequities in the selection of officers flagged for intervention.
- To increase accurate prediction of high-risk officers, agencies should adopt prediction EIS models rather than rule-based threshold models. In addition, agencies should carefully consider which outcomes are most important for them to prioritize with an EIS. This involves assessing the tradeoff between systems that strongly predict common occurrences, such as public complaints, versus those that accurately, though not quite as precisely, predict less frequent but more serious events, such as use of force.
- The thresholds or statistical measures used to identify high-risk officers should be calibrated based on the agency’s ability to provide a robust intervention. Identifying too many officers may result in the dilution of the intervention, given limited resources, and render it less effective.
- Agencies should make anonymized data on officer disciplinary histories available to researchers. This will help further refine the mix of problematic behavior that should be included in EISs and improve the predictive abilities of these systems.
- Interventions should be designed to be remedial rather than punitive, existing outside of departmental disciplinary systems with the goal of improving an officer’s wellbeing and correcting the problem behavior early on before it escalates. Punitive measures may yield unintended consequences, such as reducing officer engagement with policing activities.
- Interventions with officers that emphasize conversation and coaching by supervisors might benefit from review and discussion of body-worn camera footage from a recent incident. This process may encourage officers to reflect upon what did or did not go well and how their actions could have been altered to yield a more positive impact.
- EISs are only as good as the interventions that are used to support officers toward better performance, as well as the accountability measures an agency uses to ensure that officers follow through on remediation plans. Departments need to tailor interventions to individuals, drawing from supervisor conversations, therapy sessions, trainings, programs, and services.
Expected Impacts
Preventing Misuse of Force
Identifying officers at elevated risk of an adverse event and intervening with supportive coaching could reduce the likelihood of future use of excessive force. However, this outcome depends heavily on the accuracy of the agency’s identification of at-risk officers and the nature and implementation of the intervention.
Enhancing Transparency and Accountability
EISs are designed to improve an officer’s preparedness for difficult situations. Agencies cannot hold officers accountable for their predicted risk of some future event. However, agencies could release aggregate counts of EIS-flagged officers and the services they received, and can also be fully transparent about how officers are identified, publicly listing the specific risk factors used and what outcomes are being prioritized.
Strengthening Community Trust
If the services provided to officers identified as high risk for adverse events include measures that lead to more respectful and less excessive responses to interactions with community members, community trust in the police could improve.
Reducing Racial Disparities
There is no research indicating how EISs may reduce racial disparities in policing practices or outcomes for members of the public. Also unknown is the degree to which EISs could be designed to detect racist, biased, and sexist behaviors.
Ensuring Officer Safety
There is no research evidence on the degree to which flagged officers who participate in interventions are more or less likely to experience assault or injury.
Promoting Public Safety
There is no existing research on the direct, causal effects of EISs on public safety and officer performance outcomes in practice.
