You can’t manage what you don’t measure. This is true for all organizations, but the stakes are higher for EMS agencies, whose decisions have the potential to impact patient care.

When agencies measure clinical and operational performance, they have statistical proof of what’s working well, what needs to be improved, who should be celebrated, and where coaching needs to be implemented. In short, the agency can better understand how to positively impact patient outcomes. So, if measuring and analyzing data is that important, why isn’t everyone doing it?

Often, the primary barrier to measuring and analyzing performance is that it takes time. The busy daily operations of an EMS agency can mean performance reporting doesn’t get the time and attention it deserves. Secondly, it isn’t easy to know what you want to measure and how to measure it. This requires knowledge of what’s important to assess from a clinical and operational perspective, as well as knowledge of the agency’s software and what data points you're able to pull and report on. Finally, many agencies get backed into a corner because they try to measure too much at once and become overwhelmed by large amounts of data that are difficult to distill into actionable information.

These barriers can and should be overcome to obtain an objective assessment of agency performance and drive advancements in care.

Understanding Statistical Methods To Analyze Performance Data

Simply put, statistical analysis is a method of reviewing data to determine, based on patterns and trends, how well or how badly an agency is doing. It’s an objective evaluation framework that also “gut checks” the validity of the data using averages and standard deviations.

There are various methods of statistical analysis and a vast range of capabilities in this field that are only limited by your imagination and your willingness to experiment with and explore the data you already have at your fingertips. Some standard methods of statistical analysis include:

• Standard deviation is used to determine how far apart the data are from the mean or average of the group. If most values are close together, the standard deviation will be small, but if the values are spread apart, the standard deviation will be large.
• Analysis of variance (ANOVA) evaluates the difference between the averages of two or more groups. This statistical analysis tool separates the total variability within a data set into two components: random and systematic factors.
• Regression analysis attempts to determine the relationship between one dependent variable and one or more independent variables.
• Time series analysis examines data points collected at regular intervals over a period of time to uncover underlying patterns and trends.
• Chi-square tests examine if two categorical variables are related or independent. It helps us understand if the observed data differs significantly from the expected data. This is typically used with large data sets.

The people responsible for using these statistical analysis methods to evaluate the agency’s performance are typically the shift supervisors, managers, and—for larger companies—dedicated data analysts.

Measuring and Reporting Success

For EMS agencies, determining what qualifies as “good” or “bad” performance is typically based on a combination of internal standards and goals, as well as external benchmarks. How exactly you measure success depends on each agency, but most look at the averages for:

• Various call time intervals, such as:
  • Response times
  • Scene times
  • Wall times at the receiving facility

• Procedure success rates
• Patient outcomes

The best approach is often to measure agency-wide averages and then drill down to specific variables for a more detailed view of performance: the station, shift, unit, or individual provider. By simply measuring the standard deviation of the entire group and the average for each variable (station, shift, unit, individual provider, etc.), you can determine which variable falls inside or outside of the expected range, generally two standard deviations from the group average, and establish an objective assessment of desired or undesired performance.

For this reason, EMS agencies should ideally have software that allows them to build reports with any data point and customize the format. This ensures you can tailor performance reports with specific information your board, municipality, internal managers, or any other stakeholders need to see. Once your analysis is complete, the next important element is sharing your findings.

For example, your agency operates 24-hour shifts, and call volume has notably increased. Based on your observations, it seems like one unit is vastly over your benchmarks for unit-hour utilization (UHU), and you need another ambulance. By creating a custom report for the number of calls run by your agency and analyzing that data per unit, you can quickly determine your agency's overall UHU and compare each unit’s UHU to not only validate your assumption but provide an objective visual representation of the data that you can share with the governing board to prove the need and justify the cost for an additional ambulance.

3 Tips for Building a Data-driven Culture in EMS

Here’s how to convince your agency that using data analysis to drive improvement is worth the time and effort:

1. Make Sure What You Measure is Meaningful
Avoid putting out numbers that simply show you're doing something right or only highlighting areas that need improvement. Measure what’s meaningful to your agency and the people in it. What are the most common procedures your crews perform every day? Where are the most significant resource limitations? What feedback are you getting from patients? Target those areas for measurement. People will often resist feedback and intervention if they feel what you’re measuring and reporting is not relevant to them, but they will respond positively if you structure your reports in a way that is meaningful to their interests.

2. Continuously Measure and Monitor Your Data
Data analysis is not a “one-and-done” effort. Continuously measure your data to see trends over time and track your long-term goals. This will also help you understand performance fluctuations. For example, if you’re rolling out new, more affordable IV catheters to your crews, track first-attempt success rates over six months before and after the change. If averages drop in the first few months, perhaps crews are still getting used to the product. If they’re still lower at six months, the product’s quality is likely questionable and not only resulting in additional IV attempts being performed on the patient, but increasing the risk to patients, the risk of needle sticks to providers, and increasing the costs incurred by your agency for supplies. Regardless, you have the data to decide if and when to intervene before falling too far below established benchmarks.

3. Let the Data Do the Talking
Whether you’re assessing an individual’s performance in an internal review or sharing agency performance at a city council meeting, let the data do the talking and use it to guide decisions. Using objective numbers and analyses removes subjectivity and personal bias from strategic conversations, offering a more straightforward path toward improvements and solutions. It’s worth noting that data should always be paired with qualitative information to give context to the analysis. Visual representations of the data are always more impactful than simply showing numbers on a spreadsheet or printout.

Conclusion

Data analysis is the best way to legitimately and objectively improve EMS agency performance and patient outcomes. The knowledge gained from statistical analysis sheds light on clinical and operational elements that would otherwise be hidden, ensuring the agency can continue to operate and provide high-quality care to the community.