This is Part 2 of a series of articles on ROI for Quality Improvement (QI). Part 1 provided some background for ROI and some basic definitions and processes, and also discussed in detail what is meant by the “scope” and “perspective” of an ROI analysis. In this article, we will talk specifically about identifying and quantifying the costs and benefits of a QI initiative and how these calculations apply to the overall ROI analysis.
At the heart of a good ROI analysis is accurately and validly quantifying the costs and benefits associated with the QI initiative. However, how difficult and complex this process ends up being depends on several characteristics of your project. The relevance of certain costs and benefits is linked to the scope and perspective (see Part 1), and there are multiple methods for quantifying or monetizing costs and benefits for their use in the ROI calculation. We’ll consider each of these as we discuss the process of identifying and collecting costs and benefits.
The impact of scope and perspective
As described in Part 1 of this series, the scope and perspective of your ROI analysis relate (in general) to who incurs the costs and receive the benefits, in what setting, and over what time period. Therefore, to identify the relevant monetary values, you are required to identify what is relevant for the perspective you have chosen. For example, if you are applying for funding for a QI program from CMS and want to demonstrate the anticipated ROI CMS can expect, then the appropriate benefits are those that translate to lower Medicare expenditures, such as reduced utilization, lower incidence and prevalence of disease, and shorter treatment time. The relevant costs might simply be the amount of the award you are applying for.
Alternatively, the relevant costs and benefits for a locally-funded initiative would be very different. If your facility or organization is funding a QI initiative, then the costs to quantify are those related to staff time, materials, training, and/or any facility or operational investments (structural, electronic, new equipment, etc.). Realized benefits may result from changes in costs, revenue, or efficiency (e.g., required FTEs and/or overtime). Some benefits may be easy to identify and quantify, while others, such as changes in payments from insurance risk-pools or quality reporting programs, or the impact of increased bed capacity, may be more nuanced. Additionally, your group may experience non-monetary benefits such as improved staff and/or patient satisfaction, which you may not be able to monetize at all. We’ll dive into the details of how to quantify these types of costs in a minute, but you can see that the relevance of costs and benefits are directly tied to the perspective of your ROI analysis.
The scope also plays a part in identifying the appropriate costs and benefits. For example, if a hospital implements a new discharge process requiring more staff time, then the additional staff-related costs associated with that intervention continue for as long as that facility maintains the new process. In this case, the cost is directly proportional to the length of time (i.e., the scope) involved in the ROI calculation. Alternatively, investing in a new piece of equipment is (typically) a one-time purchase regardless of the timeframe considered for the ROI, and therefore its cost remains the same regardless of the time period chosen to examine ROI. When identifying the relevant benefits to include in the ROI analysis, the defined scope also dictates the generalizability of the intervention and whether more than just a specific subgroup of patients will experience the benefit. That is, if you’re targeting hospital-acquired infections, for example, you have to ask: is it reasonable to assume that a successful intervention would reduce infections across all settings and patients included in the intervention? Or is it more likely to succeed only in a specific setting and/or among a subset of patients? The answers to those questions directly impact the costs and benefits you will want to include in the ROI calculation. If you’re unsure about who might be impacted, it will be an important aspect to explore in the sensitivity analysis, which we’ll address in the next article in this series. As a side note, it is entirely possible that a successful intervention could reduce revenue for a facility – at least in the short term – because of shorter stays and reduced utilization. Whether this is a benefit or a cost is directly related to the scope and perspective: a payer would consider it a benefit (reduced cost) while the facility itself would consider it a cost (reduced revenue).
As mentioned last time, it is important to think about the ROI analysis when designing the intervention itself, because it can help you determine the appropriate scope and perspective to demonstrate the ROI you are hoping to achieve.
Identifying Costs and Benefits
Once you have identified the scope and perspective, you then need to identify all of the relevant costs within that framework. Sometimes this is straightforward (e.g., the cost to CMS for funding a QI initiative is simply the amount awarded). However, often it can be quite complicated. In the example of a locally-funded QI initiative that will require staff time, training, and resources, there may be dozens of sources of costs that need to be accounted for. In this case, it can be helpful to list out possible cost categories and brainstorm all of the potential cost sources. An initial list might include: Staffing, Time (calls, training sessions, overtime), Materials (tools, reports), Travel, and Facility or Operational (structural, electronic). Within these categories, there may be several sub-sections and individual line-level cost sources, and accurately capturing them all will likely involve efforts of several staff. But, after you have done it the first time, you will be much better equipped to tackle a similar process for future projects.
When identifying benefits, most are considered to fall into two main categories: “direct” and “indirect” benefits. Direct benefits are exactly what you’d think: they are those benefits that are realized as direct result of a successful intervention, either from clinical improvements or from improvements in efficiency (including waste reduction) and productivity. That is, if your goal is to reduce infections, then direct benefits from the clinical improvement could include the lower costs associated with treating fewer infections, shorter hospital stays, and fewer medication costs. If your intervention attempts to reduce patient wait times in the ED, then direct benefits may involve efficiency gains related to timing of admissions or to required staffing levels and reduced staff time (or overtime).
Indirect benefits are those that may not be the direct focus of the intervention, but occur as a result of the improved delivery or coordination of care. For each example above, improved patient satisfaction may be an indirect benefit: experiencing fewer infections or shorter ED wait times may equate to more satisfied patients in each case, even though improved satisfaction was not the direct intent of either intervention. Other indirect benefits might include increased payments (or reduced penalties) from federal programs that alter reimbursement based on quality measure performance, improved staff satisfaction, or improved patient quality of life, among others.
Often, it can be difficult to monetize indirect benefits (for inclusion in the ROI analysis), but that doesn’t mean they aren’t important. Compiling a list of indirect benefits can strengthen your ROI results when presented in addition to an ROI that is calculated using only the direct benefits. A statement along the lines of, “Using direct benefits only, we estimate an ROI of X%. This does not account for the indirect benefits of improved staff satisfaction, increased efficiency, and reduced waste…” suggests that the calculated ROI is a lower bound or conservative estimate of the return that one can expect. There are times when it may be appropriate to include rough estimates of indirect benefits in sensitivity analyses. For example, perhaps it’s reasonable to conclude that improved staff satisfaction would result in lower staff turnover. If you can estimate the reduction in turnover in specific terms and assign a reasonable monetary value to it, then it would be completely legitimate to include that benefit in a “best-case scenario” ROI calculation as part of your sensitivity analysis (more on this in the next article).
Estimating Monetary Value of Costs and Benefits
Once you have identified the items that produce the relevant costs and benefits for your intervention, you need to estimate their monetary value to include in the ROI calculation. There are several ways to do this, and there are advantages and disadvantages to each.
One option is to use information obtained from setting where the intervention is taking place. That is, if you are trying to reduce heart-failure hospitalizations at your facility, you could use your facility’s internal records regarding frequency, length of stay, and the cost of these encounters when establishing the monetary benefit associated with reducing them. While this is likely to produce a value that is accurate for a that setting (i.e., your facility), it may not be generalizable to other settings or patient populations. Perhaps the setting where the intervention is occurring is rural or typically sees certain types of patients that are not representative of the state, region, or country.
Another option would be to use literature-based or policy-based estimates. In our example of heart failure hospitalizations above, this would involve obtaining estimates on frequency and cost from a national database or report, or as the result of a literature summary on heart failure-related hospitalizations. This process is likely to produce an estimate that is generalizable, but it may not reflect the real benefit experienced at the local facility where the intervention takes place.
In addition to deciding from where to pull the information for your monetary estimates, you will also need to decide how best to compile them. The method of calculating the average cost of all heart failure-related hospitalizations at your facility described previously is referred to as “gross costing,” which relies on the average as being an accurate representation of a typical or usual encounter (note: you could similarly obtain estimates of the variability to use in sensitivity analyses). Alternatively, one could also employ “micro costing.” Instead of lumping everything together and averaging, micro costing requires you to identify each of the individual elements that are typically involved in a heart failure hospitalization and add them together. This might include room charges, typical lab tests, staff time, facility resources, durable medical equipment, etc. When deciding which method to employ, you will need to consider what information you have available, the scope and perspective of your analysis, and the populations and settings you intend to generalize your results to (if any).
Other Metrics to Consider
When identifying and quantifying costs and benefits, there are additional metrics you may want to consider as part of your ROI analysis. Two common ones include “net savings per patient” and the “payback period.”
Net savings per patient represents the net benefit averaged across all patients. Its definition is simply:
(Benefit – Costs) / Number of Patients Affected
So, if a net benefit (i.e., benefits minus costs) of $10,000 was realized for an intervention involving 34 patients, the net savings per patient is $10,000 / 34 = $294.12. Reporting results this way can sometimes be more “tangible” or “real” to the reader, especially when dealing with very large monetary values that can feel abstract or be hard to conceptualize. Presenting results per patient can also humanize the return in contrast to simply reporting results in terms of expenditures for a payor or some administrative body. This metric assumes that you can accurately account for how many patients were affected, which may or may not be the case. Additionally, note that as the number of patients increases, the net savings per patient decreases, so that it may not be as effective when applied to large populations, unless your net benefit is also relatively large.
The payback period is the amount of time (often the number of months) until the benefits realized from an intervention cover the investment costs. This can be useful when analyzing the return from a large one-time investment of equipment or a facility, or when there is some delay in realized benefits from a QI intervention. It can also be effective in demonstrating how quickly the initial costs of an intervention can expect to be covered by the realized benefit. The payback period is calculated as:
Total Cost / Monthly Savings
So, if you expect a 6-month QI initiative to cost $25,000 and to produce a net savings of $3,500 per month, you can see that over the 6-month period you would experience a negative ROI since that monthly savings equates to only $21,000 of benefits by the end of month 6: ($21,000 – $25,000) / $25,000 = –16%. However, we can also see that the payback period is $25,000 / $3,500 = 7.1 months, indicating that we’ll recoup our investment a little over a month after the initiative is completed. This can be effective in offsetting negative ROI results in the mind of the reader.
There are other metrics to explore as well, but you can quickly see that depending on the project and the situation, presenting multiple metrics can strengthen your results and help to paint a clear financial picture of the impact of your QI intervention.
Next Steps
In the next article we will talk about sensitivity analyses: why they are important and how to approach them.
Note: In addition to my own experience, information from multiple sources was used in the development of this series of articles, including: “Measuring ROI in Healthcare: Tools and Techniques to Measure the Impact and ROI in Healthcare Improvement Projects and Programs” by Victor V. Buzachero, Jack Phillips, Patrician Pulliam Phillips, and Zack L. Phillips (McGraw Hill, 2013); AHRQ’s toolkit (https://www.ahrq.gov/professionals/systems/hospital/qitoolkit/index.html), and a white paper by IHI (Sadler BL, Joseph A, Keller A, Rostenberg B. “Using Evidence-Based Environmental Design to Enhance Safety and Quality” Innovation Series white paper. Cambridge, MA: Institute for Healthcare Improvement; 2009).