Accountable Care Organizations (ACOs) in the Medicare Shared Savings Program (MSSP) have been credited with modest savings, but new research suggests the savings may be lower if it comes not from better care management but because the biggest-spending physicians and their high-cost patients are leaving the program.
Another research team questions these findings, however.
Adam A. Markovitz, BS, a medical/PhD student at the University of Michigan in Ann Arbor, and colleagues used Medicare data from 2008 to 2014 to evaluate changes in spending and quality performance while accounting for selection effects in the MSSP, a different model from those typically used in prior research. They published their findings online June 17 in Annals of Internal Medicine.
Their instrumental variable model captured disproportionately high exits by physicians and patients from the ACOs, Markovitz told Medscape Medical News.
The leader of another research team that has done previous work in the area reviewed those results and told Medscape Medical News the team had “serious concerns” about the methodology and conclusions of the research. They posted their critique in a blog post this week on The Incidental Economist.
In New Model, Neither Costs Nor Quality Improve
When Markovitz and colleagues factored in the exits of high-cost clinicians and patients, they found that MSSP was not linked with improvements in spending or quality measures (three measures of diabetes management as well as mammography) or most hospital use measures.
High-spending clinicians (those in the 99th percentile) had a 30.4% chance of leaving the MSSP, more than double the 13.8% chance of exit among average-spending clinicians (50th percentile).
Their analysis was not able to determine why, but Markowitz told Medscape Medical News his research team and others say one scenario is that high-need patients are moving away from primary care and exclusively seeing specialists outside the ACO.
The more troubling possibility, he said, is that ACOs are gaming the system and dropping high spenders and high-cost patients when they submit their provider list to Medicare each year.
“These findings suggest caution in extending ACOs to other settings and patients without stronger evidence that the program saves money or improves quality of care. Our study underscores the degree to which selection bias may affect evaluations of voluntary reforms and the challenges inherent in evaluating these programs,” the authors write.
Reviewers Say Methodology and Conclusions Are Flawed
In the guest post on The Incidental Economist, researchers J. Michael McWilliams, MD, PhD; Alan M. Zaslavsky, PhD; Bruce E. Landon, MD, MBA; and Michael E. Chernew, PhD, all with the Department of Health Care Policy at Harvard Medical School in Boston, Massachusetts, say the methodology and conclusions by Markovitz’s team have serious flaws.
McWilliams first took issue with the study‘s suggestion that high-risk patients “exiting” ACOs is evidence of risk selection. Examining only the flow into or out of a population is misleading because the rest of the population changes over time too, he told Medscape Medical News.
“But obviously we would not conclude from that observation that the average age or health risk of the Medicare population is decreasing, because those who are not entering or exiting are getting older and sicker, replacing those who left and vacating spots in the distribution that the newly eligible occupy, McWilliams said. “There can be marked differences between exiters and stayers in a population whose average health risk stays constant over time.”
McWilliams and colleagues write the exit of high-risk patients has nothing to do with ACO program incentives and say they have observed the same higher rates of assignment changes for high-risk patients in non-ACO provider organizations.
“If high-risk patients are not leaving ACOs at higher rates than they are leaving other organizations, then physicians of high-risk patients also cannot be leaving at greater rates,” McWilliams continued. “Rather, high-risk patients are just a high-flux group in terms of which provider they see the most in a given year. In our prior work, we demonstrate that the characteristics of patients do not change in ACOs relative to local control groups of patients assigned to non-ACO providers.”
McWilliams said for all analyses, except the instrumental variable analysis, Markowitz and colleagues “estimate savings that are actually greater than savings we previously reported. For example, when they remove any contribution of patient exit from their estimate of savings (the “patient fixed effects” model), they estimate savings that are nearly four times greater than the savings we estimated (3.8% savings [$90/quarter per Figure 2] vs. the 1.1% savings we reported).
“Thus, concluding that their results suggest that prior evaluations overstated the savings is a gross mischaracterization of the literature,” McWilliams added.
McWilliams also emphasized that ACOs cannot simply drop physicians from participant lists.
“ACOs are defined by a group of practices (taxpayer identification numbers or TINs), including all physicians billing under them,” he told Medscape Medical News. “To drop a physician, an ACO must fire the physician or somehow have the physician bill under a different TIN.”
McWilliams’ team concludes that “evidence to date — including the study by Markovitz et al — provides no clear evidence of a costly problem and suggests that ACOs have achieved very small, but real, savings.”
McWilliams and colleagues write, “Causal inference is hard but necessary to inform policy. When conclusions differ, opportunities arise to understand methodological differences and to clarify their implications for policy.”
The Markovitz et al study was funded by the Horowitz Foundation for Social Policy, Agency for Healthcare Research and Quality, and the National Institute on Aging. Coauthors report support from the University of California, San Francisco that had no effect on the analysis presented in this manuscript.
McWilliams is a consultant for evaluation of the ACO Investment model for Abt Associates. Chernew is working with the Centers for Medicare & Medicaid Services to examine spending patterns and risk adjustment trends in the next-gen ACO models.
Ann Int Med. Published online June 17, 2019. Abstract