A key Obamacare program that uses bonus payments to improve medical care didn't work in its first year, new research suggests.

The program is called Hospital Value Based Purchasing, and it's meant to reward hospitals that do really well at 20 metrics, including things like responding quickly to heart attacks and giving patients discharge instructions when they leave the hospitals.

Beginning in 2011, hospitals stood to lose as much as 1 percent of their Medicare revenue if they did badly at these measures — or earn an additional 1 percent if they outperformed their competitors.

This put a total of $1.1 billion at risk for hospitals, who could either earn part of that pool of funds by doing really well — or end up paying into it when they didn't hit the right targets.

The aim of the penalty and reward system was to encourage all hospitals to improve the care they deliver. That might not have happened.

Hospitals with incentives didn't do any better

Weill Cornell Medical College's Andrew Ryan worked with a team of researchers to look at how hospitals did on quality measures since this new program began. The research looks at the hospitals in the incentive program and compares them against a smaller number left out (Maryland hospitals, for example, didn't participate because they have a totally different payment scheme worked out with the feds).

Ryan and his team matched up hospitals in and out of the program that had similar quality metrics before the incentives kicked in. Then they looked at whether the hospitals in the incentive program improved faster than the ones left out. They found no difference.

"We found that the timing of the hospital value-based purchasing system wasn't associated with any additional improvement," Ryan says. "We would have hoped to have seen more responsiveness."

Why didn't it work?

It's possible that the incentives in this particular program might be too small to encourage hospital administrators to make big investments. They could be making a calculation: it would be more work than it's worth to do better. In the United Kingdom, for example, a full 20 percent of doctors' pay is tethered to how well they do their job.

This particular bonus program will grow, but only slightly: in 2017, hospitals will have 2 percent of their Medicare revenue at risk with these quality metrics.

"I still think that's really pretty small but I hope it's enough to get hospitals focused," says Ryan. "They have to make a cost benefit analysis and say, it makes sense to invest in improving these things."

"WE WOULD HAVE HOPED TO HAVE HAD MORE RESPONSIVENESS"

One other possibility: Ryan and his colleagues noticed, in the data, is that when you look back at 2008, there is some evidence that hospitals were improving back then at a faster rate than the non-participants. This suggests that hospitals may have begun improving quality in preparation for these policy changes a few years in advance, and are now reaping the benefits of that advance planning.

"It's hard to say whether the effect we observed is responsiveness to the new policy or it's just that they happened to start improving at that particular moment," Ryan says.

This would have meant the hospitals were preparing for the program two years prior to the Affordable Care Act passing, and would have had a sense whether they would fall in the bonus pool or not.

At the same time, there has been talk of a program like this since about 2003, when some initial results from a small group of hospitals suggested these type of incentives could make a difference. So it's possible, but by no means certain, that advanced preparation did improve outcomes — but hospitals couldn't make any further gains when the reward program actually came into effect.

(Hat tip to Jordan Rau at Kaiser Health News for noticing this study first)