Electronic health records deliver benefits that may automate administrative tasks, but electronic medical records don’t improve the quality of patient care overnight.

In fact a new study of electronic health records systems over five years of use at more than 3,000 hospital nationwide finds that patient deaths—or mortality rates—actually increased after hospitals made the switch to medical records automation.

An examination of how the adoption of electronic health records (EHR) affected the quality of hospital care between 2008 and 2013 found that mortality rates were initially higher among hospitals with more digital capabilities, but fell over time, as hospitals learned how to work with the technology and adopted new capabilities, say researchers from UC San Francisco, the Harvard T. H. Chan School of Public Health and the University of Michigan

“In other industries, widespread digitization took a decade to realize improvements,” says Julia Adler-Milstein, an associate professor in the UC San Francisco Department of Medicine and the Philip R. Lee Institute for Health Policy Studies. “It’s a major transformation of the healthcare system to go from paper to digital. We are seeing those rewards, but it has taken time and work.”

All hospitals did not see equal benefits from digitization, however, and the trend was driven mainly by smaller and non-teaching hospitals, the research shows. “Larger and teaching hospitals had ongoing efforts to improve hospital quality, and therefore had less room to improve with the adoption of health records,” the research study says. “In contrast, for smaller and non-teaching hospitals, EHR adoption may have represented a large, highly visible quality improvement initiative that also prompted broader quality efforts.”

Researchers examined data from 3,249 hospitals across the country, measuring quality by looking at 30-day mortality rates for 15 common conditions for patients who were 65 years and older.

They study timeframe began in 2008, because that is when national data was first collected about the adoption of EHRs. While many hospitals, particularly large and teaching hospitals, already had EHR capabilities by then, many adopted new technology following passage of the HITECH Act—Health Information Technology for Economic and Clinical Health—which provided $30 billion in 2009 to stimulate a broad national investment in new technology, the study says.

“Hospitals implement functionality over time, because it’s really hard to go from fully paper to fully electronic overnight,” Adler-Milstein says. “We measured EHR adoption in a way that was truer to the way adoption likely occurred. As hospitals added functionalities over time, there was benefit from each of those new features.”

The adoption of an electronic health records system was associated with slight gain (0.011%) in some mortality rates. But over time, the maturation of the EHR’s functions saw slight improvements in mortality rates of 0.09% to 0.21%.

“There’s been a lot of frustration with EHRs and these findings suggest that national investment in hospital EHRs should yield improvements in mortality rates, but achieving them will take time.,” Adler-Milstein says. “Our study shows they are improving care—It just may not be as much as providers or policymakers wanted or come as quickly, or as easily, as they would have liked.”