Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2014 Nov 14:2014:506-15.
eCollection 2014.

The Number Needed to Remind: a Measure for Assessing CDS Effectiveness

Jonathan Einbinder  1 , Esteban Hebel  2 , Adam Wright  1 , Morgan Panzenhagen  3 , Blackford Middleton  1
Affiliations

The Number Needed to Remind: a Measure for Assessing CDS Effectiveness

Jonathan Einbinder et al. AMIA Annu Symp Proc. .

Abstract

Background: Clinical decision support (CDS) is associated with improvement in quality and efficiency in healthcare delivery. The appropriate way to evaluate its effectiveness remains uncertain.

Methods: We analyzed data from our electronic health record (EHR) measuring the display frequency of eight reminders for Coronary Artery disease and Type 2 Diabetes and their associated performance according to a predefined methodology. We propose two key performance indicators to measure their impact on a _target population: the reminder performance (RP), and the number needed to remind (NNR), to evaluate the impact that Clinical decision support reminders have on the adherence to guideline derived CDS interventions on the entire patient population, and individual providers receiving the interventions.

Results: Data were available for 116,027 patients and a total of 1,982,735 reminders were displayed to a subset of 65,516 patients during the study period from January 1 to December 31, 2010. The evaluation framework assessed provider acknowledgement of the CDS intervention, and the presence of the expected performance event while accounting for patients' exposure to the CDS reminders. The total RP was 2.7% while the average NNR was 3.1 for all the reminders under study.

Conclusions: The proposed framework to asses of CDS performance provides a novel approach to improve the design and evaluation of CDS interventions. The application of this methodology represents an indicator to understand the impact of CDS interventions and subsequent patient outcomes. Further research is required to evaluate the impact of these systems on the quality of care.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Timeline detailing the reminder lifecycle.
Figure 2
Figure 2
CDS performance evaluation framework for passive reminders for HbA1c to be completed every six months to the entire population of diabetic patients found in the Partners Healthcare System. (A) Distribution of all the active diabetic patients depending if they have received a reminder and whether they are in compliance with the performance measure. (B) HbA1c compliance among diabetic patients in Partners population. (C) Computation of the Number Needed to Remind based on CDS performance framework.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

References

    1. Blumenthal D, Glaser JP. Information technology comes to medicine. N Engl J Med. 2007;356(24):2527–2534. doi: 10.1056/NEJMhpr066212. - DOI - PubMed
    1. Bates DW, Gawande AA. Improving safety with information technology. N Engl J Med. 2003;348(25):2526–2534. doi: 10.1056/NEJMsa020847. - DOI - PubMed
    1. Osheroff JA, Teich JM, Middleton B, Steen EB, Wright A, Detmer DE. A roadmap for national action on clinical decision support. J Am Med Inform Assoc. 2007;14(2):141–145. - PMC - PubMed
    1. Chaudhry B, Wang J, Wu S, et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med. 2006;144(10):742–752. - PubMed
    1. Berner E. AHRQ White Paper: Clinical Decision Support Systems: State of the Art. 2009. [Accessed November 7, 2011]. Available at: http://healthit.ahrq.gov/images/jun09cdsreview/09_0069_ef.html.

Publication types

Substances

LinkOut - more resources

  NODES
INTERN 5
twitter 2