PURPOSE: As part of the VHA's focus on improving Veteran access, the ESP Coordinating Center (ESP CC) is responding to a request from the Acting Deputy Under Secretary for Health for Operations Management (DUSHOM) through the Access and Clinic Administration Program (ACAP) for an evidence brief on the comparative effectiveness of appointment recall reminder procedures for established patients returning for follow-up appointments. Recall appointments are defined as future patient appointments in which the patient needs to be seen in more than 90 days. The main purpose of this brief review is to summarize the evidence on the comparative effectiveness of the current Class I VHA recall reminder software, alternative recall reminder software or approaches, and scheduling follow-up appointments at the time of leaving the office in reducing missed opportunities for Veteran follow-up appointments without negatively impacting opportunity costs.

The ACAP will use the findings from this evidence brief to help inform refinement of clinical manager training development and scheduling policies, processes, and standard operating procedures (SOPs). In addition, findings will drive recall reminder software development intended to increase the ease of patient scheduling, decrease patient no-show rates, cancellation rates, and loss to follow-up, and enhance health care delivery and access.

BACKGROUND: Missed health care appointments are a major source of potentially avoidable cost and resource inefficiency that can adversely affect organizational workflow and increase clinic wait times. Missed appointments also may reflect needed care that was not delivered that can result in delays in diagnosis and appropriate treatment and decrease patient health outcomes.-

In 2008, an audit by the Office of Inspector General (OIG) found that the Veterans Health Administration's (VHA) efforts to reduce unused outpatient appointments were inadequate and recommended establishment of procedures to (1) measure and track unused outpatient appointments, (2) measure the effectiveness of processes for reducing missed opportunities and implement best practices nationwide, and (3) require facility directors to ensure unused appointments are used. A few examples of VHA efforts to address the 2008 OIG's recommendations include (1) 2010 implementation of a standardized computerized system for tracking and reducing missed opportunities for >90-day follow-up appointments (Class I Recall Reminder software) and (2) in 2011, the Pittsburgh Healthcare System Veterans Engineering Resource Center (VAPHS VERC) developed the National Initiative to Reduce Missed Opportunities (NIRMO) for tracking missed opportunities, understanding and analyzing factors that predict them, and developing and deploying strategies for improvement.

In the VA, the farther out an appointment is scheduled, the less likely the appointment will happen (45%-60% for appointment age >90 days vs 70-80% for appointment age <14 days; J. Prentice, S.D. Pizer, unpublished data, 2015). Starting in 2010, VHA Directive 2010-027 prohibited continued scheduling of greater than 90-day follow-up appointments at the time Veterans were leaving the clinic and mandated the use of strategies for contacting patients closer to the time of the needed visit to remind them to schedule the appointment (‘Recall Reminder’). Under this system, when a patient checks out after seeing a provider, a future appointment is made only if the patient is to return to clinic within 90 days. Otherwise, (1) VA staff use the software to schedule the Recall Reminder, (2) 2-4 weeks prior to the recall date, software automatically notifies clinic staff that it is time to remind the Veteran of the need to schedule a follow-up visit, (3) a mail or phone reminder is sent to the patient, and either (4) the patient calls to schedule, or (5) patient does not call to schedule and goes onto a delinquency list and has to be contacted by VA staff.

After 5 years of mandated use, the 2010 Recall Reminder policy is being revisited because VHA Support Service Center data have shown no significant decreases from 2010 to 2014 in no-shows overall (7.1% vs 7.4%), canceled by clinic rate (9.2% vs 9.9%), or canceled by patient rate (14.8% vs 15.0%) and some staff using the Recall Reminder system have criticized the system, asserting that it is very time- and resource-intensive (written communication, March 2015). For example, as a result of requiring Veterans to call in to schedule their follow-up appointment rather than scheduling the follow-up as they are leaving the office, the VISN 8 Call Center reported a 10% increase in their call volume from 2010 to 2014 (written communication, March 2015). Another common complaint from the field is that manual management of the delinquency list is very labor-intensive (VISN3, email communication, December 24, 2014). Other potential unintended consequences of using the Recall Reminder system include negative impacts on patient follow-up, Veteran satisfaction (if Veterans want to leave with an appointment), cost and other organizational outcomes (eg, productivity, turn over, grievances, training requirements, infrastructure requirements, etc). Recall Reminders could be implemented in several different ways, potentially leading to increased or decreased scheduler burden and other adverse events, so how best to implement the system is an important question. For example, if sending Veterans due for follow-up appointments a notice at 60 days, 30 days, and calling them at 20 days, 18 days, and 15 days doesn't significantly reduce no-shows beyond a less intensive approach of calling Veterans at 18 days and 15 days, then the less intensive approach may be preferable due to the decreased workload burden.

Although NIRMO has not yet evaluated the specific impact of the Recall Reminder system, they surveyed Veterans about reasons for missed appointments and VA staff regarding scheduling practices and implementation of strategies to reduce missed appointments. NIRMO's survey of 4,749 Veterans found that the top reported reasons for missed appointments were that they forgot (19%), miscellaneous (reasons other than those listed, 16%), not aware of the appointment (15%), no transportation (8%), poor weather (7%), sick (7%), something unexpected came up (7%), and cancelled the appointment beforehand (5%). NIRMO also surveyed 1,493 VA staff to identify scheduling practices and strategies used to reduce missed appointments. Compared to a missed appointments rate of 18.7% for the strategy of negotiating appointments only, reminder calls reduced the missed appointments rate to 16.0%. The rate was reduced to 14.7% with both reminder calls and promotion of provider continuity, and to 13.7% with reminder calls, promoting provider continuity, and receiving appointment cancellations through main facility phone line instead of individual clinic phone line.

Taking into account Veteran and staff surveys, NIRMO has recommended 10 general strategies for reducing missed appointments: (1) cancel appointments as they are received, (2) negotiate all appointments, (3) coordinate appointments with transportation and other appointments already scheduled, (4) offer open access or same-day access, (5) use nontraditional modes of care, (6) manage the schedule to ensure clinics run on time, (7) improve interactions between patients and providers, patients and clerks, and between staff members, (8) disseminate educational posters, (9) use the Recall Reminder software to schedule follow-up visits beyond 90 days, and (10) perform live _targeted reminder calls to patients that they've identified as having a 20% or greater no-show probability using a predictive model they developed.

VHA is taking several steps to determine how to optimize the use of Recall Reminder processes. First, the VA is undertaking a quality improvement initiative that will evaluate the effectiveness of various Recall Reminder approaches across 6 pilot sites, comparing sequence and timing of reminder postcards and calls and the traditional approach allowing Veterans to schedule appointments before leaving the office (up to a year in advance). Second, through the ACAP, the DUSHOM requested that the ESP CC conduct a rapid evidence brief to evaluate the comparative effectiveness of the current Class I VHA recall reminder software, alternative recall reminder software or approaches, or scheduling follow-up appointments at the time of leaving the office in reducing missed opportunities for established Veteran follow-up appointments without negatively impacting opportunity costs.

SCOPE: The objective of this evidence brief is to synthesize the literature on the comparative effectiveness of appointment recall reminder systems. The ESP Coordinating Center investigators and representatives of the Access and Clinic Administration Program (ACAP) worked together to identify the population, comparator, outcome, timing, setting, and study design characteristics of interest. The ACAP approved the following key questions and eligibility criteria to guide this review:

KEY QUESTIONS: Key Question 1: For adult patients who are _targeted for follow-up appointments, what is the comparative effectiveness of the current Class I VHA recall reminder software, alternative recall reminder software or approaches, or scheduling follow-up appointments at the time of leaving the office?

Key Question 2: For adult patients who are _targeted for future appointments, does the comparative effectiveness of Appointment Recall Reminder (RR) procedures versus other kinds of follow-up appointment scheduling systems differ according to:

1. Patient factors: Preference, clinical characteristics

2. Appointment scheduling systems engineering design and management factors: Mode of notification (mail, phone, electronic), threshold for notification (1 month, 2 weeks), mode of patient response, reminder type

3. Facility characteristics: Efficiency, backlogs.

ELIGIBILITY CRITERIA: The ESP included studies that met the following criteria:

1. Population: Adult patients who are _targeted for follow-up appointments.

2. Intervention: Any procedures for scheduling established patients' follow-up appointments. We accepted any type of procedures. These included, but were not limited to strategies incorporating the following procedures:

   1. 365 scheduling: Negotiation of follow-up visit upon leaving the clinic, regardless of how far in the future the appointment is needed;

   2. Strategies for reducing appointment age – after patients leave the office, contacting the patients closer to the time the future appointment is needed by:

      1. Recall Reminder: Sending a notification requesting that the patient contact the office to schedule an appointment

      2. Blind scheduling: Sending a notification of an appointment that has been scheduled on behalf of the patient without their input about preference on date or time.

      The highest-priority studies were those that most directly addressed our questions about the comparative effectiveness of different systems for scheduling established patients' follow-up appointments, with or without reminders. To address gaps in the highest-priority evidence, we also accepted lower-priority studies that either (a) focused on scheduling new patients for initial visits or (b) focused only on the reminder component.

3. Comparison: Mandated versus flexible use of a recall reminder system; comparison of different recall reminder engineering designs; comparison of recall reminder versus other strategies for reducing appointment age.

4. Outcomes: Primary outcomes of interest include no-show rates and cancel rates. Secondary outcomes of interest include appointment wait times, patient loss to follow-up (undelivered needed care), scheduler learning and behavior, organizational outcomes (eg, productivity, turnover, grievances, training, infrastructure requirements), and patient satisfaction.

5. Timing: No restrictions.

6. Setting: Within and outside of the VA. We will prioritize VA studies, but will look outside of the VA to fill gaps in VA evidence, including international studies.

7. Study design: Longitudinal studies. Using a Best Evidence approach, we will prioritize evidence from systematic reviews and multisite studies that adequately controlled for potential patient-, provider-, and system-level confounding factors. Inferior study designs (eg, single-site, inadequate control for confounding) will only be accepted to fill gaps in higher-level evidence.

We are aware of the large volume of evidence on the effectiveness of reminders for increasing vaccine and screening test uptake. However, we excluded those from this review because they generally focus on ultimate uptake regardless of whether it encompassed multiple failed appointment attempts and/or the procedures can sometimes be completed in walk-in clinics and don't always involve any scheduling.

ANALYTIC FRAMEWORK: The analytic framework below illustrates the populations, interventions, outcomes, and adverse effects that guided this review and their relationship to the Key Questions.