National Registry of Evidence-Based Programs and Practices


The National Registry of Evidence-based Programs and Practices was a searchable online database of interventions designed to promote mental health or to prevent or treat substance abuse and mental disorders. The registry was funded and administered by the Substance Abuse and Mental Health Services Administration, part of the U.S. Department of Health and Human Services. The goal of the Registry was to encourage wider adoption of evidence-based interventions and to help those interested in implementing an evidence-based intervention to select one that best meets their needs.
The NREPP website was phased out in 2018. The included phaseout section has more information.

Overview

In the behavioral health field, there is an ongoing need for researchers, developers, evaluators, and practitioners to share information about what works to improve outcomes among individuals coping with, or at risk for, mental disorders and substance abuse. Discussing how this need led to the development of NREPP, Brounstein, Gardner, and Backer wrote:
The focus of NREPP was on delivering an array of standardized, comparable information on interventions that are evidence based, as opposed to identifying programs that are "effective" or ranking them in effectiveness. Its peer reviewers used specific criteria to rate the quality of an intervention's evidence base as well as the intervention's suitability for broad adoption. In addition, NREPP provided contextual information about the intervention, such as the population served, implementation history, and cost data to encourage a realistic and holistic approach to selecting prevention interventions.
As of 2010, the interventions reviewed by NREPP had been implemented successfully in more than 229,000 sites, in all 50 States and more than 70 countries, and with more than 107 million clients. Versions of its review process and rating criteria had been adopted by the National Cancer Institute and the Administration on Aging.
The information NREPP provided was subject to certain limitations. It was not an exhaustive repository of all tested mental health interventions; submission was a voluntary process, and limited resources may preclude the review of some interventions even though they meet minimum requirements for acceptance. The NREPP home page prominently stated that "inclusion in the registry does not constitute an endorsement."

Submission process

NREPP held an open submission period that ran November 1 through February 1. For an intervention to be eligible for review, it was required to meet four minimum criteria:
  1. The intervention has produced one or more positive behavioral outcomes in mental health, mental disorders, substance abuse, or substance use disorders use among individuals, communities, or populations.
  2. Evidence of these outcomes has been demonstrated in at least one study using an experimental or quasi-experimental design.
  3. The results of these studies have been published in a peer-reviewed journal or other professional publication, or documented in a comprehensive evaluation report.
  4. Implementation materials, training and support resources, and quality assurance procedures have been developed and are ready for use by the public.
Once reviewed and added to the Registry, interventions were invited to undergo a new review four or five years after their initial review.

Review process

The NREPP review process consisted of two parallel and simultaneous review tracks, one for the intervention's Quality of Research and another for the intervention's Readiness for Dissemination. The materials used in a QOR review are generally published research articles, although unpublished final evaluation reports could also be included. The materials used in an RFD review included implementation materials and process documentation, such as manuals, curricula, training materials, and written quality assurance procedures.
The reviews were conducted by expert consultants who had received training on NREPP's review process and rating criteria. Two QOR and two RFD reviewers were assigned to each review. Reviewers worked independently, rating the same materials. Their ratings were averaged to generate final scores.
While the review process was ongoing, NREPP staff worked with the intervention's representatives to collect descriptive information about the intervention, such as the program goals, types of populations served, and implementation history.
The QOR ratings on a scale of 0.0 to 4.0, indicated the strength of the evidence supporting the outcomes of the intervention. Higher scores indicated stronger, more compelling evidence. Each outcome was rated separately because interventions could target multiple outcomes, and the evidence supporting the different outcomes could vary. The QOR rating criteria were:
  1. Reliability of measures
  2. Validity of measures
  3. Intervention fidelity
  4. Missing data and attrition
  5. Potential confounding variables
  6. Appropriateness of analysis
The RFD ratings were also given on a scale of 0.0 to 4.0, indicating the amount and quality of the resources available to support the use of the intervention. Higher scores indicated that resources were readily available and of high quality. These ratings applied to the intervention as a whole. The RFD criteria were:
  1. Availability of implementation materials
  2. Availability of training and support resources
  3. Availability of quality assurance procedures

    Reviewers

QOR reviewers were required to have a doctoral-level degree and a strong background and understanding of current methods of evaluating prevention and treatment interventions. RFD reviewers were selected from two categories: direct services experts, or experts in the field of implementation. Direct services experts must have had previous experience evaluating prevention or treatment interventions and knowledge of mental health or substance abuse prevention or treatment content areas.

Products and publications

NREPP published an intervention summary for each intervention in the Registry. The summaries, accessed through the Registry's search engine, contained standardized information:
  • A brief description of the reviewed intervention, including targeted goals and theoretical basis
  • Study populations
  • Study settings and geographical locations
  • Implementation history
  • Funding information
  • Comparative evaluation research conducted with the intervention
  • Adaptations
  • Adverse effects
  • List of studies and materials reviewed
  • List of outcomes
  • Description of measures and key findings for each outcome
  • Research design of the studies reviewed
  • Quality of Research and Readiness for Dissemination ratings
  • Reviewer comments
  • Costs
  • Replication studies
  • Contact information
NREPP maintained an online Learning Center that included learning modules on implementation and preparing for NREPP submission; a research paper on evidence-based therapy relationships; and links to screening and assessment tools for mental health and substance use.

Predecessor system

The registry originated in 1997 as the National Registry of Effective Prevention Programs and has gone through several changes. This predecessor to the NREPP was developed by SAMHSA's Center for Substance Abuse Prevention as part of the Model Programs initiative. Procedures under this earlier registry were developed to review, rate, and designate programs as Model, Effective, or Promising. Based on extensive input from scientific communities, service providers, expert panels, and the public, the procedures were revised. Reviews using the new NREPP system began in 2006, and the redesigned Web site debuted in March 2007.

Phase out in 2018

According to an email from SAMHSA:
"SAMHSA is committed to advancing the adoption of evidence-based interventions related to mental health and substance use. Consistent with the January 2018 announcement from the Assistant Secretary for Mental Health and Substance Use related to discontinuing the National Registry of Evidence-based Programs and Practices, SAMHSA has now phased out the NREPP website, which has been in existence since 1997. In April 2018, SAMHSA launched the Evidence-Based Practices Resource Center that aims to provide communities, clinicians, policy makers, and others in the field with the information and tools they need to incorporate evidence-based practices into their communities or clinical settings. The Resource Center contains a collection of science-based resources; however, it does not replace NREPP and does not contain all of the resources that were previously available on NREPP.
"The Resource Center is a component of SAMHSA’s new comprehensive approach to identify and disseminate clinically sound and scientifically based policy, practices, and programs. Under this new approach, we are continuing to develop and add additional resources to the Resource Center as they become available. In the meantime, please use our Resource Center as well as the SAMHSA Store to find information on evidence-based practices and other resources related to mental health and substance use. For products and resources not developed by SAMHSA, please contact the developers for more information."