Summary Implementation and Evaluation – HEP4205 (2021)
Summary Implementation &
Evaluation
1. 10,000 Steps to go
1. Definitions
Innovation = an idea, practice, or object that is perceived as new by an individual or other unit of
adoption
Evidence-based intervention (EBI) = the subjects of D&I activities, interventions with proven
efficacy and effectiveness (i.e., evidence-based).
o Interventions=broadprograms, practices, processes, policies, guidelines
Diffusion = the passive, untargeted, unplanned, and uncontrolled spread of new interventions
a spontaneous process
Dissemination = an active approach of spreading EBIs to the target audience via determined
channels using planned strategies. The targeted distribution of information and intervention
materials to a specific public health audience more active
Adoption = the decision of an organization or community to commit to and initiate an EBI
Implementation = The process of putting to use or integrating EBIs within a setting. Increasing
the integration of EBIs into routing real-world practices.
Institutionalization = the extent to which the EBI is integrated within the culture of the recipient
setting or community through policies and practice
Maintenance = The ability of the recipient setting or community to continuously deliver the
health benefits achieved when the intervention was first implemented.
2. Models/frameworks
Stage models describe the steps of dissemination and implementation (D&I). The D&I of
interventions occurs as a series of successive phases: dissemination adoption implementation
sustainability.
2.1 RE-AIM framework (Reach, efficacy, adoption, implementation, and maintenance)
Goal: encourage program planners, evaluators, readers of journal articles, funders, and policy-
makers to pay more attention to essential program elements (including external validity) that
can improve the sustainable adoption and implementation of effective, generalizable, evidence-
based interventions
Uses:
o Reporting: helps with reporting all important aspects when writing an article
o Enhance the quality, speed, and public health impact of efforts to translate research into
practice
o Evaluating the public health impact of health promotion interventions RE-AIM
score for the letters between 0 and 1 (0 and 100%)
o Planning of D&I project helps researchers in development of multistage (R, A, I, M),
and multilevel (individual, setting) indicators
,Summary Implementation and Evaluation – HEP4205 (2021)
The five dimensions occur at multiple levels (individual and setting):
o REACH: how do I reach those who need this intervention? proportion of the target
population that participated in the intervention
The ability of a program to engage its ultimate target audience, both in terms of
quantity (number of participants) and quality (representativeness of participants)
reach can greatly influence the level of public health impact of the program
Reach is on the individual level
o EFFICACY: how do I know my intervention is working? success rate if implemented as
in guidelines, defined as positive outcomes minus negative outcomes
Behavioral outcomes, quality of life, participant satisfaction outcomes, and
physiologic end points, all in addition to biological measures
Efficacy is on the individual level
o ADOPTION: refers to the proportion and representativeness (number of employees etc.)
of settings (such as worksites, health departments, or communities) that adopt a given
policy or program how do I develop organizational support to deliver my
intervention?
Adoption is on the organization level
o IMPLEMENTATION: how do I ensure the intervention is delivered properly? extent to
which the intervention is implemented as intended in the real world (fidelity)
Efficacy x implementation = effectiveness effectiveness is used to describe
evaluations conducted in real-world settings by individuals not part of research
staff
Organization level (but individuals also have to actively participate)
o MAINTENANCE: how do I incorporate the intervention so it is delivered over the long
term? extent to which a program is sustained over time
Individual level: whether the individual still performs in things that maintain the
behavior booster sessions
Organizational level: extent to which a program/policy becomes institutionalized
or part of the routine update training for nurses
The product of the 5 dimensions is the public health impact score (population-based effect)
Limitations of the RE-AIM framework:
o Economic factors are not explicitly included
o Precise nature of the relationships among the 5 RE-AIM dimensions, and how they
combine to determine overall public health impact, is unknown
o There is no evidence for the suggested time intervals for assessing implementation (6
months-1 year) and maintenance (2+ years)
2.2 Theory of Rogers: diffusion of innovations
Diffusion of innovations is a theory that seeks to explain how, why, and at what rate new ideas and
technology spread. His work gives insight into the factors that influence adoption and strategies that
can be used to enhance adoption. planning
The diffusion system an external change agency and its paid change agents who, if well
trained, correctly seek out and intervene with the client system’s opinion leaders and
paraprofessional aides, and support the enthusiasm of unpaid emergent innovation champions
o A change agent is an individual who influences clients’ innovation-decisions in a direction
deemed desirable by a change agency (usually thus to start using the intervention)
,Summary Implementation and Evaluation – HEP4205 (2021)
o A champion is a charismatic individual who throws his or her weight behind an
innovation, thus overcoming indifference or resistance that the new idea may provoke in
an organization.
Innovation decision process: the process through which an individual passes from gaining initial
knowledge of an innovation, to forming an attitude toward the innovation, to making a decision
to adopt or reject, to implementation of the new idea, and to confirmation of this decision. There
are five phases:
1. Knowledge: occurs when an individual (or other decision-making unit) is exposed to an
innovation’s existence and gains an understanding of how it functions (what?, how? and why?).
There are three types of knowledge:
o Awareness-knowledge: knowing of the intervention/being aware that it exists
o How-to-knowledge: information about how to use an innovation correctly
o Principles-knowledge: how and why an innovation works mechanisms
2. Persuasion: occurs when an individual (or other decision-making unit) forms a favorable/an
unfavorable attitude towards the innovation. Five perceived characteristics of the innovation play
a role here:
o Relative advantage: the degree to which the EBI is perceived by potential adopters as
better (e.g. more effective, less costly) than the process or practice it replaces
o Compatibility = how well does it fit with the norms and values of the organization?
o Complexity: how easy/difficult is it to understand the intervention?
o Trialability: can you try it out before adopting it?
o Observability: the extent to which the outcomes can be seen
3. Decision: activities that lead to a choice to adopt or reject the innovation leads to either
adoption (and then later continued adoption or discontinuance) or rejection (and then later
continued rejection or later adoption)
4. Implementation: active process of putting the new idea into use
5. Confirmation: seeking reinforcement (try to reduce or eliminate internal disequilibrium) for the
innovation-decision already made or possible reversion of the decision if exposed to conflicting
messages about the innovation
, Summary Implementation and Evaluation – HEP4205 (2021)
The adoption rate is the relative speed with which an innovation is adopted by members of a
social system (external and internal factors influencing adoption) number of individuals who
adopt a new idea in a specified time period. The adoption rate is influenced by:
o The perceived attributes of an innovation mainly relative advantage and
compatibility (but also the other 3) if positive higher adoption rate
o Type of innovation-decision decisions by individual more rapid than adoption
decision by organization (more people slower)
Optional innovation decision: made by an individual who is in some way
distinguished from others
Collective innovation-decision: made collectively by all individuals
Authority innovation-decision: made for the entire social system by individuals in
positions of influence or power
o Type of communication channel interpersonal channels are slower than mass media
channels
o Type of social system the nature of the social system in which the innovation is
diffusing such as it norms, and the degree of interconnectedness in the communication
network. The social system is a combination of external influences (e.g. mass media) and
internal influences (e.g. distance from opinion leaders) influences adoption
o Extent of change agents’ promoting efforts
Rogers distinguishes five types of adopters based on their degree of innovativeness. This
depends on their needs, interests, motivation, SES, demographics, personality, etc. Adopters are
the minimal unit of analysis (individuals or organizations).
o Innovators (2.5%): they are willing to take risks (deal with uncertainty) and are the first
ones to adopt an innovation. They need to have substantial financial resources to
compensate for failure. Their role is to launch the new idea into the system by importing
it from outside of the system’s boundaries.
o Early adopters (13.5%): they have
the highest degree of opinion
leadership among the adopter
categories. Potential adopters go
to them for advice and information
about an innovation. They serve as
a role model for many other
members of a social system and
help trigger the critical mass when
they adopt an innovation (critical
mass is when the number of
individual adopters ensures that
the innovation is self-sustaining). Grey line = rate of adoption, black line = proportion of people
o Early majority (34%): they adopt new ideas just before the average member of a system.
Because they are in the middle, they provide interconnectedness in the system’s
interpersonal networks.
o Late majority (34%): they are more skeptical and need to be persuaded by close peers in
the interpersonal network. Economic necessity and peer pressure may lead them to the
adoption.
o Laggards (16%): they are more skeptical about innovations and change agents than the
late majority. They want to make sure an innovation works before adopting it, as they
have limited resources and lack of awareness-knowledge. Laggards decide after
successful adoption by other members of the social system in the past.
Criticism on Rogers’ theory:
