Measuring Progress on
HCAHPS
Before we start…
Reminders:
• Letters of commitment
• IHI Open School
Your feedback is very important for us. So please continue
to share it with us. We truly appreciate the time you take
to give us your thoughts and input.
2
Important notes
Within3 Community
3
Experts From the Field
Ed
Mendez
RN, MPH
NSN
Improvement
Coach
Jane
Hooker
RN, MN, CPHQ
AVP for
Quality &
Innovation,
NAPH
Carrie
Brady
JD, MA
Principal, CBrady
Consulting
Jerod
Loeb
PhD
Executive VP
for Healthcare
Quality
Evaluation,
Joint
Commission
Sherri
Loeb
BSN, RN
Personal
Navigator
5
Polling Question:
How often do you review your HCAHPS reports’ data?
1-3 months
4-6 months
7-9 months
10-12 months
6
How to Assess Quality?
Donabedian Framework of Quality
Structure
equipment,
building, personnel
Outcomes
Process
Actions to evaluate
and treat patients
7
How is Donabedian Framework Related to
HCAHPS?
Donabedian Framework
HCAHPS Domains
Structure Process Outcomes
Hospital
environment:
Ex: cleanliness, quietness
Nursing communication Overall rating of hospital
& willingness to
recommend
Doctor communication
Responsiveness of staff
Pain management
Communication of
medication
Discharge information
8
Case Study
Problem: Low HCAHPS scores on the nursing communication domain.
Intervention: Implement a whiteboard within visual range of each patient
bed.
Implementation: Nurse education, simulation, and scripting to ensure
maximum effectiveness.
Audit:
1. Ensure the data on the whiteboard was accurate and timely.
2. Evaluate the effectiveness of the use of the whiteboard by staff when
discussing care plans with the patient and family.
9
Historical (baseline) Data:
Reviewing data shows low nursing
communication scores below the
national average of 77%.
10
Why is it Important to Plot Data in Time Order?
• Summary statistics hide information (patterns, outliers)
• In improvement efforts, changes are not fixed, but are adapted over time
• Time series graphs annotated with changes and other events provide evidence of
sustained improvement
11
• Line graph
o series of data over time
connected by lines
• Run chart
o line or points on a graph with a
median
o allows application of rules to
detect process change
• Control chart
o series of data over time with a
mean (average) center line
o upper and lower control limits
o allows statistical identification
of change
Three types of visual displays
12
0
1
2
3
4
5
6
7
8
1 3 5 7 9 11 13 15 17 19 21 23 25
CRBSI/1000 Line Days
Day
Example Line Graph
0
1
2
3
4
5
6
7
8
1 3 5 7 9 11 13 15 17 19 21 23 25
CRBSI/1000 Line Days
Day
Example Run Chart
Median
Example: Run chart
Example: Line graph
Example: Control chart
12
Base-line Data
13
Whiteboard Implementation
Process measurement:
• % compliance to whiteboard completion
• % of nurses compliant to discussing whiteboard
with patients
Proxy outcome measure:
• % of patients who understand the plan of care
presented on the whiteboard
14
Process Measures and Proxy Outcome
15
Process Measures
Proxy Outcome Measure
16
Whiteboard Implementation
17
Making Improvements “The Norm”
To ensure that improvements are permanent and
steady, the new way of working should become
the regular way of working.
It is important to address sustainability because
improved outcomes achieved during the
implementation phase of a project do not
automatically result in lasting improvements.
Thomas, S., Zahn, D. (2010). Sustaining Improved Outcomes: A Toolkit. Asian Pacific Islander American Health Forum, the Community Health Foundation of Western and Central New York, and the
New York State Health Foundation.
18
Continuous Auditing
Process Measures
Proxy Outcome Measure
19
Improved Outcomes
20
Random (Common Cause) Variation
• “Unassigned” variation
• Is present in all processes –
reflects “business as usual”
• Does not judge whether the
process is “good” or “bad”
• Is predictable
Example: Arrival time to work varies when driving
due to traffic lights and weather conditions.
0
5
10
15
20
25
30
35
40
45
50
55
60
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
Commute time in minutes
Date
Katie's commute time to work
Commute time (mins) Medians GoalNone Defined
21
Non-Random (Special Cause) Variation
• “Assignable” variation
o Is assignable to a specific
cause
o Is a special circumstance that
is not part of the process – not
“business as usual”
• Helps you determine if your
change is an improvement
Example: Arrival time to work varied one time due to a
breakdown of the car or involvement in an accident.
0
5
10
15
20
25
30
35
40
45
50
55
60
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
#NAME?
Commute time in minutes
Date
Katie's commute time to work
Commute time (mins) Medians GoalNone Defined
22
Rules for Detecting a Process Change in a Run Chart
(special cause in the variation of data points displayed)
Source: The Data Guide by L. Provost and S. Murray, Austin, Texas, February, 2007: p3-10.
A Shift:
6 or more
An
astronomical
data point
Runs: Too
many or too few
runs
A Trend:
5 or more
23
Orientation to Run Chart Tools’ Three Tabs
26
Step 1: Clear data entries in sample template. Save with new name
on your own computer.
Automatically
clears here
OR…be careful
if want to save
month/year labels.
You need to
manually clear all
other white fields.
and
and
27
Step 2: Edit the titles and legends fields as suggested in sample
template.
Type in your edits to best
reflect the metric
specifics for your
improvement project
(note ratio multiplier in
cell E8).
28
Step 3: Enter numerators and denominators (or just rates if using
final values in column C).
29
Step 4: Click on the “Run Chart” tab to see if the display makes sense. Edit font sizes
as needed -OR- return to “1st line” tab to edit title / axis / legend fields.
30
Step 5: Edit / add annotations to run chart.
31
Step 6: Return to “Run Chart” tab. Use run chart rules to identify special cause
variation. Readjust median line as applicable.
32
Next Steps
For more information on run charts, please read The run
chart: a simple analytical tool for learning from variation
in healthcare processes , also found on our reference list.
The case study and tools will be posted on our website
soon. Please make sure you download the tools and
practice using them.
Please take advantage of your valuable IHI Open School
access.
33
Next Steps
Please look forward to our next Webinar on March 27
th
:
Ensuring Leadership Engagement
Pre-work:
Take complimentary IHI Open School Course* on “The Human Side of Quality
Improvement” (Contact your Team Leader for registration information
regarding complimentary IHI Open School registration)
1. Overcoming Resistance to Change
2. What Motivates People to Change
3. Culture Change vs. Process Change
34
Next Steps
What topics would you like to read about on our
community? Help us provide you with what you want.
Should you have any further questions, please contact:
Mai AlSokair
Email: [email protected]
Phone: (202) 495-3350
Jane Hooker
Email: [email protected]
Phone: (202) 585-0134
35
