The evidence describes the information in support of solution(s) within a system and the means used to measure the performance of the system.
Contents
Introduction
The need to Collect the Evidence is the fourth of the seven strands in the improvement model. It underlines the importance of collecting the evidence that the system has been improved, following understanding the system, defining the problem and developing the solution. As a result, it is expected that such a definition will be developed in the intermediate stages of the improvement process and revised, as appropriate, as the process develops.
Purpose
Collecting the evidence has particular importance to systems improvement: defining the measures required to evaluate the performance of the system of interest; undertaking a range of evaluation activities to evaluate effectiveness, patient safety and patient experience; and synthesising the information to evidence the extent of the improvement to the system.
Activities
The process of collecting the evidence may include a wide range of activities including, but not limited to: Define Measures, Agree Quality Targets, Review Effectiveness, Review Safety, Review Experience and Synthesise Evidence.
Tools
The practice of collecting the evidence may draw on a wide range of tools including, but not limited to: Exclusion Audit, Expert Review, User Trials, Root Cause Analysis, Failure Mode and Effects Analysis, Fault Tree Analysis, Hazard and Operability Analysis, Structured What-if Technique and Risk Matrix.
Getting Started
Collecting the evidence is key to the successful evaluation of an improvement project. It is particularly important to collect the evidence starting early in the improvement process and to update the description as necessary during the remainder of the process.
The Understand, Design, Deliver and Sustain stages of an improvement process were previously described within the Introduction section, where each stage will likely comprise a preliminary activity, followed by a number of stage activities.
For the Collect the Evidence strand of the Understand phase, the preliminary activity is entitled Identify the Big Wins. For the Design, Delivery and Sustain stages, the evidence should be further developed into a full account of the effectiveness (or otherwise) of the implemented solution(s).
Identify the Big Wins
This preliminary activity encourages people to identify the big wins and justify the resources required to make the system changes proposed and deliver a description of the most likely outcome(s) from delivering the proposed solution(s) for improving the system.
Worksheets: Improvement Questions, Improvement Stakeholders, Improvement Strands Canvas, Improvement Plan
Top tips:
- Quantify how many patients could benefit from the changes
- Identify the potential benefits, costs and risks for the stakeholders
- Consider the alignment of risks and benefits for each stakeholder
Following on from this preliminary activity, the remaining stage activities are: Define Measures, Agree Quality Targets, Review Effectiveness, Review Safety, Review Experiences and Synthesise Evidence. Each of these are now described in turn, together with the tools that may be used to support them.
Activities
Define Measures
The translation of the core themes for improvement into the definition of appropriate and robust measures of success
Purpose: to define performance measures to evidence the successful delivery of a measurably better system
Inputs: Define Requirements, Agree Quality Targets, Review Effectiveness, Review Safety, Review Experience
Tools: tbd
Outputs: a clear definition of the performance measures required to evidence success
Top tips:
- Establish a clear rationale for the criteria for success
- Identify performance measures necessary to demonstrate success
- Align evaluation activities to the measures and criteria
Agree Quality Targets
The translation of the core themes for improvement into the definition of appropriate and robust quality targets to ensure success
Purpose: to define service quality targets to ensure the successful delivery of a measurably better system
Inputs: Define Requirements, Define Measures, Review Effectiveness, Review Safety, Review Experience
Tools: Literature Review, Soft Systems Method, Causal Loop Diagram, Entity Relationship Diagram, Swimlane Diagram, Dependency Structure Matrix
Outputs: a clear definition of the quality targets required to ensure success
Top tips:
- Define appropriate targets for clinical and cost effectiveness
- Define appropriate targets for patient safety
- Define appropriate targets for patient experience
Review Effectiveness
The systematic review of new concepts, identifying their strengths and weaknesses in meeting the prioritised stakeholder needs
Purpose: to identify clinical and cost effectiveness risks in the adoption of new concepts by their intended users
Inputs: Define Requirements, Filter Ideas, Develop Concepts, Make Models, Define Measures, Agree Quality Targets, Review Experience, Review Safety
Tools: Exclusion Audit, Expert Review, User Trials
Outputs: a list of the strengths and weaknesses of the new concepts in meeting the stakeholder needs
Top tips:
- Ensure concepts meet the demands in the requirements
- Check if concepts meet the wishes in the requirements
- Note requirements that relate to different stakeholders
Review Safety
The systematic review of the safety of the system, identifying hazards and the subsequent risks they pose to users
Purpose: to identify safety concerns in the adoption of new concepts by their intended users
Inputs: Define Requirements, Filter Ideas, Develop Concepts, Make Models, Define Measures, Agree Quality Targets, Review Experience, Review Effectiveness
Tools: Root Cause Analysis, Failure Mode and Effects Analysis, Fault Tree Analysis, Hazards and Operability Analysis, Structured What-if Technique, Risk Matrix
Outputs: a list of the hazards present within the proposed system and the safety risk they pose to users
Top tips:
- List the hazards presented to users along the care journey
- Estimate the likelihood of harm posed by these hazards
- Identify the likely extent of harm posed by these hazards
Review Experience
The systematic review of the demands made by the system on patients and other stakeholders, and their corresponding capability to respond
Purpose: to identify potential barriers to the adoption of new concepts by their intended users
Inputs: Define Requirements, Filter Ideas, Develop Concepts, Make Models, Define Measures, Agree Quality Targets, Review Safety, Review Effectiveness
Tools: Exclusion Audit, Expert Review, User Trials
Worksheets: Improvement Design Wall
Outputs: a list of accessibility issues within the proposed system and the challenge they pose to users
Top tips:
- Understand the demands made of users along the care journey
- Estimate the users’ capability to respond to these demands
- Identify where specific demands exceed users’ capabilities
Synthesise Evidence
The drawing together and summary of a body of evidence that describes the evaluation and selection of concepts for improvement
Purpose: to provide evidence of evaluation of new concepts against the system requirements
Inputs: Define Requirements, Consider Pre-existing Solutions, Stimulate Ideas, Filter Ideas, Develop Concepts, Make Models, Define Measures, Agree Quality Targets, Review Experience, Review Safety, Review Effectiveness
Tools: tbd
Outputs: a clear summary of evidence relating to the evaluation of the new concepts
Top tips:
- Draw together, summarise and communicate all of the evidence
- Do this in a systematic way to support the review criteria
- Use the evidence to support the choice of solution concept
Tools
Feedback
We would welcome your feedback on this page:
Privacy policy. If your feedback comments warrant follow-up communication, we will send you an email using the details you have provided. Feedback comments are anonymized and then stored on our file server
Read more about how we use your personal data. Any e-mails that are sent or received are stored on our mail server for up to 24 months.