Comprehensive quality improvement by external expert assessment
QIP is a comprehensive quality improvement tool for health promotion and prevention projects. It uses external experts to assess a detailed documentation form filled in by the project. The questionnaire can also be used as a guide for the self-assessment of projects, programs or strategies.
QIP has been quality checked, tested in practice and adapted to the context of HIV prevention. It was developed in close partnership between the Federal Centre for Health Education (BZgA) and the University Medical Centre Hamburg-Eppendorf (UKE) in Germany.
QIP stands for Quality in Prevention and is a comprehensive, evidence-based questionnaire. It is completed by the project or programme and assessed by external expert reviewers co-ordinated by BZgA in Cologne.
QIP is a quality improvement tool that uses 7 quality assurance dimensions with 22 sub dimensions.
The external QIP assessment offers a detailed profile of the project or programme against the quality dimensions and sub-dimensions.
QIP helps to ensure that prevention work is implemented in a targeted, effective and sustainable way.
QIP is more comprehensive and detailed than Succeed and includes an external assessment.
How will QIP help improve quality?
QIP delivers practice-oriented, detailed quality profiles and generates ideas for improving quality.
QIP identifies strengths as well as opportunities for improvement and prioritises areas for future action.
QIP offers the advantage of independent, external quality assessment.
QIP helps document that a project is quality-assured and based upon quality dimensions and up-to-date knowledge.
Background and evidence of effectiveness
QIP was first developed for use in general health promotion and then adapted for use in HIV prevention.
The generic version of QIP has undergone systematic field tests that examined its objectivity, reliability and validity over several years.
QIP is based on evidence about the efficacy, effectiveness and efficiency of prevention, health promotion and education interventions.
QIP has been evaluated, tested in practice and adapted for use in HIV prevention projects and programmes.
HIV prevention specialists have reviewed and pilot-tested the QIP version used for HIV prevention.
What are the benefits of QIP?
QIP helps improve your work according to evidence-based dimensions.
QIP provides the opportunity for external assessment and feedback on the quality of projects. An external viewpoint can be more objective than self-assessment. It can highlight “blind spots” which are easily overlooked internally and can also contribute new questions, suggestions and ideas.
QIP identifies opportunities for improvement at every stage of the project cycle, including the early planning stages.
QIP supports continuous quality improvement so as to increase efficiency, effectiveness and sustainability over the long term.
QIP helps improve the success of interventions by identifying gaps in the project and shortfalls in skills and knowledge.
QIP helps identify the links between needs, objectives, methods and effects.
QIP highlights areas where the quality of the intervention is already high.
QIP provides feedback on where operating environments need to change in order to improve projects and services.
QIP helps improve the quality of evaluation design so that projects and programmes can have more meaningful descriptions of measureable results, outcomes and operating environments.
When can QIP be used?
QIP has been validated for all professional prevention and health promotion tasks.
QIP examines the quality of programmes, projects, campaigns, setting-based interventions, one-off interventions, health education and training.
QIP can be used for innovative and recently initiated projects or programmes of any size.
QIP can be used to improve quality at any level and every stage of the project cycle: from planning to implementation to improving evaluation design.
QIP: Applying in practice
What is the structure of QIP?
QIP focuses on key components for effectiveness in prevention and health promotion which are used internationally: project description and concept; personnel and their qualifications; target groups (beneficiaries and intermediaries); planning and preparation; dissemination and promotion; process design and results.
QIP uses yes-/-no or multiple choice questions where possible. It also uses open questions to stimulate responses on specific areas so that information about every aspect of the project or programme is documented.
How can QIP be used?
You complete a detailed questionnaire. It may either be used as a self-assessment tool or it may be sent to an external review panel selected by the QIP team at BZgA. Only the nominated contact persons of the project/programme will receive the results of the external review. Confidential information is handled by a data manager not involved in Quality Action.
When you receive the feedback and recommendations, you can use them to decide on actions for quality improvement. You can use methods from other tools, such as PQD for example, to improve aspects of the project. And you can apply the QIP tool again later to document your improvements.
Stakeholders – Who should be involved?
Anyone who is familiar with the project can complete the questionnaire. Your team may decide to complete the whole document together, or you may divide it into sections and then meet up to agree on a final version.
As completing the document is an opportunity for self-reflection, you and your team will gain a more complete picture when there is broad participation of the relevant stakeholders.
We have found from experience that using a facilitator who is familiar with the tool helps improve participation and communication.
You may like to use methods from PQD to help identify relevant stakeholders and the levels of participation best suited to the application.
Documentation
You will complete a questionnaire where you describe the structures, concepts, processes and outcomes of the intervention in detail.
See the QIP tool for details of the documentation to be completed.
Completing the documentation in itself provides an opportunity for self-reflection and conversations within your team.
The more detail you provide, the more complete the picture reviewers will get of your project / programme and the more specific their feedback.
What is the review process?
The review process includes assessment using the quality dimensions, analysis and specific feedback.
Assessment:
Experienced, independent, trained QIP reviewers use a detailed assessment guide with evidence-based criteria to assess structure, processes and outcomes according to set criteria outlined in the quality dimensions.
To ensure objectivity and validity, each project is assessed by at least three reviewers. The process is co-ordinated at BZgA, in Cologne.
The assessment creates a detailed profile of the project/programme against the QIP quality dimensions and sub-dimensions, with feedback on where the project/programme is doing well and suggestions about areas for improvement.
Reviewers rate each dimension using a set of clearly defined quality levels.
Levels of achievement are assigned ranging from 0 (problem zone) to 3 (exceeds standards) for each dimension. A score of near or above 2 indicates that the project operates at a good level of quality and can expect to succeed.
Analysis:
Reviewers send their assessments to the Federal Centre for Health Education (BZgA) in Cologne, Germany where the data is statistically analysed, feedback is compiled and sent directly and exclusively to the project. QIP ensures the statistical validity of the assessment.
BZgA pools the assessments and characteristics of all projects/programmes that have applied QIP in a database. The QIP database only contains non-identifiable, aggregate data, ensuring confidentiality for projects and programmes.
Benchmarks are arrived at through comparisons with the average scores among the different projects stored on the database. The QIP database is not yet large enough to derive benchmarks for comparative analysis in the field of HIV prevention, so projects/programmes participating at this stage will be important contributors to the database.
In addition to the assessment of the individual project, the analysis can show structural influences on quality by comparing projects from similar fields of activity. Examples of these include the impact of financial constraints or the benefits (or otherwise) of a targeted approach.
Feedback and Quality Profiles:
QIP provides project/programme-specific advice and suggestions for improvements based on the documentation submitted and current professional standards.
QIP provides an overall picture of the achievements, results and probable effectiveness of the project, indicating starting points for improvement so that actions may be taken quickly.
QIP reports only to the designated individual in the project.
What resources are required?
Preparation: You may find it useful to send the questionnaire to everyone involved in the application in advance of the meeting/s so that everyone has prepared their input for the discussions.
Decide how much time is available and when meetings will be held. It is advisable to tell people in advance that using QIP needs every participant’s full attention and that breaks are provided for urgent phone and computer uses.
Decide whether you need an external facilitator and nominate a note-taker who documents the discussion.
Time: In addition to the preparation time, the completion of the documentation by the projects or programmes may take regular dedicated time over up to two weeks. The preparation work may require additional time if stakeholders receive advance copies of the questionnaire. The review process takes approximately seven weeks, after which you need to schedule time to reflect on the feedback and plan actions to improve quality.
Facilities: Creating an open and supportive environment for reflection is essential. Time-tabling, allowing for sufficient breaks, food, comfortable surroundings with good light and air, can all be considered when choosing the best venue for implementing the tool.
Finances: For the duration of Quality Action, there is no fee for the review process for European HIV prevention projects and programmes. Apart from any organisational expenses required for facilities, the application of the tool itself need not require additional resources. QIP results may however point to the need for additional resources for the improvement of the project itself.
People: As the application of QIP is quite detailed, it is important to delegate duties and responsibilities. See section on Stakeholders involved for further information on who to involve.
What are the challenges of using QIP?
Some people may be resistant to applying a more structured way of thinking and working. Here, it helps to establish a positive and open atmosphere for reflection.
Some participants may feel intimidated by the questionnaire as it requires time and a level of detail that can be off-putting initially. Consider breaking up the task or use Succeed, as a more accessible tool to start with.
QIP may highlight serious deficiencies and/or lack of sufficient data to support the intervention. This could impact on morale unless there was a positive commitment to the challenge of quality improvement beforehand. It is better to keep things simple in the early days of quality improvement than lose the support of important stakeholders.
QIP reviews the quality of the intervention – It does not assess the entire organisation.
