Welcome to the Quality Action E-Learning tool.

The overall aim of Quality Action is to improve the quality of the response to HIV and AIDS in Europe by increasing the effectiveness of HIV prevention using practical Quality Assurance (QA) and Quality Improvement (QI) tools. This E-Learning tool aims to support the practical application of the Quality Action tools for HIV prevention.

Quality Action offers:

Five QA/QI Tools

Five practical and knowledge-based QA/QI tools, with guidance and training materials, have been selected, developed and adapted on the basis of the best available scientific, theoretical and practice-based evidence. We present three (PQD, QIP and Succeed) in this E-Learning tool and the other two (PIQA and SCHIFF) are available on the Quality Action website.

See the Tool Selection Guide for further information on the selection of the tools.

See the Rationale for Quality Action and references for the evidence of effectiveness of the tools (coming soon)

Training

Quality Action held European-wide training workshops in four cities, Dublin, Barcelona, Ljubljana and Tallinn, attended by over 100 participants. In addition, some countries organise national training on one or more tools.

If you would like to offer training on quality assurance and quality improvement in your country, the Quality Action Facilitation Guide (LINK) has examples of training activities and instructions to support you.

See also National Training – some considerations on this E-Learning tool.

Website, core documents and translations

The website contains information about Quality Action tools, materials, activities etc. and is regularly updated. Translations are/will be available of some core documents for Quality Action. Further details are available on the website.

More comprehensive information is also available on the partner’s section of the website.

Forum

An interactive forum exists on the website so that those applying the tools can engage with each other and the trainers in relation to questions on and experiences during the tool application.

Click here to visit the forum

Collection of case studies

Some tools offer case studies as examples of how others have used them. You can find them in the sections on each tool on the website.

There will be new case studies added to the website as they are applied post Quality Action training.

PQD case studies.

Succeed case studies, Saphy Project and MMSM Project.

Charter for quality in HIV prevention and a Policy kit

Quality Action is developing these documents to help integrate QA/QI in HIV prevention at the programme, policy and strategic levels. Check the website for updates.

Key Terms and Concepts used in Quality Action

Here is a brief glossary of some key terms used in Quality Action to help with applying the tools. A document entitled Quality Action, Key Principles will also be useful and will be available on the website at a later stage.

Epidemiology and Data

Epidemiology refers to the study of the causes and distribution of infections, diseases or health problems in populations and the application of this study to their control.

National epidemiological data on HIV prevention will provide statistical information about the incidence and distribution of HIV (surveillance). You can also search for analytical and behavioural studies on health determinants, health promotion, HIV prevention and other data relevant to your project/programme.

Insights gained by comparing the effects of different approaches in different places, case studies and the experience of stakeholders complement our knowledge.

Evidence

Evidence for making sound decisions in HIV prevention is available from a range of sources. Choosing the approaches, interventions and methods most appropriate to a given situation and implementing them at the highest possible level of quality are important for effective prevention and health promotion.

Such narrowly defined experimental evidence is not always available or conclusive, especially for structural, social and behavioural interventions. A lack of availability of what is regarded as scientific evidence does not mean an approach or method is ineffective, nor should it stifle innovation.

In addition to evidence-based practice, other forms of evidence are required, such as cultural appropriateness for example. Quality Improvement tools can help articulate and document these.

Evidence-based practice and Practice-based evidence

These two concepts differ in terms of how evidence is generated and applied.

With evidence-based practice, evidence is generated according to scientific standards, often using randomised control trials, with a focus on quantifiable, measurable effects. This type of evidence is often published in peer-reviewed journals.

Practice-based evidence derives information on the effectiveness of interventions from the structure and logic of the practical work. This model is internationally recognised as Action Research, or community-based participatory research.

Practice-based evidence means that indications for the effectiveness of interventions are tested in a particular context, at a particular point in time and at a particular location.

This localising of evidence has the potential to produce new insights which can be immediately integrated into practice and contribute to a process of on-going improvement. This evidence may or may not be generalised into larger settings.

Evidence-based practice is informed by evidence that is scientifically generated (although the extent to which it is actually scientific is a topic of debate in psycho-social research fields). The evidence may change in light of new evidence, such as when new and better medication becomes available, for example.

With practice-based evidence, evidence and practice inform each other in a continuous cycle of quality improvement.

Goals, Sub-goals, Objectives, Activities and Indicators

The definitions of goals, objectives, activities and indicators differ in the literature. For the purpose of distinguishing between these terms when applying the QA/QI tools we suggest the following:

Goals (or aims) are the big picture changes we are trying to achieve. Your goal is the change you want to see happen, your terminal point or what you aspire to do. An example is the UNAIDS 90-90-90 goal to increase to 90% the proportion of people who know they have HIV, the proportion on treatment and the proportion with suppressed viral load.

Objectives or sub-goals are shorter term measureable changes on the way to achieving the long term or strategic goal. They should be articulated as SMART goals (e.g. defined in the PQD tool as Specific, Measureable, Attractive, Realistic and Time bound). Examples are ‘increase the number of MSM who have never had an HIV test accessing our service by 60% within the next year’, or ‘By, 2016, increase the number of needles and syringes distributed per client to an average of 250 per year’, or ‘By 2016, fully train three new peer educators who can reach sub-Saharan African migrant communities with HIV prevention messages in our city’.

Activities are the actions taken to achieve the goals and objectives.

Indicators are the measurements and targets that will tell you whether you have achieved the objectives. You can identify key performance indicators (KPIs) for your project/programme.

See some practical examples in the original case studies linked to the tools.

Key and Vulnerable Populations and Target Groups

Key populations are those people the most affected populations. People living with HIV are always considered a key population in HIV prevention.

In Europe, the key populations are gay men and other men who have sex with men (MSM), people who inject drugs, migrants from countries with generalised HIV epidemics and sex workers.

Some populations are more vulnerable to HIV in specific situations or contexts. They may not be affected by HIV uniformly in all countries. Examples of vulnerable populations include adolescents or those in prision.

You can identify the key and vulnerable populations in your own country from epidemiological and other data.

Target groups

Your target groups are those who serve as the focal point for a particular project/programme. There are two types of target groups:

Beneficiaries are those who you directly want to target and

Intermediaries are those you involve in the project/programme because they can effectively reach the beneficiaries.

Some people may belong to both categories, especially in peer-based interventions. For example, sex workers are beneficiaries and are also important as intermediaries who can approach other sex workers.

Some people regard the term ‘target group’ as conveying a non-participatory or top-down approach. That is not the intention in its use in the application of QA/QI tools and participation is a key principle of Quality Action.

Key Principles of Quality Action

Two key principles of Quality Action are Participation and Self-Reflection

Participation is important because no single point of view can give an accurate picture of the context in which a HIV prevention activity operates.

The participation of the target group is especially significant and a key element of the Quality Action tools. If the project/programme does not respond to the needs and context of the target groups it is less likely to be effective.

Participation involves

• Stakeholder involvement
• Communication, consultation and facilitation
• Team and group work
• Empowerment

Self-reflection means stepping back to critically examine how well our efforts actually work.

We already reflect on our work in our own minds and from time to time in conversations with colleagues. However, these reflections are rarely structured, rarely look at all aspects of a project/programme, rarely include all relevant stakeholders, are rarely documented and rarely lead to systematic changes.

Rigorous self-reflection is a prerequisite for improving quality because the assumptions we protect most fiercely are often the most rewarding to question. Self-reflection cannot be imposed by or on others; it is a voluntary process. It can take courage and a supportive environment to take this position.

It is important to document that we have and will continue to do our best with our interventions.

Self-Reflection involves:

• Self-determination
• A supportive environment
• A structured process
• Satisfaction

Practical application

This term is used in Quality Action to describe the process of practically applying QA/QI tools to projects/programmes.

Project/Programme Cycle

An important characteristic of projects/programmes is that they happen in cycles.

A HIV prevention activity, project or programme can be thought of as a continuous cycle consisting of plan, do, check, act (See Quality Action website for further details of this cycle).

Quality Improvement can be applied to any phase of the cycle. It is most effective if it is applied throughout the cycle.

Quality Assurance and Quality Improvement QA/QI

QA/QI is a process that shows what we are doing well while offering clear steps to take to improve quality.

Quality Assurance (QA) monitors the quality of services and activities against standards, including review, problem identification and corrective action.

Quality Assurance is particularly suitable where standardised services are offered, such as medical and clinical procedures (e.g. testing and counselling). However, most HIV prevention interventions such as information provision, education, behavioural interventions and community development are more complex.

Quality Improvement (QI) methods are more flexible than Quality Assurance. They can be applied to complex interventions that are tailored to particular contexts and settings.

Quality Improvement tools help identify, implement and evaluate strategies to increase the capacity to fulfil and exceed quality standards.

Quality assurance and improvement tools aim to ensure that decisions about what to do and which methods to use are based on the best available evidence, knowledge and opinion.

Quality Assurance (QA) and Quality Improvement (QI) tools at project/programme level are used to ensure that the chosen activities are planned, implemented, monitored and evaluated to maximise their potential effect.

Stakeholder participation

Increasing the number of perspectives means meaningfully involving stakeholders. Those who have an interest in the project/programme, especially the key populations and target group, are asked to contribute their point of view.

Many Quality Improvement tools ask questions that different stakeholders will answer differently, depending on their particular perspective. Eliciting these answers requires skills in communication, consultation and facilitation. Engaging in Quality Improvement creates time and space for communication and exchange within teams.

One of the outcomes of using quality improvement tools in a participatory fashion is increased team interaction and the opportunity for productive group work.

Building the participation of disadvantaged target groups in a significant and committed way contributes to greater ownership of the project/programme.

Standards

Standards are a set of criteria against which an intervention is compared or measured. Standards are normally based on general consent or established by custom or authority as being the benchmark for quality.

In the wider quality field, standards are used where activities can be described in detail and reproduced accurately and repeatedly. They originate in manufacturing and exist to some extent in clinical medicine (quality control, quality assurance).

HIV prevention is very context-dependent and the rigid transfer of standardised methods from one context to the next may not work in the same way. Standards that emerge during local quality improvement practice can be documented to provide practice-based evidence.

Standards may be specific to an individual project or they may be useful for a particular method across a range of contexts or for a variety of methods within a single context.

Deirdre Seery

Javier Toledo

Chantel De Mesmaeker

What is Succeed?

Succeed is an easy to use quality improvement tool using a guided questionnaire to help projects or programmes assess three aspects of their project: the structure, the process and the results. It is a self –diagnostic / self-assessment approach to quality improvement, helping the project to meet its own goals.

How will Succeed help improve quality?

Succeed documents what is working well, what improvement actions need to be taken, by whom and in what time frame.

Positive responses to questions generally indicate that a project or programme has a good structure and is being run in a way which will lead to intended (and quality) results. The more positive answers there are, the greater the likelihood that the project is of high quality. But it is not an examination, so there is no point in faking positive responses. The tools help to show what is working well and what can be improved upon, so honesty really is the best policy.

Negative answers will point to areas for improvement for planning and implementing on the project. Negative responses may need careful consideration as people may feel attacked or defensive which can result in resistance to improvements. Because negative responses provide opportunities for discussions on what improvements can be made, they are an essential part of the process.

You may decide that a negative response to a question requires actions that are unrealistic given the resources required, the values of the project or accessibility issues relating to the target group, for example. Or you may decide to review your goals as a result of a negative response. That is part of the value of this self-diagnostic approach and Succeed helps to make such decisions explicit.

Documentation is a key part of the application of the Succeed tool and the improvement process. Record all agreed recommendations and actions for improvement. You can then incorporate them into future action plans and strategies and review them within agreed timeframes.

Do not forget to note the achievements of the project to date where the questions are answered with yes or another positive answer! Succeed highlights areas of good quality as well as areas for improvement.

Background and evidence of effectiveness:

Succeed is a Quality Improvement questionnaire based on scientific research about success factors in the field of health promotion and has been evaluated for its effectiveness. It has been adapted from general health promotion for specific use in the field of HIV.

Evaluations of the original tool ‘How to Succeed in Health Promotion’ have shown that applying the tool helped different stakeholders improve their understanding of the project, as well as their own role in its activities and results.

Berensson, K., Granath, M. and Urwitz, V. (1996) Succeeding with Health Promotion Projects – Quality Assurance. Landstingsförbundet (The Association of County Councils).

Swedish National Board of Health and Welfare (1999) Uppföljning av kvalitetsmanualen “Att lyckas med folkhälsoprojekt”.

Maj Ader, Karin Berensson, Peringe Carlsson, Marianne Granath and Viveca Urwitz (2001) Quality Indicators for health promotion programmes. Health Promotion International. Vol 16, No.2. Oxford University Press.

What are the benefits of Succeed?

Succeed helps highlight achievements of the project and focuses on good practice and the quality of the work being done in a transparent way. It helps focus on questions such as: Are the goals clear? Are the right populations being reached? Are there sufficient resources and expertise to meet the goals? Is the message getting across to the key populations?

Succeed helps re-define and improve areas of work that may have become habitual. It also helps identify gaps in current practice.

Succeed is easy to use and accessible. It allows for different levels of application, from simple (using available data) to medium (collecting additional data) to in-depth (providing indicators for evaluation).

Succeed also allows for different levels of participation from high to low. It involves stakeholders in the change process, improves knowledge of the projects and clarifies the roles of participants.

Succeed workshops can motivate stakeholders so they become more engaged with the project.

Succeed helps produce of better documentation and reporting. A completed report identifying key success factors, clear and measureable goals and quality improvement actions can provide opportunities for stakeholders to participate more fully in the project and help embed the quality improvement process as a way of working.

When can Succeed be used?

You can use Succeed to review existing projects and programmes and help to ensure that the work of a project is steered in the planned direction. While it is not in itself a planning tool, it can be used to review a plan for a new project.

Succeed is designed primarily for organisations implementing HIV prevention interventions in projects or programmes. It is equally useful for large or small organisations, governmental or non-governmental, for projects working on the prevention of new HIV infections and those projects working with people living with HIV. Any project, programme or organisation working with population-based health promotion and prevention can use Succeed. The only requirements are a defined goal (however unclear), a defined plan and actions.

Case studies

Click here to view the Saphy project case study

Click here to view the MMSM project-case study

Click here to view the NSP project case study

What is the structure of Succeed?

Structure of Succeed?

Succeed covers three aspects of quality and the questionnaire is divided into these three parts: Structure, Process, Results. Each part has several sections. See Applying in Practice for further information.

‘Structure’ involves the set-up of the project, the organisation of operations and resources; it has 6 sub-sections: Goals, Key Populations, Approach, Responsibility, Organisation and Resources.

‘Process’ focuses on the activities of the project or programme; it has 3 sub-sections: Support and Participation, Networks and Reach & Response.

‘Results’ focuses on the difference a project is making and what can be measured; there are 3 sub-sections: Measuring Effects, Environmental, Operational and Social Changes and Sustainability.

Each section includes yes/no questions, open questions and action questions.

How can Succeed be used?

The scientific evidence base for the tool relates to its original format.

It is ideal if you answer the questionnaire as a whole. If a question is not (yet) relevant for your project, make sure to note why it is not (yet) relevant. Don’t be tempted to just answer the easy questions!

It is essential that you remain faithful to the methodology, where a positive answer indicates good quality and a negative answer indicates areas for improvement.

The more detail you provide, the better the picture you get about what improvements can be made to the project.

You need to support your answers. You may not have all the information you need and may have to collect it to support your answers, using more structured and/or extensive investigations, measurements, surveys, literature reviews etc., depending on the scope of the project. You can use them to support the evidence base for the particular project.

Stakeholders – Who should be involved?

You can complete Succeed either alone or, preferably, as part of a group where you discuss it with the participants. The participation of relevant stakeholders is an important ingredient in the success of the implementation of Succeed. The more different and appropriate perspectives you involve, the more complete will be the picture the tool provides for your project. The evaluation of Succeed as being effective relates to its application in a group discussion.

As the tool is easy to use and accessible, Succeed makes it possible for members of staff, the target group and other significant stakeholders to work together to jointly review the project, with a focus on improving its implementation. The results are likely to be more useful and interesting the more different perspectives you manage to invite to respond to the questionnaire.

Decide who to involve in the tool application: Do you want to involve the project team only? The project team and other members of your organisation such as the manager or other decision makers, other people close the operational level, board members etc? The project team, volunteers and some representatives from the key population who know your project and a manager? Some from the project team, target groups, key stakeholders, manager etc? Some stakeholders can contribute more to some sections than others. It may help to divide them into smaller working groups and then discuss their responses together.

It may be useful to appoint an independent person (they could be from another part of the organisation if resources are tight) to facilitate the implementation of the tool and/or a discussion about the responses to the questionnaire. The greater the diversity of participants you invite and the more participants you invite, the greater the need for a qualified facilitator.

What resources are required?

Preparation:

It can be useful to send the questionnaire to everyone involved in the application in advance of the meeting/s so that everyone has prepared input for the discussions.

If there are 12 people or more to be involved, it might be useful to have more than one group, using 10 as the recommended maximum group size. If you are not experienced with the tool, we recommend having not more than 8 people in a group.

Decide what time is available and when meetings will be held. It is advisable to tell people in advance that the time for the application should be made completely free and that phones and computer access should be reserved for the breaks.

Decide whether you need an external facilitator and nominate a note-taker so that the discussion is documented.

Time:

In addition to the preparation time, you can complete the application of Succeed within a timeframe of between one or two days, depending on levels of detail required and the numbers of people involved in the process. It can sometimes be helpful to have an initial meeting to go through the questionnaire. You will also need time to gather additional data on open questions and to complete the questionnaire. We recommend you plan for a second meeting some time later to review an action plan for quality improvement.

Facilities:

Creating an open and supportive environment for reflection is essential. Time-tabling, allowing for sufficient breaks, food, comfortable surroundings with good light and air, can all be considered when choosing the best venue for implementing the tool.

Finances:

The application of the tool itself need not require additional resources. It may be possible to use meeting rooms that are free of charge, for example. Food and travel expenses may or may not be an issue, depending upon geographical spread. Using Succeed may however point to the need for additional resources for the improvement of the project itself.

People:

the number of people involved depends on your decisions about participation. See Stakeholders above.

Challenges of using Succeed?

Succeed may highlight serious deficiencies and/or lack of sufficient data to support the project. This could impact on morale unless there was a positive commitment to the challenge of quality improvement. It is better to keep things simple in the early days of quality improvement than lose the support of important stakeholders.

Some participants may feel intimidated by the size of the questionnaire. It can appear less daunting if you point out all the empty space included for the answers. Otherwise, the questionnaire is very accessible.

Identifying areas for improvement may cause some people to become defensive and therefore resistant. Some people may also be resistant to applying a more structured way of thinking and working. Here, it helps to establish a positive and open atmosphere for reflection. Plan in advance how to handle the possibility of negative responses and resistance to change. Differences of opinion can help clarify assumptions. See Values Clarification or the Shift tool (links) for methods that may help.

People often struggle with the difference between the implementation of a self-diagnostic Quality Improvement tool and a formal evaluation process. Those involved in the application of the tool need to have trust in the process and must be able to decide how the information gathered is communicated to decision-makers and / or funders. You might consider how to help funders and decision makers to understand the value of Quality Improvement.

Checklist for implementing Succeed

What is PQD?

PQD stands for Participatory Quality Development. You can use PQD as a Quality Improvement tool for prevention and health promotion projects.

The PQD Toolkit is an integrated set of theory, eleven practical methods and a set of case studies. PQD aims to involve stakeholders meaningfully in improving Health Promotion and HIV Prevention and support projects/programmes, create practice-based evidence and improve the effectiveness of interventions.

How will PQD help improve quality?

PQD uses local knowledge to improve the relevance of projects/programmes and provide feedback on the effectiveness of interventions.

PQD methods can be used to improve the quality of interventions at all stages of the cycle – from needs assessment to planning, implementation and evaluation.

PQD methods can help gather data that supports evidence-based practice and in turn build practice-based evidence.

Although the PQD methods can be used to enhance the quality of a project/programme, they do not necessarily offer specific quality benchmarks. Stakeholder collaboration and participation can, however, be an indication of the quality of a project/programme.

PQD can be used with other Quality Action tools, for example the Succeed tool, which will provide a structure to guide quality improvement of your project/programme.

Background and evidence of effectiveness:

PQD is an approach used by Deutsche AIDS-Hilfe (DAH), initiated with WZB (the Social Science Research Centre, Berlin) and supported by BZgA, the German Federal Centre for Health Education. The feasibility of PQD has been tested in German HIV and Health Promotion organisations.

The methods and processes of PQD are based on evidence from the fields of health and social science, theory and practice. The four-phase PQD cycle is adapted from the Public Health Action Cycle.

PQD stems from the internationally recognised Action Research model (or community-based participatory research), which demonstrates the effectiveness of interventions through generating practice-based evidence.

What are the benefits of PQD?

PQD is a flexible and adaptable quality improvement tool that offers a choice of methods for all stages of the project/programme cycle.

PQD provides practical, step-by-step methods that are tailored, feasible, useful, participatory and evidence-based.

PQD helps you gather the local knowledge of target groups to help identify problems and provide insights into implementing interventions that are relevant and appropriate to the local situation.

When you use PQD methods, you can document local knowledge and theories and produce practice-based evidence.

PQD is a means to strategically increase participation of key stakeholders.

When you involve stakeholders, you create the opportunity for more informed and expansive thinking. Stakeholders then have greater ownership of interventions and their capacity to identify with the project/programme is strengthened. This can lead to better outcomes for the interventions.

When can PQD be used?

Quality improvement is a long-term commitment and different methods from the toolkit may be used at all stages of planning, implementation and evaluation.

You can use PQD in combination with other Quality Action tools.

You can use PQD in a variety of prevention and health promotion interventions such as information campaigns, outreach, group work, social support, self-help groups, community development as well as community-based health services and clinics.

You can use PQD if your projects/programmes are engaged directly with your target groups, or if you want to move in that direction.

Case studies

Click here to view PQD Case Studies

Click here to view the HIV toolkit for migrants case study

Click here to view the HCDCP case study

Click here to view PAAARC case study

Click here to view the BHA HIV prevention & community testing project case study

Click here to view the “AH Plus” case study

Local knowledge

You can use PQD to include the observations of the target group alongside the perspectives of service providers and other experts because they each possess local knowledge. By using the local knowledge (i.e. expertise) of key stakeholders, your project/programme will be more fully informed about the needs and environments of your target groups.

Based on this local knowledge, you may develop a local theory or understanding of the characteristics of the health problem and its context and causes, so as to devise appropriate interventions.

Local knowledge and local theories exist but are often implicit (unspoken). You can make these more explicit and clarify them through the participatory data collection, planning and evaluation methods in PQD, which are:

• Tailored to the specific local conditions such as the composition of the target group and the approach and capacity of your organisation.
• Feasible so that the time you put into the appbrcation is proportionate to the results you get from it.
• Useful so that you can act to improve the situation.
• Evidence-based (practice-based evidence)so that you can check if your intervention methods are appropriate to the local context.

Collaboration

Building and sustaining collaboration between stakeholders is a core principle of PQD, informed by the view that no individual stakeholder has a full picture of the local problem.

You can facilitate collaboration between different stakeholders in your project/programme, to help ensure the diverse views of stakeholders are heard, common interests identified and solutions mediated.

The stakeholders usually include the project team, members of the key groups or individuals, intermediaries in close contact key populations, funding bodies and others identified as key to the success of the project/programme.

You and your team can identify the key stakeholders for your project/programme by assessing whom you want/have to collaborate with for this intervention and how they should be involved. See Collaboration on the PQD website for further information.

You and your team can use PQD methods and other Quality Action tools to clarify structures and agree procedures and levels of authority in the decision-making process for the project/programme. For example, you can use the Circles of influence method to clarify who contributes to the decision-making process and at what level they are best involved.

Participation

PQD focuses on the strong and meaningful participation of key populations at any stage of the intervention (needs assessment, planning, implementation, evaluation).

Participation means more than being involved in the project/programme. It also facilitates ownership by target groups and service providers who possess the local knowledge required for the success of the interventions.

Participation depends on both the abilities and experience of all of the stakeholders and the various contextual factors of the intervention.

Strong and meaningful participation requires involvement in the decision-making process.

People are more likely to participate in your project/programme when they see the benefits of their involvement and that their contributions are valued and respected.

Participation requires planning to ensure that the key people are involved and that the collaborations are sustainable. This can be quite a time-consuming process and you will need to ensure that you and your team have the resources to support it. The benefits of participation for the project/programme outweigh this.

You can use some PQD methods to assess the level of participation best suited to your project/programme and your target group under current conditions. Other PQD methods focus on increasing the level of participation.

The staged levels of participation are as illustrated below:

What is the structure of PQD?

While the other tools provide a very clear structure to guide you through their application, PQD provides a variety of participatory methods for use at any stage of the intervention.

You can select the most appropriate methods and apply them to suit the problems, skills and interests of the stakeholders and the phase of the intervention.

The PQD cycle describes the four phases of interventions: needs assessment, project planning, implementation and evaluation/analysis, as you can see in the diagram below.

Needs Assessment refers to what the people need in order to improve their health and well-being.

When you use PQD methods for needs assessment, you will generate specific information about your key populations from the key stakeholders. Participation of stakeholders in needs assessment means that information is not only collected about the target group, but also from and with the target group.

In addition to gathering local knowledge using PQD methods, you can also use other data to help with needs assessment, such as epidemiological data, research, official reports etc.

Project planning with PQD methods means involving the target group and other stakeholders in setting the goals, objectives and strategies of the project/programme, in response to the results of the needs assessment.

Meaningful participation of stakeholders improves project planning by clarifying what is desirable at societal level (vision); the values, principles, ideas, methods and expectations of the organisation (mission); and the objectives of the intervention.

Evaluation with PQD methods means the participation of stakeholders in all stages of the evaluation process.

What stakeholders want/require from an evaluation can differ. If you involve the stakeholders from the beginning you can focus on the approach you need to the evaluation which best suits everyone’s (appropriate) needs.

How can PQD be used?

You can organise training workshops on the principles and methods of PQD to enable the project teams to facilitate meaningful and purposeful participation using the methods.

You will need to make a decision which stage(s) of the quality improvement cycle to work on and then select the most appropriate methods for that purpose. If you are not sure which part of the cycle to focus on or which aspect of your project/programme needs improvement, one of the assessment-based Quality Action tools, such as Succeed, can help you make that decision.

Ideally, your project will aim to improve all parts of the cycle and you will select one or more appropriate methods for each.

PQD Toolkit – Link to PQD on Quality Action website for Toolkit: All Tools at a Glance

The toolkit consists of eleven methods to improve interventions through participation. The methods are accessible on the right hand side of this page.

The methods include the following:

• Service User Advisory Committee

• Guided Working Group

• Open Space

• Focus Groups

• Rapid Assessments

• Enquiries and Concerns Register

• Circles of Influence

• Programme Logic

• SMART Criteria

• Participant Observation

• Developing Local Objectives and Strategies – ZiWi Method

The toolkit describes each method in detail with sections as follows:

Brief Description
• Prerequisites Applications
• Process Overview
• Resources Required (Time, Personnel, Materials, Other Costs)
• Detailed Working Steps
• Further Advice
• Further Reading and Links

While you can choose the methods which are most suitable for your project/programme, the toolkit highlights methods which are of particular use during each of the four phases of the cycle.

PQD also contains a collection of case studies that illustrate the use of specific methods in a range of health promotion and prevention projects.

Some methods require a high level of participation from the service providers and target group and others require a lower level of participation. This allows you to choose the method most appropriate to your project/programme.

You can also use PQD methods for facilitating or increasing stakeholder participation in the application of other Quality Action tools.

What resources are required?

Preparation:

It is essential that you are familiar with the PQD Toolkit. This will help you select and apply different methods to a variety of situations and at the different stages of the process.

You need to consider whether the meetings you organise to apply PQD are better facilitated by a neutral (external) person to allow for diverse views to be expressed and respected.

Decide what time is available and when meetings will be held. It is advisable to tell people in advance to keep the time for applying PQD completely free and that phones and computer access will be reserved for the breaks.

People:

The number of participants involved depends on the PQD method you select and your decisions about participation. See Levels of Participation above.

You will need people with experience and skills in facilitation and leadership to build and maintain mutual trust and respect between the various stakeholders in the project/programme.

Some methods require research expertise. You will at a minimum need people to take notes during meetings and make the documentation available to participants afterwards.

You can consider ways in which to support the on-going participation of key stakeholders, such as facilitating environments of mutual respect to ensure positive experiences of their involvement.

Time:

Some methods require a high degree of commitment from stakeholders and others require relatively low levels. The application of PQD methods can range from hours to days to weeks, depending on the aims and methods you choose. The PQD toolkit states the time required in the step-by-step guides to each method.

The greater the levels of stakeholder participation, the more time may be required to support it.

You will need to be prepared to contribute a significant amount of time to ensure sustainable, strong and meaningful participation at all stages of the cycle.

Facilities:

Creating an open and supportive environment for reflection is essential. Time-tabling, allowing for sufficient breaks, food, comfortable surroundings with good light and ventilation, can all be considered when choosing the best venue for meetings.

Finances:

The application of the tool itself need not require additional resources. It may be possible to use meeting rooms that are free of charge, for example. Food and travel expenses may or may not be an issue, depending upon geographical spread of participants and the ethos of the organisation.

You may consider providing financial assistance to support participation of those who are not employed by the project/programme, such as travel or childcare expenses.

What are the challenges of using PQD?

PQD requires leaders and facilitators who are skilled and share the principles of collaboration and participation.

There may be differences in power relationships and responsibilities between stakeholders (for example between funders and recipients; drug users or sex workers and government officials). You will need careful planning and facilitation to ensure all viewpoints are heard and respected.

PQD requires considerable time and a commitment to collaboration and participation which can be difficult to sustain given the diversity of interests involved.

PQD may require that you train people in using the methods so as to sustain collaboration and ensure participants engage with the process effectively.

You will need to ensure that you have the right people involved and consider how to identify them and how to motivate them to be involved.

Checklist for implementing PQD

What is QIP?

QIP stands for Quality in Prevention and is a comprehensive, evidence-based questionnaire. It is completed by the project or programme and assessed by external expert reviewers co-ordinated by BZgA in Cologne.

QIP is a quality improvement tool that uses 7 quality assurance dimensions with 22 sub dimensions.

The external QIP assessment offers a detailed profile of the project or programme against the quality dimensions and sub-dimensions.

QIP helps to ensure that prevention work is implemented in a targeted, effective and sustainable way.

QIP is more comprehensive and detailed than Succeed and includes an external assessment.

How will QIP help improve quality?

QIP delivers practice-oriented, detailed quality profiles and generates ideas for improving quality.

QIP identifies strengths as well as opportunities for improvement and prioritises areas for future action.

QIP offers the advantage of independent, external quality assessment.

QIP helps document that a project is quality-assured and based upon quality dimensions and up-to-date knowledge.

Background and evidence of effectiveness

QIP was first developed for use in general health promotion and then adapted for use in HIV prevention.

The generic version of QIP has undergone systematic field tests that examined its objectivity, reliability and validity over several years.

QIP is based on evidence about the efficacy, effectiveness and efficiency of prevention, health promotion and education interventions.

QIP has been evaluated, tested in practice and adapted for use in HIV prevention projects and programmes.

HIV prevention specialists have reviewed and pilot-tested the QIP version used for HIV prevention.

What are the benefits of QIP?

QIP helps improve your work according to evidence-based dimensions.

QIP provides the opportunity for external assessment and feedback on the quality of projects. An external viewpoint can be more objective than self-assessment. It can highlight “blind spots” which are easily overlooked internally and can also contribute new questions, suggestions and ideas.

QIP identifies opportunities for improvement at every stage of the project cycle, including the early planning stages.

QIP supports continuous quality improvement so as to increase efficiency, effectiveness and sustainability over the long term.

QIP helps improve the success of interventions by identifying gaps in the project and shortfalls in skills and knowledge.

QIP helps identify the links between needs, objectives, methods and effects.

QIP highlights areas where the quality of the intervention is already high.

QIP provides feedback on where operating environments need to change in order to improve projects and services.

QIP helps improve the quality of evaluation design so that projects and programmes can have more meaningful descriptions of measureable results, outcomes and operating environments.

When can QIP be used?

QIP has been validated for all professional prevention and health promotion tasks.

QIP examines the quality of programmes, projects, campaigns, setting-based interventions, one-off interventions, health education and training.

QIP can be used for innovative and recently initiated projects or programmes of any size.

QIP can be used to improve quality at any level and every stage of the project cycle: from planning to implementation to improving evaluation design.

What is the structure of QIP?

QIP focuses on key components for effectiveness in prevention and health promotion which are used internationally: project description and concept; personnel and their qualifications; target groups (beneficiaries and intermediaries); planning and preparation; dissemination and promotion; process design and results.

QIP uses yes-/-no or multiple choice questions where possible. It also uses open questions to stimulate responses on specific areas so that information about every aspect of the project or programme is documented.

How can QIP be used?

You complete a detailed questionnaire. It may either be used as a self-assessment tool or it may be sent to an external review panel selected by the QIP team at BZgA. Only the nominated contact persons of the project/programme will receive the results of the external review. Confidential information is handled by a data manager not involved in Quality Action.

When you receive the feedback and recommendations, you can use them to decide on actions for quality improvement. You can use methods from other tools, such as PQD for example, to improve aspects of the project. And you can apply the QIP tool again later to document your improvements.

Stakeholders – Who should be involved?

Anyone who is familiar with the project can complete the questionnaire. Your team may decide to complete the whole document together, or you may divide it into sections and then meet up to agree on a final version.

As completing the document is an opportunity for self-reflection, you and your team will gain a more complete picture when there is broad participation of the relevant stakeholders.

We have found from experience that using a facilitator who is familiar with the tool helps improve participation and communication.

You may like to use methods from PQD to help identify relevant stakeholders and the levels of participation best suited to the application.

Documentation

You will complete a questionnaire where you describe the structures, concepts, processes and outcomes of the intervention in detail.

See the QIP tool for details of the documentation to be completed.

Completing the documentation in itself provides an opportunity for self-reflection and conversations within your team.

The more detail you provide, the more complete the picture reviewers will get of your project / programme and the more specific their feedback.

What is the review process?

The review process includes assessment using the quality dimensions, analysis and specific feedback.

Assessment:

Experienced, independent, trained QIP reviewers use a detailed assessment guide with evidence-based criteria to assess structure, processes and outcomes according to set criteria outlined in the quality dimensions.

To ensure objectivity and validity, each project is assessed by at least three reviewers. The process is co-ordinated at BZgA, in Cologne.

The assessment creates a detailed profile of the project/programme against the QIP quality dimensions and sub-dimensions, with feedback on where the project/programme is doing well and suggestions about areas for improvement.

Reviewers rate each dimension using a set of clearly defined quality levels.

Levels of achievement are assigned ranging from 0 (problem zone) to 3 (exceeds standards) for each dimension. A score of near or above 2 indicates that the project operates at a good level of quality and can expect to succeed.

Analysis:

Reviewers send their assessments to the Federal Centre for Health Education (BZgA) in Cologne, Germany where the data is statistically analysed, feedback is compiled and sent directly and exclusively to the project. QIP ensures the statistical validity of the assessment.

BZgA pools the assessments and characteristics of all projects/programmes that have applied QIP in a database. The QIP database only contains non-identifiable, aggregate data, ensuring confidentiality for projects and programmes.

Benchmarks are arrived at through comparisons with the average scores among the different projects stored on the database. The QIP database is not yet large enough to derive benchmarks for comparative analysis in the field of HIV prevention, so projects/programmes participating at this stage will be important contributors to the database.

In addition to the assessment of the individual project, the analysis can show structural influences on quality by comparing projects from similar fields of activity. Examples of these include the impact of financial constraints or the benefits (or otherwise) of a targeted approach.

Feedback and Quality Profiles:

QIP provides project/programme-specific advice and suggestions for improvements based on the documentation submitted and current professional standards.

QIP provides an overall picture of the achievements, results and probable effectiveness of the project, indicating starting points for improvement so that actions may be taken quickly.

QIP reports only to the designated individual in the project.

What resources are required?

Preparation: You may find it useful to send the questionnaire to everyone involved in the application in advance of the meeting/s so that everyone has prepared their input for the discussions.

Decide how much time is available and when meetings will be held. It is advisable to tell people in advance that using QIP needs every participant’s full attention and that breaks are provided for urgent phone and computer uses.

Decide whether you need an external facilitator and nominate a note-taker who documents the discussion.

Time: In addition to the preparation time, the completion of the documentation by the projects or programmes may take regular dedicated time over up to two weeks. The preparation work may require additional time if stakeholders receive advance copies of the questionnaire. The review process takes approximately seven weeks, after which you need to schedule time to reflect on the feedback and plan actions to improve quality.

Facilities: Creating an open and supportive environment for reflection is essential. Time-tabling, allowing for sufficient breaks, food, comfortable surroundings with good light and air, can all be considered when choosing the best venue for implementing the tool.

Finances: For the duration of Quality Action, there is no fee for the review process for European HIV prevention projects and programmes. Apart from any organisational expenses required for facilities, the application of the tool itself need not require additional resources. QIP results may however point to the need for additional resources for the improvement of the project itself.

People: As the application of QIP is quite detailed, it is important to delegate duties and responsibilities. See section on Stakeholders involved for further information on who to involve.

What are the challenges of using QIP?

Some people may be resistant to applying a more structured way of thinking and working. Here, it helps to establish a positive and open atmosphere for reflection.

Some participants may feel intimidated by the questionnaire as it requires time and a level of detail that can be off-putting initially. Consider breaking up the task or use Succeed, as a more accessible tool to start with.

QIP may highlight serious deficiencies and/or lack of sufficient data to support the intervention. This could impact on morale unless there was a positive commitment to the challenge of quality improvement beforehand. It is better to keep things simple in the early days of quality improvement than lose the support of important stakeholders.

QIP reviews the quality of the intervention – It does not assess the entire organisation.

Checklist for implementing QIP