RANAS Approach

Phase 1: RanasEXPLORE

Specify behaviours, behavioural factors, and context

We first define the exact behaviour to be changed and the specific population group to be targeted – we specify who exactly should change which behaviour (Step 1.1: Define target behaviours and target audience). Then, we collect information on the behavioural factors that might influence the target behaviour in the specific population, for example by conducting qualitative interviews (Step 1.2: Explore relevant psychosocial and contextual factors). Thereby, we gain a first impression of the behavioural factors that potentially determine the target behaviour in the specific population and context. In the following, the potential behavioural factors that we have identified are included in the RANAS model (Step 1.3: Complement the RANAS factors); this means adapting and extending the RANAS model to the local context. To finalise, we decide how the project design will look like (Step 1.4: Define project design) as a basis for the next phases.

The steps of this phase are:

1.1: Define target behaviours and target population

1.2: Explore behavioural and contextual factors

1.3: Specify the RANAS behavioural factors

1.4: Define project design

Step 1.1: Define target behaviours and target population

In this step, we will define the specific behaviour to be changed and the specific population group to be targeted. In other words, we specify who exactly should change which behaviour.

Key actions

Define the target behaviour

It is important to precisely define what behaviours need to be targeted by the behaviour change campaign. Take the example of safe water consumption, one needs to define within the local context whether the campaign should target where drinking water is collected, how it is stored at home, if it needs to be treated at point of use etc. The more precisely a target behaviour is identified and described, the more effective a campaign can be designed. For more examples of behaviours relevant for health and the environment see Resource 1.1.

Often, one behaviour depends on other behaviours. To stop open defecation, first a latrine has to be constructed and kept clean. For handwashing to occur, soap and water have to be available. For safe water to be consumed, it may be necessary to first disinfect the water and then store it safely. For proper waste management, separate bins and separated recollection needs to be organised and available.

To select the appropriate target behaviour, it is useful to answer certain key questions. First, you want to think about what health and environmental problems are currently happening and what people would like to see change:

• What is the current disease burden and what behaviours are linked to those diseases?
• Which environmental problems is the community exposed to or contributing to?
• Which behaviours would community members like to change?

Once you have a draft list of important behaviours to potentially address, then you want to ask some follow-up questions on each behaviour to learn more and be able to make a final selection:

• What are existing governmental programs and policies recommending about that behaviour?
• What are ongoing activities in the community about that behaviour?
• Does the behaviour require infrastructure? Is that infrastructure available?
• How common or rare is this behaviour? Does no one do it or do a few people already show the behaviour?

Ideally select one or two behaviours. Implementing the RANAS approach for more than three behaviours at a time within the same project is difficult to do. If you need to, we advise to create implementing teams for three behaviours each.

Conduct key informant interviews

Even if your project team is familiar with the preferences of community members and can give responses to the key questions above, it makes sense to check your assumptions and collect additional information. For this, select 5 to 10 key informants, who are individuals who you believe have an in-depth knowledge of the target community and project context. Key informants include members of the target group and decision makers. You may also select few key informants only and then ask those who else they would recommend you to interview. You can adapt and complement the example questions for key informant interviews (Resource 1.2).

Conduct fast behaviour checks

It makes only sense to target a behaviour for which the required infrastructure is present, but which is not yet practiced consistently in the target community. Promoting latrine use, for example, does neither make sense if there are no accessible latrines nor if everybody is already using latrines for defecation. If you do not have reliable data about the prevalence of a behaviour and availability of infrastructure, then you need to collect them. We recommend to conduct spot check observations in 50 to 100 randomly selected households, see Resource 1.3 for examples of spot-check questions.

Describe all the components of the target behaviour

If you have successfully identified your target behaviour, you need to specify it further. A behaviour is a sequence of actions; an action is an observable single act. To define a behaviour comprehensively, we have to describe all the actions involved. Many behaviours are practiced in response to certain cues, such as handwashing with soap is practiced in relation to other activities such as eating. Resource 1.4 provides two example descriptions ‘to use a latrine’ and ‘to wash hands with soap’.

Define the target group

Next, we need to define the population group to be targeted. Some behaviours have different main actors while other behaviours should be practiced by everyone. Fetching water, for example, often is the task of girls or mothers while latrine construction usually falls to the domain of the male head of household. Handwashing or latrine use, on the other hand, should be practiced by everyone. But even in those cases, the behaviour of a particular group of people may have a greater influence on the household’s or population’s health, either directly (e.g. handwashing before cooking by primary caregivers) or indirectly by influencing others’ behaviour (e.g. teachers or natural leaders as role models).

Therefore, the specific group to be targeted by a campaign typically depends on the behaviour to be changed. Different campaigns may be necessary for different target groups, see Resource 1.5 for examples of potential target groups.

In order to select the appropriate target group, it is useful to answer certain key questions:

• Who are the persons to practice the target behaviour?
• Whose behaviour has the greatest influence on the family’s health or on the environment?
• Whose influence on other people’s behaviour is highest (who are potential role models)?
• Who are the persons most at risk if the behaviour is not practiced?

Outputs

The behaviour(s) to be changed is/are defined.

The target population(s) is/are defined.

The project context is explored

Step 1.2: Explore behavioural and contextual factors

This step describes how to collect information on behavioural and contextual factors in order to adapt the RANAS model to the project context. The chapter Introduction provides an overview about the RANAS behavioural and context factors.

Behavioural factors are elements in the mindset of a person, such as knowledge, beliefs, and emotions which can be motivators or barriers for behaviour performance. Because every population is unique, behavioural factors beyond those contained in the RANAS model may be relevant in a specific population. In addition, some behavioural factors of the RANAS model are very generic, such as “feelings” or “barriers” and need to be specified.

Contextual factors, in contrast, are conditions outside of a person’s mindset that may facilitate or hinder a behaviour, such as existing water infrastructure or information provided at a health centre. We have started to explore these factors already during the previous step through spot-check observations and key information interviews. In this step, we will complete these insights through interviews with members of the target population.

Key Actions

Get familiar with the RANAS behavioural factors

First, gain familiarity with the RANAS behavioural factors. Resource 1.6 provides definitions of behavioural factors along with examples of typical thoughts related to each factor.

Conduct individual qualitative interviews or focus group discussions

There are different ways of collecting data in this step. Focus group discussions are widely used, because they are time-efficient and allow discussion between participants. The group interactions bring out the the social norms around the behaviour and give you the chance to better understand them. However, these group processes and social pressure (Resource 1.7) will also hinder participants from expressing their opinions and beliefs freely. So, while well suited for better understanding the social norms, focus group discussions are less useful to explore the remaining behavioural factors. Resource 1.8 provides general instructions on conducting focus group discussions.

Individual qualitative interviews have the advantage of concentrating on only one participant at a time, they allow to gather data in a more private setting so a person is more likely to share freely, and the course of the data collection can be adapted to the participant’s individual responses. Although they are more time consuming, for these reasons we recommend to conduct individual qualitative interviews if possible. Resource 1.9 provides general instructions on how to conduct individual qualitative interviews.

We recommend that you prepare a question guide in advance (Resource 1.10), although the course of the interview or focus group discussion does not have to strictly follow the question guide. The guide helps to ensure that detailed information on the current practice is gathered and all behavioural factors that require further specification are explored. Irrespective of which tool you use, collect data from approximately 20 participants. If you chose focus groups, the size of each group should not exceed 10 participants.

Data quality obtained from both tools is highly dependent on the skills of the data collectors. So, ideally, the project team conducts the interviews themselves instead of hiring and training interviewers. If you require additional data collectors, select (Resource 2.14) and train (Resource 2.15) them carefully. Consider ethical principles for conducting individual qualitative interviews or focus group discussions, Resource 1.11, and train your team accordingly.

It is not necessary to prepare a full transcript of an interview or a group discussion. However, it is critically important to take notes during the interview. This is best done by a designated note taker.

Outputs

Qualitative data on behavioural and context factors

Step 1.3: Specify the RANAS behavioural factors

In the following phase (RanasMEASURE) you will develop a quantitative questionnaire to measure all behavioural factors, contextual factors and behaviour. The questionnaire that you will use will have to be adapted to your specific project context. In the present step, we use the qualitative data collected so far to adapt the RANAS model and specify the behavioural and contextual factors accordingly. The resulting adapted RANAS model is thus both context specific and based on existing scientific evidence provided by the RANAS model.

Key Actions

Enter the data

After returning from the data collection, adapt the example data entry table (Resource 1.12) so it contains all questions that you asked during the qualitative data collection. Then consult your notes and, for each respondent, write a short summary of their response to the respective questions into the data entry table.

Specify existing factors and include additional factors

Use the filled-out data entry table (Resource 1.12) and, in the final column, summarise the responses to each question. Then allocate these summaries to the corresponding RANAS factors (Resource 1.13). For example, the responses to the question “What are your positive feelings towards …?“ reveals which specific feelings may be relevant for the target behaviour in the specific population and should be noted under the RANAS factor feelings in the attitude factor block. If the responses do not fit to any of the existing behavioural factors in the RANAS model you may also want to add a new factor. Also make sure to note frequently mentioned contextual factors. This will then allow you to develop the quantitative RANAS questionnaire, as you will learn in Phase 2: RanasMEASURE.

Outputs

Potential motivators and barriers to the target behaviour in your specific target population are specified.

The behavioural factors of the RANAS model are adapted to the local context and can be used to adapt questions and answer options in the quantitative questionnaire (see Phase 2: RanasMEASURE).

Step 1.4: Define project design

Before moving on to the next phase, we recommend to define the project design. When planning a project, you have to define in the very beginning if and how reliably you want to evaluate change in behaviour and behavioural factors. Such evaluation allows you to continuously improve the campaign, prove its impact, and scale it up. The method you chose for the evaluation already requires attention when starting the project, because the project design depends on it. Your evaluation’s reliability is mostly determined by the comparison group you use in the project design. A strong comparison group lets you better attribute changes in behaviour and behavioural factors to the campaign (and rule out the effect of background context like regulations, seasonality, or socio-economic conditions). We recommend three options to evaluate changes, each with a different type of comparison group: independent comparison, natural comparison, and no comparison. Please see Resource 1.14 for a visual representation of the following explanations.

Key actions

Define how to evaluate changes 

Option 1: Phased implementation with before-after measurement and independent comparison group

The most reliable method to evaluate your behaviour change campaign is by comparing changes in people’s behaviour and behavioural factors in areas where you implemented the campaign (campaign group) to areas where you will implement the campaign later (comparison group). Practically, this means that, first, you measure behaviour and behavioural factors in the entire project area (baseline survey), second, implement the campaign in only one half of the project area, third, measure behaviour and behavioural factors again in the entire project area (follow-up survey), and fourth, implement the campaign in the other half of the project area. This means that at the time of the follow-up survey, only half of the project area has received the campaign. Using baseline and follow-up data, you can compute the changes in behaviour and behavioural factors for each of the two areas and compare them. This method allows for the strong conclusion that the differences between groups that you measure are due to the RANAS campaign (and not due to background changes in the context). We recommend using this method whenever possible.

Option 2: Regular implementation with before-after measurement and a natural comparison group

However, resources, time and other considerations may not always allow to have a real comparison group as described above. In this case, your behaviour change campaign can be evaluated using a “natural comparison group”. To do so, conduct the baseline survey, implement your campaign in the entire project area and conduct the follow-up survey. In the follow-up survey, you include a “campaign check”. This means that for each participant you determine whether they have received the campaign or not. Individuals who have received the campaign or most parts of it are the campaign group. Individuals who have not received the campaign or only limited parts of it are the comparison group. You can then compute the changes in behaviour and behavioural factors for each of the two groups and compare the changes. This method provides some evidence that differences between the groups are due to the RANAS campaign (and not due to background changes in the context). However, it is possible that specific people tend to participate more readily in your campaign than others, such as those already aware of the topic. This can bias your results.

Option 3: Regular implementation with before-after measurement without comparison group.

The easiest but most unreliable method is to measure behaviour change before and after the campaign implementation in the campaign group only. To do so, you conduct the baseline survey, implement your campaign in the entire project area and conduct the follow-up survey without using the campaign check. This method may be used in stable contexts and in cases where the time between baseline and follow-up is short. However, even then, regulations, seasonality, or socio-economic conditions may have changed at the same time, and you cannot disentangle the effects of the background changes in the context from the effects of the RANAS campaign. We recommend this method only if none of the others are feasible.

Summarising, like visualised in Resource 1.14, we have the following project designs to evaluate behaviour change:

1. Phased implementation with before-after measurement and independent comparison group,
2. Implementation with before-after measurement and a natural comparison group,
3. Implementation with before-after measurement without comparison group.

Consider ethical standards in campaign planning 

To adhere to good ethical standards, all individuals participating in the project need to equally benefit from it. Thus, participants that are part of a comparison group as mentioned in this guideline, should also receive a campaign. To be able to nevertheless receive the necessary data and findings of a comparison group, we can give the campaign to the comparison group after the implementation of the campaign and the endline evaluation (phased implementation). If a phased implementation is not possible, for example when working in an emergency, the comparison group could receive a standard campaign, or different campaign groups could be compared which have received different aspects of the campaign. The participating communities should be consulted to make the respective decisions conjointly or with the respective input.

Outputs

Project design

Phase 2: RanasMEASURE

Collect data on behaviours, behavioural factors, and context

In the previous Phase 1: RanasEXPLORE, we have learned about the different behavioural and contextual factors that form part of the RANAS model for behaviour change. The chapter also discussed the importance of carefully selecting the target behaviour and deciding for a project design that fits your context and resources. It also introduced the different steps to conduct qualitative interviews to explore the behavioural factors potentially steering the target behaviour and finally provided guidance on how to specify the behavioural factors of the RANAS model according to your project context.

In Phase 2: RanasMEASURE, we will build on the findings from Phase 1: RanasEXPLORE to develop the tools needed for a quantitative baseline survey and discuss how to plan and implement the survey. The first Step 2.1 is to develop a questionnaire measuring the behaviour and the behavioural factors, which have now been adapted to the specific context. The information we gained from Phase 1: RanasEXPLORE will guide the development of the quantitative questionnaire. If required, we develop a protocol of structured observations of the target behaviour. In the second Step 2.2, the questionnaire and observations are implemented in a baseline survey. The results of Phase 2: RanasMEASURE, are quantitative data on behaviours, behavioural factors, and context for the specific target audience.

The steps of this phase are:

2.1: Develop a questionnaire and a behaviour observation protocol

2.2: Conduct a baseline survey

Step 2.1: Develop a questionnaire and a behaviour observation protocol

In this step, we develop a survey tool to measure the target behaviour and the behavioural factors. To measure the behaviour, we have three options: direct observations, spot checks, and self-reports (i.e. questionnaires). For the behavioural factors, we have only one option: self-reports measured with the help of a questionnaire. The same is true for some of the contextual factors (e.g. a respondent’s age) while others are observable or measurable by spot checks (e.g. distance of the water source).

Beware that the development of the questionnaire is a crucial step: it requires rigor, time, knowledge and skills. It may be advisable to seek assistance from a RANAS expert for this essential step, even if you have experience in using KAP – knowledge, attitude, practice – surveys as a thorough understanding of the RANAS model is required. See Resource 2.1 for a comparison of RANAS surveys to KAP surveys.

Key Actions

Select mode of data collection

There are different ways to conduct interviews: face-to-face, via phone call or online. If the survey includes a direct conversation (face-to-face or phone-based) with the participant, the answers can either be recorded using paper-pencil or electronically. Direct interaction has the advantage of being more direct and adaptable to the needs of the participant. Whereas internet-based surveys reduce social desirability and are time-saving but participants need to be literate. It is advisable to think through all possible advantages and disadvantages before selecting on mode of data collection, see Resource 2.2.

Introduction and consent

The questionnaire starts with an introduction that briefly explains the general purpose of the survey to the household. It outlines the importance of the respondent’s participation, a statement guaranteeing confidentiality, and a section obtaining informed consent, see Resource 2.3. It also includes information on how participants can hand in complaints regarding the survey.

Development of behavioural, psychosocial and contextual questions

To measure the behaviour as well as each behavioural and contextual factor, we formulate at least one and often several questions. Different question formats, see Resource 2.4, can be used: open questions with or without predefined response options, or closed questions with uni- or bipolar rating scales. Formulating meaningful questions and response options is very important and possible pitfalls are manifold. It is essential to follow some basic rules for formulating questions, Resource 2.5, response options, Resource 2.6, and rating scales, Resource 2.7. One option to make rating questions easier to be answered by participants is by using a visual scale, Resource 2.8.

Further, the questions have to be comprehensible to the target population. Therefore, it is essential that people familiar with the local languages are involved in the questionnaire development. If the questions cannot be developed directly in the local language (e.g. because the focal person of the project does not speak the local language), we have to translate them, see Step 2.2: Conduct a baseline survey.

The sample questionnaire, Resource 2.9, presents questions to measure behaviour, all RANAS behavioural factors and some contextual factors for water treatment with chlorine. Please bear in mind that these are sample questions, and it is not a ready-to-use questionnaire. The questions have to be adapted to the specific behaviour and population group and to the specific local conditions.

Consult results from Phase 1: RanasEXPLORE

During Phase 1: RanasEXPLORE, we gathered qualitative insights and used them to specify the behavioural and contextual factors. Now, these insights will help to develop the quantitative questionnaire. Create questions which correspond to the specified factors, e.g. one question for each identified feeling associated with the behaviour. Also, choose answer categories depending on the data you obtained, e.g. what kind of barriers are most commonly named and which kinds of solutions.

Identification of households

In case we decided in Step 1.4: Define project design to use a project design with a follow-up survey, we have to find the same households and target persons surveyed at baseline again for the follow-up survey. Therefore, we need questions that collect sufficient information to unambiguously identify an individual person even after a long time. The type of information that is suitable depends on the context, we often need to ask for the name of the participant and the name of her/his father. Household ID numbers or GPS coordinates can also be helpful. For data protection it is crucial to later store the interview data separate from personal identification measures, see Step 3.1:  Prepare the dataset for analysis.

Behaviour observation protocol

The questionnaire measures behaviour by self-reports. A more reliable and valid way, however, is to use direct observations and spot checks, Resource 1.3. For example, observe where a person goes to defecate, whether a person washes hands with soap after defecation, or how much litter can be found in the streets. Direct observations are usually very time-consuming and thus costly. However, they are more objective than self-reports. We recommend to complement self-reports with at least short spot-check observations.

Spot checks measure the behaviour indirectly; they measure proxy indicators of the behaviour (e.g. soap and water at the handwashing station to measure handwashing) and outputs of the behaviour (e.g. PET bottles in the sun to measure solar water disinfection, SODIS). Therefore, they are somewhat less precise than direct observations. However, they are very quickly and easily collected and thus very cost effective. Examples of spot checks include the water level in the water filter to measure water filtering, cleanliness of hands to measure handwashing, the amount of correctly separated garbage in different garbage containers to measure waste separation, and cleanliness of latrines to measure toilet cleaning.

For both direct observations and spot checks, we prepare a protocol that includes specific instructions on what and how to observe as well as a checklist to record the observations, Resource 1.3. Usually, spot checks can be included in the same document as the questionnaire; for direct observations, it is advisable to prepare a separate manual.

Arranging questions in a meaningful order

Once all the questions have been formulated, we compile the questionnaire by arranging all the questions into a meaningful sequence, Resource 2.10. If an observation protocol is used, it usually makes sense to place this in the end of the questionnaire after the interview questions.

To assist the interviewer, we can also include hints on question types, household selection or definition of terms used in the questions.

Translate the questionnaire and observation protocols into the local language

Unless the questionnaire has been prepared in the local language, we have to translate it, taking into account the specific vocabulary and dialect of the target population. The translation is vital; simply providing data collectors with the original, untranslated questionnaire and letting them translate the questions individually is not an option. In such a scenario, each data collector would ask the questions slightly differently and perhaps even change the wording from interview to interview. To be able to compare the data for analysis, all the data collectors have to ask the questions identically; therefore, we need a translated questionnaire.

We have two options for the translation; we can hire a translator, or we can translate the questionnaire together with the data collectors during training, Resource 2.11.

Outputs

Survey tool that includes a structured questionnaire and, optionally, an observation protocol. With these, we can ensure that we collect the same types of information from all participants in the same way.

Step 2.2: Conduct a baseline survey

After the questionnaire and observation protocols are finalized, the next step is to conduct the baseline survey. The data will be used for the doer and non-doer analysis in Phase 3: RanasANALYSE, before we then derive the behaviour change techniques in Phase 4: RanasDESIGN. It is important to survey a relatively large and representative sample of the population to receive a clear picture of the frequency of the behaviour and the psychosocial factors. The sample of individuals selected in this step will be surveyed again in Phase 6: RanasEVALUATE after the intervention. Thus, we can follow their changes in behaviour and psychosocial factors over time.

The key actions presented here do not all need to be executed sequentially; some can occur in parallel and, depending on the mode of data collection, some can be omitted.

Key actions

Define the sample size and the sample selection procedure

Whenever the target population is too large to be surveyed in its entirety, we have to select a part of the population and survey this sample. To receive a high-quality sample, two aspects are relevant: first, the sample size and second, the selection procedure, Resource 2.12.

Schedule the data collection, define the number of data collectors to be employed and supervisors to be appointed

When the sample size and sample selection procedure have been defined, we can schedule the data collection and define the number of data collectors to be employed. It is necessary to know the approximate daily capacity of a data collector, Resource 2.13. For a team of 10 data collectors, you need at least one local supervisor, who organizes the data collection and supervises the team. A local supervisor should have the same mother tongue as the target population and be familiar with local customs and social protocols.

Employ data collectors

The next key task is to select and employ data collectors. Resource 2.14 provides some information on the requirements for data collectors and the advantages and disadvantages of appointing health promoters as data collectors. We recommend employing one or two additional data collectors; they serve as stand-ins during data collection.

Organise the data collection

A visit to all the communities to be surveyed is essential to inform them about the upcoming data collection, to meet the relevant authorities, and to receive their consent and support. In some cases, it may be helpful to ask for a letter of support from the authorities to be distributed to the data collectors. In case of phone-based or internet data collections, this action can be omitted. Under certain circumstances, ethical clearance might have to be sought.

Provide the questionnaire

We need to prepare the questionnaire for data collection, either program it for use on electronic devices or print the questionnaire for paper-pencil data collection.

Programming the questionnaire for electronic data collection needs some preparation and skills. Common tools are free services like KOBO collect or ODK collect or paid services like Survey CTO. A detailed instruction for programming and saving electronic data using KOBO collect can be found on the UNHCR webpage.

Train the data collectors

The collection of data requires intensive training of the data collectors in which all supervisors take an active part. Note that this is a crucial step; it might be advisable to seek assistance from an expert for this, especially when applying the RANAS approach for the first time.

The duration of the training depends on the length of the questionnaire and the mode of data collection and lasts between 3 and 5 days. The training includes a pretest day in the field. It lasts longer when the survey instruments contain direct observations and when the questionnaire is translated jointly with the data collectors. Resource 2.15 provides an exemplary outline for training data collectors.

It may be helpful to ask the team to complete a short evaluation form and/or exam daily to detect any difficulties in understanding the training content. Use role-plays to ensure that data collectors get used to the tool and handle participants respectfully at all times. Role plays also help to develop strategies to ask difficult or sensitive questions.

In case the training cannot be realized face-to-face, it can also be provided using an online communication platform (e.g. zoom or teams). In this case, plan for more breaks and stretch the training over more time, if possible. Resource 2.16 lists some recommendations on remote/online trainings. All organizational aspects for the training are listed in Resource 2.17.

Pretest the survey instruments

The training ends with a pretest day in the field or remote in case of phone- or internet-based data collection. It is conducted with participants which are not part of the actual baseline sample but which share the key characteristics of the study participants (e.g. their situation is also rural). The pretest day has two goals. First, it is an important exercise for the data collectors. Second, we can test the survey instruments: the questionnaire, the spot checks, and the direct observation manual. We can verify whether the interview partners understand all the questions, whether all questions are answerable, and whether the questions are correctly and completely understood by the population. We can also check whether the spot checks and the direct observation manual are applicable and correspond to the situation in the field. Feedback from the data collectors is essential.

Revise the survey instruments

In nearly all cases, the survey instruments have to be revised after the pretest day. Plan at least one or two days to update the questionnaire, including the observation protocols. Bear in mind that when you change questions, the new formulations have to be translated as well, then programmed and we need to make sure the interviewers work with the latest version of the questionnaire.

Conduct the data collection

During data collection, it is essential that the data collectors are accompanied every day by one or, depending on the team size, several local supervisors. The tasks of the supervisors are outlined in Resource 2.18. Instructions for the supervisors during data collection. If data collectors are not supervised, data quality may suffer; survey instruments may be (1) incorrectly used due to misunderstandings, (2) left incomplete due to an error, resulting in missing data, or (3) falsely completed due to cheating. Only through adequate supervision can we guarantee to collect data of high quality.

Outputs

Survey data from a sample of the target population group.

Phase 3: RanasANALYSE

Conduct doer/non-doer analysis

In Phase 2: RanasMEASURE, we discussed the development of a quantitative questionnaire and observation tools based on the findings of Phase 1: RanasEXPLORE. In a second step, how to conduct the data collection.

In this Phase 3: RanasANALYSE, we will learn how to process the obtained data from the baseline survey and how to determine those behavioural factors that steer the target behaviour. Based on this information, the according behaviour change techniques will be selected and the campaign designed in Phase 4: RanasDESIGN.

The steps of this phase are:

3.1: Prepare the dataset for analysis

3.2: Conduct the doer/non-doer analysis

Step 3.1: Prepare the dataset for analysis

For preparing the dataset, the data gathered in the survey is entered into a data file, cleaned, and processed.

Key actions

Clean and process collected data

After data collection, the dataset needs to be prepared for data analysis. In case of electronic data collection, the dataset can be downloaded and imported into Excel. When all data is imported, we have to process it. Cleaning the data is the first step. Missing or erroneous values need to be identified and corrected, as can be seen in Resource 3.1. To further process the data, we need to identify the behaviour-question which can be used to determine doers and non-doers, code open answers into categories and recode multiple answer options into yes/no answers to each answer option, like presented in Resource 3.2. Finally, data entry needs to be done by hand in case data is collected by paper-pencil. In an excel table, each row represents one participant, each column is one question, and each cell thus equals the response of one person to one question, Resource 3.3.

Delete personal information

In order to keep the interview data anonymous, we need to delete any personal data from the data file. If you have not done so already, assign every respondent one ID number. Then copy the ID number together with any personal information on the individual (name, phone number, nicknames) to an extra file. Delete the personal information from the datafile. Only selected individuals should have access to the file containing ID number and personal information.

Combine responses for data analysis

Sometimes, it is necessary to combine the responses to some questions or to some question-parts before analysing the data, thus building scales.

• Calculating the mean value of the responses to all questions measuring self-reported handwashing at different key times for each participant gives a self-reported handwashing score.
• To sum the responses to the questions on health knowledge for each participant creates a health knowledge score.

Outputs

Dataset ready for analysis, anonymised and cleaned.  Please do this in the same format as the other outputs per step.

Step 3.2: Conduct the doer/non-doer analysis

After having prepared the dataset, we conduct a doer/non-doer analysis. A doer/non-doer analysis compares the responses of people who do a behaviour (doers) to the responses of those who do not (non-doers). A large difference between doers and non-doers in responses to a question about a behavioural factor indicates that this factor is critical, meaning that it steers the target behaviour. Small or no differences mean that these factors are not relevant for the behaviour in question, thus no resources need to be spent to address them. A doer/non-doer analysis involves three steps. First, the sample is divided into doers and non-doers. Second, mean scores are calculated separately for doers and non-doers. Third, the mean scores are compared. Following, these steps are explained in more detail.

Key actions

Select questionnaire items for the definition of doers and non-doers of the target behaviour

There are different possibilities to decide which participants classify as doers and which as non-doers. This decision can either be made based on questions related to the target behaviour (e.g., The last time you defecated; did you use a toilet, or did you defecate in the open?) or questions that are part of the observation protocol (e.g., does the toilet look used?). In case the target behaviour is not yet performed by the target audience, it is also possible to use the intention to perform the target behaviour as the item to classify doers and non-doers (i.e., intenders and non-intenders). For example: In a community, where no toilets exist and people practice open defecation, no doers exist. Therefore, it is possible to use the intention to use a toilet instead of the actual behaviour “toilet use”. Each of these mentioned options has different advantages and disadvantages in terms of social desirability, bias, variance and reliability: Resource 3.4.

Divide the sample into doers and non-doers of the target behaviour

If the behaviour was assessed by a dual response question, the distinction of doers and non-doers is simple. For example, the question: “The last time you defecated, did you use a toilet, or did you defecate in the open?” leads to one group which used a toilet (doers) and a second group which defecated in the open (non-doers). However, for most behaviours, there is no predefined value or cut-off point at which to divide the sample into doers and non-doers. Instead, a cut-off point has to be determined based on the data. For this decision, the following rules of thumb are proposed:

• The first choice is to define people as doers that perform the behaviour in 100% of the occasions. For avoiding arsenic contaminated drinking water, for example, only people classify as doers that collect 100% of their drinking water from safe water sources. People who consume less than 100% arsenic-free water are considered non-doers.
• However, for sound data analysis, none of the two groups of doers and non-doers should contain less than 30% of the sample or less than 30 cases. If classifying doers as people performing the target behaviour in 100% of the occasions leads to less than 30% doers (or less than 30 cases), we need to decide for a more meaningful cut-off point.
• For handwashing, for example, we can also decide to categorise only people who wash their hands at 100% of key events as doers, and all who wash their hands less than 100% as non-doers. However, 100% handwashing might be an unrealistic cut-off point for many populations. Therefore, a more reasonable cut-off point might be 90% handwashing prior to and after key events. In this case, people who wash hands at 90% of key events and more are doers; people who wash hands at less than 90% are non-doers.

When we have defined a cut-off point, we divide the sample into doers and non-doers. In most cases, we divide the sample into doers and non-doers based on one measure; however, it is also possible to combine several measures.

Sort the dataset according to doers and non-doers

Sort your data according to your main behavioural outcome variable (column B1, Behaviour: Chlorination, measured in % of household’s chlorinated water, see Resource 3.5). In more detail, the following steps need to be performed:

• Select the whole table; selecting your data only partially will distort your data dramatically as only some columns will get sorted and others remain (!), make sure to choose the option “expand the selection” if requested.
• Make sure you have selected the option “my data has headers” within the sort command; so that your variable names will always remain in the first row
• Now sort your data according to the item that you will use to divide doers and non-doers; this could be a binomial (yes/no) or a linear measure (how much?)

For formatting cells, using colours under “cell styles” or use the “conditional formatting” options to highlight your groups. Resource 3.5 shows visually, and in more detail, how to perform this step.

Calculate the mean scores of each behavioural factor separately for doers and non-doers

For each behavioural factor (i.e. for each question), the mean score in the responses is calculated separately for doers and non-doers. Calculation and interpretation of mean scores is quite straightforward for questions with rating scales or about factors such as age; it simply means the average of responses. For yes/no questions, the mean score equals the percentage of yes responses and should be displayed in Excel as a percentage. For open multiple-response questions, we treat every response option as a separate yes/no question; ‘yes’ means that that response was mentioned and ‘no’ means that that response was not mentioned. Resource 3.5 provides more details and visuals on how to calculate means for doers and non-doers.

Compare the mean scores between doers and non-doers to identify the behaviour-steering factors

Next, we compare the mean scores of doers and non-doers for each behavioural factor. We calculate the differences between mean scores for doers and non-doers by subtracting the means of the non-doers from the means of the doers. The critical behavioural factors are those with the largest differences between doers and non-doers. These are thus selected to be targeted with the respective BCTs. Please see Resource 3.5 for more details on how to compare the mean scores between doers and non-doers for different kinds of question formats. Resource 3.6 demonstrates how to use the comparison results for interpretation and campaign planning.

Analyse contextual factors

We can analyse the data that we collected about contextual factors in the same way as we analyse the data about the behavioural factors. For the interpretation of the results, however, we have to distinguish two types of contextual factors. The first type of contextual factors is those that can be changed by a behaviour change campaign, such as the availability of chlorine for water disinfection or tippy taps for handwashing (physical context) or the social cohesion within a village or neighbourhood (social context). If we find a meaningful difference between doers and non-doers for such a contextual factor, we will select a BCT to change it in the subsequent RanasDESIGN phase. The second type of contextual factors are those that cannot be changed by a behaviour change campaign, such as the age of a respondent, the household income, or the level of education (all three are personal context factors). If we find differences between doers and non-doers in such a contextual factor this often means that we have identified a vulnerable sub-group of our target group. For example, if we find that those households that chlorinate their drinking water have a higher income, this also means that low-income households chlorinate their drinking water less. Consequently, low-income households are currently more at risk of contracting a communicable disease and require specific attention when designing the behaviour change campaign. In such a case, we recommend conducting a second doer non-doer analyses which includes only individuals from that vulnerable sub-group. For example, if we have found a relevant difference in household income between doers and non-doers in our first analysis, we should now conduct a second analysis and compare doers and non-doers on the behavioural factors in this sub-group that only includes low-income households. This allows us to identify the behavioural factors which are specifically relevant to them. By selecting BCTs according to these behavioural factors, we make sure that our campaign is tailored to the mindset of this especially vulnerable group.

Outputs

The behavioural factors steering the target behaviour are determined. These are the factors that we want to tackle with our campaign.

Phase 4: RanasDESIGN: 

Develop the behaviour change campaign

In Phase 3: RanasANALYSE, we identified a) relevant behavioural factors that steer our target behaviour, b) influential context factors, and c) communication channels favoured by our target population.

In Phase 4: RanasDESIGN, we will choose Behaviour Change Techniques (BCTs) from the RANAS catalogue of BCTs to target the identified behavioural factors. Combining them with insights from our data about preferred and trusted communication channels, we will develop a behaviour change campaign strategy. It is important to include our implementing partners in this process to assure activities are acceptable, practicable, and adapted to the local context. Subsequently, we will develop campaign instructions offering concrete, step-by-step guidance for implementing the campaign. These campaign instructions also serve as a tool to monitor the implementation. Finally, campaign materials (e.g., leaflets, poster) are necessary to be professionally designed and all activities have to be pre-tested. As a result, we have a fully prepared behaviour change campaign ready for implementation.

The steps of this phase are:

4.1: Develop a campaign strategy

4.2: Develop campaign instructions and materials

Step 4.1: Develop a campaign strategy

The campaign strategy is an overview table linking each relevant behavioural factor with the selected BCT and communication channel, and briefly describes campaign activities. The campaign strategy is the heart of the future campaign containing all its elements in a nutshell. Therefore, it should be developed jointly with or at least get feedback from the implementing partners.

Key actions

Pre-select communication channels

As a first key action to develop a behaviour change campaign, we recommend reflecting on different ways of how your organisation could deliver the campaign. We call these different ways of delivering information “communication channels”. Communication channels can be mass media or interpersonal communications. Interpersonal channels are household visits or group meetings, mass media ranges from leaflets, wall paintings or stickers to radio spots, parades or social media channels. A more detailed overview can be found in Resource 4.1. See Resource 4.2 for more specific forms of using mass media, for example, mass media role modelling or behavioural journalism.

Interpersonal communications often allow to best respond to the needs of the participants. Many studies have shown that interpersonal channels are more effective than mass media, but a larger number of people can be reached with mass media. Decide on whether you want to use mass media or interpersonal communication channels in your campaign. Of course, you can also combine both.

An important criterion for selecting communication channels is the preference of your target audience. Based on the information from Phase 3: RanasANALYSE and your organisation’s preferences you can now pre-select communication channels fitting to the context of your target audience. We recommend to use the following guiding questions in addition to the results of the data analysis:

• What worked well in your previous experience? Reflect which communication channels you used previously and which ones were well accepted by your specific target audience.
• Which communication channel fits to the topic? Take into account that certain communication channels might be appropriate for one topic (e.g., sharing information about COVID-19 through public posters) but not for another topic (e.g., very sensitive topics like menstrual hygiene management might rather be discussed one-by-one). Also think of gender sensitivity and inclusion (e.g., Is everyone able to read or come to a community meeting?) while reflecting on communication channels.
• Which communication channels are practicable? Evaluate which communication channels fit to your resources. One organization may be capable and used to conducting household visits whereas another organization has expertise in social media campaigns.
• Which communication channels have the best reach? In some communities, for example, street theatre may work very well, whereas in other places no one will show up.

Select BCTs that correspond to the behavioural factors that steer the behaviour

Behaviour change techniques (BCTs) are the mechanisms of action of a campaign, designed to change the behavioural factors steering behaviour. Behaviour change techniques (BCTs) are the smallest component of a campaign. In behaviour change practice, BCTs and communication channels are frequently confounded; it is important to remember that communication channels are only the mode of delivery of BCTs. The BCTs are the content transferred through the communication channels.

Each BCT affects a specific behavioural factor: for example, BCT 13 Prompt public pledging addresses the behavioural factor Other’s behaviour. See a list of all BCTs in the Methods fact sheet about the RANAS behaviour change techniques (Resource 4.3). Each BCT is described in more detail in the catalogue of RANAS behaviour change techniques (Resource 4.4). Several of our BCTs also work as nudges, i.e. they use the same mechanism as nudges to change behaviour (a more detailed explanation can be found in Resource 4.5).

The BCTs we will select for our behaviour change campaign have to target those behavioural factors that were found to steer our target behaviour. Therefore, of the relevant behavioural factors identified in Phase 3: RanasANALYSE, we recommend selecting those five behavioural factors for the campaign development that show the biggest differences between Doers and Non-doers. The aim is not to overwhelm participants with too many BCTs but rather focus on the most important behavioural factors during the campaign.

With the list of the five most influential behavioural factors, you can now select one BCT for each factor using the Methods fact sheet (Resource 4.3) or the detailed catalogue (Resource 4.4). If the Doer and Non-doer analysis, for example, revealed Factual knowledge as an important factor, then BCT 1 Present facts or BCT 2 Present scenarios could be selected. If more than one BCT is suitable to change a behavioural factor, then decide which BCT you would like to use considering your target group, their cultural specifics and existing rules or taboos.

After this step, you should have the first two columns of the behaviour change strategy: the five most relevant behavioural factors and one BCT for each of them. For those not so familiar with the terminology of the behavioural factors, it is helpful to add two more columns: 1) containing the questionnaire item and 2) containing the meaning of the result (e.g., for the behavioural factor Feeling: people who always wash their hands feel prouder than people who wash their hands less frequently).

Match BCTs and communication channels

The next step is to match the BCTs with communication channels, see Resource 4.6 for an example. By that, we define how the BCTs are being communicated to the target group. You now need to make a final decision on the communication channels.

There are some recommendations when matching BCTs with communication channels:

• Check how the BCTs fit in with the communication channels you have pre-selected – which channel is best suited to communicate a certain BCT?
• Several BCTs could be delivered through the same communication channel, for example, a radio spot can convey several messages that relate to different BCTs.
• The same communication channel could be used several times, for example, plan for several household visits. This is helpful for following up on more complex BCTs or for a more intensified campaign.
• BCTs can also be repeated in different communication channels, for example, BCT 9 Inform about other’s (dis)approval, targeting Other’s (dis)approval, could be delivered through a poster at the community wall, through a radio spot and a community meeting.
• The selection of BCTs is a creative process allowing to think in different directions, for example, you may try out new BCTs or use well-known ones with new communication channels.

See Resource 4.7 for recommendations which BCTs are best used through interpersonal communication channels.

Create behaviour change activities

The BCTs from the previous key action now need to be unpacked into behaviour change activities that fit the selected communication channel. By activities, we mean the specific actions to engage with the target audience. For example, a health promoter and a participant, together with other household members, could jointly develop a detailed action plan on when, where, and how to segregate waste (BCT 34 Prompt specific planning; communication channel: household visit). For some BCTs it is advisable to use additional materials like posters or pre-designed sheets where people can fill in certain information (e.g., an action planning sheet). Take note of these additional materials (that you may already have or need to develop) in your campaign strategy. Other BCTs you will rather translate into specific messages. Here we recommend getting yourself familiar with persuasive communication. A short description can be found in Resource 4.8.

Adding short descriptions of activities, materials needed and messages completes the campaign strategy table, see Resource 4.9 for an example of a campaign strategy. As you can see, the table now groups all BCTs using the same communication channel together.

Present campaign strategy to stakeholders

To ensure that the campaign strategy and its activities are contextualised and meet the needs of the participants, we recommend discussing it intensively with different stakeholders. This group of stakeholders should include at least the implementing partners or experienced promoters. Their feedback helps to evaluate practicability and ideally creates ownership for the campaign. Additionally, representatives of the participating communities should be invited to give their feedback on the planned campaign activities. Voices from community leaders and community members are very valuable to ensure acceptance and potentially increase the impact of the campaign. Only after receiving feedback, the campaign strategy can be finalised.

Outputs

Campaign strategy

Step 4.2: Develop campaign instructions and materials

The campaign instructions are a detailed guideline to implement the behaviour change activities of the campaign. The main users of the instructions are those who implement certain campaign elements (e.g., promoters visiting households, organisers of a community event), however, they can also be used by trainers (for the training) and supervisors (for monitoring) alike. Materials are additional items we need for the campaign like posters, sheets, stickers etc.

Key actions

Develop campaign instructions

The campaign strategy developed in the previous step contains all relevant elements for the behaviour change campaign. We now have to develop the exact content and sequence of the campaign activities by developing step-by-step campaign instructions – a script about how the activities should be implemented.

For example, we have decided in the campaign strategy that we want to implement BCT 13 Prompt public pledging during a household visit and use a sticker as sign of a household’s pledge to separate their waste – the campaign instructions describe step-by-step what the promoter has to say, when to give the sticker and how to guide the household on the decision where to put it. To give another example, we may have decided to implement BCT 9 Describe feelings about performing and about consequences of the behaviour through a mindfulness exercise – the instructions will contain the details about how the promoter guides this exercise.

The campaign instructions have several purposes: The first purpose is to provide clear and concise guidance for the implementers of the campaign. This is important so that all team members carry out activities in the same way. The second purpose is to define the parameters for monitoring the campaign implementation. The campaign instructions will serve as checklist for both, the promoters and their supervisors.

When developing the campaign instructions, we need to make sure that everything we write is based in the BCTs and corresponding activities that we have outlined in our campaign strategy. A common mistake is to fall back into old habits of simple risk communication or awareness raising. This can be avoided by a) noting down again the behavioural factors and corresponding BCTs that are the basis of each activity and b) by frequently checking back the descriptions of the BCTs in the Methods fact sheet and the catalogue (Resource 4.3, Resource 4.4). Resource 4.10 provides an example for a campaign instruction that you can adapt.

Design campaign materials

Some behaviour change activities will require the use of materials, such as a pre-recorded radio spots or a visualisation of the target behaviour’s benefits on a flyer or a poster. To come back to our previous example of BCT 13 Prompt public pledging, we need a sticker as the public sign of a household’s pledge that will be put publicly visible outside the house (e.g., on the door).

For producing audio or video clips, detailed scripts have to be prepared. Or if a live radio message is planned, the speaker needs to have a script about what to say according to the selected BCTs. So, no matter which materials we use, we need to make sure our materials are in line with the relevant BCTs.

Furthermore, it is important to tailor the materials to the local context and culture. For example, in one context, a sticker at the door of a house may be a good commitment sign, whereas in another context, a flag on the roof of a house may be better.

We can adapt and create those materials ourselves, or we partner with a creative agency who designs and produces the materials. Usually, the design is more appealing when involving professionals. Also, we recommend working with a local creative partner – they usually know better how to attract the attention of the target group and about cultural go’s and no-go’s. To save resources, check whether you can revise and adapt materials that have been developed for previous campaigns, see Resource 4.11 on how to integrate existing activities and materials.

A common challenge when working with creative agencies is to make sure that the materials they produce are in line with the campaign strategy and BCTs. We recommend informing the agency in detail about the RANAS steps that have led to the campaign strategy, including the results of the doer/non-doer analysis. We also suggest to develop a first draft defining the content of a material and then to discuss in detail with the agency. Resource 4.12 provides some examples of past campaign materials.

Pilot campaign

Once we have a first version of the campaign instructions and required materials, it is very important to carry out a pilot. Piloting means to conduct the campaign in a small sample of 5 to 10 participants that are not part of our target communities but should have similar characteristics (e.g., if we plan to do a campaign for farmers to fertilise trees, we should do the pilot with farmers who are somewhat familiar with the topic).

We should focus the pilot on two guiding perspectives:

1. Are the activities and materials feasible for the implementing staff?
2. Are the activities and materials acceptable to the target population?

These two perspectives include aspects like: does the sequence of several activities make sense (e.g., during a group session), is the time it takes to carry out the activit(ies) appropriate, are participants getting bored or getting lost or getting irritated about something, are the materials easy to understand for the participants, are there any redundancies in the instructions.

If we can answer these questions for all activities and materials, for implementing staff and target population with “Yes”, we are ready for the next phase, Phase 5: RanasIMPLEMENT. Usually, some adaptations are necessary after the pre-test. Pay attention to whether a certain communication channel or BCT is generally not practicable or not acceptable. Then we need to go one step back to the campaign strategy. Sometimes, several iterations of piloting and revisions may be necessary.

The pilot should be conducted by members of the team who have already been involved in the previous RANAS phases and who will continue to work for the project during the implementation. We do not recommend working with external promoters for the piloting. Also, it is helpful to work in pairs when piloting trainings, meetings or household visits. One person can carry out the activity and the other observes, takes notes and may also help with the instructions if necessary. Feedback form Resource 4.13 is a useful extension of the campaign instructions to take notes.

Before starting the pilot, inform participants that they are part of a pilot and get their informed consent to participate and provide feedback afterwards. Feedback from participants can be gathered using short qualitative interviews or focus group discussions (see Resource 1.7 and Resource 1.8 from Phase 1: RanasEXPLORE for instructions).

Examples for guiding questions:

• What did you like/dislike?
• Which activities were easy/difficult for you?
• What could be improved?
• Do you have any general feedback/comments?

Add the key findings from the interviews or focus group discussion to the corresponding activity and step in the feedback form (Resource 4.13) so it contains all the information that you have gathered during the pilot to make the required revisions of the campaign instructions.

Outputs

Campaign instructions with detailed step-by-step guidance for implementers

Campaign materials

Phase 5: RanasIMPLEMENT

Realise the behaviour change campaign

So far, we have seen how to prepare a behaviour change project (Phase 1: RanasEXPLORE) and how to analyse and understand behaviour. We have carried out an in-depth qualitative and quantitative analysis of the situation, the context, the behaviour(s) in question, and their influencing behavioural factors (Phase 2: RanasMEASURE). Based on these insights and by comparing doers with non-doers of our target behaviour (Phase 3: RanasANALYSE), we chose BCTs to change the most influential behavioural factors. We developed a campaign strategy, wrote instructions for its implementation, and designed supporting promotional materials (Phase 4: RanasDESIGN).

In this chapter, we will look at what it needs to prepare the implementation of the campaign, what to consider during implementation and how to monitor implementation.

The steps of this phase are:

5.1: Prepare to implement campaign

5.2: Implement and monitor campaign

Step 5.1: Prepare to implement campaign

In this step, we prepare the campaign implementation. We decide which community or group will receive which activities, create an implementation schedule, train personnel, and establish a monitoring system.

Key actions

Assign the behaviour change activities to project communities or project groups

First, we need to decide which project communities or groups receive which behaviour change activities. To do so, we refer to the results of Step 1.1: Define project design.

If you chose a phased implementation with before-after measurement and independent comparison group, you need to decide in which group you will roll out the campaign first and which group will be your comparison group. These two groups should show similar behaviours to begin with, so that the evaluation results are not confounded by differences that existed already before the campaign. To avoid spill-over effects, the campaign and comparison group should not interact with each other. This can be ensured by choosing communities that are geographically far enough away from each other to avoid regular communication and family bonds. Ensure that of the campaign group, all participants of the baseline survey receive the campaign.

If you chose a regular implementation with before-after measurement and a natural comparison group or without a comparison group, you can roll out your campaign with the entire target population. The evaluation will have to be adapted accordingly, see Phase 6: RanasEVALUATE.

Whether you have an independent comparison group or a natural one, you may decide to compare specific activities with each other instead of comparing people who have received the overall campaign with those who have not. This allows you to gain more specific insights into the effectiveness of single activities. However, you need to decide which area, group or community receives which activity.

Select promoters

Promoters are the people who will implement community meetings or household visits, for example. We suggest separating promoters from interviewers to reduce social desirability bias. This can be achieved by using different teams for interviews and promotions or by rotating roles between areas or interviewees. Also, choose promoters who have a profound knowledge about and familiarity with the target area. If promoters are already active in the region, clarify how to integrate the new activities into the existing ones and what differentiates this campaign from previous ones. Similar criteria as described in Resource 2.14 on the selection of data collectors apply for selecting promoters.

Train promoters

No matter whether you work with experienced promoters or external ones, a training on the new activities is a must. Resource 5.1 suggsuggests a content outline for a promoter training. A promoter training should last between one and two days and should include role-playing with feedback and a pre-test with persons outside of the project, as well as discussions with and participation of all team members. Ensure that all promoters will implement the activities according to the campaign instructions. Before finalizing the training, verify that promoters are able to independently implement the campaign activities. The same ethical principles as for interviews, see Resource 1.11 apply to the campaign implementation and promotors need to be trained accordingly. As part of the preparations, every promoter receives printed campaign instructions and knows why it is important to check off completed steps during implementation.

For preparing live delivery of mass media communications, the responsible persons do not need a full promoter training but should have knowledge about behaviour change and the planned campaign.

Prepare a detailed schedule

A detailed schedule will help to understand the time needed for rolling out the campaign, necessary resources and who might need to be contacted. Resource 5.2 shows an exemplary schedule, which details who does what, on which days, where, and with which resources. The whole implementation team should decide these details together and make sure that all members are aware of their responsibilities, are committed to realizing them and have the time and resources to do so. Find out the appropriate times for household visits or community meetings to ensure that the target group is available. Integrate participating communities or their leaders into the detailed planning of the implementation, they have important insights and experiences and will help you notify community members.

Prepare monitoring

Thorough monitoring of the campaign implementation is very important because even the best campaign will not be effective if not implemented correctly. Especially for experienced promotors it may be difficult to apply new kinds of strategies. There is always a danger to revert to promoting simply risk awareness or conveying knowledge, even if analyses have shown repeatedly that this kind of strategy is not effective. Thorough monitoring can prevent this reversion and ensure that the campaign is implemented in all its planned aspects. Out of the group of trained promoters, a number of supervisors can be chosen who partake in implementing the campaign, but also record monitoring data with the help of the campaign instructions. Good supervisors stand out from the group through leadership skills, natural authority, a good understanding of the issues at hand and the planned campaign and/or outstanding social skills.

If you choose to collect monitoring data, use the campaign instructions for recording. A simple yes/no check for each step offers quantitative insights into campaign quality and promoter adherence. If you want to analyse campaign implementation quantitatively, it is advisable to use a digitalized monitoring tool instead of the paper instructions. You can simply program each step into a KoboToolbox survey.

Outputs

Campaign implementers who understand the campaign and their exact tasks

If applicable: Digitalised monitoring questionnaire

Step 5.2: Implement and monitor campaign

Step 5.2 is all about implementation and monitoring of the planned behaviour change campaign – the goal of the project and the integration of all knowledge we have gathered so far. In the following, details are given about how to roll out the planned activities and how to systematise analysis and feedback.

Key actions

Community participation and feedback

Before starting to work in a community, leaders and/or officials of the area should have been contacted to ensure a good introduction into the community. Ideally, the community has already been participating in the whole process leading up to this point. Additional to participation in decision making, there should be a feedback system in place that ensures that participants of the campaign can be heard and the campaign can be adapted to possible problems. Depending on the local context, this can take the form of a sort of letter box with paper provided and a pen attached or contact details like an email-address and telephone number of a responsible person specifically for complaints. The feedback system should be explained to the community. Determine beforehand how to integrate such feedback and criticism into the existing project.

Implement campaign

To implement the campaign, we need to roll out the planned activities according to the campaign strategy and instructions. Appointments with the participants may be necessary. Team leaders should be assigned to groups of promoters. For a successful implementation, regular meetings between groups of promoters and team leaders are necessary (e.g., at the end of each day) to organise logistics, give feedback, exchange of experiences, and ensure early solution of problems.

Document campaign implementation

Capture photos or videos of each promotion activity. These can be used in reports and project advertisements as well as to identify implementation errors. When taking pictures or videos, permission to publish needs to be given by all visible participants (community members and promoters).

Conduct monitoring visits

You can either choose independent monitoring supervisors or work with the team leaders to fulfil this role. In either case, supervisors accompany the promoters or implementers unannounced and in a randomised way to observe campaign implementation. Supervisors check whether each step and activity of the campaign instructions is realised as it was planned. They continuously and systematically gain insight into the correct implementation of the ongoing campaign but also difficulties, which allows for adjustments. Monitoring can fulfil two purposes: (1) monitoring data can be collected and analysed, see next key action “Quantitative analysis of monitoring data”, and (2) observations of promoters can be used to give feedback, see key action “Feedback”.

Feedback

Feedback to promoters and campaign implementers can be given in two ways: instant feedback to individual promoters and regular debriefings with the team. Instant feedback to individual promoters can be realised directly after a supervision visit. The supervisor talks about steps that have not been implemented correctly, potential improvements and what aspects were successful. Regular debriefing with the group can take the form of an open discussion. Implementers share their experiences and difficulties, other group members may have ideas for possible solutions. In case monitoring data has been collected systematically, feedback can be given using the data. For example, what are the reasons for steps that have frequently been left out. Maybe those steps are not well understood, are perceived as inappropriate, participants frequently don’t understand the meaning of the step etc. Based on the insights, promoters can be trained more in-depth, or instructions can be altered. Independently of the form it takes, feedback allows for continuous improvement of the campaign implementation.

Quantitative analysis of monitoring data (optional)

To avoid errors when transferring data from paper to data sheets (e.g. Excel), use a digital system (e.g. KoboToolbox) to collect data with the help of the digitalised instruction items. This will directly produce a data sheet. Alternatively, if this is not possible, have two people work together on the transfer of written to digital data (see Resource 3.3 on how to enter data collected by paper-pencil). Basic percentages of checks (indicating proper implementation) for each step in the campaign instructions show how well each activity is implemented. The higher this percentage, the better the implementation quality. Regular revision of the collected monitoring data is important to detect campaign problems early on and can also be used to give feedback to promoters (see the previous key action “Feedback”).

Outputs

Behaviour change campaign implemented as planned

Data about the quality of the campaign implementation

Phase 6: RanasEVALUATE

Quantify change in behaviour and behavioural factors

In the previous phases, we have seen how to set the parameters of the project (Phase 1: RanasEXPLORE), how to analyse the target population’s behaviour, context and behaviour-influencing factors (Phase 2: RanasMEASURE and Phase 3: RanasANALYSE). The results of the explorative phase and the baseline survey have been used to choose behaviour change techniques and these build the basis for a campaign strategy (Phase 4: RanasDESIGN). With the help of the campaign strategy and intervention checklists, the campaign has been implemented and monitored (Phase 5: RanasIMPLEMENT).

After having implemented the developed behaviour change campaign, most projects evaluate whether the targeted behavioural factors and behaviours have changed as anticipated. In some projects, a before and after measurement is not possible, in which case this phase does not fully apply. To measure long-term effects, the follow-up survey should be conducted 6, 12, 18, or even 24 months after campaign implementation. For this evaluation, a follow-up questionnaire has to be prepared (Step 6.1: Prepare follow-up questionnaire), the follow-up survey(s) have to be implemented (Step 6.2: Conduct follow-up survey(s)) and the data has to be used to quantify the change that has taken place (Step 6.3: Quantify change). Finally, the findings of the evaluation are used to improve the existing campaign, scale it up to a bigger target population and / or to plan future behaviour change campaigns (Step 6.4: Integrate findings). Evaluation is important for accountability and learning because it examines the achieved outcomes, the efficiency and the wider impact on people’s lives and allows to find recommendations to improve the project itself and to improve future policy and practices.

The steps of this phase are:

6.1: Develop follow-up questionnaire
6.2: Conduct follow-up survey(s)
6.3: Quantify change
6.4: Integrate findings

Step 6.1: Develop follow-up questionnaire

In order to evaluate the wider impact of the campaign, as well as the outcomes in terms of changes in behaviour and behavioural factors, first a follow-up or endline questionnaire needs to be developed. Questionnaire development depends on the planned project design and respective evaluation design. Based on those designs and the baseline questionnaire, the actual creation of the questionnaire needs to take place.

Key actions

Review project design

During Phase 1 (RanasEXPLORE, see Step 1.1) you have defined your project design. Review it now and check if it still fulfils your needs. If you did not plan to or were not able to establish an independent comparison group that was not reached by your campaign, you now have the chance to add one just for the follow-up data collection. Of course, you will not have baseline data from this group, but you could still compare the follow-up data between this comparison group and the campaign group. This is also useful, in case you wanted to work with a natural comparison group, but you expect that all participants were reached by the campaign, so no participants for the natural comparison group would remain.

Creating the follow-up questionnaire(s)

A follow-up questionnaire consists of the same observations of and questions about the relevant behaviour(s) and behavioural factors as the baseline (see Phase 2, RanasMEASURE, for how to create a baseline survey). In addition, the follow-up questionnaire contains a separate part called “campaign check”: To evaluate a campaign, it is important to know how many members of the target group received which parts of the campaign and how it was perceived. This is particularly important in case there is no comparison group. For example, some people participated in community meetings while others only received household visits, and some may have had both. To find out who has participated in which campaign activities, we measure participants campaign recall and campaign recognition. Campaign recall is the participant’s memory of different activities that they have been a part of. Campaign recognition is whether they remember specific messages and content. To improve participants’ recall, it is useful to show some materials used during the campaign implementation. Some generic campaign check items can be found in Resource 6.1: Examples for campaign check items.

Outputs

A developed and programmed or formatted follow-up questionnaire ready to be used.

Step 6.2: Conduct follow-up survey(s)

Conducting follow-up survey(s) follows the same structure as conducting the baseline, as was introduced in Phase 2 (RanasMEASURE). Thus, in the following, you will find the same key actions listed and references to resources from Phase 2 (RanasMEASURE), completed by recommendations for this present step. For details, please refer back to Phase 2 (RanasMEASURE).

Key actions

Define the sample size and the sample selection procedure

Your sample should be the same as for the baseline. However, if your project design did not include a comparison group, or in case you experienced a lot of drop-out, you might want to include an additional sample which can serve for comparison.

Schedule the data collection, define the number of data collectors to be employed and supervisors to be appointed

Use Resource 2.13 to determine the number of data collectors needed and also choose one or several supervisors.

Employ data collectors and organize the data collection

Data collectors should not be the same person as the implementers of the campaign(or promoters) to avoid social desirability effects. Thus, if there is only one team available, a cross pattern can be planned where those persons from the target population who came into contact with a certain campaign implementer, are being interviewed by a different person. Resource 2.14 provides information on the general requirements for data collectors. For data collection, community entry, necessary permits, transport and logistics need to be planned, just as for the baseline. See Phase 2 (RanasMEASURE) for details.

Provision of the questionnaire and train the data collectors

Provide your evaluation questionnaire, just as you did with your baseline questionnaire in Phase 2 (RanasMEASURE). Phase 2 provides the necessary resources for training the data collectors. Resource 2.15 provides an exemplary outline for training data collectors. Resource 2.16 lists some recommendations on remote/online trainings. All organizational aspects for the training are listed in Resource 2.17.

Pretest and revise the survey instruments

Pre-test the evaluation questionnaire, along the lines of what you did for the baseline questionnaire during Phase 2 (RanasMEASURE). Revise the evaluation questionnaire in accordance with your findings from pre-testing. Make sure the data collectors download and use the latest version of the questionnaire.

Conduct the data collection

Ideally, the same persons who were interviewed for the baseline should be interviewed during follow-up, using the follow-up questionnaire(s) developed. Household or identification numbers, photos of the households, mobile phone numbers and even GPS coordinates collected during the baseline survey are helpful to identify the same participants. Data collectors should be accompanied by supervisors. Their tasks are outlined in Resource 2.18. Make sure to monitor data quality and completeness, especially during the first few days of data collection. If you find mistakes, alert supervisors and data collectors, so that they can be avoided going forward.

Outputs

Data sheet from collected follow-up survey.

Step 6.3: Quantify change

In order to quantify the effectiveness of the campaign to change behaviour and the behavioural factors, the collected data needs to be entered, cleaned and processed. Then, mean scores are being calculated for the different groups and times. These mean scores are then compared, and the results interpreted.

Key actions

Enter, clean and process the data

First, data sheets from the follow-up survey are downloaded, entered, cleaned and processed (similar to what was done in Phase 3, RanasANALYSE). If you have chosen an evaluation design with an independent comparison group, your data should now look like Figure 1 in Resource 6.2. If you are working with a natural comparison group, you need to first compute the campaign dose for each participant. The campaign dose is the intensity in which each participant has experienced the campaign. The number of campaign activities that the participant remembered is one way of computing the campaign dose. Once computed, split your follow-up sample in two groups: Those participants with a low campaign dose are your comparison group and those participants with a high campaign dose are your campaign group.

Calculate mean scores at baseline and at follow-up separately for the comparison and the campaign group

In the follow-up data set, that we have prepared in the previous step, we now separate the comparison group and the campaign group. For each of the groups, we calculate the mean scores of the behaviour and of each behavioural factor. We do that irrespective of our evaluation design. Your follow-up data set should now look like Figure 2 in Resource 6.2. Repeat the same process for the baseline data. Your baseline data set should now look like Figure 3 in Resource 6.2.  If your evaluation design neither includes an independent nor a natural comparison group, you need to compute the mean scores for your entire follow-up sample. Computation in the baseline data set in addition to those that you already did in Phase 3 RanasANALYSE are not needed.

Calculate change scores from baseline to follow-up separately for the comparison and the campaign group

To assess whether and by how much the behaviour and the behavioural factors changed from baseline to follow-up, we calculate the change scores. Change scores are simply the mean scores at baseline subtracted from the mean scores at follow-up. Again, we do this separately for comparison and campaign group. The bigger the change score, the bigger were the changes in behaviour(s) and behavioural factors. Your calculation should now look like lines 1 to 11 in Figure 4 in Resource 6.2.  If your evaluation design neither includes an independent nor a natural comparison group, compute the change score for your entire sample. A positive change score shows that the behaviour and the behavioural factors have increased between baseline and follow-up.

Compare change scores between comparison and campaign group

We now subtract the change scores of the comparison group from the change scores in the campaign group. This yields the difference in change scores, the so-called difference in difference. Your calculation should now look like Figure 4 in Resource 6.2. If your evaluation design neither includes an independent nor a natural comparison group, you need to skip this key action and rely only on the change score computed in the previous key action.

Interpret results

Generally, a bigger change in behaviour in the campaign group than the comparison group shows that the campaign worked. Also inspect the difference in change scores for the behavioural factors to see whether the behavioural factors that we targeted in our campaign changed as we wanted.

If your project design includes a before-after measurement and an independent comparison group, you can directly attribute the changes that you measured to your campaign: Your campaign caused the changes that you measured. Findings from a project design that included a natural comparison group, are interpreted similarly. If your evaluation design neither includes an independent nor a natural comparison group, changes in behaviour and behavioural factors also suggest that your campaign was effective; however, you cannot disentangle the actual effects of your campaign from seasonal changes or from other activities that might have happened during the same time.

Outputs

Information about the effectiveness of the campaign.

Step 6.4: Integrate findings

Integrating the findings is an important part of the evaluation process. It allows you to improve the campaign and scale it in a way that is evidence-based and data-driven.

Key actions

Improving campaigns

To improve the campaign, we need to check whether the behavioural factors that we intended to target through our BCTs really changed. For example, if our campaign included BCT 9 Inform about other’s dis/approval which is targeting the factor Other’s (dis)approval, we need to check whether this factor really changed. If it did, this suggests that the BCT and corresponding activity was effective. If it did not, we should consult our monitoring data and check whether the activity was implemented correctly and, if not, identify the reasons and tackle them, e.g. by retraining the promoters. However, if implementation was correct, this suggests that we need to revise the design of the activity or conduct another pilot to identify why the activity did not work.

Upscaling

When we see that a campaign was successful, we can scale up the campaign to other members of the target group or to other similar areas, all backed up by the evidence that the campaign is effective in changing behaviour. However, we must keep in mind that the campaign was designed and proved effective for a specific target group and context and may lose its effectiveness if applied to different contexts. When upscaling the campaign in new contexts, areas or target audiences, we therefore recommend conducting a short baseline survey to verify the selected behavioural factors and BCTs. If this yields major differences in relevant behavioural factors, the campaign needs to be adapted accordingly.

Outputs

Improved campaign.

Upscaled campaign.