Reflecting on our RCT: How measuring our program’s impact made it even stronger
“Transformative,” “meaningful,” “educational,” “enriching,” “challenging”… These are just a few of the words PJL team members used to describe their work on the cluster Randomized Controlled Trial (cRCT) we conducted over the past two years.
It's been a massive effort. We wanted to understand whether our comprehensive sexuality education (CSE) program — especially when implemented in some of Madagascar's most rural schools — can foster healthier, safer relationships among Malagasy adolescents. We worked with over 50 research assistants, who engaged more than 8,000 participants in the study across the country.
As we approach the finish line, we hosted a workshop last month in Antsirabe with the PJL research and evaluation team, along with our collaborators Céline Mazars (Social Development Direct) and Nancy Perrin (Johns Hopkins University), to reflect on our experiences of the journey.
One conviction stood out from these discussions: the research was a “seed of change and improvement for PJL,” as perfectly summed up by one team member.
A catalyst for improving the way we work
Shortly after we secured funding from What Works II to carry out the cRCT, two experienced members of the Monitoring, Evaluation and Learning (MEL) team brought up a general challenge they had been facing with Laura, our Director of Program Strategy and Impact: often, during interviews, young people become emotional as they discussed tough personal issues. MEL team members didn't know how to best support them. Around the same time, PJL had just recruited a psychologist to strengthen our safeguarding and protection efforts.
“It was a real turning point, just before the research started. I thought: ‘We have the time, we have the resources, and we need to make a major change in how we do programming and evaluation. Everything clicked.” – Laura, Director of Program Strategy and Impact
We seized the cRCT opportunity to enact a suite of improvements. We strengthened our research ethics with improved informed consent procedures and trauma-informed research methods, introduced a safety and security protocol, set up a reporting hotline, created a self-care journal for research assistants, and arranged regular check-ins with our psychologist. We also overhauled our Safeguarding and Protection Policy and encouraged more open conversations about risks, challenges and how team members were feeling. Embedded in front of the mind of every PJL team member was the fundamental principle: Do No Harm.
The ethical principles we developed for research can also be applied to our everyday lives. Consent matters in the workplace too, and the way we communicate with others can be guided by Do No Harm principles. We need to choose our words carefully, just as we did when we considered every single word while drafting the research questions.” – Mihaja, Monitoring and Evaluation Systems Integration Manager
And the changes didn't stop with the Research Department! Little by little, they spread across the entire organization. The Communications Department, for instance, developed an Ethical Communications Charter, while our Technical Department now includes trauma awareness and self-care in our training of teachers delivering our program.
“The proudest moment of my career was training more than 1,500 public school teachers in trauma-informed approaches and self-care.” – Laura, Director of Program Strategy and Impact
These efforts also helped PJL join the CHS Alliance, an international network committed to improving quality and accountability in humanitarian action by promoting the development and use of ethical standards.
“Safeguarding isn't just about preventing violence. It's about avoiding harm in every possible way.” – Saraha, Psychologist and Research, Accountability and Safeguarding Lead
Of course, the research journey wasn't always easy. Unexpected situations, stress, logistical hurdles, and interpersonal challenges meant the teams constantly had to adapt and find solutions along the way. But with PJL’s strong learning mindset at the organization, these challenges also sparked innovation and integration of new levels of feedback and reporting into our processes.
"I really enjoyed being part of the research because there's always something new to learn. Sometimes we think certain activities are working well, but the research helps us see what needs to change to better reflect what's actually happening in communities. Feedback from participants is essential to helping us improve our program.” – Ravaka, Former Research Assistant, now Research Project Management Assistant
A story of trust
As our team reflected on the research process, one reflection came up again and again: its success was built on the trust developed between everyone involved. Research assistants, students interviewed, school principals, local communities, PJL teams, Johns Hopkins researchers, the What Works II team…
And that trust made a difference far beyond the research itself. It helped students feel heard and valued, better understand their rights, and know who to turn to when they needed help, even for participating students from control schools!
It also brought PJL closer to the schools involved in the research and helped build a strong network of experienced research assistants who know and share PJL's values — a real asset for future research and evaluations.
“The research helped strengthen our partnerships with schools, particularly through the workshops we organized with school principals. Our research assistants have also remained very loyal to PJL. I once bumped into a former research assistant and suggested they apply for a position I saw advertised by another institution. But they told me they would rather keep working with PJL because of our management style and the way we treat people with care and respect.” – Tahiry, Evaluation, Learning and Accountability Coordinator
But trust isn't just about relationships between people. It's also about trust in yourself. The October workshop gave PJL team members a chance to look back on what this experience had brought them individually. They shared that they developed technical skills, from trauma-informed research to team coordination and leadership, as well as essential soft skills such as emotional intelligence and setting personal limits and boundaries. All of this helped them grow in confidence and feel better equipped to take on future research projects.
“As a research assistant, I developed so many soft skills. I was really impressed by how much the training covered. It wasn't just about technical skills, but also active listening, recognizing signs of distress, avoiding assumptions and jumping to conclusions, and learning to adapt.” – Tahina, Former Research Assistant, now Communications Assistant
“This experience taught me that when you have access to a psychologist, you should make use of that support. Some of the stories are really difficult, and in the past, I used to mentally and emotionally take on too much of the load of what the students shared with me.” – Tahiry, Evaluation, Learning and Accountability Coordinator
Perhaps one of the biggest changes: Asking for help, speaking up when something is wrong, or admitting that you're facing difficulties is no longer seen as a failure or a sign of weakness. It's increasingly understood as a normal and essential part of looking after ourselves and our well-being.
“We now recognize and talk about risks much more openly. The silence has been broken, and staff members and other collaborators speak up more when they’re facing difficulties.” – Saraha, Psychologist and Research, Accountability and Safeguarding Lead
A great team effort
Teamwork was at the heart of this research and played a huge part in making it a success. Navigating uncertainty, dealing with unexpected challenges, and finding solutions together brought people closer.
“When I was a team leader, the research assistants would come to me when they needed support, and we would work things out together. Some participants wouldn't say anything, wouldn't respond or would start crying. It was really tough, especially when dealing with such difficult situations. So we would put our heads together and find ways to handle them.” – Ravaka, Former Research Assistant, now Research Project Management Assistant
“It was such a wonderful team experience! I'm really proud to have been part of this research, and I feel ready to take on new projects!” – Fandresena, Research and Special Projects Coordinator
During the workshop in October, several team members shared how nostalgic they felt looking back on this period, with such pride and appreciation.
“Sometimes I feel a little nostalgic when I think back to that time. It could be stressful, but I learned so much and had the chance to work with so many different people. I'm really happy with the results and proud of everything we accomplished together.” – Sarobidy, Former Research Assistant Team Leader, now Data Processing and Analysis Assistant
A huge thank you to Céline and Nancy for traveling to Antsirabe to support this reflection workshop.
And thank you to What Works II for the opportunity to lead meaningful research throughout these two years.
Stay tuned for the RCT results, coming soon!
