Best Practices in Global Research and Program Evaluation

Although the Field Center for Children’s Policy, Practice & Research focuses much of its work on child welfare and wellbeing in the United States, occasional opportunities for global collaboration become a natural fit for our portfolio. The CAFO-funded Research Challenge Grant is one such program that pairs researchers with non-profits to improve child and family programming worldwide. The Field Center team is honored to have participated in this prestigious collaboration, where we worked to evaluate a new program serving vulnerable families in rural Kenya.

The Family Strengthening Program at Agape Children’s Ministry provides time-limited family counseling by trained providers to recently reintegrated families following a child’s return home from the streets. The program was developed in 2020 to stabilize families during the critical period after reunification and promote long term family stability. With a couple of years under its belt, the team at Agape Children’s Ministry began to ask reflective questions: Is the Family Strengthening Program working? Is the program successful in its goal to keep children at home with their families and communities?

The Research Challenge Grant paired researchers around the globe with promising practices from NGOs to answer questions like these and assist providers in evaluating the services they provide to children and families. Assessing these programs not only allows for improvement within the organization, but the work could inform human service programs with similar populations worldwide.

Global partnerships can be incredibly challenging but immensely rewarding. Effective program evaluation requires skill and preparation, and doing this work internationally only adds to the complexity. As we reflect on the past year of our work in Philadelphia, PA and Kisumu, Kenya, we wish to share some of our key takeaway points and lessons learned during the year-long collaboration.

Recommendations for Engaging in Global Research and Program Evaluation:

Focus on lived experience

Effective program evaluation should include the perspectives of both those who provide and receive programming. Recognizing these individuals or groups as the experts of their own lived experience will help frame your evaluation. Include program staff in developing a data collection plan and consider available avenues for speaking directly with program participants or clients. Individual interviews or focus groups will provide greater insight than anonymous surveys.

Select and develop appropriate tools

There is no need to reinvent the wheel if the data you desire to obtain can be collected using existing resources. Conduct a review of the published literature and search governmental and NGO websites to locate surveys and measures that may have already been used in similar environments. If you are directly collecting qualitative data from providers or participants, developing culturally-appropriate tools is essential. Work with members of the local community to ensure that your measures, such as semi-structured interview guides, will be well-received by participants.

Understand the local context

If at all possible, ensure that the evaluators become familiar with the community, culture, customs, and local context where services are provided. This can be accomplished through site visits or by recruiting/selecting evaluation team members that hail from the same community. Understanding key terms, traditions, and cultural norms is essential when analyzing your data. Language varies across communities and it may take time to contextualize a speaker’s intent.

Utilize Technology

Working across time zones requires creativity and flexibility from all team members. Conferencing technology is indispensable when connecting with program providers, which should ideally happen on a regular basis. Cell phone communication may be more difficult in many countries and you should inquire about preferred methods. Develop a safe and secure plan for data sharing and communicating confidential information.

Seek Buy-In From Staff and Participants

Oftentimes, nonprofit program staff find requirements outside of the traditional job tasks hard to accommodate as the work is already challenging and time consuming. Clearly communicating the goals of a program evaluation, and tying it to how it might positively impact their work in the future, is an important step in securing staff investment in the process. Identify respected leaders in the organization that will reiterate your message. If program participants are included in the evaluation plan, culturally appropriate compensation should be provided along with an explanation about how their input could impact them or others.

Highlight areas for growth

No program is perfect, and it is already a great strength if a nonprofit is seeking to evaluate its effectiveness. Following the analysis of the data you collected during the evaluation, the organization should receive information about what they are doing well and what areas could be improved. Evaluators can use the data to make evidence-informed recommendations, as well as seek out other promising practices that might demonstrate success in areas that prove challenging.

—

While there is no “one size fits all” approach to program evaluation, the strategies described above can help frame your work, especially if you are working across cultures, countries, or continents. We are grateful for the opportunity to have worked alongside Agape Children’s Ministry and Christian Alliance for Orphans and we are immensely excited to see the Family Strengthening Program increase its already impressive impact on the children and families it serves in Kenya.

To learn more about the four research pairs that participated in the Research Challenge Grant, click here.