How can a program like the Think Tank Initiative (TTI) contribute to stronger and more sustainable think tanks? And do stronger and more sustainable think tanks lead to changes in policy and practice? These are two of the big questions that TTI’s external evaluation is exploring. We are keen to share what we are learning with the broader community interested in the work of think tanks, and welcome the chance to learn from others’ experiences.
[Editor’s note: This blog post refers to key findings and emerging lessons from the 1st Interim Report of the External Evaluation for Phase 2 of the Think Tank Initiative, prepared by NIRAS inDevelop and released on July 22nd, 2016. The full report is available here.]
What has TTI set out to achieve?
The Think Tank Initiative is a 10-year program dedicated to strengthening the capacity of policy research institutions, also known as think tanks, in the developing world. At the centre of TTI’s theory of change is the belief that think tanks are uniquely positioned to effect change in their societies, by contributing local evidence to inform improved policies and practice on challenging social and economic issues.
A group of promising think tanks were selected for Phase 2, all of whom have received support since TTI began in 2008. Through a mix of core funding and focused capacity development, it is believed that these think tanks will be able to strengthen their research quality, policy linkages, and overall organizational performance.
TTI believes that as think tanks become more effective, sustainable, and agile in pursuit of their own research agendas, they will be better positioned to inform policy within their countries, gaining credibility in the process. Ultimately, evidence-driven and transparently-derived policies and practices are expected to help shape more equitable and prosperous societies.
What do we hope to learn from the evaluation?
Over the course of the program’s 2nd phase, the external evaluation will be investigating the relationship between TTI support and the strengthening of think tanks, and the extent to which the latter has an impact on changes in social and economic policy.
The evaluation takes into account the diversity within the cohort of think tanks that we work with, both institutionally and concerning the complex political, cultural, and socio-economic environments in which they operate. As such, the evaluation is closely examining the context surrounding specific think tanks (13 have been included in the sample), taking care to consider both internal and external factors. In particular, the evaluators are analyzing how context, combined with the use of core funding and technical assistance, determine the overall effectiveness of TTI support in improving think tank performance.
Although we and our donor partners believe this kind of support can lead to institutional sustainability, it is hugely important that this hypothesis be rigorously questioned and interrogated. TTI’s theory of change was largely validated in the Final Evaluation Report for TTI’s Phase 1, however we now wish to explore further to better understand when, where, and why the theory of change holds in practice. The Phase 2 evaluation seeks to do just this, accompanying the program from the start and providing in-depth reflections along the way. This approach will help us to:
Integrate what we are learning: Insights from interim evaluation reports confirm many of our efforts and also shed light on things that we can do differently. Ultimately, they provide us an opportunity to reflect on how we can tweak the kinds of support that we provide to think tanks to better meet their needs.
Measure change: By taking stock of our progress at different points over the course of the five years, we are able to get a better sense of how TTI is performing in terms of achieving its program outcomes.
Share lessons on what works and what doesn’t: By sharing what we’re learning with others, we hope to contribute to the global knowledge base on how different conditions, factors, and types of support affect think tank performance. Lessons from the TTI experience will no doubt serve as valuable input for donors who are thinking about future programming options and/or providing funding to policy research institutions.
What are some of the early findings?
This first interim report provided a wealth of early insights, which are broadly summarized in these three key areas:
Value of core funding: To date, core funding is seen as greatly facilitating think tank efforts to plan, think, and act strategically. These non-earmarked resources are providing institutions with breathing room for making decisions about organizational development, and increasing their confidence to independently design credible and relevant research agendas. The evaluation shows that TTI support in this area is resulting in progress towards improved capacity for producing quality research, better functioning organizations, and proactive engagement of think tanks in policy dialogues. However, the evaluation is also showing that core funding alone is insufficient for informing longer-term financial and organizational sustainability. One recommendation was for TTI to facilitate action research to help think tanks develop more comprehensive strategies for financial sustainability.
Effectiveness of support for capacity development: The evaluation is unpacking how TTI’s capacity development efforts are helping think tanks, and the extent to which this is resulting in positive outcomes. For example, it is investigating connections between research quality at the think tank level and TTI support towards: building researcher capacity, attracting and retaining qualified staff, focusing on policy-relevant research, and strengthening internal processes for research coordination and control. One recommendation was for TTI to refine its needs assessment approach, focus resources for greater impact, and further engage think tanks at the design stage of capacity development activities.
Better understanding of the context: Early insights suggest that the role think tanks play in policy development and debate, and the ways that TTI has contributed, vary depending on context. Key factors include: independence from government and international actors, closeness of relations with civil society and the broader research community, demand for evidence from policy actors, and freedom of expression. The evaluators will continue to build understanding of the complex interplay between these different factors, and how the effectiveness of TTI’s efforts can be best extrapolated.
What happens next?
With only 3 years remaining until TTI comes to a close in mid-2019, we are keenly interested in the interim findings that have surfaced to date, and also look forward to receiving the next round in the 2nd interim report due in late 2017. We are already planning adjustments to our programming to address some of the recommendations that were shared by the evaluators. We will continue to share our plans and what we are learning along the way through upcoming blog posts.
If you would like to share thoughts from your own experiences in evaluating the impact of support to policy research institutions, we would welcome hearing from you. The more we collectively know about what works (and what doesn’t), the better we can support think tanks in developing countries to inform positive change.