twitterFacebooklinkedin

Pitfalls of “Slippery Indicators”: The Importance of Reading Between the Lines

Date: August 2022
Author(s): Julia Fischer-Mackey and Jonathan Fox
Publication type: Scholarly journal article
Published by: Development in Practice

Within the field of social accountability, studies about community monitoring have made broad claims about what works” – or not in practice, with significant implications for practitioners and policymakers. Interpretation of these findings is complicated when studies rely on slippery indicators that do not measure the real-world processes they claim to address. This article illustrates the problem of slippery indicators, which has two main elements. First, some studies rely on indicators that do not actually measure community monitoring. Second, studies that claim to show a failure of community monitoring to deliver improvements may actually show a failure to deliver community monitoring in the first place. While complex research methods may obscure these two related problems, readers can still assess whether studies claims are supported by their empirical data by checking whether the findings are grounded in indicators that actually measure what they claim to study

Julia Fischer-Mackey was a researcher with the Accountability Research Center who received her PhD from American University’s School of International Service. She has conducted development program research and evaluation in several countries, including Guatemala. She is interested in questions of power, knowledge and evidence, and has experience with thematic areas including health, gender, and environmental justice. juliamackey@gmail.com   Jonathan Fox is a professor at the School of International Service at American University, where he directs the Accountability Research Center. He works with a wide range of public interest groups, social organizations, private foundations, and policymakers to learn from transparency, participation, and accountability initiatives. His most recent articles were published in World Development, Journal of Peasant Studies and IDS Bulletin. His books about accountability include Accountability Politics: Power and Voice in Rural Mexico; Mexico’s Right-to-Know Reforms: Civil Society Perspectives; Demanding Accountability: Civil Society Claims and the World Bank Inspection Panel and The Struggle for Accountability: The World Bank, NGOs and Grassroots Movements. He serves on the boards of directors of Bank Information Center, Community Agroecology Network, and Controla tu Gobierno (Mexico). He is also an honorary board member of Fundar (Mexico) and an honorary associate of the Institute of Development Studies. He previously served on the board of Oxfam America and as a founding member of the International Expert Panel of the Independent Reporting Mechanism of the Open Government Partnership. For publications, see: www.jonathan-fox.org.