by Melinda T. Tuan, Principal, Melinda Tuan Consulting

Over the past decade, there has been an increased emphasis on impact measurement throughout the nonprofit sector. Since co-founding REDF fifteen years ago with George Roberts and Jed Emerson, I have been a big fan of the appropriate use of evaluation and measurement. I believe it is a good thing for us to find out whether a program works (or not) to improve people's lives and the world around us. Funders and nonprofits alike must continue to invest in their capacity to measure the impact of their efforts to inform their practices and better meet their missions.

In recent years, however, and especially with the federal government's 2012 publication of guidelines on the need to use evidence and rigorous evaluation in making budget, management and policy decisions, some funders seem to be enamored with using a specific evaluation methodology to measure impact: randomized control trials or RCTs. Known as the "gold standard" of evaluation methodologies, RCTs are commonly used in clinical trials to attribute change to a particular intervention. For example, patients in a study are randomly assigned to receive a new prescription drug or a placebo in order to determine cause and effect. Using a RCT, one should be able to demonstrate that a particular outcome can be attributed to the specific intervention, regardless of other internal or external influences.

Fay Twersky, REDF's long-time partner in developing our evaluation methodologies and capacity and now Senior Fellow at the Hewlett Foundation, recently published a piece that describes the limitations of RCTs in influencing policy, nonprofits and philanthropy (see http://www.effectivephilanthropy.org/blog/2012/02/time-for-a-gold-standard-of-use/). In addition to Fay's excellent analysis, another significant limitation and perhaps unintended consequence of this increased emphasis on RCTs in the social sector, is that they serve to reinforce organization-centric behavior. It will always be important to measure the impact of individual organizations and programs and to build the capacity of individual nonprofits to assess their work to inform and improve their practice. However, increasingly, nonprofits and funders alike are recognizing they cannot achieve their goals alone and change across a system cannot be attributed to a single organization's efforts. We need to understand impact on a larger scale and place individual organizations' contributions in the context of the larger ecosystems in which they operate.

Herein lies the challenge and opportunity for the field in thinking about and measuring impact into the future. We have by no means mastered the first ability to measure impact at the program or organizational level and this work should continue in order to ensure our limited resources are invested in the highest impact solutions. We are in the very beginning stages of exploring an ecosystem-wide approach to evaluation; and measuring impact at the field or system level is undoubtedly even more challenging. Yet, if we believe our best work is done in concert with others in a field, system or network, to effect lasting, positive change; we will need to go beyond measuring individual impact toward understanding our collective impact.