Read time: 4 mins
Based on Optimally generate policy-based evidence before scaling, published February 2024.
Many ideas that succeed in small-scale testing struggle to deliver the same benefits when applied to larger populations. This is known as a ‘voltage drop’. New research from ANU explores how voltage drops happen and offers policymakers a path to strengthening their evidence base.
Read time: 4 mins
Based on Optimally generate policy-based evidence before scaling, published February 2024.
1
Ideas proven to work at small scale often dramatically lose their efficacy when fully implemented, and this is undermining evidence-based policymaking.
2
False positives, unrepresentative data, and unforeseen consequences are significant threats to the scalability of policy evidence.
3
Starting the policy research process by imagining how an idea would work when fully implemented could improve efficacy and prevent voltage drops.
Issues with scaling are a major problem in the social sciences and are affecting the efficacy of public policy, according to new ANU research.
The status quo in policy research is a ‘learning-by-doing’ approach. Whether and how a solution would work at a larger scale is only considered after it has passed efficacy tests at a small scale.
This is efficient, and in many cases, good enough. But according to new ANU evidence, it has led to between 50 and 90 per cent of programs, across a range of disciplines, experiencing a voltage drop when scaled up.
When a proposed solution has a high, fixed cost, voltage drops are especially expensive.
But new ANU research suggests an additional first step that could reduce the likelihood of an idea suffering a voltage drop and identifies five threats to scalability that it would address.
They are:
Mitigating these risks involves researchers and subject-matter experts considering what a successful, fully implemented intervention looks like at the very start of their experimental process.
Because their evidence directly informs policymaking, this would provide policymakers with more ideas that are less likely to suffer a voltage drop.
“According to new ANU evidence, between 50 and 90 per cent of programs, across a range of disciplines, experience a voltage drop when scaled up.”
