The role of data and evidence in Canada’s economic recovery
by Sarah Doyle, Karen Myers, Kelly Pasolli. Originally published on Policy Options
September 8, 2020
Swift and large-scale investments are being made to restart Canada’s economy in a context of disruption and uncertainty about the future. To ensure these investments have a real impact on people’s lives, on businesses and on the economy, investments in evidence generation are also needed that are commensurate with the scale of Canada’s recovery needs and ambitions. These investments should focus on evaluating the effectiveness of programs and services designed to help individuals, families, firms and communities impacted by COVID-19, and on monitoring progress against defined economic recovery goals.
Invest in evidence alongside recovery
Canada should not aim to return to normal. The pandemic has highlighted cracks in our economic and social policy landscape that are limiting the potential of many to participate in and benefit from economic activity. Risks, opportunities, harms and benefits are not distributed evenly, and this uneven distribution is hampering Canada’s overall economic potential.
Government leaders have rightly signalled a commitment to investing in an economy that is more resilient, greener and with prosperity that is more widely shared. This will require new policies, programs and partnerships, and it will be no small feat to get these right. In the context of ambition twinned with the uncertainty associated with this unprecedented time, governments should ensure that a proportion of this investment goes toward generating evidence on what is working, where recovery efforts are falling short, how needs are changing and when a pivot is needed. A “data-driven mindset” will ensure that bold ideas for the future are effectively translated into impact.
Investing in evidence to understand how well responses are working may not be top of mind at the moment. Responding to the new and complex problems presented by COVID-19 requires urgency and a willingness to try what has not been tried before. Generating and using evidence about what works, on the other hand, can be seen as a lengthy, resource-intensive process.
We believe evidence should, in fact, become part of the DNA of Canada’s recovery efforts. Specifically, we think governments should commit to directing at least 1 percent of economic recovery investments toward funding evidence-generation activities.
Creative, agile and pragmatic approaches to generating evidence exist and can be integrated into policies and programs, allowing us to learn and adapt rapidly to maximize the effectiveness and efficiency of responses to COVID-19. Timely, meaningful and actionable evidence will accelerate and increase the impact of recovery efforts. Given the challenges people across Canada are facing, governments and their non-profit and private sector partners need to know whether what they are doing is working.
Learn what is working on the ground
Investing in evidence can take many forms, some more effective than others. Traditional approaches to evaluating programs and services generally focus on measuring the extent to which they have achieved a predefined set of outputs and outcomes. In the context of COVID-19, there is also a need for more flexible, agile and participatory approaches to evidence generation that foster experimentation and continuous learning and improvement. These approaches should focus on collecting the information that service-delivery practitioners want and need, when they need it, to ensure they are serving participants effectively. Building a virtuous cycle of testing, learning and adapting would ensure that program responses are both agile and evidence-informed.
Specifically, we recommend that funding and requirements for evidence collection and use be incorporated into all services and programs aimed at supporting Canada’s economic recovery. Requirements and appropriate budget allowances should be embedded in departmental plans prior to securing Treasury Board approval, and in grant and contribution agreements and other contracts with third-party service providers.
Second, program design and evaluation requirements must be flexible. Instead of requiring service providers to participate in rigidly prescribed output- and outcomes-focused evaluations, providers should be supported in developing evidence and learning plans that outline what they want to learn about a program or service and how they will use the findings to inform further developments. These plans could include measurement of outcomes that align with the ultimate goals and objectives of the program — for example, sustained employment after one year. But, to allow for rapid learning and improvement, they should also include other evidence-generation activities such as ongoing, real-time monitoring of participants’ experiences in the program; rapid-cycle evaluation to test improvements to a program model; and measurement of how effectively a program or service is being implemented. Project agreements should have the flexibility for programs and services to be adjusted based on what practitioners learn as they implement.
Third, we recommend investments in capacity building. To ensure service providers are empowered to ask and answer the right questions about how well their programs are working, governments should invest in coaching and capacity-building support for practitioners. This could involve matching service providers with technical experts who can support them in developing an evaluation and learning plan for an intervention. Experts could also help providers build their capacity to use data to support rapid learning.
Fourth, investments in evidence should prioritize the user voice, placing a strong emphasis on collecting data about the needs, experiences and perspectives of target populations. COVID-19 has exacerbated inequalities and had an outsized impact on already vulnerable groups, including those who face systemic oppression and racism. In developing evidence and learning plans, practitioners should be required to include a strategy for how they will meaningfully and continuously engage users. The strategy should ensure that users have opportunities to co-design and participate in the implementation of evidence-generation activities, and that users’ voices are reflected in the design, implementation and adaptation of the program or service.
Finally, decision-making should be informed by evidence. Strong feedback loops should connect the evidence generated about programs and services to ongoing government decision-making. Specific teams within government could be tasked with synthesizing findings generated by service providers about what is working, and with translating these findings into adjustments to program design and allocation of resources, to ensure that resources are directed toward the most promising solutions.
Service providers should also be encouraged to share findings and insights with each other, building strong communities of practice and accelerating the adoption of evidence-informed insights in program delivery. Feedback loops between evidence generation and decision-making will ensure that government resources are allocated effectively and efficiently, and that service providers are empowered to apply evidence to their day-to-day practices.
Measure, monitor and report on progress
An effective recovery on a broader scale, beyond individual programs and services, also requires setting concrete, ambitious goals for Canada’s future economy, monitoring progress and adjusting investment as needed to achieve these goals.
Echoing recent federal government statements, we believe that a successful recovery should lead to a more resilient, inclusive, equitable and sustainable economy. To ensure that recovery investments align with this vision for the future, a clear picture of Canada’s performance with respect to economic, inclusion, equity and environmental-sustainability indicators is required, as is a commitment to track and report on progress against these indicators. This picture of Canada’s performance should build on existing tools, including the federal government’s Sustainable Development Goals Data Hub and the Brookfield Institute for Innovation + Entrepreneurship’s forthcoming Inclusive Innovation Monitor.
Regular monitoring would allow government leaders to understand how Canada’s economic recovery is performing relative to defined goals and to peer countries. It would also direct attention to where Canada’s progress is lagging, and inform decisions about how to adjust existing investments; it could help determine whether new policy initiatives are needed and how to effectively target any such initiatives.
Subscribe to our newsletter.
The impacts of COVID-19 on the social and economic well-being of Canadians are drastic and far-reaching, and governments have already responded swiftly to mitigate these effects. As governments shift their focus from crisis response to long-term recovery — and the opportunity to build back better than before — evidence and rapid learning can and should play an important role in enhancing the effectiveness of these measures. The recommendations outlined here would ensure that Canada is equipped with the evidence needed to shape recovery efforts in real time and ensure meaningful impacts for those most affected by the COVID-19 crisis.
Developed with support from Blueprint and the Brookfield Institute for Innovation + Entrepreneurship.
This article first appeared on Policy Options.