Indigenous Evaluation Strategy

• After decades of developing policies and programs designed to improve the lives of Aboriginal and Torres Strait Islander people, we still know very little about their impact, or how outcomes can be improved.
• Evaluation can answer questions about policy effectiveness, but both the quality and usefulness of evaluations of policies and programs affecting Aboriginal and Torres Strait Islander people are lacking.
  • Evaluation is often an afterthought rather than built into policy design (and this can affect data collection and evaluation design and result in evaluations that tell you very little).
  • Many evaluations focus on the wrong things (compliance rather than measuring impact, which means findings are often not useful).
  • Aboriginal and Torres Strait Islander people have minimal input into evaluations.
• There is also no whole‑of‑government approach to evaluation priority setting. And while policy makers agree that evidence is critical for good policies, in practice there is little reliance on evaluation evidence when designing or modifying policies.
• The draft Indigenous Evaluation Strategy (the Strategy) sets out a new approach. It provides a whole‑of‑government framework for Australian Government agencies for evaluating policies and programs affecting Aboriginal and Torres Strait Islander people.
• The Strategy puts Aboriginal and Torres Strait Islander people at its centre. To achieve better outcomes, what Aboriginal and Torres Strait Islander people value, their expertise, and lived experiences needs to be reflected in what is evaluated, how evaluation is undertaken, and the outcomes policies and programs seek to achieve.
• The Strategy sets out evaluation principles and provides principles‑based guidance for Australian Government agencies for selecting, planning, conducting and reporting evaluations. The principles‑based guidance aims to ‘lift the bar’ on the quality of evaluations and improve their usefulness.
• An Office of Indigenous Policy Evaluation (OIPE) is proposed to monitor agencies’ performance against the Strategy, provide evaluation leadership, and identify potential cross‑agency/topic evaluations. A central clearinghouse for the body of evidence on the effectiveness of policies and programs affecting Aboriginal and Torres Strait Islander people would improve the accessibility of evaluation evidence.
• The Strategy also sets out a whole‑of‑government approach to evaluation priority setting, including the establishment of an Indigenous Evaluation Council to partner with the OIPE to identify evaluation priorities.
• The case for central evaluation leadership and oversight is wider than the evaluation of policies and programs affecting Aboriginal and Torres Strait Islander people. Longer term, a new independent agency — a Centre for Evaluation Excellence — could be established to provide evaluation leadership and external oversight for all social and health policy evaluations across Australian Government agencies. If such a Centre was established, the OIPE could move to the Centre as a standalone branch (with the Indigenous Evaluation Council continuing its role).

Centring Indigenous people in evaluation is key to better outcomes

The Productivity Commission today outlined a plan to improve the evaluation of policies and programs affecting Aboriginal and Torres Strait Islander people.

“It is a stark reality that despite decades of new policies and programs aimed at improving the lives of Aboriginal and Torres Strait Islander people, we know very little about their impact,” Productivity Commission Chair Michael Brennan said.

“Evaluation is too often an afterthought. We need to lift the bar on evaluation quality, embed it at the outset of policy design and make sure Aboriginal and Torres Strait Islander people are closely involved throughout.”

The draft Indigenous Evaluation Strategy puts Aboriginal and Torres Strait Islander people at its centre. It recognises that to achieve better outcomes, what Aboriginal and Torres Strait Islander people value, their expertise, and lived experiences need to be reflected in what is evaluated, how evaluation is undertaken and the outcomes policies and programs seek to achieve.

“The Strategy is about Australian Government agencies engaging effectively with Aboriginal and Torres Strait Islander people to better design and evaluate policies and programs. The Strategy must improve evaluations to improve policies and the lives of Aboriginal and Torres Strait Islander people,” Commissioner Romlie Mokak said.

The draft Strategy includes detailed guidance material, outlining the strengths of different evaluation approaches and how to ensure evaluation is part of every stage of policy making and program delivery.

The Commission also proposes new governance arrangements to monitor Australian Government agencies’ performance against the Strategy, provide evaluation leadership, and identify potential cross-agency and topic evaluations. The Strategy also sets out a government‑wide approach to setting evaluation priorities.

The Commission consulted widely when developing the draft Strategy, including with Aboriginal and Torres Strait Islander people, communities and organisations. The Commission is again seeking feedback on the proposed approach.

The draft Strategy, guidance material and background paper are available at www.pc.gov.au