Indigenous Evaluation Strategy

• After decades of developing policies and programs designed to improve the lives of Aboriginal and Torres Strait Islander people, we still know little about their impacts, or how outcomes can be improved.
• Evaluation can answer questions about policy effectiveness, but both the quality and usefulness of evaluations of policies and programs affecting Aboriginal and Torres Strait Islander people are lacking.
  • Evaluation is often an afterthought rather than built into policy design (and this can affect data collection, evaluation design and result in evaluations that tell you very little).
  • Many evaluations focus on the wrong things (compliance rather than measuring impact, which means findings are often not useful).
  • Aboriginal and Torres Strait Islander people have minimal input into evaluations.
• There is also no whole-of-government approach to evaluation priority setting. And while policy makers agree that evidence is critical for good policies, in practice there is little reliance on evaluation evidence when designing or modifying policies.
• The Indigenous Evaluation Strategy (the Strategy) sets out a new approach. It provides a whole-of-government framework for Australian Government agencies for evaluating policies and programs affecting Aboriginal and Torres Strait Islander people.
• The Strategy puts Aboriginal and Torres Strait Islander people at its centre. To achieve better outcomes, what Aboriginal and Torres Strait Islander people value, their knowledges, and lived experiences needs to be reflected in what is evaluated, how evaluation is undertaken, and the objectives of policies and programs. A strategy embodying these principles will help achieve the outcomes in the new National Agreement on Closing the Gap.
• The Strategy provides principles-based guidance for Australian Government agencies for selecting, planning, conducting and reporting evaluations. It aims to ‘lift the bar’ on the quality of evaluations and improve their usefulness.
• The Strategy’s proposed governance arrangements (which are essential architecture for an effective Strategy) include an Office of Indigenous Policy Evaluation (OIPE) and an Indigenous Evaluation Council (with all Aboriginal and Torres Strait Islander members). The OIPE and the Council would work in partnership to: monitor and report on agencies’ progress implementing the Strategy; identify evaluation priorities and potential cross-agency/topic evaluations; and provide evaluation leadership and guidance.
• A central clearinghouse for evidence on the effectiveness of policies and programs affecting Aboriginal and Torres Strait Islander people would strengthen accountability for undertaking good evaluations and improve diffusion of knowledge.
• The case for central evaluation leadership and oversight is wider than the evaluation of policies and programs affecting Aboriginal and Torres Strait Islander people. Longer term, a new independent agency — a Centre for Evaluation Excellence — could be established to provide evaluation leadership and external oversight for all social and health policy evaluations across Australian Government agencies. If such a Centre was established, the OIPE could move to the Centre as a standalone branch (with the Indigenous Evaluation Council continuing its role).

Aboriginal and Torres Strait Islander people should be at the centre of evaluation

The Productivity Commission today released a proposed Indigenous Evaluation Strategy. The Strategy, which has been delivered to the Government, sets out a new approach to evaluating Australian Government policies and programs.

“Policies and programs affecting Aboriginal and Torres Strait Islander people are not working as well as they need to. Evaluation can play an important role filling this gap, but regrettably it is often an afterthought and of poor quality,” Commissioner Romlie Mokak said.

“Importantly, Aboriginal and Torres Strait Islander people are rarely asked about what, or how to evaluate, or what evaluation results mean,” Mr Mokak said.

The Strategy puts Aboriginal and Torres Strait Islander people at its centre. Better outcomes will be achieved if the values, expertise and lived experiences of Aboriginal and Torres Strait Islander people are reflected in what is evaluated, how evaluation is undertaken, and the outcomes policies and programs seek to achieve.

“The Commission has for many years advocated for better evaluation to inform decisions about policies and programs,” Chair Michael Brennan said.

“Evaluation is about understanding what is working, what is not working, and what changes could be made to make policies and programs work better. Evaluation can also inform the direction of new policies and programs. This Strategy will improve the quality and usefulness of evaluations and strengthen the evidence base,” Mr Brennan said.

“Working in partnership with Aboriginal and Torres Strait Islander people is fundamental to lifting the quality of evaluations, as is planning early so that the right questions are asked and the right data collected,” Commissioner Mokak said.

The Strategy does not endorse particular evaluation approaches or methods, but rather acknowledges that different evaluation questions and contexts will require different approaches. There is also guidance material for Australian Government agencies to use when they are selecting, planning, conducting and using evaluations of policies and programs affecting Aboriginal and Torres Strait Islander people.

The Strategy and supporting documents are available at www.pc.gov.au/ies.