Contents

Please note: Aboriginal and Torres Strait Islander people are advised that this paper may link to other websites which contain names, images and voices of deceased people.

About the Strategy

For decades there have been calls to better understand how policies and programs are affecting Aboriginal and Torres Strait Islander people. But the reality is that evidence about what works and why remains thin. And yet, to design policies and programs that improve the lives of Aboriginal and Torres Strait Islander people, evidence is central.

Improving outcomes for Indigenous Australians depends on agencies with responsibility for policies and programs affecting Indigenous Australians undertaking meaningful evaluations. The Commission is to develop a strategy to guide that evaluation effort..

Frydenberg (2019)

The Australian Government recently committed to better evaluation of policies and programs affecting Aboriginal and Torres Strait Islander people. Recognising the link between evaluation and better outcomes, the Australian Government asked the Productivity Commission to develop an evaluation strategy to be used by all Australian Government agencies, for policies and programs affecting Aboriginal and Torres Strait Islander people.

To develop the Indigenous Evaluation Strategy (the Strategy), the Commission engaged widely, and worked with Aboriginal and Torres Strait Islander people and organisations, government agencies, and people administering, delivering and evaluating policies and programs. We looked at the strengths and weaknesses of the current arrangements and how a Strategy could improve evaluation, policies and programs, and the lives of Aboriginal and Torres Strait Islander people, and Australians more generally.

The Strategy was informed by:

• what Aboriginal and Torres Strait Islander people and organisations, government agencies, and people undertaking and using evaluations told us about their experience with evaluation, how evaluation could be done better, and how the findings from evaluation could be used more effectively to improve policies and programs
• a workshop on ‘Evaluation Priority Setting’ and roundtables — one on ‘Objectives, Principles and Defining Success’ and another on ‘Culture, Capability and Governance’
• results from a questionnaire to Australian Government agencies on recent evaluations, evaluation policies and practice
• the literature on good practice evaluation approaches and methods, other evaluation strategies, ethics and research guidelines, standards and strategies, and governance arrangements for evaluation  adopted in other similar countries
• feedback provided to the Commission on the draft Strategy, background paper and guidance.

The Strategy sets out a new approach to evaluating policies and programs affecting Aboriginal and Torres Strait Islander people aimed at improving the quality and usefulness of evaluation. It puts Aboriginal and Torres Strait Islander people at its centre, and emphasises the importance of drawing on the perspectives, priorities and knowledges of Aboriginal and Torres Strait Islander people when deciding what to evaluate and how to conduct an evaluation.

It provides a whole-of-government framework for Australian Government agencies when they are evaluating both Indigenous-specific and mainstream policies and programs affecting Aboriginal and Torres Strait Islander people.

The Strategy has principles-based guidance for Australian Government agencies to use when selecting, planning, conducting and using evaluations of policies and programs affecting Aboriginal and Torres Strait Islander people. It also outlines the governance arrangements needed to strengthen accountability and support the centring of Aboriginal and Torres Strait Islander people, perspectives, priorities and knowledges. The Strategy’s monitoring and reporting arrangements also embed incentives for agencies to improve the quality of evaluations, to learn from, share and use evaluation findings to inform and improve policy decisions.

The Strategy has been developed at a time when relationships between government and Aboriginal and Torres Strait Islander people are changing. All Australian governments have committed — as part of the National Agreement on Closing the Gap (the National Agreement) — to change the way they work to accelerate improvements to the lives of Aboriginal and Torres Strait Islander people.

The Strategy aligns with the National Agreement and supports Aboriginal and Torres Strait Islander people’s right to self‑determination and to actively participate in decision making about matters that affect them — as outlined in the United Nations Declaration on the Rights of Indigenous Peoples — by:

• putting Aboriginal and Torres Strait Islander people, perspectives, priorities and knowledges at its centre
• proposing an Indigenous Evaluation Council, made up of Aboriginal and Torres Strait Islander members, be established to provide leadership and oversight for the Strategy
• providing practical ways for Australian Government agencies to change the way they work with Aboriginal and Torres Strait Islander people as they develop and evaluate policies and programs
• disseminating, synthesising and translating evaluation findings in accessible forms that can be used by Aboriginal and Torres Strait Islander people to support decision making and assess progress in achieving the aims of the National Agreement
• strengthening and supporting the capability of Aboriginal and Torres Strait Islander people, organisations and communities to engage in and use evaluation.

The Strategy should be read in conjunction with:

• Indigenous Evaluation Strategy, Background Paper — this paper describes the Commission’s approach to developing the Strategy and provides further detail on the governance arrangements and actions outlined in the Strategy.
• A Guide to Evaluation under the Indigenous Evaluation Strategy — this document provides additional guidance for Australian Government agencies on what applying the Strategy’s principles means in practice.

What is evaluation and why evaluate?

Evaluation is the systematic process of making a judgment about the merit or worth of a policy or program. It is about assessing the effectiveness of policies and programs and identifying the factors that drive or undermine their effectiveness. It answers questions such as — how was the policy or program delivered, what difference did the policy or program make, what would have happened without the policy or program in place, and do the benefits of the policy or program justify the costs?

Evaluation is undertaken to:

• better understand what policies and programs work, why, when and for whom — to support evidence-informed policy development, program design and implementation
• inform improvements in policies and programs — to support learning by doing
• support accountability to funders and the community — shedding light on how best to use taxpayers’ money, including whether policy and program funds are properly spent, are delivering value for money and are meeting recipients’ needs.

Evaluation can also help to build trust in government, if the findings are used to support ‘learning by doing’ and governments find ways to work with those affected by the policy or program to achieve better outcomes. Trust between government and Aboriginal and Torres Strait Islander people, organisations and communities is particularly important in light of Australia’s colonial history and its ongoing impacts.

Why an Indigenous Evaluation Strategy?

There are many Australian Government policies and programs that are designed to improve the lives of Aboriginal and Torres Strait Islander people. But after decades of developing new policies and programs and modifying existing ones, we still know very little about their impact on Aboriginal and Torres Strait Islander people, or how outcomes could be improved.

We need to know what is working and if policy and programs which apply to Indigenous individuals, families and communities are having an impact. We also want greater insight into why policy or program implementation is not effective, and we need early opportunities for correction or reinvestment of funds and effort to ensure that funding is directed to where it is needed most.

Empowered Communities (2019, p. 5)

While evaluation can provide answers on the effectiveness of policies and programs, both the quality and usefulness of evaluations of policies and programs affecting Aboriginal and Torres Strait Islander people are variable. Evaluation is often an afterthought rather than built into policy design and this can affect data collection and evaluation design. Many evaluations also ask the wrong questions. Crucially, Aboriginal and Torres Strait Islander people currently have limited input into evaluation planning, conduct and reporting.

There is currently no Australian Government‑wide approach to priority setting for evaluations of policies and programs affecting Aboriginal and Torres Strait Islander people. And while policy makers agree that evidence is critical for good policies, many admit that in practice they do not rely heavily on evidence, or past experience, when formulating or modifying policies and programs.

Who does the Strategy apply to?

The Strategy applies to all Australian Government agencies with responsibility for designing and/or implementing policies and programs affecting Aboriginal and Torres Strait Islander people.

While the Strategy is to guide Australian Government agencies when they are selecting, planning, commissioning, conducting and using evaluation, in practice it affects everyone involved in the evaluation of Australian Government policies and programs affecting Aboriginal and Torres Strait Islander people, including:

• individuals and communities who are affected by the policies and programs being evaluated who may participate in interviews and/or surveys as part of the evaluation
• external evaluators who are commissioned to conduct evaluations
• service providers who deliver policies and programs who may be involved in collecting data, identifying evaluation participants, participating in, and implementing recommendations from, evaluations
• peak bodies and community representatives who may conduct, partner in, or lead evaluations
• users of evaluation, including ministers, policy and program administrators and other individuals and groups who make decisions about policy and program design and implementation.

The Strategy could also have implications for state, territory and local governments given the crossover between governments’ responsibilities, and the aims of the National Agreement. Because the Strategy aligns with the ambition of the National Agreement for all Australian governments to change the way they work with Aboriginal and Torres Strait Islander people, state and territory government agencies will be able to use the Strategy when they are selecting, planning, commissioning, conducting and using evaluations of policies and programs affecting Aboriginal and Torres Strait Islander people.

The objective of the Strategy

The objective of the Strategy is to improve the lives of Aboriginal and Torres Strait Islander people by having policy and program decisions informed by high‑quality and relevant evaluation evidence (figure 1).

The clear objective of all government action that impacts Aboriginal and Torres Strait Islander people should be to improve wellbeing, to ensure that Indigenous people have the capabilities and opportunities to live the life they value, in a society that values and affirms Indigenous peoples’ identities, cultures and contributions to Australian nationhood.

Australian Institute of Aboriginal and Torres Strait Islander Studies (2019, p. 3)

Evaluations should contribute to a high‑quality, useful and accessible body of evidence that government and Aboriginal and Torres Strait Islander people, organisations and communities have confidence to use when designing, modifying and implementing policies and programs.

An Indigenous Evaluation Strategy can improve policies and programs, and ultimately the lives of Aboriginal and Torres Strait Islander people, by:

• centring Aboriginal and Torres Strait Islander people, perspectives, priorities and knowledges in all stages of evaluation
• ‘lifting the bar’ on the quality of evaluations of policies and programs affecting Aboriginal and Torres Strait Islander people
• enhancing the use of evaluations to inform policy and program design and implementation by supporting a culture of evaluation and building an accessible body of evidence and data on the effectiveness of policies and programs
• promoting a whole-of-government approach to priority setting and evaluation of policies and programs affecting Aboriginal and Torres Strait Islander people.

Figure 1. The objective of the Strategy is to improve the lives of Aboriginal and Torres Strait Islander people

Guiding principles

The guiding principles for the Strategy set out what good practice looks like and what agencies should consider when undertaking evaluations, while also allowing flexibility for evaluations to be tailored to the particular circumstances of policies, programs and communities. The principles are to guide what agencies and evaluators do when they are planning, conducting, reporting and using evaluation. They should also guide priority setting, knowledge sharing and translation, building evaluation capability and culture, monitoring and review.

The power of principles for policy development, program delivery and evaluation is that they are adaptable to different contexts. … Principles allow both the local-level experts and high-level abstract managers to have a shared understanding of how ‘good’ can be judged. Thus, principles can be framed in a way that can guide decisions, operations and evaluation across all levels…

Chelsea Bond et al. (2019, p. 11)

The overarching principle of the Strategy is centring Aboriginal and Torres Strait Islander people, perspectives, priorities and knowledges (figure 2). This principle is about recognising the strengths and diversity of Aboriginal and Torres Strait Islander people, communities, knowledges, histories and cultures. It is also about building partnerships with Aboriginal and Torres Strait Islander people to define policy and program objectives, decide on evaluation questions, how evaluations will be conducted and how evaluation findings will be interpreted. This will improve the quality and use of evaluations and better align policies and programs with the needs and priorities of Aboriginal and Torres Strait Islander people.

The overarching principle is also the lens through which the Strategy’s other principles — credible, useful, ethical and transparent — should be interpreted. These principles frame how agencies should plan and conduct evaluation and how evaluations will be assessed. Together, the principles aim to enhance the relevance of evaluation to Aboriginal and Torres Strait Islander people, the quality and usefulness of evaluations, as well as guiding the implementation, governance and review of the Strategy itself.

The Strategy does not replace, but rather complements and builds on, Australian Government agencies’ evaluation processes and frameworks. It provides a whole-of-government framework and consistent accountabilities for all Australian Government agencies developing and implementing policies and programs that affect Aboriginal and Torres Strait Islander people.

Further guidance to agencies on implementing the principles of the Strategy is available in A Guide to Evaluation under the Indigenous Evaluation Strategy.

Figure 2. Guiding principles of the Indigenous Evaluation Strategy

An overarching principle:
Centring Aboriginal and Torres Strait Islander people, perspectives, priorities and knowledges

In order to be effective, the evaluation strategy must honour and adopt Aboriginal perspectives and approaches to health, wellbeing and policy development. … Aboriginal Communities are well positioned to inform policy changes as they are at the forefront of issues in their Communities.  In order to achieve effective evaluation and long term outcomes, we must place Aboriginal people and leadership at the centre of an [Indigenous Evaluation Strategy].

Victorian Aboriginal Community Controlled Health Organisation (2019, p. 11)

This principle is at the core of the Strategy. Centring Aboriginal and Torres Strait Islander people, perspectives, priorities and knowledges is fundamental to achieving the objective of the Strategy of improving the lives of Aboriginal and Torres Strait Islander people. What Aboriginal and Torres Strait Islander people value, their knowledges, and lived experiences needs to be reflected in what is evaluated, how evaluation is undertaken, and the objectives of policies and programs.

Centring involves recognising the diverse cultures (languages, knowledge systems, beliefs and histories) and the impacts of contemporary and historical policies and programs on the wellbeing of Aboriginal and Torres Strait Islander people. It supports Aboriginal and Torres Strait Islander people’s rights to self‑determination and aligns with the National Agreement on Closing the Gap by emphasising the need for governments to change the way they work with Aboriginal and Torres Strait Islander people. In practice the principle means:

• evaluations are undertaken in the areas, and address the issues, that are most important to Aboriginal and Torres Strait Islander people
• Australian Government agencies routinely consider the impacts of mainstream policies and programs on Aboriginal and Torres Strait Islander people
• Aboriginal and Torres Strait Islander people, organisations and communities have the opportunity to decide how they want to be involved in evaluations
• non‑Indigenous evaluators have the necessary knowledge, experience and awareness of their own biases to work in partnership with, and to draw on the knowledges of, Aboriginal and Torres Strait Islander people
• evaluation processes strengthen and support the evaluation capability of Aboriginal and Torres Strait Islander people.

Table 1. Centring Aboriginal and Torres Strait Islander people, perspectives, priorities and knowledges in practice

What to evaluate	Aboriginal and Torres Strait Islander people are engaged to decide what policies and programs have the greatest impact on their lives and should be subject to evaluation. Evaluations consider impacts of policies and programs on Aboriginal and Torres Strait Islander people and how agencies are working with Aboriginal and Torres Strait Islander people to develop and deliver policies and programs.
Evaluation planning, design and conduct	Evaluations draw on the perspectives, priorities and knowledges of Aboriginal and Torres Strait Islander people and communities. Mainstream policies and programs routinely consider impacts on Aboriginal and Torres Strait Islander people and evaluate where the impact is considered significant. Aboriginal and Torres Strait Islander people, organisations and communities have the opportunity to decide how they want to be involved in evaluations. Sufficient time and resources are allowed for meaningful engagement and capability strengthening with Aboriginal and Torres Strait Islander people during evaluation. Engagement between commissioners, evaluators, participants and users is respectful of differences and undertaken in culturally safe ways. Evaluations are conducted by Aboriginal and Torres Strait Islander evaluators and/or non-Indigenous evaluators with skills and experience working with Aboriginal and Torres Strait Islander people. Decisions about data planning, collection and use are undertaken with Aboriginal and Torres Strait Islander people so that the right data are collected, data are of high quality, and governance arrangements are in place for ownership and use of data. Evaluation design and reporting reflects the diversity of Aboriginal and Torres Strait Islander people, perspectives, priorities and experiences.
Reporting evaluation findings	Aboriginal and Torres Strait Islander people engage, partner or lead in translating evaluation findings so they are meaningful, accessible and useful. Evaluation reports describe how Aboriginal and Torres Strait Islander people engaged, partnered or led during the evaluation process. Evaluators and commissioners ensure that evaluation findings are communicated back to the Aboriginal and Torres Strait Islander people, communities and organisations that participated.
Building capability and a culture of evaluation	Evaluation teams have the capability to incorporate Aboriginal and Torres Strait Islander knowledges into their evaluative thinking. Evaluation processes strengthen evaluation capability among Aboriginal and Torres Strait Islander people, organisations and communities. Agencies provide opportunities for staff to strengthen their cultural capability. Agencies build strong and sustainable relationships with Aboriginal and Torres Strait Islander people so there is a shared understanding about evaluation priority areas. Agencies encourage review and feedback from Aboriginal and Torres Strait Islander people, organisations and communities about their evaluation practices.

Principle:
Credible

Aboriginal and Torres Strait Islander people are more likely to accept an evaluation as credible if it is carried out in a culturally responsive manner, with the issues that are important to Aboriginal and Torres Strait Islander people being explored in a way that Aboriginal and Torres Strait Islander people see as relevant.

Yulang Indigenous Evaluation (2020, p. 14)

Evaluation approaches, methods and processes must be credible if policy and program design and implementation decisions are to be based on evaluation findings. Evaluations should be conducted by evaluators who are technically and culturally capable. Credibility is grounded on rigorous methodology, and includes Aboriginal and Torres Strait Islander values, perspectives and knowledges.

The Strategy does not endorse particular evaluation types, approaches or methods. Agencies and evaluators should use evaluation types, approaches and methods that:

• are rigorous and fit‑for‑purpose
• answer the questions that policy makers and Aboriginal and Torres Strait Islander people want answered
• suit the context in which the policy or program is operating, the size and importance of the policy or program, and the timeframe and resources available for evaluation.

Using a combination of evaluation types, approaches and methods — including both qualitative and quantitative, and Western and Indigenous methods — can maximise the strengths and compensate for limitations of any single evaluation type, approach or method.

Evaluation users should have confidence that evaluation findings are robust and any limitations to the analysis are clearly identified. Aboriginal and Torres Strait Islander people and the broader Australian community should have confidence that policies and programs are being assessed objectively and independently.

The credibility of evaluation practices and findings will be enhanced when Aboriginal and Torres Strait Islander people and organisations are engaged, partner or lead in the interpretation of findings, evaluations are peer reviewed, and data are available to allow evaluation results to be replicated and interpreted by external researchers.

Table 2. Credible evaluation in practice

What to evaluate	Evaluation priorities are identified in a systematic way based on what is important to Aboriginal and Torres Strait Islander people, the impact of the policy or program, its risk profile, strategic significance and budget.
Evaluation planning, design and conduct	Aboriginal and Torres Strait Islander people can engage in all stages of the evaluation process. Evaluations draw on the perspectives, priorities and knowledges of Aboriginal and Torres Strait Islander people and communities and acknowledge the diversity of Aboriginal and Torres Strait Islander people. Decisions about whether to use an internal or external evaluator (or some combination) take into account independence, objectivity and policy or program knowledge. The decision about whether to use an internal or external evaluator should be based on who is best placed to undertake a particular evaluation. The resources available for evaluation are proportionate to the size and importance of the policy or program being evaluated. Sufficient time and resources are allocated for engaging or partnering with Aboriginal and Torres Strait Islander people during evaluation. Evaluation is planned early before policies and programs are implemented so that the right data can be collected to undertake rigorous analysis and measure the things that are important to Aboriginal and Torres Strait Islander people. Evaluations employ appropriate approaches and methods given the evaluation questions being asked, data, time and resources. Data used for evaluation are of high quality and are collected in a culturally safe manner. Data can be disaggregated to reflect the diversity of Aboriginal and Torres Strait Islander people and communities. Evaluations have clear quality assurance processes, including peer and community review.
Reporting evaluation findings	Evaluation methods and data are described in detail in evaluation reports. The limitations of evaluation analysis, data and results are clearly documented in evaluation reports.
Building capability and a culture of evaluation	Agencies provide opportunities for staff to improve their capability in planning, commissioning, conducting, reporting and using evaluations of policies and programs affecting Aboriginal and Torres Strait Islander people. Evaluations strengthen the evaluation capability of Aboriginal and Torres Strait Islander people, organisations and communities, including by resourcing their engagement in an evaluation. Evaluation teams have the capacity, cultural and technical capability and experience to deliver high-quality evaluation. Evaluative thinking and high-quality evaluation is valued by agencies and Aboriginal and Torres Strait Islander people, organisations and communities.

Principle:
Useful

Irrespective of the framing, design, quality and conduct of Indigenous evaluation or evaluation with Indigenous peoples; the evaluations come to nothing if the findings do not consistently inform Indigenous policy and planning processes — locally, regionally and nationally.

Australian Evaluation Society (2020, p. 3)

Evaluations that do not provide useful results or are not used to improve policies and programs affecting Aboriginal and Torres Strait Islander people are a missed opportunity and a waste of resources. When Australian Government agencies plan, commission or conduct evaluations, the intention should always be to use the evaluation findings to inform policy and program decisions.

Useful evaluations consider the needs of a range of end‑users — including governments, Aboriginal and Torres Strait Islander people, organisations and communities — and tailor questions, approaches, interpretation and reporting accordingly.

Usefulness is also important when deciding which policies and programs to evaluate. What is important to Aboriginal and Torres Strait Islander people should be a primary consideration when deciding what to evaluate.

Evaluations are most useful when they are of high quality, findings are available to decision makers, and evaluations are timed to contribute to policy‑making and implementation decisions.

Evaluations should be embedded in the policy cycle. This will improve the quality of evaluations by ensuring that adequate resources are devoted to evaluation, baseline and monitoring data are fit‑for‑purpose, and evaluation questions are linked closely to policy and program goals. It will also mean that evaluation findings are available at key decision points during implementation, and to support continuous quality improvement.

High‑quality evaluations will also contribute valuable knowledge to the policy evidence base for future policy development. The usefulness of individual evaluations can be enhanced by synthesising evaluation findings into the existing evidence base and using meta‑analysis of multiple evaluations to provide broader insights into policy and program effectiveness and evaluation quality.

Table 3. Useful evaluation in practice

What to evaluate	The policies, programs and outcomes most relevant for improving the lives of Aboriginal and Torres Strait Islander people are evaluated. Evaluations fill knowledge gaps about what works, why, for whom and in what circumstances. Evaluations support the National Agreement on Closing the Gap by providing evidence on how policies and programs affecting Aboriginal and Torres Strait Islander people are contributing to the National Agreement’s objectives and priorities.
Evaluation planning, design and conduct	Evaluation is embedded in the policy and program design and delivery cycle and is planned for early. Evaluations are planned and conducted with the intention that the findings will be available at key decision points and used to inform decision making by governments and Aboriginal and Torres Strait Islander people, organisations and communities. Evaluation questions address issues that are important to Aboriginal and Torres Strait Islander people and other intended users of evaluation findings. Evaluation findings and lessons feed into planning cycles, policy formation, program delivery and learning processes for agencies and Aboriginal and Torres Strait Islander people, organisations and communities. Data collected and used for evaluation align with Aboriginal and Torres Strait Islander data priorities.
Reporting evaluation findings	Evaluation reports are written and communicated in ways that are useful and accessible to evaluation users, including policy makers, Aboriginal and Torres Strait Islander people and policy and program providers. Evaluation findings are synthesised and used in meta-analyses to contextualise findings and provide broader insights.
Building capability and a culture of evaluation	Evaluation is core business for agencies. Agencies understand the benefits of evaluation as well as how to ask questions that will inform decision making. Agencies welcome potentially negative or unexpected evaluation findings and recognise that they are an opportunity to learn and improve policies or programs. Agencies have processes that allow the findings of evaluations to be incorporated into policy or program changes and new policy development. Agencies build their capability to use and respond to evaluation findings. Agencies support Aboriginal and Torres Strait Islander people, organisations and communities to use and implement evaluation findings.

Principle:
Ethical

For me, [engaging ethically] is yindyamarra, a Wiradjuri concept which means to act with honour and respect, wisdom, to go slowly and act responsibly, be gentle and polite and honest with each other, be careful of the words and actions you put out to the world and understand the impact they have.

Michael McDaniel, AIATSIS Council Chairperson (AIATSIS, 2018)

All stages of evaluation — planning, commissioning, conduct, reporting and use — should be conducted in an ethical way. Applying ethical standards improves the quality and consistency of evaluation and ensures that evaluation has a positive impact on Aboriginal and Torres Strait Islander people.

Ethical practice during evaluation should be guided by established ethical guidelines for research involving Aboriginal and Torres Strait Islander people. These include:

• the AIATSIS Code of Ethics for Aboriginal and Torres Strait Islander Research (AIATSIS 2020)
• Ethical Conduct in Research with Aboriginal and Torres Strait Islander Peoples and Communities (NHMRC 2018)
• the National Statement on Ethical Conduct in Human Research (NHMRC, ARC & UA 2018).

Formal review by an ethics committee will be required for some, but not all, evaluations conducted under the Strategy. However, all evaluation projects should include systematic and well‑documented assessments of ethical risks. In cases where an evaluation does not require formal review by an ethics committee, agencies and evaluators must still follow ethical practices. Evaluation quality will be improved by getting feedback from an ethics committee with experience in Aboriginal and Torres Strait Islander research during the ethics review process. Ethical evaluation is the joint responsibility of agencies and evaluators, regardless of whether an evaluation is conducted internally or externally.

Agencies should ensure that evaluation budgets and timeframes are sufficient for evaluators to fulfil ethical requirements. This means allowing enough time for ethics risk assessment and formal ethical review, if required, as well as for evaluators to engage meaningfully with Aboriginal and Torres Strait Islander people, build capability and report back to participants.

Based on current practice, agencies will need to build their capability to plan, commission and conduct ethical evaluation, as well as ensure that external evaluators have sufficient skills and experience to put this principle into practice.

Table 4. Ethical evaluation in practice

What to evaluate	Aboriginal and Torres Strait Islander people are engaged in identifying priorities in an ethical way.
Evaluation planning, design and conduct	Evaluations are conducted according to the values and ethics identified in established guidelines for research and evaluation with Aboriginal and Torres Strait Islander people, organisations and communities. The evaluation plan allows sufficient time and resources for evaluators to meet ethical requirements. Evaluations include a systematic assessment of ethical risk to determine whether evaluations should be subject to formal review by an ethics committee. When evaluations are subject to formal ethical review, this should be done by an ethics committee with expertise in research with Aboriginal and Torres Strait Islander people. Data are collected, stored and shared in ways that benefit and do not harm Aboriginal and Torres Strait Islander people.
Reporting evaluation findings	Ethical conduct during evaluations is clearly documented in evaluation reports. Publication practices do not impede the ability of evaluators to engage in ethical practices, including sharing findings with evaluation participants. Evaluation participants’ contributions are recognised in evaluation reports. Ownership, management and communication of data and results is negotiated with Aboriginal and Torres Strait Islander people and processes are agreed to at an early stage.
Building capability and a culture of evaluation	Agencies develop processes for systematically assessing ethical risks associated with evaluation. Agency staff understand what is required to uphold ethical practice when evaluating policies and programs affecting Aboriginal and Torres Strait Islander people. Agencies provide opportunities for staff to improve their capabilities in addressing ethical issues during evaluation.

Principle:
Transparent

An underlying tenet of evaluation practice is knowledge transfer and the uptake of evaluation findings in future decision making. Where evaluation is undertaken and subsequently fails to be made available, this represents a waste of resources. Moreover, transparency in evaluation is an ethical requirement, as Aboriginal and Torres Strait Islander communities have the right to know about the effects of programs that affect them, and a right to the findings from evaluations they have participated in.

Margaret Kelaher et al. (2018, p. 25)

Transparency improves accountability and allows the lessons from evaluation to feed into decision making. It involves publishing evaluation findings, information on how evaluations were planned and conducted, and how the findings were used to improve policies and programs. Governance arrangements for the Strategy — including the priority‑setting and monitoring processes — also need to be transparent.

Publishing evaluation reports allows a range of users to learn from evaluation insights. Aboriginal and Torres Strait Islander people should be able to access evaluation results. Evaluators should work with Aboriginal and Torres Strait Islander people in the early stages of evaluation to determine how evaluation findings and data will be shared in ways that suit the needs of Aboriginal and Torres Strait Islander people, organisations and communities.

Transparency provides incentives for agencies to commission and conduct high‑quality evaluations that meet the needs of Aboriginal and Torres Strait Islander people. Publishing details of how evaluations were planned and conducted allows evaluation users to judge the credibility, ethics and rigour of evaluation techniques used and see how Aboriginal and Torres Strait Islander people were engaged in the evaluation process.

Where there are concerns about privacy, cultural or commercial sensitivity, agencies can publish a summary rather than a full evaluation report. However, agencies should ensure that the insights from such evaluations are shared as widely as possible. It will often be possible for relevant findings to be shared with Aboriginal and Torres Strait Islander people, organisations or communities even if a full evaluation report cannot be published. Evaluation summaries should include sufficient detail on evaluation methods and processes for users to understand how the results were obtained.

Agencies should publish a response to evaluation findings in a timely manner so that the community can see how evaluation findings are contributing to policy and program improvements.

Table 5. Transparent evaluation in practice

What to evaluate	Agencies publish evaluation forward work plans that detail the process and criteria for deciding what policies and programs are high priority for evaluation.
Evaluation planning, design and conduct	Agencies’ evaluation frameworks, strategies and policies are publicly available. Evaluation teams are selected through an open and transparent process. When commissioning evaluations, the process and criteria used to make tender decisions are transparent. Evaluation participants receive information about the purpose and conduct of the evaluation, what participating in the evaluation means for them, procedures for the collection and use of data and other information, before seeking their consent. Agencies and evaluators work with Aboriginal and Torres Strait Islander people early in the planning process to determine how evaluation findings and data will be shared in ways that suit the needs of Aboriginal and Torres Strait Islander people, organisations and communities. There are clear processes in place for facilitating access to, and release of, data (including for peer review, participants and communities) while also protecting privacy and confidentiality.
Reporting evaluation findings	Evaluation reports are published and easy to find. Evaluation findings are communicated in ways that meet the needs of various users, including the diverse needs of Aboriginal and Torres Strait Islander users. Appropriate attention is given to concerns about privacy, confidentiality and culturally sensitive information, while at the same time making evaluation findings public. Evaluation reports include clear documentation of methods, data, ethical practices and the limitations of an evaluation and its results. Evaluation reports outline how Aboriginal and Torres Strait Islander people were engaged during the evaluation process. The basis for evaluation findings is transparent, clearly argued and supported by evidence. Any conflicts of interest, and how such conflicts were managed during the evaluation, are disclosed. Agencies publish a response to evaluation findings. Agencies share evaluation reports with a central clearinghouse of evaluations.
Building capability and a culture of evaluation	Agencies have processes to routinely share the lessons from evaluation internally and with others. Where planned evaluation activities have not commenced or been completed on schedule, agencies explain why and provide revised plans.

Independent leadership and oversight

Australian Government agencies should continue to have primary responsibility for conducting evaluations under the Strategy. This emphasises the need for evaluation to be embedded in the policy cycle and places accountability for the conduct and use of evaluation with the agencies who have responsibility for developing and delivering policies and programs affecting Aboriginal and Torres Strait Islander people.

However, for the Strategy to make a difference to evaluation processes and practice, it will need centralised leadership, support, coordination and oversight. A new Office of Indigenous Policy Evaluation (OIPE), in partnership with an Indigenous Evaluation Council (with all Aboriginal and Torres Strait Islander members), should undertake this role (figure 3).

Figure 3. Roles and responsibilities

Joint functions of an OIPE and Indigenous Evaluation Council

An OIPE and Indigenous Evaluation Council should provide oversight and leadership for the Strategy by:

• monitoring the performance of Australian Government agencies against the Strategy and identifying good and/or innovative practice in the evaluation of policies and programs affecting Aboriginal and Torres Strait Islander people
• monitoring how effective the Strategy has been in encouraging high‑quality and useful evaluations and providing recommendations on how the implementation of the Strategy could be improved
• updating government‑wide evaluation priorities, including identifying areas for cross‑agency evaluations.

The OIPE and the Indigenous Evaluation Council should produce regular, public reports based on their monitoring of the Strategy’s implementation. The first report should be released two years after the Strategy is endorsed, then biennially.

Independence needs to be enshrined in the monitoring arrangements for the Strategy if they are to be viewed as credible.

Additional functions of an OIPE

An OIPE should provide stewardship for the Strategy and advise Australian Government agencies on what the Strategy’s principles mean in practice, including by reviewing and updating the Guide to Evaluation under the Indigenous Evaluation Strategy. The OIPE could also fulfil other central roles of the Strategy such as managing a new Indigenous Evaluation Clearinghouse (action 10) and working with agencies to build evaluation and cultural capability across the Australian Public Service (action 4).

The OIPE should sit within an existing independent statutory authority of the Australian Government.

The Productivity Commission currently has the role of monitoring and reviewing the performance of agencies against the Strategy.

Additional functions of the Indigenous Evaluation Council

As well as fulfilling its joint functions with the OIPE, an Indigenous Evaluation Council could also advise the OIPE on:

• evaluation planning, commissioning, conduct, publication and use, capability building and cultural safety
• the translation, dissemination and synthesis of evaluation findings
• engaging with Aboriginal and Torres Strait Islander people, communities and organisations to facilitate their input into the Strategy’s monitoring
• the OIPE’s stewardship of the Strategy.

Government-wide evaluation priorities

It is not practical or feasible to rigorously evaluate all policies and programs. Setting priorities for evaluation ensures the limited resources Australian Government agencies have are directed to where evaluation can best inform the development of policies and programs to improve the lives of Aboriginal and Torres Strait Islander people.

The OIPE, working in partnership with the Indigenous Evaluation Council, should establish and maintain a formal set of government‑wide evaluation priorities. Government‑wide evaluation priorities will help guide agencies’ decisions about what to evaluate. It is expected that evaluation priorities — including opportunities for cross-agency evaluations — will continue to be better defined over time with better information on:

• the implementation of policy and reform priorities included in the National Agreement
• the evaluation and policy landscape of agencies as they report more and better quality information through forward work plans and completed evaluation reports on key policies and programs
• other evaluation priorities identified during the monitoring process.

An interim set of priority areas has been established based on the National Agreement (figure 4). Under the proposed interim government‑wide evaluation priorities, the four priority reforms under the National Agreement will help identify what actions within (and across) the identified priority policy domains should be prioritised for evaluation.

Figure 4. Proposed interim government‑wide evaluation priority areas
Based on the National Agreement on Closing the Gap

The Strategy should prioritise evaluation of government efforts …
… delivered as part of a formal partnership and/or shared decision making arrangement (Priority 1) …	… that build up the Aboriginal and Torres Strait Islander community-controlled sector (Priority 2) …	… (particularly mainstream ones) that eliminate racism, embed and practice meaningful cultural safety, deliver services in partnership with Aboriginal and Torres Strait Islander people (Priority 3) …	… involved in data sharing arrangements (Priority 4) …
… and focused across (and within) the following policy domains:
Education Housing Health and wellbeing Land and water Justice Culture and languages Safety (Families, childre and youth) Employment (Economic development)

Source: adapted from Joint Council on Closing the Gap (2020).

Mainstream policies and programs

Many mainstream policies and programs affect Aboriginal and Torres Strait Islander people. The National Agreement highlights the importance of transforming mainstream institutions if outcomes are to change for Aboriginal and Torres Strait Islander people.

Evaluations of mainstream policies and programs should assess impacts on Aboriginal and Torres Strait Islander people. Findings from evaluations of these policies and programs will play an important role in measuring progress towards meeting the objectives of the National Agreement.

The approach taken to evaluating mainstream policies and programs under the Strategy will depend on the type of policy or program being evaluated, the number or proportion of Aboriginal and Torres Strait Islander people affected by the policy or program, and what is already known about the impacts of the policy or program on Aboriginal and Torres Strait Islander people.

Assessing the impact of mainstream policies and programs on Aboriginal and Torres Strait Islander people could include:

• examining policy or program outcomes for Aboriginal and Torres Strait Islander people and non‑Indigenous people to determine if there are different impacts
• focus groups or case studies to look more closely at particular issues of importance to Aboriginal and Torres Strait Islander people
• undertaking a targeted evaluation that looks specifically at the impact of the policy or program on Aboriginal and Torres Strait Islander people.

In line with the principle of centring Aboriginal and Torres Strait Islander people, perspectives, priorities and knowledges, evaluations of mainstream policies and programs should consider the impacts of the policy or program on Aboriginal and Torres Strait Islander people at the policy development/early evaluation planning stage so that the right evaluation questions are asked and data are collected.

Evaluators should engage with Aboriginal and Torres Strait Islander people on the relevant parts of a mainstream policy or program evaluation. This could mean having Aboriginal and Torres Strait Islander people on the evaluation team or steering committee and engaging with Aboriginal and Torres Strait Islander people to ensure a diversity of views are considered. Agencies should also consider the needs of Aboriginal and Torres Strait Islander people and communities when reporting on and using the findings of a mainstream evaluation.

Actions to support an evaluation culture

The Strategy would be most effective in an environment where evaluation is valued for accountability, learning and evidence‑based decision making. This is where agencies want to know about the performance of their policies and programs, are prepared to experiment, share learnings, and use evaluation results in policy making.

The Strategy would support a culture of evaluation through a series of actions taken by the Australian Government and its agencies to ensure that:

• the policies and programs that are most important to the lives of Aboriginal and Torres Strait Islander people are evaluated through a systematic and transparent process of setting priorities at the agency  level
• evaluation is planned early
• agencies build their capability — including cultural capability, skills and data — to support high‑quality evaluation practices
• evaluation findings are accessible and used to improve policies and programs.

Agencies’ progress implementing the actions should be monitored by the OIPE in partnership with the Indigenous Evaluation Council.

Ensuring the most important policies and programs are evaluated

Central to achieving the Strategy’s objective of improving the lives of Aboriginal and Torres Strait Islander people, and ensuring evaluations are more useful, is to conduct evaluations in the areas that are most important to Aboriginal and Torres Strait Islander people and where there are gaps in the evidence base..

Action 1: Agencies should systematically identify evaluation priorities

To identify evaluation priorities at an agency level, Australian Government agencies should assess all new policies and programs, and undertake a stocktake of existing policies and programs, to determine which contribute to the government‑wide evaluation priorities identified under the Strategy.

Agencies should adopt a criteria‑based priority setting process to determine the extent to which the policies and programs that are expected to impact the lives of Aboriginal and Torres Strait Islander people (for example, that contribute to government‑wide evaluation priorities) should be prioritised for evaluation. Priorities should be determined based on policy and program impact, risk profile, strategic significance and expenditure, and Aboriginal and Torres Strait Islander people’s priorities.

Action 2: Departments should develop and publish Three Year Evaluation Forward Work Plans

Australian Government departments should develop and publish, on an annual basis, a rolling Three Year Evaluation Forward Work Plan, which details:

• policies and programs within their portfolio that contribute to government‑wide evaluation priorities aimed at improving the lives of Aboriginal and Torres Strait Islander people
• how the department identified high priority policies and programs
• a plan for how and when over the next three years the department’s identified policies and programs will be evaluated (or how they will become ready for evaluation)
• how Aboriginal and Torres Strait Islander people, perspectives, priorities and knowledges were centred as part of the prioritisation process.

The plan should be published on the department’s website, and provided to the OIPE so it can review the department’s performance against the Strategy. There should also be mechanisms in place for the community to provide feedback on the adequacy of these plans. This will strengthen the incentives for departments to ensure the quality of their prioritisation process.

Departments should decide whether agencies within their portfolio should publish their own forward work plans.

Planning early for evaluation

Evaluations should be planned early — ideally during policy and program development — to ensure that the right questions are asked, useful baseline and monitoring data are collected, opportunities for engaging Aboriginal and Torres Strait Islander people in the evaluation are identified, and adequate resources are available to undertake high‑quality evaluation.

Action 3: Agencies should prepare an Aboriginal and Torres Strait Islander Impact Assessment and Evaluation Plan for new policy or program proposals

To embed evaluation planning and engagement into the early stages of the policy design process, agencies should prepare an Aboriginal and Torres Strait Islander Impact Assessment and Evaluation Plan for new policy or program proposals. The Impact Assessment and Evaluation Plan should detail:

• the Aboriginal and Torres Strait Islander people, organisations and communities that would be affected by the proposed initiative
• a proposed engagement plan (including timeframes and cultural safety considerations)
• a proposed approach (including the scale of evaluation required)
• the data required to assess the policy’s impact and how they would be collected
• an estimated evaluation budget.

Building capability to conduct and manage high-quality evaluations

Good data and access to evaluators with technical and cultural capability are essential to support good evaluation practice.

Action 4: Agencies, supported by an OIPE, should ensure they have access to the skills required to undertake or commission evaluations consistent with the Strategy

To undertake high‑quality evaluations in line with the Strategy, agencies need to ensure that those designing, commissioning, managing and conducting evaluations that affect Aboriginal and Torres Strait Islander people have the capability — including the skills, experience and understanding — to comply with the Strategy.

The OIPE should provide support and resources that agencies can access to build and maintain their evaluation capability. The supports should include:

• providing training for commissioning, conducting and using evaluations of policies and programs that affect Aboriginal and Torres Strait Islander people
• facilitating an Australian Public Service‑wide community of practice for people who are involved in evaluating such policies and programs
• establishing processes through which evaluators can seek secondments or other opportunities for mobility, to broaden or deepen their experience evaluating policies and programs affecting Aboriginal and Torres Strait Islander people
• developing a strategy to build a cohort of Aboriginal and Torres Strait Islander evaluators within the Australian Public Service which may include recruitment, structured training, networking, mentoring and secondment opportunities.

Action 5: Agencies should ensure that they have access to, or are able to collect, the data they need to effectively undertake evaluations under the Strategy

Good data are essential for high‑quality evaluation. Agencies should plan early to identify data needs for an evaluation. Data planning should consider:

• what data are needed to answer evaluation questions
• what data are needed to produce credible results (including the use of quantitative and/or qualitative data, and sampling methods)
• what existing data are available and suitable for the evaluation and what additional data should be collected

To ensure data used for evaluation are relevant to Aboriginal and Torres Strait Islander people and collected and managed in a culturally safe manner, agencies should develop and/or use:

• appropriate Indigenous data governance arrangements, including partnering with Aboriginal and Torres Strait Islander people in the development, collection, use and management of data
• appropriate data standards
• appropriate data sharing and release protocols
• ethical and culturally-safe data collection processes.

Action 6: Agencies should strengthen and support Aboriginal and Torres Strait Islander people’s capability to engage, partner and lead in evaluation

Agencies will only be able to effectively centre Aboriginal and Torres Strait Islander people, perspectives, priorities and knowledges in evaluation if Aboriginal and Torres Strait Islander people, organisations and communities have the time and resources to engage, partner or lead in evaluation. While many Aboriginal and Torres Strait Islander people, organisations and communities are well placed to do this, agencies can support and strengthen Aboriginal and Torres Strait Islander capability by:

• developing strong and sustainable relationships with Aboriginal and Torres Strait Islander people that are responsive to Aboriginal and Torres Strait Islander perspectives, priorities and knowledges
• providing the resources and time needed for Aboriginal and Torres Strait Islander people and organisations to effectively engage in evaluations and evaluation processes
• working with Aboriginal and Torres Strait Islander people, organisations and communities to strengthen evaluation capability, including by engaging Aboriginal and Torres Strait Islander evaluators and community researchers and providing training and development opportunities
• building agencies’ cultural capability — both at an organisational and individual level — to work effectively with Aboriginal and Torres Strait Islander people.

Enhancing the use of evaluations

The use of evaluation evidence will be improved by having high‑quality, timely and credible evaluations that answer questions that are relevant to policy makers and Aboriginal and Torres Strait Islander people. Policy makers and other users of evaluation evidence also need to be able to access evaluation findings in formats that are useful.

Action 7: All evaluation reports should be published

All evaluations of policies and programs affecting Aboriginal and Torres Strait Islander people undertaken or commissioned by Australian Government agencies should be published and made available on agencies’ websites within three months of being completed, unless there is a compelling reason to do otherwise (such as where publishing the report will compromise confidentiality or privacy, or where there is culturally sensitive information).

Where there are reasons for not publishing an evaluation report, a summary of the findings of the evaluation should be published. The summary report should include an explanation for why the full evaluation report could not be published. Where the concerns only apply to part of the evaluation, the rest of the evaluation should be made public.

Published evaluation reports and summaries should be provided to the Indigenous Evaluation Clearinghouse (action 10).

Action 8: Agencies should publish accessible evaluation report summaries

All evaluations of policies and programs affecting Aboriginal and Torres Strait Islander people undertaken or commissioned by Australian Government agencies should include a clear and concise summary of the evaluation report. Evaluation summaries should document how the planning, commissioning and conduct of the evaluation adhered to the principles of the Strategy.

Other ways of sharing evaluation findings, such as oral feedback or information sessions, should be agreed with Aboriginal and Torres Strait Islander people and communities at the evaluation planning stage.

Action 9: Agencies should publish a management response to evaluation findings

Agencies should publish a timely management response to evaluation findings. Management responses can be published as part of the evaluation report or separately, and should include an explanation about what agencies have learned, what they have changed in response to the findings, and any further action they intend to take.

Action 10: The Australian Government should establish an Indigenous Evaluation Clearinghouse

Good knowledge management is critical for the systematic use of evaluation findings across Australian Government agencies, and for the dissemination of evaluation findings more broadly to inform decision making, share lessons from evaluation and increase accountability.

The Australian Government should establish and maintain an online evaluation evidence clearinghouse — the Indigenous Evaluation Clearinghouse — as a repository for reports of evaluations of policies and programs affecting Aboriginal and Torres Strait Islander people.

The Clearinghouse should also be responsible for coordinating the synthesis of new evaluation evidence into the existing evidence base, and for translating the knowledge into forms that are accessible to different audiences.

Monitoring and reviewing the Strategy

Behavioural change only comes when people have skin in the game through some measure of accountability or responsibility for the outcomes of their actions. There is no shortage of goodwill in departments but that is not enough. The public service needs a reason to move beyond what it is doing now, to consider and include evaluation in a more comprehensive sense, which will only come with greater incentivisation.

Maggie Walter (2019, p. 1)

To change the culture around how evaluations of policies and programs affecting Aboriginal and Torres Strait Islander people are undertaken, agencies need to be accountable for implementing the Strategy’s principles and actions. The Strategy itself should also be subject to regular monitoring and review to assess how well it is meeting its objectives.

Monitoring agencies’ progress in implementing the Strategy

An OIPE would monitor and report on the performance of agencies against the Strategy and assess how effective the Strategy is in encouraging high‑quality and useful evaluations. The monitoring process will provide incentives for agencies to implement the Strategy’s principles and actions and allow good practice to be identified and lessons shared.

Changing the culture of evaluation will take time. Agencies will be starting with different levels of skills and experience in evaluation and engagement with Aboriginal and Torres Strait Islander people. Agencies should plan a pathway to better evaluation by assessing their current practices and looking for areas where improvements can be made.

To assist agencies to implement the Strategy’s principles, a ‘progression pathway’ has been developed for each principle. The progression pathway identifies what unsatisfactory, developing, mature and leading practice looks like for each of the Strategy’s principles for the various stages of an evaluation.

The progression pathway provides clear and transparent benchmarks for agencies to look to as they seek to improve their evaluation practices. It is also a tool that will be used to assess agencies’ conduct as part of the monitoring process.

Progression pathways for each of the Strategy’s principles are available on the Commission’s website. The OIPE and the Indigenous Evaluation Council should continue to review and develop the progression pathways as part of the monitoring process.

The OIPE and the Indigenous Evaluation Council should produce regular, public reports that outline their findings and recommendations based on their monitoring of the Strategy’s implementation. The first report should be released two years after the Strategy is endorsed, then biennially thereafter. These reports should:

• assess the extent to which Australian Government agencies have complied with the Strategy and have implemented the Strategy effectively
• identify good and/or innovative practice in the evaluation of policies and programs affecting Aboriginal and Torres Strait Islander people
• assess the extent to which the Strategy has been effective in encouraging high‑quality and useful evaluations
• formalise evaluation priorities, including identifying areas for cross‑agency evaluations
• provide recommendations on how the implementation of the Strategy could be improved.

Independent review of the Strategy

The Australian Government should commission an independent review of the Strategy led by an Aboriginal or Torres Strait Islander person five years after it is first endorsed.

Reviewing the Strategy is an opportunity to assess:

• whether the Strategy’s principles remain fit‑for‑purpose, and if not, the ways they should be updated
• the extent to which the Strategy has been effective in encouraging better and more useful evaluations of policies and programs affecting Aboriginal and Torres Strait Islander people, including the effectiveness of the Strategy’s actions
• the performance of those tasked with overseeing the Strategy and whether changes to oversight arrangements are required.

Implementation timeline

Implementing the Strategy’s principles and actions and putting in place the governance arrangements to support the Strategy should be undertaken within two years of the Australian Government endorsing the Strategy (table 6).

The initial focus (within the first six months) should be on establishing the governance arrangements and tools for implementing the Strategy. Agencies should also start developing their data and capability for evaluation, noting that these will be ongoing functions.

Within the first year, agencies should start planning for and conducting evaluations in line with the Strategy’s principles and actions. The focus should then move to sharing evaluation findings and lessons from implementation and establishing transparent accountability mechanisms. The independent review of the Strategy should take place five years after the Strategy is endorsed.

Table 6. Proposed implementation timeline for the Strategy

Actions by agencies	Central functions
Within first six months
Agencies develop prioritisation tools and undertake stocktake (Action 1) Agencies begin to develop capability and data for evaluation, including measures to strengthen and support capability for Aboriginal and Torres Strait Islander people, organisations and communities (Actions 4, 5 and 6)	OIPE and Indigenous Evaluation Council established OIPE and Indigenous Evaluation Council develop guidance on Aboriginal and Torres Strait Islander Impact Assessment and Evaluation Plans (Action 3)
Within one year
Agencies adopt the Strategy’s principles for evaluations of policies and programs affecting Aboriginal and Torres Strait Islander people Agencies begin to use prioritisation tools for evaluation planning; departments develop and publish forward work plans (Actions 1 and 2) Agencies prepare an Aboriginal and Torres Strait Islander Impact Assessment and Evaluation Plan for all new policy or program proposals (Action 3) Agencies begin publishing all evaluation reports (Action 7)	OIPE provides guidance to agencies on implementing the Strategy and support for capability building (Actions 4 and 6) OIPE and Indigenous Evaluation Council monitor progress implementing the Strategy
Within two years
Departments revise and publish annual updates to work plans (Actions 1 and 2) Agencies begin to publish evaluation summaries and management responses (Actions 8 and 9) Agencies begin to share evaluation reports with the Indigenous Evaluation Clearinghouse (Action 10)	Indigenous Evaluation Clearinghouse established (Action 10) Government-wide evaluation priorities reviewed and formalised First State of Evaluation report published (and every two years subsequently)

References

• AES (Australian Evaluation Society) 2020, Submission to the Productivity Commission Indigenous Evaluation Strategy project, sub. DR174.
• AIATSIS (Australian Institute of Aboriginal and Torres Strait Islander Studies) 2018, ‘Respect and honour’ will underpin ethical research guidelines review, Media Release.
  —— (Australian Institute of Aboriginal and Torres Strait Islander Studies) 2019, Submission to the Productivity Commission Indigenous Evaluation Strategy project, sub. 72.
  —— (Australian Institute of Aboriginal and Torres Strait Islander Studies) 2020, AIATSIS Code of Ethics for Aboriginal and Torres Strait Islander Research, Canberra.
• Bond, C., Brady, K., Hassall, K., Macoun, A., Mukandi, B., Singh, D., Staines, Z. and Strakosch, E. 2019, Submission to the Productivity Commission Indigenous Evaluation Strategy project, sub. 40.
• Empowered Communities 2019, Submission to the Productivity Commission Indigenous Evaluation Strategy project, sub. 41.
• Frydenberg, J. 2019, Indigenous Evaluation Strategy: Letter of Direction.
• Joint Council on Closing the Gap 2020, National Agreement on Closing the Gap, Canberra.
• Kelaher, M., Luke, J., Ferdinand, A., Chamravi, D., Ewen, S. and Paradies, Y. 2018, An Evaluation Framework to Improve Aboriginal and Torres Strait Islander Health, The Lowitja Institute, Melbourne.
• NHMRC, ARC and UA (National Health and Medical Research Council, Australian Research Council and Universities Australia) 2018, National Statement on Ethical Conduct in Human Research, National Health and Medical Research Council, Canberra.
  —— (National Health and Medical Research Council) 2018, Ethical Conduct in Research with Aboriginal and Torres Strait Islander Peoples and Communities: Guidelines for Researchers and Stakeholders, National Health and Medical Research Council, Canberra.
• Victorian Aboriginal Community Controlled Health Organisation 2019, Submission to the Productivity Commission Indigenous Evaluation Strategy project, sub. 44.
• Walter, M. 2019, Oral submission to the Productivity Commission Indigenous Evaluation Strategy project, sub. 112.
• Yulang Indigenous Evaluation 2020, Submission to the Productivity Commission Indigenous Evaluation Strategy project, sub. DR150.

About the artist

Luke Penrith’s ancestry is connected through the Wiradjuri, Wotjobaluk, the Yuin and the Gumbaynggirr Nation. His passion is mentoring and nurturing Indigenous Australian job seekers and supporting Aboriginal Businesses. Lore, culture and heritage are paramount to Luke. His art reflects what he sees, hears and can smell and touch; he is a modern contemporary Aboriginal Artist living in Brungle NSW. Luke’s bloodlines are connected through the rivers, the mountains, the coastline and the plains.

The artwork used in this publication is adapted from River of Knowledge by Luke Penrith.

Copyright and publication detail

The Productivity Commission acknowledges the Traditional Owners of Country throughout Australia and their continuing connection to land, waters and community. We pay our respects to their Cultures, Country and Elders past and present.

© Commonwealth of Australia 2020

Read more about copyright

Except for the Commonwealth Coat of Arms and content supplied by third parties, this copyright work is licensed under a Creative Commons Attribution 3.0 Australia licence. To view a copy of this licence, visit creativecommons.org/licenses/by/3.0/au. In essence, you are free to copy, communicate and adapt the work, as long as you attribute the work to the Productivity Commission (but not in any way that suggests the Commission endorses you or your use) and abide by the other licence terms.

Use of the Commonwealth Coat of Arms

Terms of use for the Coat of Arms are available from the Department of the Prime Minister and Cabinet’s website: www.pmc.gov.au/government/commonwealth-coat-arms

Third party copyright

Wherever a third party holds copyright in this material, the copyright remains with that party. Their permission may be required to use the material, please contact them directly.

Attribution

This work should be attributed as follows, Source: Productivity Commission, Indigenous Evaluation Strategy

If you have adapted, modified or transformed this work in anyway, please use the following, Source: based on Productivity Commission data, Indigenous Evaluation Strategy

Publications enquiries

Strategic Communications and Engagement, phone: (03) 9653 2244 or email: communications@pc.gov.au

The Productivity Commission

The Productivity Commission is the Australian Government’s independent research and advisory body on a range of economic, social and environmental issues affecting the welfare of Australians. Its role, expressed most simply, is to help governments make better policies, in the long term interest of the Australian community.

The Commission’s independence is underpinned by an Act of Parliament. Its processes and outputs are open to public scrutiny and are driven by concern for the wellbeing of the community as a whole.

Further information on the Productivity Commission can be obtained from the Commission’s website (www.pc.gov.au).

Strategy homeEnd