The method will continue to be reviewed and refined over time.

Overview of key steps and statistical techniques

The method has four key steps, with step 3 currently relevant at the national level only.

• Step 1: Assess if the trend in the data represents a ‘no change’ or ‘change’ scenario (that is, whether there is change or not from the baseline year to the current year)
• Step 2: If change is identified, assess if the change is an ‘improvement’ or a ‘worsening’
• Step 3: If the change is an improvement at the national level, assess if the improvement is sufficient to ensure the target is ‘on track’ or ‘not on track’ to be met in the future target year
• Step 4: Assess the reliability of (that is, our confidence in) the derived assessments of progress.

Within these steps three key statistical techniques are used:

1. Regression analysis, with ordinary least squares regression used. This analysis simply calculates a line that will best fit the data and then describes the line using an equation. For example, the equation for a straight-line is:

   𝑥_time = α + β time

   where:

   • 𝑥_time and time are data-pairs at baseline, Time 1, Time 2 and so on. 𝑥_time is the actual data for the indicator and time is the predictor variable (e.g. year)
   • α and β are estimated using regression analysis: α is the intercept and β is the ‘slope’ or the annual change in the indicator per year.
2. Likelihood ratio test comparing nested regression models to determine which model best fits the data (linear models only have been used to date) and assess if the trend in the data represents a ‘no change’ versus a ‘change’ scenario.
3. Confidence intervals around the regression line and significance testing of the likelihood ratio test to understand the reliability of the assessments (once five or more datapoints are available).

Steps for assessing progress

Step 1: Assess if the data reflects a ‘no change’ or ‘change’ scenario.

A likelihood ratio (LR) test is used to test the ‘no change’ scenario comparing nested regression models¹ in two stages:

1. Perform the following regression models:
   • flat line that represents the ‘no change’ scenario. This is an intercept-only model that restricts the parameter β to be zero.
   • full model that best fits the data:
     • the linear model shown above (i.e. intercept model nested with a fitted model that includes the time parameter) is the default for all targets
     • for targets with five or more data points, we will evaluate whether the linear regression model is the best fit, by testing it against alternative non‑linear models (quadratic, exponential). No targets have moved to using a non‑linear model.
2. Once a full model is chosen, a ‘likelihood ratio test’ is performed to determine whether the flat line or the full model is the best fit for the data. The likelihood ratio test provides a statistical basis to reject the null hypothesis H₀ in favour of the alternative hypothesis H_a :
   • H₀ : The full model (linear, quadratic or exponential) and the nested (intercept-only) model fit the data equally well.
   • H_a : The full model fits the data significantly better

   The LR test statistic is expressed as a difference between the log-likelihoods of the two models:

   LR (chi2(k − 1)) = 2(L₁ − L₀)

   where:

   • L₁ is the log-likelihood value for the full model.
   • L₀ is the log-likelihood value for the intercept-only model.

   The LR test has a chi2(k-1) distribution, where (k-1) is the number of parameters estimated (other than the intercept).

   The p-value of the test is used to reject the null hypothesis and conclude that the full model better fits the data. If the p-value of the likelihood ratio test is:

   • … greater than 0.5, the assessment is no change
   • … less than or equal to 0.5 the assessment is change.

Step 2: If change is identified, assess if the change is an improvement or a worsening

Once ‘no change’ is ruled out, the values on the regression line for the baseline year to the current or target year are compared. A change is identified as an improvement or worsening using the conditions in table 1.

Step 3: If the change is an improvement at the national level, assess if the improvement is sufficient to ensure the target is ‘on track’ or ‘not on track’ to be met in the future target year

The regression trend line is projected out to the target year and compared with the value of the target in the target year, according to the conditions in table 2.

Step 4: Assess the reliability of (that is, our confidence in) the derived assessments of progress.

Step 4A: For targets, with less than five data points note that the assessments of progress should be used with caution

The following note will be added when assessments of progress are provided.

• Note: These assessments of progress should be used with caution as they are based on a limited number of data points.

Step 4B: For targets with five or more data points², the level of confidence in the assessment will be provided

For the state and territory assessments, the level of confidence (High or Low) is based on table 3 conditions, using the likelihood ratio test from step 1 (point 2) that compares the likelihood of the full model to the no change model. These conditions will also be used for determining the level of confidence for the national assessment for no change or worsening:

Confidence intervals around the regression trend line estimates are provided at the state, territory and national levels. However, these confidence intervals are only used for the assessments of progress at the national level to assign a level of confidence for assessments that are an improvement (on track and not on track). The conditions for assigning a level of confidence are in table 4.

a All conditions are based on comparisons made at the target year.

Footnotes

1. Nested models are those where the full model can be transformed into the simpler model by imposing constraints on the parameters of the full model. Locate Footnote 1 above
2. To assign a level of confidence five valid data points are needed. This means for non-annual infrequent data sets levels of confidence may not be able to be assigned even up to the final target year. For the purposes of statistical regression, valid data points are numeric only, with ‘not applicable’, ‘not available’, or ‘not published’ considered invalid and are treated as missing values. Locate Footnote 2 above