Robodebt: losing quietly to avoid losing publicly

TL;DR. From July 2015 to November 2019 the Australian government raised welfare debts by comparing annual tax income against fortnightly income reported to the welfare agency. Where they disagreed, it divided the annual figure into equal fortnights and treated the difference as an overpayment. For anyone with irregular work, that produces a debt that does not exist. Around 470,000 unlawful debts were identified; roughly $751 million was recovered from about 381,000 people and repaid under a settlement totalling $1.8 billion. A Royal Commission reported in July 2023 that the scheme was neither fair nor legal. And the detail that matters most: the administrative tribunal had been striking down income averaging for years, and the department did not appeal those rulings. Losing individual cases quietly meant no binding precedent was ever created, and the scheme continued.

---

Status: established. Primary source: the Report of the Royal Commission into the Robodebt Scheme, Commissioner Catherine Holmes AC SC, presented 7 July 2023, together with the Federal Court's approval of the class action settlement in Prygodicz v Commonwealth. Characterisations in quotation marks are the Commissioner's or the Court's.

---

Australian welfare payments are calculated on actual fortnightly earnings. That is what the legislation specifies, because a payment is meant to reflect what someone earned in the fortnight it covers.

The tax office holds annual income totals. The two datasets do not align, and they were never designed to.

The scheme resolved that by dividing the annual figure into 26 equal fortnights and comparing the result against what the recipient had reported. Where the assumed fortnight exceeded the reported one, the system raised a debt.

For anyone paid the same amount every fortnight of the year, this works. For anyone else it invents money. Seasonal work, casual shifts, three months employed and nine months not: all produce an average that never matched any actual fortnight, and a debt calculated from a fortnight that never happened.

Justice Murphy put it plainly in approving the settlement: where a recipient does not earn a constant fortnightly wage, the assumed income based on averaging is unlikely to match the actual income, and it should have been plain that the system might indicate an overpayment when none existed.

The burden was reversed

Once a debt was raised, the recipient had to disprove it.

That meant producing payslips and bank statements going back years, from employers who may no longer exist, for periods when the person may have been homeless, ill, or simply not keeping records. The agency held the data that generated the claim. The person receiving the letter held nothing.

This is the same structure as Horizon, where the accused had to rebut a system whose evidence sat inside the prosecutor's building, and the same as the Dutch benefits scandal, where a flagged family had to prove a negative to an agency with no discretion to listen.

Three countries, three systems, one design: the institution asserts, the individual disproves, and the evidence lives with the institution.

What the Commission found

The Royal Commission reported on 7 July 2023: 990 pages, 57 recommendations, and a sealed section referring individuals for civil and criminal consideration.

Commissioner Holmes described the scheme as a crude and cruel mechanism, neither fair nor legal, which made many people feel like criminals, and wrote that people were traumatised on the off chance they might owe money.

The report found the use of income averaging inconsistent with social security legislation.

The Commission heard evidence from families of people who died, including mothers of children who died by suicide after receiving debt notices. The report does not establish a total, and figures circulating publicly are not Commission findings. What the record contains is testimony, and it is in the report in the families' own words.

Justice Murphy, approving the class action settlement, called the scheme a shameful chapter in the administration of the Commonwealth social security system and a massive failure of public administration.

The finding: they lost every appeal and never appealed

Here is the mechanism that kept it running for four years, and it is the most transferable thing in the case.

The Administrative Appeals Tribunal repeatedly struck down income averaging. Recipients who challenged their debts and reached the tribunal won.

The department did not appeal those decisions.

Losing at first-tier tribunal has no precedential effect. The individual gets their debt cancelled and nothing else changes. Appealing would have produced a binding ruling on whether the method was lawful, and the department declined to seek one.

So the scheme absorbed a stream of individual losses in order to avoid a single ruling that would have stopped it. Every person who fought and won had their debt quietly cancelled. Every person who did not fight, which was most of them, paid.

That is not a technology failure and it is not a mistake. It is a legal strategy, and it works precisely because the cost of challenge falls on individuals while the benefit of a precedent would be shared by everyone.

Warnings existed throughout. A departmental lawyer had flagged the method as likely unlawful in 2014, before the scheme began. External advice from a major law firm in August 2018 said the same. The Solicitor-General's advice, prompted by a legal challenge in 2019, found it unlawful, and the scheme stopped in November of that year.

Four years, with the answer available at the start.

There is no AI in this either

Like Horizon, and worth stating for the same reason.

The mechanism is division. Annual income divided by 26. There is no model, no training data, no learned parameters, not even a complicated rule set. A spreadsheet does it.

What made it catastrophic was automation of scale plus reversal of proof plus a strategy that prevented review. Take any of the three away and it fails: a manual process could not have raised 470,000 debts, a normal burden of proof would have collapsed most of them, and one appeal would have ended it years earlier.

None of those three requires artificial intelligence, and adding it would change nothing except the volume.

Four things this establishes

A statistical assumption can be a legal error. Averaging is a legitimate technique with a stated assumption: that the underlying distribution is roughly even. Applied to a population selected for irregular income, the assumption is false by construction, and applying it against a statute requiring actual fortnightly earnings made the output unlawful rather than merely inaccurate.

Reversal of proof multiplies every other flaw. A system with a modest error rate and a normal burden produces disputes. The same system with the burden reversed produces payments, because most people cannot discharge it.

Non-appeal is a governance failure mode with no technical component. Where an institution can absorb individual losses to avoid a precedent, tribunal review stops functioning as a check.

And warnings are not a safeguard unless something acts on them. The scheme was flagged as likely unlawful before it started, again in 2018, and definitively in 2019. The existence of internal legal concern changed nothing about its operation for four years.

What it does not establish

A death toll. The Commission heard testimony from bereaved families and recorded it. It did not establish a causal total, and the numbers circulating are not findings.

That averaging is always improper. It is a standard method. The failure was applying it to a population defined by irregular income, against a statute requiring actual figures, and treating the output as a debt rather than as a reason to ask.

That the problem ended in 2019. A separate practice, income apportionment, affected files going back decades and was found unlawful by the Commonwealth Ombudsman in 2023, with remediation contested through the courts since.

And who is individually accountable. The sealed section referred individuals onward, and outcomes from those referrals are not public.

What is unresolved

Whether the recommendations are implemented. 57 were made. Implementation is tracked and incomplete.

Whether automated suspensions continue. Legal services have noted that decisions leaving people without income remain automated, and that debt letters remain opaque about how a figure was reached.

The scope of pre-2015 debts. Averaging-type practices are documented well before the scheme, and the government's position has been that identifying those affected is impractical.

And whether the non-appeal strategy is addressed anywhere. No reform obviously prevents an agency from declining to appeal in order to avoid a precedent.

The counter-argument

Calling this an algorithmic harm overstates the software's role. The unlawfulness was in the policy decision to use averaging against a statute requiring actual income. That decision was made by people who received advice saying it was unlawful. The automation determined how many people were affected, not whether the scheme was wrong.

Some debts were real. A proportion of the population targeted had genuinely been overpaid. The scheme's failure was that it could not distinguish them, and remediation returned money to people who did owe some of it, which is a cost worth acknowledging rather than eliding.

The reversal of proof was not novel. Requiring a person to substantiate their own income against a departmental record is longstanding practice in welfare administration and tax. What changed was the volume and the absence of any human assessment before the letter went out.

And non-appeal is ordinary litigation strategy. Parties routinely decline to appeal losses they expect to lose again. Describing it as a deliberate evasion of precedent assumes an intent the Commission examined and which remains contested for particular individuals.

The short version

From 2015 to 2019 Australia raised welfare debts by dividing annual tax income into 26 equal fortnights and comparing that against what recipients reported. Social security law requires actual fortnightly earnings. For anyone with irregular work, the average matched no real fortnight, and the difference became a debt that did not exist.

Once raised, the recipient had to disprove it, producing years of payslips and bank statements. The agency held the data. The person held nothing.

Around 470,000 unlawful debts. Roughly $751 million recovered from about 381,000 people, repaid under a $1.8 billion settlement. A Royal Commission reported in July 2023 that the scheme was neither fair nor legal, that it made many people feel like criminals, and that people were traumatised on the off chance they might owe money.

And the detail that kept it alive for four years: the administrative tribunal repeatedly struck down income averaging, and the department did not appeal. A first-tier loss sets no precedent. Appealing would have produced a binding ruling on lawfulness, and none was sought. The scheme absorbed a steady stream of individual losses precisely to avoid the one ruling that would have ended it. Those who fought had their debts quietly cancelled. Those who did not, which was most, paid.

There is no AI in this. The mechanism is division. What made it catastrophic was scale, reversed proof, and a strategy that prevented review, and none of those three needs a model. Three countries now in this record, Australia, the Netherlands and the United Kingdom, have produced the same failure with three different technologies, one of which is arithmetic.

---

This article records a case involving deaths. If any of it is affecting you personally, talking to someone you trust or a professional is worth doing, and I can help find appropriate resources.

Common questions

What was Robodebt? An Australian government scheme running from July 2015 to November 2019 that raised welfare overpayment debts automatically. It compared annual income data held by the tax office against fortnightly income reported to the welfare agency, and where the two disagreed it divided the annual figure into 26 equal fortnights and treated the difference as an overpayment. Around 470,000 debts raised this way were later identified as unlawful.

Why was income averaging unlawful? Because social security legislation calculates entitlement on actual fortnightly earnings, not on an assumed even distribution of annual income. The Royal Commission found the use of averaging inconsistent with that legislation. Justice Murphy, approving the settlement, noted that for anyone not earning a constant fortnightly wage the assumed figure was unlikely to match the actual one, and that it should have been plain the system might show an overpayment where none existed.

What did the Royal Commission conclude? Commissioner Catherine Holmes reported on 7 July 2023 in a 990-page report with 57 recommendations and a sealed section referring individuals for civil and criminal consideration. She described the scheme as a crude and cruel mechanism, neither fair nor legal, which made many people feel like criminals, and wrote that people were traumatised on the off chance they might owe money.

Why does the non-appeal detail matter so much? Because it explains how a scheme found unlawful could run for four years while losing case after case. The administrative tribunal repeatedly struck down income averaging, and the department did not appeal those decisions. A first-tier tribunal loss cancels one person's debt and sets no precedent; an appeal would have produced a binding ruling on whether the method was lawful. By absorbing individual losses rather than seeking that ruling, the scheme continued. The cost of challenging fell on individuals while the benefit of a precedent would have been shared by everyone.

How much money was involved? Approximately $751 million was recovered from about 381,000 people, and the class action settlement totalled $1.8 billion including debts wiped and interest. Most people in the class action received nominal payments representing what they had paid plus lost interest, so the headline figure reflects the scale of the scheme rather than the depth of the redress.

How many people died? The Commission heard testimony from families of people who died, including mothers of children who died by suicide after receiving debt notices, and recorded it in the families' own words. The report does not establish a causal total, and figures circulating publicly are not Commission findings. That the record contains testimony rather than a count is itself worth stating precisely.

Is there any AI in this case? No. The calculation is division: annual income divided into 26 fortnights. There is no model, no training data, no learned parameters. What made it catastrophic was the combination of automated scale, a reversed burden of proof, and a litigation strategy that prevented review. None of those requires machine learning, and adding it would change only the volume.

What should be taken from it by anyone building automated decision systems? That a statistical assumption becomes a legal problem when the population violates it, that reversing the burden of proof multiplies every other flaw because most people cannot discharge it, that appeal mechanisms stop working as a check when an institution can absorb individual losses to avoid a precedent, and that internal legal warnings are not a safeguard unless something in the process is required to act on them.