How Canberra Stacks Up Against the World on Duplicate Image Replacement in Government Archives

Canberra's federal public service is sitting on an estimated backlog of millions of duplicate digital images spread across agency servers, shared drives and legacy content management systems — and the push to identify, replace or cull those files is becoming one of the quieter but more consequential digital housekeeping tasks inside the Australian Public Service in 2026.

The issue matters now because the Albanese government's whole-of-government digital records framework, updated in early 2025, set a compliance deadline of 30 June 2027 for agencies to complete foundational data deduplication audits. That timeline is putting pressure on departments concentrated in the Civic and Barton precincts, where agencies such as the Department of Finance and the Australian Taxation Office maintain some of the largest internal image repositories in the southern hemisphere.

What Canberra Is Actually Doing

The National Archives of Australia, based on Queen Victoria Terrace in Parkes, has been piloting an automated duplicate-detection program since February this year. The program uses perceptual hashing — a technique that compares images by their visual fingerprint rather than file name or metadata — to flag near-identical records before they are migrated into the Archives' new digital preservation environment. The Australian National University's 3A Institute in Acton has been providing technical advisory support to that pilot, according to publicly available procurement records on AusTender.

The ACT government's own records management unit, housed within the Chief Minister, Treasury and Cabinet Directorate on London Circuit, is running a parallel but smaller-scale exercise focused on planning and land-use images held by the Environment, Planning and Sustainable Development Directorate. Those records include aerial survey images of growth suburbs like Gungahlin and Belconnen — areas where rapid development since 2015 has generated thousands of overlapping cadastral photographs that planners say have complicated land title searches.

Neither program is moving fast. Public servants familiar with large-scale data remediation projects — speaking generally, not about these specific programs — typically note that deduplication work stalls when agencies disagree on which version of a duplicated image is the authoritative record. That definitional problem is not unique to Canberra.

How Wellington, Ottawa and Singapore Compare

Wellington's Archives New Zealand completed a comparable deduplication exercise across six core ministries in 2023, taking 14 months and processing roughly 4.2 million image files, according to a published case study released by the New Zealand government in March 2024. The New Zealand approach leaned heavily on a centralised cloud platform — Microsoft Azure Government — which allowed cross-agency deduplication to happen in a single environment rather than agency by agency.

Ottawa's approach under Library and Archives Canada has been more fragmented. A 2025 report from the Canadian government's Office of the Auditor General noted ongoing inconsistency in how federal departments handled duplicate digital assets, with no single technical standard mandated across the public service.

Singapore's Whole-of-Government digital records program, run through the Government Technology Agency, is widely regarded as the most mature. By 2024 GovTech had mandated a unified metadata schema across all ministries, making duplicate detection substantially easier because files could be compared on standardised fields rather than free-text descriptions.

Canberra's situation sits somewhere between Ottawa and Wellington. The National Archives pilot has a centralised logic, but it is not yet legislatively mandated across all Commonwealth agencies. That gap means departments can — and some reportedly do — defer the work without formal consequence until the June 2027 deadline arrives.

For public servants navigating this in practical terms, the advice from digital records professionals is consistent: do not wait for a whole-of-government mandate to conduct internal audits. Agencies that begin deduplication at the business-unit level now, using even basic tools available inside the Microsoft 365 Government suite already licensed by most Commonwealth departments, will be in a significantly stronger position when the 2027 compliance window closes. The National Archives has published a guidance note — NAA Advice 2025-04 — that outlines acceptable methodologies for agencies starting that process independently.