How Canberra Is Handling the Duplicate Image Problem — and Where It Stands Against Cities Doing It Better

Canberra's public sector holds one of the densest concentrations of digital image archives in the Southern Hemisphere. The Australian National Archives in Parkes, the National Library on Parkes Place, and dozens of federal departments collectively store billions of scanned documents, photographs and asset records — and a growing body of internal audits has found that duplicate image files routinely consume between a quarter and a third of allocated storage across large government repositories. The problem is neither new nor trivial: redundant files slow retrieval systems, inflate cloud costs and complicate freedom-of-information processing.

The issue has sharpened in mid-2026 for two reasons. Federal agencies are midway through the Albanese government's digital transformation agenda, which set a 2027 target for cloud migration across core Commonwealth systems. Simultaneously, the ACT government's Digital Canberra Action Plan — now in its second phase — is pushing Territory agencies toward consolidated data infrastructure. Both timelines mean that duplicate-image management, long treated as a backroom IT chore, is now a line item in budget submissions and a factor in procurement decisions.

What Canberra Is Actually Doing

The Australian National University's Scholarly Information Services division has been running a deduplication program across its image and research-data holdings since late 2024, targeting the institutional repository hosted under the ANU Figshare platform. The program uses hash-matching to flag identical files before ingestion, a method that prevents new duplicates but does not automatically clear existing ones from legacy storage pools. Staff at the Chifley Library building on the Acton campus have been involved in manual review workflows on top of the automated flagging — a resource-intensive arrangement that reflects just how embedded the problem already is.

The Australian Institute of Health and Welfare, headquartered on Bowes Street in Phillip, has taken a different approach. Its data governance framework, updated in 2025, now requires image deduplication sign-off before any dataset is cleared for public release on the AIHW website. That policy shift came after an internal review found redundant diagnostic images in at least two published data collections. The Institute has not publicly disclosed the scale of those redundancies.

Meanwhile, the ACT's own Land Information and Planning directorate, which manages aerial and cadastral imagery of the Territory, has been trialling perceptual hashing — a technique that catches near-duplicate images, not just exact copies — across its holdings of Gungahlin and Belconnen growth-corridor surveys. The trial, running since March 2026, is the most technically ambitious local effort to date.

The Global Benchmark Gap

Canberra's patchwork of agency-by-agency programs looks modest against what comparable capital cities have done. Amsterdam's municipal archive, the Stadsarchief, completed a citywide deduplication sweep in 2023 that cut its 14-petabyte image repository by roughly 18 percent and was conducted with a single centralised tool rolled out across all departments simultaneously. Singapore's National Archives ran a similar consolidated program under its ArchiveSG initiative and reported completion in under 14 months. Wellington, arguably the closest analogue to Canberra given its similar public-sector dominance and population of roughly 215,000, finished a whole-of-government image audit through the New Zealand Public Records Office in 2024.

Canberra has no equivalent whole-of-government program. Responsibility is fragmented across the National Archives, individual Commonwealth agencies operating under their own chief information officers, and ACT government directorates that do not share infrastructure with federal bodies even when they occupy the same buildings on London Circuit or Constitution Avenue.

Cloud storage costs provide one concrete reason to close the gap quickly. AWS and Azure pricing for government-tier storage in the Asia-Pacific region has risen incrementally over the past two years, and agencies locking in multi-year contracts this financial year are doing so at rates that make redundant data meaningfully expensive. Every petabyte of unnecessary image duplication represents ongoing expenditure rather than a one-time inefficiency.

For ANU researchers, ACT government project managers and federal IT leads, the practical next step is the same in most cases: push for perceptual hashing tools — not just exact-match filters — to be written into procurement standards before new storage contracts are signed. Wellington's experience showed that mandating the technology at the contract stage, rather than retrofitting it later, cut implementation time by more than half. The window to do that cleanly in Canberra is the current budget and procurement cycle, which closes in October 2026.