The hidden cost of duplicate images clogging Canberra's government databases

Canberra's public sector agencies are sitting on a sprawling problem measured not in policy failures but in petabytes. Duplicate image files — the same photographs, scanned documents and design assets stored two, five, sometimes dozens of times across shared drives and cloud platforms — are consuming a significant and measurable slice of government IT budgets across the ACT and federal bureaucracy.

The issue has gained fresh urgency in 2026 as several Commonwealth agencies complete their migration to consolidated cloud storage systems under the Australian Government's whole-of-government cloud strategy. That consolidation, rather than cleaning up the mess, has in many cases exposed just how deep the duplication runs. Analysts working in the digital records space estimate that duplicate files — images chief among them — can account for between 20 and 40 percent of total stored data in large organisations that have grown through mergers, machinery-of-government changes, or rapid pandemic-era digital uploads.

The numbers behind the clutter

A 2024 report by the Australian National Audit Office examining digital asset management across Commonwealth entities found that storage inefficiencies, including redundant files, were a contributing factor in agencies exceeding their projected cloud expenditure. The federal government's cloud spend has grown substantially year-on-year, with the Digital Transformation Agency noting in its 2024-25 annual report that cloud services now represent a major and rising line item across portfolio budgets. Specific per-agency breakdowns are not publicly disclosed, but the directional trend is unambiguous.

For context on scale: a single high-resolution scan of an A3 planning document — the kind produced in bulk by the ACT Planning directorate during the ongoing Territory Plan consultations — can run to 15 megabytes or more. Multiply that across years of community engagement processes for suburbs like Gungahlin and Belconnen, where development applications have surged alongside population growth, and the storage arithmetic compounds quickly. The ACT Planning directorate has published more than 800 individual development-related documents for the Gungahlin town centre precinct alone since 2020, many of them image-heavy PDFs filed by multiple teams and saved into separate project folders.

The Australian National University's Research Data Commons team has grappled with the same problem at an institutional level. ANU manages research image libraries — microscopy data, satellite imagery, field photography — that run into the hundreds of terabytes. Without automated deduplication tools, the same raw image file can end up in a principal investigator's personal drive, a shared project folder, a backup snapshot and an archive, all simultaneously.

What deduplication actually costs — and saves

Commercial deduplication software licences vary widely, but enterprise-tier tools capable of scanning and resolving duplicates across hybrid cloud environments typically start at several thousand dollars per year for mid-sized government agencies, scaling upward based on data volume. The return calculation, in theory, is straightforward: storage costs money, and eliminating redundant copies reduces the volume being paid for. Amazon Web Services S3 standard storage, which several ACT government units use for document management, is priced in fractions of a US cent per gigabyte per month — but those fractions add up across terabytes, and data transfer and retrieval costs compound the bill.

The practical barrier is rarely the software. It is the governance question of who owns a given image file, and therefore who has authority to delete a copy. In a jurisdiction like the ACT, where the same planning map might be legitimately held by the Environment, Planning and Sustainable Development directorate, Transport Canberra, and a contractor working on Stage 2 of the Capital Metro light rail project, determining the authoritative master copy requires inter-agency coordination that no algorithm can substitute for.

Agencies working through this now would do well to establish a file provenance register before triggering any automated deletion process — a lesson that several Commonwealth departments learned the hard way during earlier consolidation rounds. The National Archives of Australia maintains guidance on digital records disposal that applies to image files, and any deduplication program touching official records must account for those obligations before a single file is removed. The cost of getting that wrong — in compliance terms — makes redundant storage look cheap by comparison.