Canberra Archives and Agencies Race to Fix Duplicate Image Problem That's Clogged Digital Records for Years

ACT government agencies and Canberra's major public institutions are moving this week to accelerate a long-delayed cleanup of duplicate image files clogging their digital records systems — a problem that has quietly drained storage budgets and complicated Freedom of Information requests for the better part of a decade.

The timing matters. Across the public service, which employs roughly one in three Canberrans, agencies are under pressure from the federal government's broader digital asset management overhaul to demonstrate clean, auditable records before the next funding review cycle. For Canberra specifically, where so many organisations store sensitive policy documents, planning images, and environmental survey photos, duplicated files aren't just a housekeeping annoyance — they create genuine compliance headaches.

What Happened This Week

The Australian National University's Scholarly Information Services division confirmed this week it is rolling out updated deduplication protocols across its institutional image repositories, affecting collections housed at the Chifley Library on the Acton campus. The university has been consolidating research image archives that span multiple faculties, and duplicate files — often created when researchers uploaded identical images to separate project folders — have complicated access for staff working on collaborative grants.

Separately, the ACT Government's Digital, Data and Technology Solutions directorate has been working with agencies based in the Civic precinct and in Barton to implement automated duplicate-detection tools across shared document management platforms. The process, which began in earnest in the March quarter, involves scanning stored images against hash-value fingerprints to identify exact and near-exact copies without manual review. For agencies handling large volumes of planning photography — particularly those managing development applications in growth suburbs like Gungahlin and Belconnen — the volume of redundant files accumulated over years of staff turnover has been substantial.

The University of Canberra Library, based at the Bruce campus, has also flagged its own deduplication project as part of a broader digital preservation review. UC's library collections include significant holdings of historical Canberra photography and regional survey imagery, some of which exists in multiple scanned copies across different cataloguing systems introduced at different points since the early 2000s.

Why the Backlog Built Up

The root cause isn't hard to identify. When staff change roles — and in Canberra's public service, that happens constantly — they routinely copy files before handing over access, creating shadow duplicates across shared drives and cloud storage buckets. Merger of agencies, machinery-of-government changes, and the mass shift to remote working during 2020 and 2021 all accelerated the problem. Storage costs that once seemed negligible have compounded: enterprise cloud storage in Australian government environments typically runs between $0.02 and $0.05 per gigabyte per month, and when image libraries run into the tens of terabytes, the arithmetic adds up fast.

A 2024 review by the National Archives of Australia found that many Commonwealth agencies were retaining multiple copies of the same digital assets without any systematic process for identifying or removing them — a finding that prompted updated guidance circulated to ACT-based agencies earlier this year. The National Archives is headquartered on Queen Victoria Terrace in Parkes, a short walk from Parliament House, and its guidance carries significant weight for the dense cluster of agencies in the parliamentary triangle.

For public servants working in agencies along Constitution Avenue or based at Treasury Place in the city centre, the practical upshot of this week's developments is that image uploads to shared platforms will increasingly be flagged or blocked if an identical file already exists in the system. Some agencies are also implementing retention policies that automatically archive — rather than delete — duplicates, preserving auditability while reducing active storage load.

The cleanup won't happen overnight. Institutions with the largest backlogs are setting internal targets through to mid-2027 to complete first-pass deduplication. For members of the public waiting on FOI requests that involve image attachments — particularly planning documents for properties in suburbs like Tuggeranong and Weston Creek — the practical benefit should be faster search times and more reliable document retrieval as the redundant files are progressively cleared.