Duplicate Images Are Clogging Canberra's Digital Archives — Here's What Officials and Experts Are Saying

Canberra's public sector has a clutter problem — and it lives inside its servers. Duplicate digital images, accumulated across years of departmental file migrations, shared drives and legacy imaging systems, are consuming significant portions of agency storage budgets and raising questions about records management compliance under the Archives Act 1983. The issue has landed on the agendas of IT teams at multiple Commonwealth agencies clustered around the parliamentary triangle, and it isn't going away quietly.

The timing matters. The Australian Public Service is mid-way through a broader push toward streamlined digital infrastructure, and the National Archives of Australia — based on Queen Victoria Terrace in Parkes — has been pressing agencies to audit and rationalise their digital holdings since at least 2024. Duplicate image files sit awkwardly at the intersection of two pressures: the cost of cloud and on-premises storage, which has not fallen as sharply as once predicted, and the compliance burden of maintaining accurate, non-redundant records for Freedom of Information and discovery purposes.

At the Australian National University's College of Engineering, Computing and Cybernetics in Acton, researchers working on automated file-deduplication tools have noted growing interest from public-sector clients over the past 18 months. The university's proximity to federal agencies along Northbourne Avenue and in the Barton precinct has made it a natural sounding board for agencies testing practical solutions. The University of Canberra, operating out of its Bruce campus, has similarly flagged digital asset management as a research priority within its Faculty of Science and Technology.

What the Experts Are Pointing To

The core technical issue is straightforward: when staff across a large agency share, copy and re-upload images — whether scanned documents, photo records or graphic assets — identical or near-identical files multiply across network drives and cloud repositories. Detection requires either exact-match hashing, which catches byte-for-byte copies, or perceptual hashing algorithms that can identify visually similar images even if file names or metadata differ. Experts in digital records management generally recommend a two-stage approach: automated flagging followed by a human review step before any deletion, specifically to avoid destroying the only surviving copy of a record that happens to look like a duplicate.

For Canberra's public servants — a workforce that, according to the Australian Public Service Commission's 2025 State of the Service report, numbers more than 170,000 people working across the APS — the practical stakes are real. Storage contracts for large Commonwealth agencies routinely run into the tens of millions of dollars annually, and even modest reductions in redundant data can translate to measurable savings at renewal. One widely cited industry benchmark suggests that duplicate files can account for between 20 and 30 percent of total file storage in large organisations, though the figure varies significantly by sector and workflow type.

Local Programs and What Comes Next

The Digital Transformation Agency, headquartered on London Circuit in the city centre, has published guidance on whole-of-government data hygiene as part of its broader digital investment framework. That guidance touches on deduplication as a component of data quality, though it stops short of mandating specific tooling. Agencies are largely left to procure and implement their own solutions within those parameters.

For ACT government bodies, the situation is somewhat different. The ACT Government's Shared Services ICT division, which supports directorates operating out of locations including Civic and Tuggeranong, is understood to be reviewing storage rationalisation options as part of its 2026-27 budget cycle. The ACT budget handed down in June 2026 allocated funding for broader digital infrastructure upgrades, though specific line items for deduplication tools were not publicly itemised.

Experts broadly agree that agencies waiting for a centralised mandate before acting are likely to wait longer than their storage budgets can comfortably absorb. The practical advice circulating among IT managers in the Canberra public sector is to start with a file audit, prioritise the highest-volume shared drives, and build a clear chain of accountability before any automated deletion tool is switched on. Getting the governance right, in other words, matters as much as getting the software right.