How Canberra's Digital Archives Ended Up Flooded With Duplicate Images — And What's Being Done About It

Tens of thousands of duplicate digital images — many of them scanned government documents, infrastructure photographs, and heritage records — are sitting inside the storage systems of ACT government agencies and federal bodies headquartered in Canberra, consuming server capacity and complicating public records obligations. The problem did not arrive overnight. It accumulated across roughly a decade of fragmented digitisation drives, departmental mergers, and cloud migration projects that rarely spoke to one another.

The issue matters now because several of those agencies are mid-way through major digital transformation programs, and the volume of redundant image files is large enough to distort search results, inflate storage costs, and, in some cases, obscure which version of a scanned document is the authoritative one. For a city whose entire economic identity is built around federal administration and public record-keeping, that is not a minor housekeeping matter.

How the Duplication Built Up

The pattern is consistent across institutions. The ACT's own digitisation work accelerated after 2015, when the territory government began moving physical records from the Fyshwick document storage facilities toward centralised digital repositories. The Australian National University's library system undertook a parallel effort covering historical photographic collections. The Australian Institute of Aboriginal and Torres Strait Islander Studies, based on Acton Peninsula, ran its own ingestion programs for cultural material. Each initiative used different metadata standards, different file-naming conventions, and — critically — different deduplication rules, or none at all.

When agencies later merged cloud storage accounts, consolidated after Administrative Arrangements Orders changed departmental structures, or simply copied backup drives to new servers, duplicate images multiplied. A photograph of the Molonglo Valley development corridor captured by a planning team in 2018, for instance, might exist in four separate buckets: the original field upload, a backup taken before a server migration, a copy shared to a cross-agency working group, and a version re-ingested during a later records audit. None of those copies necessarily carries a flag identifying the others.

The National Archives of Australia, located on Queen Victoria Terrace in Parkes, has documented the broader challenge of digital records integrity in its regulatory guidance to agencies, though the specific scale of image duplication across the ACT's holdings has not been publicly quantified in a single report.

The Cost Is Real, and Now Measurable

Cloud storage is not free. Enterprise-grade government storage contracts in Australia typically price bulk object storage at rates that make redundant holdings a genuine budget line. The ACT Government's Digital Strategy, released in 2023, acknowledged that data sprawl was a priority problem, though it did not publish a figure for the cost of duplicate records specifically.

For context, the federal government's broader APS data and digital strategy — administered through the Department of Finance — set a target of rationalising agency cloud footprints by the 2025-26 financial year. That deadline has now passed, and compliance has been uneven, according to public reporting on agency ICT expenditure tabled in Senate estimates.

At the University of Canberra's Bruce campus, library and information science researchers have been studying automated deduplication tools as part of a research program examining digital preservation in public institutions. Their work reflects a growing recognition that the problem is structural, not accidental.

The practical path forward involves three distinct steps that affected institutions are now beginning to take seriously. First, agencies need a complete audit of their image holdings using perceptual hashing tools, which can identify visually identical or near-identical files even when filenames differ. Second, a single authoritative version must be designated and documented with clear provenance metadata. Third, governance rules need to be written into procurement contracts so that future digitisation vendors are required to run deduplication checks before delivery. None of this is technically exotic. What has been lacking is coordination — and deadline pressure. Both, finally, appear to be arriving.