The cloud backup bill has doubled again. Eighteen months ago it was small enough to ignore, then daily snapshots and a year of retained versions crept in, and now it sits on the management accounts as a line someone has to explain. When the backup vendor’s answer is deduplication, with claims of up to 95 per cent storage savings, the reasonable question is whether that number is real or a brochure figure. The honest answer is both. Deduplication delivers dramatic savings under specific conditions, and close to nothing outside them. Knowing which side of that line your business sits on is worth twenty minutes of your attention.
What is cloud deduplication?
Cloud deduplication is a storage technique that spots identical chunks of data and keeps only one physical copy. Backup software breaks files into blocks, fingerprints each block, and stores a reference instead of a duplicate whenever a fingerprint matches one it already holds. Ten nightly backups of a largely unchanged server can therefore share one set of underlying blocks rather than existing as ten full copies.
The savings build over time rather than arriving on day one. The first backups are full of blocks the system has never seen, so early runs store almost everything and the ratios look disappointing. Once a baseline exists, each new backup adds only genuinely new blocks, and both storage growth and transfer volumes fall away. Community guidance from Commvault describes exactly this pattern, slow seeding first, then markedly better throughput once the destination has baselined the source.
Two things defeat it. Compressed files, such as video, JPEG images and ZIP archives, have already had their internal repetition squeezed out, so their blocks look unique to the fingerprinting engine. Encrypted data goes further, because good encryption is designed to look random, and identical files encrypted separately may share no blocks at all. This is why serious backup products deduplicate first and encrypt afterwards, inside the product, a design detail worth confirming with your vendor.
Why does it matter for your business?
The case is cost. Where data is repetitive and backed up frequently, deduplication can shrink stored volumes dramatically. Large environments report ratios of 10:1 to 30:1, and guidance aimed at smaller firms suggests deduplication and compression together commonly cut storage use by 50 to 80 per cent. That flows straight through to your monthly cloud bill and your backup bandwidth.
The most vivid published example comes from Commvault’s CoStar case study. Consolidating onto one deduplicating backup platform took 11.2 petabytes of source data down to around 2 petabytes on disk, a five-fold global reduction and ten-fold on SQL databases, while backup success rates climbed from roughly 55 per cent to 97 per cent and recovery times fell from 24 hours to one. Your firm is a fraction of that scale, but the mechanism scales down. Nightly images of workstations and daily copies of a slowly changing database are exactly the repetitive workloads deduplication was built for.
There is a second benefit beyond the bill. Because each additional backup costs little once the baseline exists, you can afford to back up more often, which tightens how much work you would lose in an incident. A caution rides alongside the savings, though. Cheap retention makes it tempting to keep everything forever, and the ICO’s storage limitation principle points the other way. Deduplicated backups still contain personal data, deletion requests still reach into them, and keeping data simply because storage is cheap is a liability rather than a policy.
Where will you actually meet it?
You will meet deduplication inside backup products, not as something you buy separately. Vendors such as Veeam, Commvault and HPE build it into their backup software, usually switched on by default, with settings per backup set. Cloud providers also run their own storage optimisation behind the scenes, which is invisible to you and beyond your control either way.
In practice the decision shows up as configuration rather than procurement. Backup products expose deduplication per client or per backup set, sometimes with modes tuned for slow networks or fast disks. HPE quotes space savings of up to 95 per cent on suitable workloads for its deduplicating storage, and similar claims run through Veeam and Commvault material, always with the same caveat about data type.
One distinction saves confusion. If your files live in Microsoft 365 or Google Workspace, the provider already optimises its own storage and there is nothing for you to switch on. Your opportunity sits in whatever backup you run of that SaaS data, and in your own servers and machines. Removing duplicate customer records from a CRM is a different job again, done at the application level for data quality rather than storage cost.
When should you ask about it, and when should you ignore it?
Ask about deduplication when your backups are frequent snapshots of repetitive data, server images, databases, document stores and email. Ignore it, or leave the defaults alone, when your data is mostly compressed media or encrypted archives, when your total volume is a few hundred gigabytes, or when nobody in the firm has time to test that restores still work.
The counterexamples matter as much as the wins. A production company whose archive is unique, heavily compressed video will see little benefit, and would do better spending the same effort moving old projects into cheaper archive tiers. A five-person consultancy with a few hundred gigabytes of documents has so little at stake that tuning deduplication is a distraction, and defaults will do. Any firm in a regulated corner, an IFA or an insurance broker for instance, should check that deduplicated tiers still meet evidential and record-keeping expectations before leaning on them.
Two questions belong in any vendor conversation. First, does the product deduplicate before it encrypts, because the reverse order wipes out the benefit. Second, what happens if you leave, since proprietary deduplication formats can make migrating years of backups to another provider slow and expensive, the kind of switching barrier the Competition and Markets Authority has flagged in its cloud market work.
The Monday move is modest. List your data by type, roughly how much of each and how often it changes. Run your backup vendor’s cost calculator with deduplication on and off across a couple of retention settings. Enable it where the repetitive data lives, leave the media archive alone, and then restore something real from a deduplicated backup and time it. Restores rebuild files from shared blocks, so they can run slower than reading a plain copy, and the moment to discover that is a test, never an incident.
What related concepts should you know?
Four ideas sit next to deduplication and are worth keeping straight. Compression shrinks individual files. Incremental backup copies only what changed since the last run. Recovery time and recovery point objectives describe how fast you need to be back and how much loss you can accept. And retention policy governs how long any of it is kept.
Compression pairs naturally with deduplication, and order matters, deduplicate first, then compress what remains, because compressing first hides the repetition the fingerprinting relies on. Incremental backup achieves a related saving by copying only changed files, though deduplication works at a finer grain and across machines. Recovery objectives deserve an agreed number each, settled before an incident rather than during one. Retention policy is where the governance lives, since deduplication changes what storage costs, while your retention duties under UK GDPR stay exactly as they were.
Deduplication earns its place in owner-managed businesses whose backups repeat, and it earns nothing in businesses whose data resists it. If your cloud backup bill is climbing and your workloads are documents, databases and machine images, the savings are real and the effort to claim them is a configuration session and a test restore. If your world is compressed media, tiny volumes, or a team with no slack to manage another moving part, take the simpler setup and pay the modest premium for predictability.
