A services firm with a dozen virtual machines gets a storage renewal quote. The vendor mentions a “data reduction ratio of up to 10:1” and a “deduplication engine” in the same paragraph, then offers three tiers at different price points. The operations lead forwards it to the owner with a single line attached: “Do we need the dedupe option?”
That is a good question. It is also more specific than it looks.
What choice are you actually making here?
Data reduction is the umbrella for anything that shrinks how much physical space your data occupies. Deduplication removes identical copies of data blocks, storing one and replacing the rest with references. Compression encodes the remaining data more efficiently, removing statistical redundancy. Modern storage and backup platforms commonly apply both in sequence, but they work differently and deliver their biggest savings on different kinds of data.
The vendor’s “up to 10:1 effective capacity” figure almost certainly combines the two. When a supplier quotes that ratio, they mean that after applying dedupe and compression together, 10 TB of physical storage behaves like 50 TB of usable capacity. Pure Storage and VAST Data both market their platforms on effective capacity rather than raw capacity. NetApp ONTAP offers per-volume controls so you can choose which technique to apply to which workload.
For a UK services SME, this comes down to a configuration question: which settings to enable on a product you are already buying, and for which workloads. The two main failure modes are applying dedupe where it cannot help, paying CPU overhead for no saving, and failing to apply it where it could cut thousands off your capacity costs.
When is deduplication the right call?
Deduplication delivers its biggest savings when your storage holds lots of repetitive content: nightly backups of similar servers, virtual machine snapshots, shared project folders with near-identical document templates, and user home directories across a team. For these workloads, savings of 4:1 or better are achievable in real deployments. Cohesity illustrates the case: three identical 10 MB files stored across separate volumes consume 30 MB without cross-volume dedupe, and just 10 MB with it applied globally.
Dell PowerStore implements deduplication at 4 KB block level, meaning even files that are not identical can still share repeated chunks. The same principle underlies tools many businesses already use day-to-day. Git identifies identical files by content hash; Dropbox applies a similar check to avoid uploading duplicate content.
Image-based server backups are arguably the highest-value use case. If you run nightly backups of ten servers that share a common base image, deduplication removes the repeated blocks across backup sets and stores each incremental change once rather than ten times. TechTarget notes that SMB backup targets now routinely rely on dedupe to keep backup windows and storage footprints manageable, making it a reasonable default for any business running structured backup processes.
When does compression-first make more sense?
Compression works by encoding data more efficiently, regardless of whether identical copies exist elsewhere in your storage. It earns its keep on unique-but-compressible content: log files, CSV exports, database dumps, JSON feeds, and text-heavy documents can all compress at 2:1 or better. If your storage is dominated by this kind of content rather than by repetitive VM images or shared files, compression is the technique doing most of the work.
Deduplication struggles when data is already compressed or encrypted. A JPEG image, an MP4 video, a ZIP archive, or a backup job that encrypts client-side before sending data off-site: all of these look like random noise to a dedupe engine. The engine finds no repeated blocks and saves no space, but it still consumes CPU cycles and metadata overhead in the attempt.
NetApp advises that dedupe, compression, and data compaction can be run independently, noting that for some workloads compression alone is the right choice. The ICO recommends strong encryption as a safeguard for personal data sent off-site. If you follow that guidance, your backup target will receive encrypted ciphertext and gain nothing from deduplication. The working rule: turn on dedupe plus compression for virtualised workloads, shared documents, and email stores. For already-compressed media or encrypted archives, apply compression only, or rely on storage tiering.
What does it cost to get this wrong?
The most common planning error at SME scale is trusting a vendor’s headline ratio without modelling your actual workloads. If the quote promises “up to 10:1” and your workload delivers 2:1, you will need additional capacity at short notice, typically at retail prices without a project discount. A more disruptive outcome is changing dedupe block size after initial configuration and triggering a full re-baseline that temporarily doubles your storage footprint.
VAST Data documents the re-baselining problem in detail. Change the block size and the system recalculates hashes for all existing data, temporarily writing everything again. For a firm with several terabytes of backup data, this can mean multi-day disruption to backup schedules.
Cohesity also notes that different deduplication algorithms can produce anything from no savings to roughly 45 per cent savings on the same data set, depending on block size and method. Two firms buying the same product for similar workloads may see very different outcomes.
The resilience dimension matters too. The NCSC recommends at least one offline or immutable backup copy, specifically because online dedupe stores are targets for ransomware. A single dedupe store holding your only backup copies is a single point of failure. If the index is corrupted or encrypted by attackers, recovery becomes complex.
For firms regulated by the FCA under PS21/3, restore time enters the picture directly. If your operational resilience scenario requires critical data restored within four hours, and dedupe index overhead extends that window, your configuration choices are also a compliance question.
What should you ask before you commit to a setting?
Before finalising a storage or backup configuration, five questions will surface gaps in the typical vendor proposal. The aim is to ensure the settings you choose match your actual workloads, your data retention obligations under UK GDPR, and the resilience standards the NCSC expects. Each question maps to a known failure mode.
First: what deduplication ratio should you realistically expect for your specific workloads? Ask for reference customers of similar size and sector, not just a headline figure. TechTarget notes that ratios vary widely by data type, and the gap between a realistic estimate and a marketing claim can run to several times the expected capacity.
Second: is dedupe inline or post-process? Inline reduces data before it is written but consumes more CPU during the backup window. Post-process is gentler on performance but requires spare raw capacity to hold data before savings are applied.
Third: can you disable dedupe per volume, so you are not applying it to encrypted archives or already-compressed media?
Fourth: how is the dedupe index itself backed up, and what is the recovery plan if it is corrupted? VAST Data’s account of re-baselining after a block-size change shows how disruptive a rebuild can be, even without a fault.
Fifth: how does the solution support your data retention policy? The ICO expects organisations to enforce storage limitation under UK GDPR, including automated expiry for personal data. Cheap effective storage from deduplication can make it tempting to keep data beyond permitted retention periods. Your configuration should make deletion easier to enforce, not an afterthought.
One further question on vendor lock-in: the CMA has raised concerns about cloud storage egress fees and proprietary deduplication formats that raise the cost of switching providers. Ask specifically how a cross-platform restore works before you sign.
If you are deciding whether to Book a conversation about data readiness before an AI rollout, getting this configuration sorted first is worth the hour. The AI tools you add later will inherit whatever your storage hygiene looks like today.
