Deduplication and The Long Arm of the Law

Check out this press release and you'll see a couple interesting facts:

• NetApp has delivered deduplication to over 7,200 customers and over 37,000 systems
• Cohasset evaluated deduplication with SnapLock and found that it meets SEC regulations for electronic records management

Each of these items is a milestone of sorts.  Having been part of the original dedupe launch team back in the Spring of 2007, I can tell you that none of us would have imagined that within two years we would be approaching 40,000 deployments - in fact our "stretch" goal was around a thousand systems in the first year - we beat that by about 10,000 systems.

The success of deduplication at NetApp is a testament to the vision of our development team - they recognized yet another way to leverage the intelligence of Data ONTAP, and to our customers, who were willing to put their faith in our design and prove that production data could be dedupe'd just as easily as backup data.

The second part of today's announcement is Cohasset's evaluation of dedupe for compliance records.  Cohasset Associates is the acknowledged authority in electronic records management.  For decades, storage vendors have turned to Cohasset to make sure their archival and compliance products passed the sniff test. 

To validate products, Cohasset leans heavily on SEC rule 17a, probably the most stringent test of any government agency for legal record compliance.  The SEC is interested in two key criteria - A) Is the record stored in a state that cannot be altered by users? and B) Can the record always be recovered back to its original state?  When Cohasset evaluated NetApp deduplication, there were two things that stood out that eventually resulted in two thumb's up from them:

1) Before deduplication - a byte-level comparison takes place to eliminate data corruption as a result of hash collisions

2) After deduplication, the only change to the original data are re-addressed data pointers, which cannot be altered or otherwise manipulated by users, and does not involve any external hash tables or containers.

Deduplication of legal records has been a controversial topic for some time.  Hopefully our announcement today makes this topic is somewhat less controversial...and before I close I'd to make sure I say thanks to the thousands of NetApp dedupe users that have made this the fastest-growing feature in NetApp history!

Comments

[Carter George]

Dr. Dedupe:
This is great news - for NetApp and for all of us in the dedupe world. It takes a market-leading company like NetApp to not just solve the technical problems, but to get through the legal, psychological, and regulatory maze of whether dedupe can be valid in a compliance store. Congratulations. I think this will be a step forward for storage efficiency for many many customers.

[glenn]

I guess A-SiS wasn't all it was cracked up to be, by about $1.5B. Happy spinning.

[DrDedupe]

Now, now Glenn, do I detect some slight causticity in your comment? Check out my latest blog "NetApp Deduplication Revisited" and you'll find that A-SiS is everything it was cracked up to be and more.