NetApp - Ask Dr Dedupe

Questions and Answers about Deduplication

I recently sat in on a webcast Q&A panel with a few other vendors that provide deduplication technology.  Representatives from EMC, Data Domain, NEC, and Exar were on the panel, along with me.  The discussion was lively and the questions came in from the listeners rapid-fire.  Based on the large number of attendees and questions, its clear that people still have a lot of concerns about deduplication.  The webcast was sponsored by SNIA, who continues to do good work in educating people about new technologies like dedupe.  Click here for a replay of this panel discussion.

DrDedupe

Deduplication and Hyper-V are bringing NetApp and Microsoft closer together

I don't want to say that dedupe was the only reason that NetApp was named Microsoft's Storage Solutions Partner of the Year, but I can dream can't I ?....

Steve Ballmer:  DrDedupe, on behalf of our bazillion Microsoft employees, I'd like to personally thank you for your contribution to our success.

DrDedupe:  Aw shucks Steve, it was nothing really.  Just a couple hundred lines of code the team wrote over a long weekend.  We were kind of bored you see and it seemed like a fun project...

OK seriously, as Hyper-V begins to gain in popularity, there is some real synergy going on between NetApp and Microsoft.  Check out this video, where can you see a demonstration of a Hyper-V LUN being created and then easily re-sized after being dedupe'd.

Using NetApp storage with Microsoft apps seems to makes alot of sense not only for Hyper-V but also for some other apps you might have heard of such as Exchange, SQL, and Sharepoint.

SQL and its nephew Sharepoint offer some interesting (and largely unexplored) use cases for dedupe that I will discuss in upcoming blogs.  Exchange, as I noted in a previous blog, has jettisoned single instance storage (SIS) in 2010, presenting yet another compelling opportunity for NetApp dedupe.  Yes - I know that Microsoft often espouses server-attached storage for Exchange, but sooner or later they'll realize that people stopped doing that around a decade ago - for many good reasons.  (Microsoft are you listening?)

Speaking of blogs, here's a good one that Nick Triantos posted about one of our newer features, SnapManager for Hyper-V.  This feature brings automation to Hyper-V for DR and D2D Backups and is integrated with Microsoft's Volume Shadow Service (VSS).  Very cool indeed, and OK maybe the real reason we won the partner award...

Anyway, if you are a Microsoft app user (sorry Sun) you owe it to yourself to check out what NetApp can do for you.  Our latest issue of Tech OnTap also features an article on the 5 best practices for Hyper-V.

DrDedupe

The Dedupe 2.0 Pundits Are Still Swimming in Lake 1.0

I had to chuckle when I saw the concurrent announcements from Permabit and Storage Switzerland that dedupe 2.0 had arrived.  Apparently the acquisition of Data Domain by EMC had signaled the demise of dedupe 1.0, even though I failed to see any technology transformation as a result of that merger.  Data Domain is still doing the same thing they always did and EMC is still doing what they do.  There was no transformative event here guys.

Anyway, the people at Permabit seem to believe that hashing and deduplicating large archival stores is ushering a new era of deduplication and leaving point products behind.  But, er, isn't Permabit another point product?  Last time I looked in a data center there was more than archival data behind the blinking lights on those storage arrays.  In my estimation, deep archival consumes somewhere around 1% of the world's data storage.

Then there's Storage Switzerland.  I guess the Switzerland part is to have one believe that they are a neutral party amidst this vendor-against-vendor world.  But does anyone out there really think they are neutral?  I'll let my readers form their own conclusions.  Anyway, SS says that the core of dedupe 2.0 will be a foundational repository where all previously optimized data will come to rest from primary and secondary storage.  Hmmm, sounds strangely like an archival system to me.  Wait, doesn't Permabit sell archival systems, and they are also talking about dedupe 2.0?  Could there be a connection here?  Say it ain't so Storage Switzerland!

OK lets move back towards reality.  You know when dedupe 2.0 will really arrive?  When people stop talking about dedupe altogether.  Like many technologies that came before it, such as RAID, SCSI, Snapshots, and dozens of others, dedupe will become ubiquitous and acknowledged as a must-have feature from every serious storage vendor.  Inline or Post-process, source or target, local or global - it doesn't matter as long as it dedupes.  Dedupe will run silently in the background, eradicating those pesky duplicate data objects.  NetApp has proven that dedupe can run quite well on a unified platorm that services all tiers of storage (including archival), uses all standard protocols, and serves a wide breadth of applications.  Once a concept is proven, the next steps are constant refinement and optimization, and thats exactly what we are doing.  I can't tell you exactly the day dedupe 2.0 will arrive, but I am pretty sure that NetApp will be among the frontrunners.

DrDedupe

Exchange 2010 Dismisses SIS?

Microsoft Exchange revolutionized the way we communicate.  Its hard to imagine life without email.  Sure there are other email apps out there but aren't they all ladies-in-waiting to the true Queen of email - Microsoft Exchange?

Since version 4.0 was released (about the time the B-52's were singing "Love Shack") Exchange had a cool feature called Single Instance Storage, SIS for short.  SIS was the granddaddy of deduplication, and went something like this:

Email from Joe in Accounting:  "Hey check out this cool bootleg copy of "Love Shack!"

Email from Sally in Marketing, Fred in Shipping, Adam in Sales and about 10 other people "Hey check out what Joe sent me!" 

Well what Joe sent you and you sent to 10 friends and they sent to 10 friends was basically the same email with the same attachment sent over and over again.  The folks in Redmond figured this was going to be a regular thing, so they invented SIS.  With SIS, all those emails were reduced to a single instance, as long as they were held in the same Exchange Store.

Single Instancing saved storage capacity and was free - what's not to like?  Well fast-forward about 15 years and Microsoft had a problem.  More mailboxes, more attachments, and not only bootlegged audio files but attachments of all shapes and sizes - Word Docs, Powerpoints, PDF's, JPEGs, MPEGs, where will it stop?  So Microsoft did what any self-respecting software company would do, they said "you know, storage hardware is cheap these days, and all this SIS stuff is really slowing us down - so please don't use SIS anymore and just buy more storage, OK?

That's right.  Microsoft needed to destroy the monster it had created.  As of Exchange 2007 the message from Redmond was "Given current trends, we expect the value of single instance storage to continue to decline over time. It's too early for us to say whether SIS will be around in future versions of Exchange. However, we want everyone to understand that it is being deemphasized and should not be a primary factor in today's deployment or migration plans."  Worse yet, the word on the streets is that in Exchange 2010, SIS will disappear altogether.

Now before you come down with an acute case of ESISPTD (Exchange SIS Post-Traumatic Disorder) the Dr has some good news for you.  You can use NetApp deduplication to get back what Microsoft taketh away.  On tests run with Exchange 2010, without doing much optimization, we've seen seen space reductions of around 30% via dedupe.  Better yet, since NetApp dedupe is run in the background as a post-process procedure, performance impact on those users sharing their favorite files should be nominal.  I guess you could say that NetApp dedupe lets you go Back to the Future - It's free and it saves storage capacity - whats not to like?

DrDedupe

The Evolution of the Storage Brain - Applications Run Faster With Deduplication

It seems a contradiction - improving system performance with dedupe?  Usually, when someone thinks about implementing dedupe on a storage system, the first thought is "hmmmm...how much is this going to slow down my applications?"  In this blog I am going to tell you how dedupe can actually speed up your applications.

In Data ONTAP 7.3.1, NetApp introduced "Intelligent Caching."  Intelligent Caching refers to FAS and V-Series system memory, as well as secondary memory.  By secondary memory I am referring to the Performance Acceleration Module, or "PAM" for short.  NetApp introduced PAM II in this press release, although it was buried in with a bunch of other important announcements so it was easily missed.  I'll  come back to PAM II later.

So how does system memory, secondary memory, and Intelligent Caching relate to dedupe?  Simple.  When a NetApp storage system holds a dedupe'd data block in its memory, it also holds all the information about the data pointers that reference that particular block.  If this block has been deduped, say, 100 times, and is currently held in memory, there is no need to access the block from disk again, regardless of who is requesting the data block.  The result is faster access to data via reduced disk I/O latency.

The ramifications of this become very interesting.  The higher the dedupe ratio, the more dedupe'd blocks are placed into memory and the faster the system becomes.  We've already run benchmarks of this during VDI boot storms (when everyone turns on their virtual desktops at the same time and the storage system is inundated with read requests) and the results with dedupe and Intelligent Caching show orders of magnitude improvement in read response times.  With PAM II, the total system cache can exceed 2TB, and as Foxy Cleopatra might say, "Now that's a WHOLE lot of memory."  With this larger cache, the laws of probability begin to take affect - a larger cache means more blocks can be cached, and frequently requested blocks will remain in cache longer.  Meanwhile, since dedupe'd blocks have multple pointers they have a higher probability of being cached ...you get the idea.

To me, the combination of intelligent caching and deduplication is a signal that we are entering a new era in the maturity of deduplication, where the intelligence of deduplication is combined with other intelligences (intellligenci?) of the storage system to bring breakthrough results - the evolution of the storage brain.  I'll be exploring other thoughts on this topic in upcoming blogs...

Here Comes The EPA

First a little background - in December 2006 our Congress passed Public Law 109-431, directing the Environmental Protection Agency to perform a study of energy consumption in U.S. data centers.  Dutifully, in August of 2007 the EPA responded with this report, outlining an alarming trend in electricity consumption within these power hungry bits of real estate.

In their report, the EPA estimated that in 2006, data center storage devices consumed 3.6 Billion kilowatt hours of electricity. This gave storage systems the dubious role of being the #2 power hog of all data center device categories (volume servers took top honors.)  Furthermore, in their policy recommendations, the EPA suggested "ENERGY STAR specifications for data center products

— Label products that have the best energy performance on the market, to make it easy for purchasers to identify and buy these products."

As a result of these findings, for the past year the EPA has been working with SNIA'a Green Storage Initiative to define an Energy Star Rating for storage systems.  A few weeks ago, the EPA released the Energy Star Enterprise Storage Draft Specification and opened the spec to public comment.  These comments will be discussed in San Jose next week at aSNIA Symposium.

So why bring this up in my dedupe blog?  Because the EPA recognizes the value of deduplication (and other software efficiencies) in contributing to the greening of storage.  From the EPA draft specification:

"EPA understands that there are many software-based approaches to improving the energy efficiency of storage products. The benefits of virtualization, data deduplication, and other software-based data management techniques are well documented. These software solutions, perhaps even more so than the hardware itself, are heavily customized for specific customers and applications. Achieving maximum efficiency gains is highly dependent upon proper software architecture, implementation, operation, and maintenance by individual users. A key objective of the ENERGY STAR specification is to identify and reward storage solutions that seamlessly integrate software and hardware efficiency strategies that provide verifiable benefits without user intervention."

I'll be reporting periodically on the progress of this initiative as new developments warrant.  Some day you just might see an Energy Star logo on your favorite storage system...

EMC's Celerra dedupe calculator doesn't add up

I ran across this EMC dedupe calculator today.  Interested in the space savings EMC was advertising for primary storage with dedupe, thought I'd take a closer look.  I was surprised by the results.  The calculator asks you to input the "Size of Current Data" for seven different application categories.  So I ran the calculator 7 times, each time entering 10TB into the 7 categories, one at a time.  Here are the results I got:

EMC Celerra Data Duplication Calculator

(Data Duplication? I sure hope that’s a typo)

 

Potential Savings:

Active virtual machine files - 0TB (0%) saved

Active databases and messaging files - 0TB (0%) saved

Compressed files - .2TB (2%) saved

Media files - .4TB (4%) saved

Binaries – 1.7TB (17%) saved

Office documents - 3.3TB (33%) saved

Text files - 5.3TB (53%) saved

 

Wait a minute, this can't be right.  4 of the 7 categories offer between 0% and 4% savings.  Doesn't seem like its worth the effort.  Well how about Binaries at 17% savings?  But EMC only dedupes "inactive" files - which of my binaries are inactive?  Not sure, so I think I'll skip that one.  Office documents - bingo 33% saved - but again this only applies to inactive docs, so I guess my savings will be a little less.  Finally, there are text files with a whopping 53% savings - but I probably don't have 10TB in inactive .txt files though.  At this point, I am a little disappointed in EMC.

 

The way I see it, using Celerra dedupe in a real world scenario would produce maybe 5% overall savings on a good day.  The problem you see is that the majority of user data is tied up in those darned applications that Celerra dedupe can't touch.  The puzzling part is why EMC would include applications with zero percent savings in their calculator.  Trying to deceive users?  Nah there must be some other explanation.  Guess I'll leave that one up to them to answer.

 

Your voice in the search for dedupe honesty-

 

DrDedupe

 

NetApp Deduplication Revisited

It’s been quite a ride here at NetApp since we launched deduplication 2 years ago. Thought I’d take some time to reflect on what I’ve witnessed in this short period of time. As I said in an earlier blog, no one on the original dedupe launch team (at that time we were still calling it A-SIS) could have envisioned that today we’d be sitting here looking at an installed base of over 37,000 systems. Our customers are running deduplication in virtually all industry segments, across scores of workloads (including primary production workloads), with use cases offering tremendous space savings. There are very few times, if any, in one’s career where something you are responsible for becomes an "overnight" success, but this was one of those cases. Let’s look back and see how this all happened.

NetApp deduplication development began back in 2004; when our technical directors realized that we could create a catalog of “fingerprints,” and use those fingerprints to identify deduplicate data blocks. The rest was easy – once the duplicates were identified, we could use our existing system of multiple block referencing (used in all our snap features) and create many pointers to a single data block, while releasing all the duplicate blocks back to the system for re-use. NetApp deduplication was born.

Over the next year or so, the development work started. The coding was actually pretty quick, what took far greater time was the QA testing. You see, when you store some of the world’s most valuable data, you better be very careful you are not going to break anything with any new feature, especially one that removes blocks of data.  One thing I am grateful for was the fact that the early designers insisted that we do a full byte-level comparisons before any deduplication occured, that turned out to be a very wise decision indeed.

Finally, in October of 2006, we were ready for field tests. Luckily, time was on our side – deduplication was still nascent and people weren’t clamoring for it (yet.) So we decided to conduct an “early release” program with 25 of our closest friends, er customers. Everyone on the development team held their collective breath and waited for the results to come in. Were we going to see system crashes? panics? data corruption? As it turns out, over this 6-month period not a single bug report was filed. I was astounded. Having been around the block a few times at other companies I knew how these things usually go. Marketing wants the widget before Engineering is ready to give it up, and there are constant “I told you so” meetings during this rough-and-ready period. But none of that happened in this case – a Marketeer’s dream.

So in March of 2007, we released NetApp deduplication to the world. What I failed to understand in those early days was our customer’s propensity to take risks and ignore our advice. “Just use it for archival data”, we said, figuring that was safe enough – we wanted to walk before we ran. Our customers, however, seemed to be running the 100-yard deduplication dash. They immediately began to use deduplication on user files, virtual machines, and all those other areas we cautioned them against – and on thier primary production volumes no less! During this period, I got the feeling I was raising another teenage son, telling him not to do something and then watching him do exactly what I told him not to do.

Time for my second moment of astonishment. NetApp deduplication worked on these applications. And worked. And worked. To my amazement, I heard over and over again that our customers were reporting significant space saving and no performance impact. So I did what any good marketing man would do. I promoted these use cases, both within NetApp and to our other Users. You should have seen the icy stares I got in engineering meetings. But I persisted and eventually won over the mob - but it really wasn’t me, it was a very well designed feature that I was lucky enough to inherit just at the right time.

Now let’s fast forward to today’s scene. Anyone who does anything in the data storage industry knows that deduplication is hot, white hot. NetApp has proven that deduplication can be used anywhere, including primary production data. Newcomers and established storage vendors today are making a lot of noise trying to grab a chunk of the dedupe pie. Through all this clamoring, lets not forget one fact - NetApp was in the right place at the right time; and changed the way that people view deduplication and storage efficiency. Come to think of it, maybe deduplication really is the teenager of the IT family. Bordering on maturity - but still with plenty of years to grow. Time will tell…but I like our position at the table today.

NetApp Teams Up To Offer Even More Deduplication

Some of you might have noticed this press release from CommVault, where they announced support for block-level deduplication in their latest release of Simpana.  Curious to know how they viewed NetApp deduplication in light of this new feature, I contacted old friend and Marketing VP Dave West to get his viewpoint.  To my relief, Dave stated that he viewed NetApp dedupe as complimentary to CommVault's and sent along some documentation for me to review.

After looking over the docs, I agree with Dave's assessment.  CommVault user's can enable Simpana's deduplication and remove duplicate blocks at the source, the main advantage being a reduction in the amount of data transmitted over the wire.  CommVault recommends that the examined/comparison size be 50KB or larger, presumably so that Simpana can find a reasonable amount of duplicates without overtaxing media server resources - this granularity can be adjusted by the user for optimal reduction/performance.

Now here's the cool part.  If CommVault is backing up to a NetApp FAS system, Simpana recognizes this and fires off a second dedupe process, this time directly on the FAS volume.  This accomplishes two things: First, since NetApp dedupe occurs at a 4KB level, some additional duplicates will likely be found.  Second, if multiple Simpana backup jobs are aggregated into one FAS volume, NetApp dedupe will sweep through all those datasets and remove duplicates across all backup jobs. Pretty slick..

Who says storage vendors can't get along?

Deduplication and The Long Arm of the Law

Check out this press release and you'll see a couple interesting facts:

• NetApp has delivered deduplication to over 7,200 customers and over 37,000 systems
• Cohasset evaluated deduplication with SnapLock and found that it meets SEC regulations for electronic records management

Each of these items is a milestone of sorts.  Having been part of the original dedupe launch team back in the Spring of 2007, I can tell you that none of us would have imagined that within two years we would be approaching 40,000 deployments - in fact our "stretch" goal was around a thousand systems in the first year - we beat that by about 10,000 systems.

The success of deduplication at NetApp is a testament to the vision of our development team - they recognized yet another way to leverage the intelligence of Data ONTAP, and to our customers, who were willing to put their faith in our design and prove that production data could be dedupe'd just as easily as backup data.

The second part of today's announcement is Cohasset's evaluation of dedupe for compliance records.  Cohasset Associates is the acknowledged authority in electronic records management.  For decades, storage vendors have turned to Cohasset to make sure their archival and compliance products passed the sniff test. 

To validate products, Cohasset leans heavily on SEC rule 17a, probably the most stringent test of any government agency for legal record compliance.  The SEC is interested in two key criteria - A) Is the record stored in a state that cannot be altered by users? and B) Can the record always be recovered back to its original state?  When Cohasset evaluated NetApp deduplication, there were two things that stood out that eventually resulted in two thumb's up from them:

1) Before deduplication - a byte-level comparison takes place to eliminate data corruption as a result of hash collisions

2) After deduplication, the only change to the original data are re-addressed data pointers, which cannot be altered or otherwise manipulated by users, and does not involve any external hash tables or containers.

Deduplication of legal records has been a controversial topic for some time.  Hopefully our announcement today makes this topic is somewhat less controversial...and before I close I'd to make sure I say thanks to the thousands of NetApp dedupe users that have made this the fastest-growing feature in NetApp history!