NetApp - The Missing Shade of Blue

HP and Death by a Thousand VMs

Over at HP's community storage blog, HP appear to be attempting to psychoanalyze NetApp. Strange, I know, but it seems to be a tactic that keeps them happy. First, HP's Karl Dohm on WAFL; and now Jim Haberkorn on NetApp usable space and our VMware 50% Space Guarantee.

Where to start? Jim's first post and now this latest post are so long and full of hearsay, straw men and misrepresentations that it's a difficult one.

There are many, many questions that Jim asks, and I could make this an exceedingly long reply indeed. Instead, read some of the comments; I've replied (I think!) to most of the substantive points and refuted them, so they're not worth repeating here.

One is worth a mention, though. Here's Jim;

I think you can justifiably conclude that NetApp's usable capacity woes span all sorts of applications.

I could have just said "we don't have a problem" and left it there, but I actually went to some effort last time EMC accused us of this, so I said "we don't have a problem, here's the proof". The response;

NetApp has a huge usable capacity issue in many environments that it tries desperately to hide but at the same time seems driven to confess as if subconsciously trying to purge some unresolved guilt.

Very Freudian. We're apparently in denial. Need more HP evidence of NetApp in denial? Yes, it's our VMware 50% Space Guarantee

Ah, to have one in the first place. Why is this guarantee around at all? Nick Triantos sums up the whys and wherefores. Worth a read, especially to see the impressive numbers of customers and the kind of %age savings they are getting.

Meanwhile, EMC and HP are team tagging at the moment, presumably on the principle that "your a friend of mine if your enemy is my enemy". Cool, really, considering that HP had a hissy fit with EMC on the subject of usable space just recently. How short the memory, how time flies.

Anyhow, EMC and HP have kissed and made up, and they're in agreement. We still have space issues according to the good doctors at HP and EMC, with a bit of a diversionary marketing gimmick thrown in for good measure.

So how does NetApp hide all this?  Well, one way is if they have a small SAN running on the same filer as a large NAS.  With all the free space running around no one really tracks whether the SAN is being a free space hog.   

So we're still a basket case running around when it comes to those space issues, apparently.

Listen, stop playing doctors and nurses and get used to it; it's a good idea. Everyone except HP and EMC thinks it's a good idea. Gartner in a recent report on our guarantee says;

During the research phase of your next storage purchasing cycle, ask each vendor if it offers a capacity savings or utilization guarantee.

Gosh. That's a "No" for EMC and HP then. Go psychoanalyze Gartner, why don't you.

Time for me to don the white coat and stethoscope. Here's a list of psychoses the HP MSA, EVA and XP suffer from;

• RAID-5 which just isn't good enough 
• No evidence of any RAID-6 on the EVA
• HP's VMware best practices that recommend VRAID-1 (mirroring) for performance across all their platforms
• VMware over iSCSI on the EVA is not available without expensive FC/iSCSI bridges
• VMware over NFS is not available on any of the mid to high end storage offerings
• The XP has no VMware certification [Update 5Jan2009; thanks to a sharp eyed reader that pointed out thet the XP is VMWare certified.]  
• None of the systems has displayed any signs of deduplication
• Snapshots impact performance (it's that copy-on-first-write problem again)
• Lack of effective thin provisioning (the EVA's "dynamic provisioning" is actually "chubby provisioning" and anyhow it doesn't work with VMware)

Why do you need these features to remain sane in the VMware world? If there's only one link you click on in this post, make it this next one. Try getting this kind of value on any HP storage system. This is why you'll never ever see a space usage guarantee on VMware from HP on their current range of storage.

An excellent example of low cost operation is captured in this video where we host 5,440 VDI clients in 10 GB of storage and just 32 Volumes.

5,440 VDI clients in 10 GB of NetApp storage. And NetApp has got problems? I think we're fine, thanks.

But not for HP storage. This is death by a thousand VMs. And HP had better not start denying it, or we'll have grounds for thinking they've a guilty secret and got something to hide.

.

MAD Blog: The E in EMC Stands For Emulated

Yes, it does, I tell you. EMC is not what you think it is; it's emulated! The real EMC wants to hide this from you!

Only kidding. With EMC, what you see is what you get. Unfortunately.

Point in case; here's EMC's uber blogger, Chuck Hollis, on the subject of EMC's FC solutions over NetApp's, specifically about the NX4;

Like other Celerras, it does the full unified storage thing: iSCSI, NAS and "real deal" FC that isn't emulated.

The FC is native, and is not mediated by the NAS head.

In his comments, Tony Asaro can't quite fathom this out. He wasn't alone. Chuck bounces back, and over a series of comments (my bold italics); 

But you lose the thread there -- most FC care about predictable performance behavior, and emulation can often get in the way of that, as you lose the underlying geometry and device characteristics.

Regarding disk geometry, NetApp (as do all emulated environments) attempts to emulate physical disks using a file system.

The real debate is --- when you need FC and what it does well, are you better off with an emulated version, or a native implementation?

My point is that -- at the application level -- emulated FC drives behave differently than real ones. And, in many cases, it can matter.

There's a mishmash of ideas here. FC stands for everything; drives, protocols, storage heads, even disk geometry is relevant. But the L for Logical in LUN seems to have somehow passed by without a mention. Clever, clever NX4!

Sigh. Still, we did get one admission.

I have no basis to comment on NetApp's specific implementation of the FC protocol. Unless I talk to one of our FC engineers, I just wouldn't know. Or know if any deficiencies or advantages really mattered.

That would have been a good move before blogging on the subject, no?.

So go and ask an engineer to explain what "emulated FC" is. Then come back and let me know. Personally, I'm betting on "Uhmmm...", and a puzzled look.

---

I'm grateful to Kostadis for the MAD blog idea. if you're tired of reading posts where vendors try and poke each other in the eye, just skip the ones marked MAD.

.

MAD Blog: The E in EMC Stands For Emulated

Yes, it does, I tell you. EMC is not what you think it is; it's emulated! The real EMC wants to hide this from you!

Only kidding. With EMC, what you see is what you get. Unfortunately.

Point in case; here's EMC's uber blogger, Chuck Hollis, on the subject of EMC's FC solutions over NetApp's, specifically about the NX4;

Like other Celerras, it does the full unified storage thing: iSCSI, NAS and "real deal" FC that isn't emulated.

The FC is native, and is not mediated by the NAS head.

In his comments, Tony Asaro can't quite fathom this out. He wasn't alone. Chuck bounces back, and over a series of comments (my bold italics); 

But you lose the thread there -- most FC care about predictable performance behavior, and emulation can often get in the way of that, as you lose the underlying geometry and device characteristics.

Regarding disk geometry, NetApp (as do all emulated environments) attempts to emulate physical disks using a file system.

The real debate is --- when you need FC and what it does well, are you better off with an emulated version, or a native implementation?

My point is that -- at the application level -- emulated FC drives behave differently than real ones. And, in many cases, it can matter.

There's a mishmash of ideas here. FC stands for everything; drives, protocols, storage heads, even disk geometry is relevant. But the L for Logical in LUN seems to have somehow passed by without a mention. Clever, clever NX4!

Sigh. Still, we did get one admission.

I have no basis to comment on NetApp's specific implementation of the FC protocol. Unless I talk to one of our FC engineers, I just wouldn't know. Or know if any deficiencies or advantages really mattered.

That would have been a good move before blogging on the subject, no?.

So go and ask an engineer to explain what "emulated FC" is. Then come back and let me know. Personally, I'm betting on "Uhmmm...", and a puzzled look.

---

I'm grateful to Kostadis for the MAD blog idea. if you're tired of reading posts where vendors try and poke each other in the eye, just skip the ones marked MAD.

.

Web2.0, meet Presentation2.0

So there I was, scouring around to try and find some Web2.0 cloudy storagey stuff to generate a new blog entry. Partly to try and understand why EMC loves it (summary from Chuck; “Buy Atmos”) and Oracle hates it (summary from Ellison; “complete rubbish”).

Then I stumbled on this. Identity management in the new Web2.0 world, or (you guessed it) Identity2.0. Yeah, yeah, just another 2.0 pundit.

I was wrong. The message is interesting; there really are some technologies that deserve the 2.0 tag. This identity issue might be be of them, even though I have a deep and abiding suspicion of giving too many details away. Especially to governments and corporations that want to sell my kids stuff.

But as I watched the video, what really struck me was the presentation technique.

This is Presentation2.0 for the brave new web world we’re creating. Web2.0, meet the newest member of the family. Play, enjoy and learn how to make your boring PowerPoint positively sizzle.

.

99.99999999999999999% Availability

In what can only be described as a truly amazing demonstration of Sun’s latest attempt to deliver robust, nay, unburstable storage, they’ve just declared that downtime on its latest range of reduced fat, cures all known diseases, includes no artificial colourings, no-tangle hair curling tongs...

Sorry, got carried away there. Not as carried away as the Sun marketing bods, though. Apparently, the downtime on Sun's latest storage, is – wait for it – 

3 femto seconds

That’s right,19 9s availability. If that's per year (and let's be generous with a year here, even though the stuff hasn't been out more than a few months), then it’s only ever down for the amount of time it takes a beam of light to travel approximately 0.1mm.

Click on the graphic for an IDC paper that shows NetApp's audited 5 9s.

Wow.

Either the laws of physics have been suspended at Sun, or the numbers are – just ever so slightly – wrong. Even if Sun correct the number and it falls down from the realms of the fantastic, there appears to be no evidence, no track record and no independent audit that Sun can point to for even one nine, far less duotrigintillions of them.

.

ABC and Storage Costs (Part 2)

In part 1 I covered some of the basic terminology of Activity Based Costing.  In this entry, I'll apply these principles to storage costs.

First, some general observations.

Making Costs Manageable

To make costs manageable, it's desirable to be able to predict, for every volume of output, the needed costs to support that output. Do nothing; costs nothing. Do X; costs Y, And ideally, if I do 2X, it costs no more than 2Y.

To do that, we need to get rid of fixed costs and turn them into variable costs. (See part 1 for definitions.) The difference between a fixed cost and a variable cost is one of time period; taken over the long run, all costs begin to look variable. Shorter time periods, and they tend towards fixed. I'd suggest a sensible window here; say 3 years (although in the current economic climate, that feels like a very long time indeed). 

Fixed costs, especially indirect fixed costs, are the enemy of good cost management. In general, we want to turn fixed costs into variable and indirect costs (a cost not associated with a specific activity) into a direct cost.

We also want our variable costs to act "sensibly". So, for instance, we want 2X to be less than cost 2Y; we want to achieve economies of scale.

[That leads into a separate issue, one not covered here.

"Thus, while a decision to increase its scale of operations may result in decreasing the average cost of inputs (volume discounts), it could also give rise to diseconomies of scale if its subsequently widened distribution network is inefficient because not enough transport trucks were invested in as well."

The effect is well known to IT capacity planners and performance analysts; remove a bottleneck (for instance, IO bandwidth on a SAN) and it simply pops up somewhere else down the line (network bandwidth to deliver the data).]

Storage Cost Drivers

• Capacity; the request for space
• Performance; IO bandwidth the need for speed
• Availability; access frequency and times of access to the data
• Activity; writing and reading the data
• Reliability; Recovery, Backup & Restore
• Archive; retaining the data

That's a high level list, but there's one thing to note here; these are from the consumer's perspective. These are activities that drive your costs, so management of storage isn't included here, because it's not a driver. Users don't care how you manage, and it's not something they measure.

This is a pretty simple model. For instance, the category Archive might be broken out into sub-categories, which I won't do here.

Cost of Storage Example

Let's take a simple example.

Department X requires to store and retrieve engineering drawings. They've estimated the following;

• Capacity. Files are on average 10MB in size, with an initial requirement of 2TB of space. Growth is anticipated at 20% per year (400GB).
• Performance. Low to medium requirement. CAD software loads one file at a time. Engineers will wait for several seconds for a file load or save. No IO requirement while it's being edited.
• Availability; access to current and the last 30 days versions of drawings is required during normal weekday work hours
• Activity; anticipated 10 CAD workstations with a new or modified existing engineering drawing every 20 minutes.
• Reliability; 30 days backup, 1 hour granularity, self recovery preferable.
• Archive; every engineering drawing and version must be archived for at least 10 years.

---

The next in the series will take this example and map the drivers to the costs.