Extensible NetApp: Thinking outside of the box

Shut Down for the Holidays

This is my last post for 2008.

To all of my readers, be happy and be safe.

And I can’t wait to get going in 2009.

 

SANscreen 5.0

Today NetApp announced the latest version of our SANscreen product, SANscreen 5.0. This is a big release that enhances, what I think is, the best heterogeneous SRM product.

Delivering a service oriented view of storage

We are in the process of watching the evolution of storage from an interesting set of devices with capabilities to a service driven infrastructure. What do I mean by service driven infrastructure? Prosaically, it's the aggregation of a set of capabilities, things like performance, reliability, protocol, cost and a commitment that the storage will provide those capabilities, also known as the Service Level Agreement.

For some companies this has already happened. For others, it's a work in progress.

One of the key challenges is how to provide the visibility into the storage tier to ensure that the services promised are the services delivered. 

SANscreen, originally built by Onaro, has continued to evolve and extend it's core value proposition, the ability to quickly and easily see what is going on throughout the storage infrastructure. Using an agentless infrastructure, SANscreen is able to see more, faster and more reliably than any other tool.

Building on top of this very efficient data collection mechanism, SANscreen allows storage teams to monitor the service levels of their storage infrastructure.

So what's new?

The latest release extends SANscreen in three interesting ways:

1. A data warehousing infrastructure that allow storage architects to slice and dice their data
2. Extend SANscreen to iSCSI infrastructures.
3. Deliver the kind of analytic tools necessary to execute a storage tiering strategy.

iSCSI and SANscreen 

It's more than just an FC world out there, and any SRM tool needs to understand iSCSI infrastructure. And SANscreen is no different.

Storage architects that have used SANscreen understand how providing visibility into the infrastructure in an almost real time way makes it possible to mange the storage sprawl. Onaro got it's start clarifying the complex SAN fabrics that were put together over the last decade.

An increasingly important part of the storage landscape is iSCSI. An increasingly challenging part of deploying iSCSI is how to manage the fabric and the storage attached to that fabric.

With the latest release of SANscreen, NetApp is providing a tool that although not providing an end-to-end fabric view, does provide insight into the fabric end-points.

Storage Tiering 

When you're flush, time is money, so you spend money to save time. When you're short on cash you spend time to save money. The trick is to get enough bang for the buck for the time you spend.

Storage analysts talk about the need to have a comprehensive storage tiering strategy that spans the entire enterprise, and then stare dumbly as storage architects ask the inevitable question; how? Hetero SRM tools never really got enough information to be useful, homogeneous tools only told you how to use their stuff effectively.

And don't get me started on the question of FC vs iSCSI.  Never mind what is real FC, how about the basic question of how much FC do I need and how much iSCSI do I need?

To produce a reasonable tiering strategy you need complete, up-to-date, comprehensive data and the analytics necessary to do something with that data. SANscreen has always had the comprehensive, up-to-date and complete part covered, now we added the analytics.

What exactly do I mean by analytics?

1. Provide reports into how capacity is being used

    

2. Map capacity used by business group to some notion of dollars per terabyte.

    

Using both of these mechanisms it's possible to understand the current behavior in your storage infrastructure, and more importantly change it.

Let's be precise, if you want to drive to a particular global view of $/TB, understanding what's going is the first step. But once you have that information, it's really important to act on it. And once you've acted on the information see what happened.

I call this the find, act, verify cycle of storage management.

Too many tools make the find and verify phase an unending adventure. Make those two phases quick and painless, and you'll spend more time doing and less time trying to figure out what to do. Net result, you'll get more efficiency out of your infrastructure.

What's really interesting is that when you combine SANscreen's ability to monitor storage SLA's with the new tiering tools, it becomes possible to correctly identify storage that is on the wrong tier.  And even better, if storage is moved to the wrong tier, then the storage group can discover that and move the storage back without having to deal with the dreaded scream at storage group, storage group fixes problem, scream storage group for creating problem cycle.

But for any tiering strategy to be effective, the consumer has to be involved. And that's where SANscreen's new charge back capabilities come into play.

 

If you measure behavior, and show how the measured behavior is deviating from the goal, you'd be surprised at how quickly the behavior moves towards the goal.

For more information check out the website where you'll find a couple of new whitepapers that describe the product in more detail.

Backup, the new storage tiers and real snapshots

When I first described my new terminology for storage tiers, Tim Burlowski, asked how backup and recovery fit into that model.

It turns out that that is an interesting question.

Backup, the only working ILM/HSM solution

Step 10 feet away, and you realize that backup is HSM.

The backup process creates cold data, the backup image. The backup software them moves the backup copy to cheaper tier of storage. When a restore is performed, the backup software moves the backup copy from the cheaper tier to a more expensive tier.

The question on the table, then, is where does this form of HSM still make sense?

Captive IOPS

One obvious place where HSM as part of backup process is for the backup of data on a Captive IOPS tier. The tier is built using very expensive storage. Storing cold data that will probably never get accessed seems odd and a waste of money.

But that's not necessarily true.

If the Captive tier is built using disk drives, and the application  has a high IOPS data density, a lot of the capacity of the disks is  unused. The cheapest place to store backup copies, in that case, would be on the same disks you're using for primary data.

If the Captive tier is built using SSD's like EMC storage, then it makes a lot of sense to use a backup solution to move backup images off of the flash onto cheaper storage. If the application has a low IOPS data density, then a better approach is to use flash cache in front of disk.

If the Captive tier is built using a flash cache in front of disk, the cheapest place to store the backup copies may be on the disk drives.

More generally, the backup methodology of a captive tier made up of flash cache and disk, will be the same as the backup methodology of the capacity tier.

So what about the capacity tier and backup?

In the general case, the capacity tier will be backed up much in the same way that it always has been.

A capacity tier that has real snapshots can eliminate the HSM part of the backup process.

In fact, in an earlier series on backup I proved that for a VMware infrastructure, the cheapest place to store backup copies was on the same disks that stored the active data.

Now I want to generalize that comment for all storage..

If you consider IOPS density, disk drives must contain large amounts of cold data. A subset of the cold data in a data center is the backup data. That data can either be stored on a separate set of disks or on the same disks that are being used to serve IOPS.

To store the data on the same disks the following things need to be true

1. Creating the backup image must consume minimal IOPS
2. The existence of the backup image must impose no performance penalty
3. Be able to store a large number of images.
4. The storage must be configured using some form of RAID-6 or RAID 1-0
5. The cost of the backup images must be cost effective

It turns out that for NetApp systems all five are true.

So as disk capacities expand, storage architects who have the option to use real snapshots will stop moving data off of the capacity tier to some cheaper tier.

Said in a slightly more poetic way:

The final resting place for data on disk will be the disk where the data first got created.

The net effect will be a much simpler and more cost effective storage infrastructure to support backup.

One minor addendunm added after I wrote this post.

As Martin G says in the comments below, having the backup copies on the local disks doesn't protect you from site failures. So, of course, some form of storage replication is required. Thankfully NetApp has a cool technology called SnapMirror that replicates all of the snapshots to a remote destination in a space efficient way.

In an earlier post on VMware and backup I show how all of this fits together.

I should have included this need to replicate the data in my original blog post. So I am rectifying that error now.

MAD Blog: Chuck, there you go again.

Today is a fun day.

Chuck Hollis has taken me to task for not understanding what real fiber channel is. And he claims that because he's been selling FC for longer, we should trust him.

Fine, I don't know what real fiber channel is and, frankly, I don't care.

But guess what, I've been at a company selling unified storage for a lot longer than EMC has.

So appealing to my seniority, much like Chuck appeals to his seniority in the FC space, the  NX4 is not real unified storage. The value proposition of the NX4 is not that of unified storage, no matter how hard he and EMC wishes it was.

MAD Blog: Onaro is not as much rubbish!

I couldn't help but laugh when I read Martin's blog about ECC.

When I first started working in the storage management space, i tried to understand who the competition was and understand why they were successful.

It was only after I experienced ECC in all of its pointless glory, did I truly appreciate how much opportunity for innovation there was in this space. And it was precisely because of that opportunity that I worked in storage management for about 5 years. One of the things we tried to do at NetApp was to try and build products our customers wanted to use. Products that solved real problems. And I wish I could say we had the perfect product, but that would be foolish. I can say that there were a lot of customers who really liked what they got. I hope some of them didn't think it was rubbish...

But I was delighted to read in a later post that Onaro may be useful. That was indeed high praise from someone so obviously and justifiably frustrated with the state of the art in vendor SRM tools.

When I first met the Onaro guys i was genuinely impressed with what they had. This was a very smart team, with a very cool product underpinned by some very interesting technology that solved a problem that, frankly, EMC and ECC should have solved a years ago.

To this day, Chuck's comments about NetApp's Onaro acquisition remind me of this parable.

Back from two days in New York

Just spent two days in New York talking to customers about Disk to Disk backup.

Had a great meal at artisanal cheese.

Bought a stuffed animal at the last remaining FAO Schwartz. p>

Flash and the new storage classification model

So I'll be out-of-the-office  this week visiting some customers, which may mean that my blog writing may slow down.

After defining some new storage tiers, repeated here for clarity of exposition

Two IOPS tiers:

• Captive (used to be Captive IOPS)
• Shared (used to be Shared IOPS)

And a capacity tier called

• Capacity (used to be Capacity Efficient)

If you followed my post from last week, you'll remember that I made the case for disk being the media of choice for any capacity tier, because the IOPS density of disk matches the economics of this tier.

Flash, unlike disk, has an IOPS density, that is closer to 1 than to 0. That suggest that Flash belongs elsewhere ...

Captive and Shared Tiers Go Flash?

Chuck Hollis has made the case that Captive and Shared tiers will go Flash. Ironically enough, I agree with Chuck but I disagree on significant details.

EMC has positioned flash only as a stable storage. Chuck has argued that the entire dataset of Captive and Shared tiers belongs on SSDs. I think that's wrong.

NetApp has positioned flash as both a new layer in the memory hierarchy and stable storage and intends to deliver both, and I, rather unsurprisingly, think that's right.   

Why the difference?

If you read my post on IOPS data density, you'll recall that I argued that some pretty important datasets actually have a very low IOPS data density. What that means is that the dataset requires a medium that has an IOPS density that is closer to zero than to one.

But flash has an IOPS density that is closer to one than to zero...

Following that logic, you would  have to conclude that disk not flash is the right medium for those datasets..

Which suggests that Flash as Stable Storage is not the right answer for every byte of every dataset that gets deployed on Captive storage tiers!   

So where does Flash fit in?

If you read my post on The Principle of Uniform Resource Latency, you'll see that I argue that it's not the absolute performance that matters but the worst case performance. Storage arrays provide significant amount of memory to cache read and write operations. If an entire dataset fits into the memory of either the client system or the storage array, then the performance of the system is defined by the memory operations. On the other hand, if the dataset does not fit into main memory, then performance is constrained by the ability to push data to and from disk.

So now we have three kinds of datasets;

1. Low IOPS data density
2. High IOPS data density but fit into memory
3. High IOPS data density but don't fit into memory

And only one of those datasets, the third, appears to be a candidate for (Enterprise Flash Drives) EFD. But it's not that simple.

Remember the goal is to improve performance. Putting the dataset on an EFD is an approach another is to put enough memory, like a PAM module, in the storage array to avoid having to go to disk.

Which brings me to NetApp's flash strategy.

1. Provide Flash as an alternative to disk for applications that really require that kind of storage.
2. Provide Flash as a cheaper read cache for every other application

So the role of Flash?

In my opinion, I believe that flash will play two roles:

1. Be a memory cache
2. Be stable storage

And that because of the economics of (1), will probably play a role in all storage tiers, including capacity tiers. As for (2), I believe, that the role will be a lot more limited than we imagine today.

Which wins out, of course, is a matter of conjecture today, but we'll definitely know the answer in a couple of years.