Solid State Storage – the Sinatra Way

NetApp has built our business on leading via innovation, instead of following conventional wisdom.  In our earliest days, we bypassed file-serving via a general-purpose OS in favor of an efficient stripped down micro-kernel which delivers far more performance and efficiency, while offering more stability without the extra code bloat.  We chose native kernel-mode protocol implementations for NFS and CIFS instead of user-mode protocol emulation.  That enabled maximum performance and even true multi-protocol locking on the same file if necessary.

Thinovation

Other even more profound innovations feature the industry’s only RAID without compromise (enabling maximum performance, availability and efficiency in the same configuration), giving life to low-cost ATA technology in enterprise settings.  We stood out with native Thin functionality (Provisioning / Cloning / Replication) implemented at the core of our I/O engine to elegantly complement our still-exclusive dedupe capabilities on ALL tiers of storage, featuring primary instead of merely backup and archive.

Deep application integration and scalable policy-based management helps you actually apply this storage innovation to your most common enterprise apps.  Modular and consistent controller architectures with in-place upgrades make these sometimes exotic features all the more approachable to every IT department.

Unified Storage 2.0

The advent of Solid State announcements among Enterprise Storage vendors over the past year offered NetApp once again an opportunity to bypass conventional wisdom.  So we chose to redefine the concept of Unified Storage which NetApp coined years ago with our first combined SAN/NAS/iSCSI array.

Welcome to NetApp Unified Storage 2.0 -  doing Solid State Storage – our way.

The Big Picture

Solid state drives (SSD’s) are very cool technology, but the potential for Solid State Storage (SSS) in an enterprise context is far greater.  Earlier last summer, Nik Simpson from the Burton Group wrote an excellent blog outlining the strategic opportunities entailed.  Let’s review each one below.

Form Factors

RAM and Flash-based transistors can be packaged in more optimal ways that the conventional 3.5” or 2.5” SFF hard disk canisters commonly found in other enterprise storage arrays.  Breaking those molds helped NetApp introduce the benefits of SSS via rack-optimized shelf-configurations instead of retrofitting shoehorned SSD’s into disk shelves optimized for the mechanical, electrical and performance characteristics of spinning media.

Tomorrow (Tuesday Feb 3rd) NetApp will introduce the latest in SSS innovations, combining an unparalleled 30 years of SSS experience with the industry's best platform for storage provisioning & efficiency, data management and protection.  Our unique partnership with TMS offers NetApp customers unparalleled simplicity, safety, predictability, performance and efficiency for their performance-hungry applications.

NetApp’s modular storage architecture enables new dimensions of Unified Storage (US).  Other US 1.0 offerings hint at niche products with front-end multi-protocol support (SAN / NAS / iSCSI) via emulated protocol implementations built on top of one “native” protocol, often FC with the vendor lock-in of only one shelf type.

The NBC Superbowl Ads Proved - 3D is all the Rage

NetApp’s Unified Storage 2.0 not only leads the pack with truly native primary storage implementations of *all* protocols (SAN / NAS / iSCSI *and* FCoE), but also adds a 2nd dimension by enabling backup and (compliance-ready) archive storage via these same protocols as well.

With our open V-Series platform, NetApp adds the critical 3rd dimension to Unified Storage by now including Solid State Shelves to our repertoire of certified heterogeneous storage arrays.

File System Considerations

Nik Simpson of the Burton Group goes on to highlight the necessary evolution required of file systems in order to take full advantage of SSS, while avoiding this new technology’s early pitfalls.

At a high level this means optimizations in general-purpose filesystems like NTFS or UFS and ext3.

At the lowest level, this means device-level optimizations via Flash Translation Layers.

Often feared by our competitors, NetApp’s Write Anywhere Flexible Layout (WAFL) continues to prove its innovative value with each new generation of technology.  Years ago, WAFL enabled NetApp to pioneer the concept of ATA drives which perform and protect enterprise data with FC-like characteristics at dramatically lower prices.  That revolutionary product was called NearStore.

Today WAFL enables a new generation of storage innovation around Solid State.  You see despite all the hype, flash technology’s well-known limitations around write performance and endurance require sophisticated load-balancing and wear-leveling (LBWL) algorithms in order to help that technology match expectations of enterprise storage users.  Many Flash-based SSD’s today employ basic static LBWL algorithms to increase the longevity and performance of their devices.  These are often well-suited to single-tasking deployments for end-user laptops, etc…

More sophisticated SSD’s are beginning to emerge, which use dynamic LBWL algorithms in order to further enhance the performance and longevity of SSD’s in enterprise settings consisting of multi-tasking workloads such as MS Exchange, Oracle, SAP, etc…  These dynamic LBWL algorithms try to minimize the significant flash overhead reserved capacity required and inherent flash fragmentation side-effects of managing the wear & tear of flash under demanding enterprise workloads.

“File System” Serendipity

WAFL’s log-structured architecture implements native load-balancing of write operations via write-aggregation to solid state NVRAM.  This includes an innovative data layout engine which enables WAFL to “write anywhere” in order to optimize the placement of data across the appropriate media.  For flash, that means native built-in wear-leveling optimized to spread writes over as many flash cells as possible in parallel, with minimum wear to each individual flash cell.

Interestingly enough, the very same prophetic algorithms chosen by WAFL’s founders almost 18 years ago to help overcome the limitations of mechanical spinning hard drives are also intrinsically optimized to overcome the very different limitations of solid state storage media.  Some call that irony.  I call that serendipity.

Regardless of what you call it, this long-standing NetApp innovation gives us an 18-year head start on the rest of the enterprise storage industry when it comes to optimizing the performance, efficiency and reliability of SSS containing Flash technology.

Protocols and Timing

In his seminal blog, Nik further states that SSS capabilities have outgrown legacy SCSI, FC and SAS bus throughput & latency overheads.  10GbE, InfiniBand and PCIe interconnects are emerging as popular alternatives which better enable the potential of SSS technology.

NetApp pioneered the DAFS Collaborative as a way to optimize shared I/O in a memory optimized manner as RDMA transfers over capable transports.  I’ve always believed DAFS was ahead of its time, but the advanced concepts introduced by DAFS are perfectly aligned with the needs of SSS, so it would seem now that DAFS’ time has finally come.

Memory and OS Design

Nik concludes with flash futures and where it fits in the memory hierarchy.  Common sense dictates that adding flash closer to the application (within the server or storage controller) yields superior benefits to placing an SSD at the very end of the I/O chain.  NetApp agrees and has optimized our award-winning Data ONTAP for SSS with the Performance Accelerator Module (PAM) Family.

This design delivers especially useful benefits for today’s most common performance-sensitive enterprise workloads such as virtualization boot storms or write-storms.

Where’s the Beef?

Proof points are a hallmark of NetApp product launches, and Flash / SSS is no different.  I am proud to announce that NetApp is the first vendor to publish transparent SSS benchmarks in an independent and audited forum.  Our SPEC results demonstrate not only improved latency, but more importantly improved efficiency which IMHO will be the dark-horse justification for enterprise SSS deployments in 2009.

SSS Dedupe - To Boldly Go Where No One Has Gone Before!

Perhaps the most exciting innovation NetApp has enabled with our SSS / Flash launch on Tuesday Feb 3rd is the ability to dramatically increase usable SSS capacity using our industry-leading dedupe technology.

NetApp dedupe technology is SSS / Flash-aware, meaning our system cache, PAM and TMS RamSan technologies have the potential for an order of magnitude improvement in usable capacity.

While not all data types lend themselves to extensive deduplication, the fastest-growing segment of them do.  Using a common 10:1 ratio means that NetApp customers can enjoy a quantum leap increase in SSS / Flash value.

As Vice-Chair of SNIA's Solid State Storage Initiative, it gives me great pride to introduce NetApp's most comprehensive SSS / Flash lineup to date.  2009 is still young, and we are not nearly done.  Expect more exciting announcements around this subject later in the year as we continue to expose the possibilities of SSS / Flash!

UPDATE Feb 3rd: Official Press Release & Landing Page links added.

Comments

[Chuck Hollis]

Wow.

Somehow I was expecting a bit more than a flash memory card and a bolt-on 3rd party product.

I'll grant you that this approach certainly allows NetApp to "check the box" with an absolute minimum of R+D investment. And, given your situation, maybe this was the right thing for you to do.

But -- really -- can you make an argument that this will be attractive to customers vs. other approaches?

I've read through your post twice now, and -- for some reason -- there isn't any mention of block protocols -- did I miss something?

And Val, as always, you get an A+ for creative writing -- you've got a good knack for this sort of thing.

Best of luck, guys ...

-- Chuck

[Geert]

Well Chuck, read again:

"NetApp’s Unified Storage 2.0 not only leads the pack with truly native primary storage implementations of *all* protocols (SAN / NAS / iSCSI *and* FCoE), [...]"

What else did you miss twice reading this article?

[John]

Is this the response to Netapp's perceived flash gap ? maybe a case of, "How I Learned to Stop Worrying and Love RamSan"

http://en.wikipedia.org/wiki/Missile_gap

[Chuck Hollis]

Hi Geert

There's a difference between "supports" and "recommends", and that's what I couldn't figure out.

For example, all of the charts talk about file system performance, not block I/O.

But maybe you haven't gotten around to testing that part yet ... just checking!

Must be tough putting lipstick on this one ..

-- Chuck

[the storage anarchist]

Val -

Just checking to be sure I've got this straight:

1) Netapps is shipping a DRAM-based PAM that can be used with (most) Netapps Filers, which gets only the largest Filer anywhere close to the global memory supported by a DMX or an USP-V.

2) Netapps now supports putting a TMS RamSan 500 cached Flash RAID controller (http://www.superssd.com/products/ramsan-500/) behind a Netapps Vseries controller.

3) Netapps customers still CANNOT add high-performance flash devices - storage OR cache - to any currently shipping or installed Netapps Filers.

Yawn!

13 months and counting since EMC kicked off this whole new Era of Flash, and Netapps still haven't gotten out of the gate with integrated Flash solutions...

And then you have the gumption to claim that WAFL is the secret sauce that Flash needs, when in fact the TMS RamSan 500 ALREADY BUFFERS WRITES to minimize wear-out of the underlying flash devices...that's why it's called "Cached Flash RAID"!!! (see for yourself http://www.superssd.com/products/ramsan-500/).

Zow-wee, boys, we got some Creative Marketing going on here now, don't we?

Barry

P.S.: Bet you couldn't wait to use that SSSI title in a blog ;^)

[Storagezilla]

Where's Virgo?

I was expecting the 512GB FLASH based PAM module but it looks like I'm too early.

[Stephen Foskett]

Val, I can see how WAFL could be used as a native/integrated replacement for off-the-shelf wear leveling and flash allocation tech, and that this would provide solid benefits in terms of performance and reliability of flash chips. But certainly the TMS is doing its own wear leveling and allocation, so it's not used there. And the PAM isn't really WAFL, right? So where will we see integrated WAFL/flash like you wrote about?

[ThePragmatist]

Who cares how "integrated" at the drive or shelf or cache-level this offering it is!

All I care about is the overall physical density and resulting power drain per system performance.

If NetApp can trump EMC (or others) on these metrics then kudos (and $$) to them. Otherwise all this back-and-forth sniping is moot.

[John]

Pragmatist, well if that's all your bothered about, why gild the lily, just lose the Vfilers and just go with the RamSan. Rack Density will be better as will power consumption and subsequent cooling. Single support contract with no dodgy cross vendor support agreements or overheads.

[John F.]

It's somewhat amusing to watch EMC pound square pegs into round holes (and bash anyone else that doesn't follow suit with a big mallet as well), but there comes a time when you need to step back and think about the underlying technology. After Chuck's littlev tirade about what is physical and what is virtual (he still doesn't comprehend the L in LUN), how can he advocate flash? Hypocracy runs deep at EMC it seems.

At it's most basic level, flash has extremely poor write performance which is limited by the number of erase blocks, and that fact that cells wear. To overcome these limitations and produce the desireable performance levels, you need multiple erase blocks, wear leveling, over provisioning, and sophisticated controllers to handles those tasks. When you look at wear leveling algorithims, guess what, they don't overwrite. Yes, this is hidden from the storage controller when you pound flash into a disk form factor (find the L yet?); still, that goes directly against everything Chuck's been preaching for a year or more... Can I get a halleluah from ya preacher?

I'd personally love to see WAFL applied to the wear leveling algorithms. I think that where we stand today, we're only scratching the surface of where the technology can go. If, in the other hand, you insist on pounding square pegs into round holes, yet again forcing overly complex and contraining APIs upon the user, you'll never get there.

Pound away EMC.