Over the years, potential NetApp customers have been mystified about the relationship between WAFL and SAN. In particular, are NetApp LUN’s a file on top of a file-system?
For someone who has worked at NetApp for many years, the question always struck me as odd, because my one experience using WAFL was that it provided mechanisms that allowed you to build a file system, but was not itself a file system.
My first project at NetApp was to build a streaming media cache using WAFL. And for us WAFL was a way to move data to and from disk and a way to store our on-disk format, but was most certainly not a file system. For a LEGAL description you can check out our patent.
Consider that a file system has rules about how you access and find objects the file system manages. Conventionally, you first have to do a directory look up, and then open the file, and then do the read or write operations on a file handle returned by the file system. WAFL imposed no such constraints on our design. We could directly write to or read from an object. We were free to use what ever name lookup scheme we wanted.
As a concrete example, when we were serving a piece of a media stream we would use a clever hash to find the object on disk without doing a name look up, effectively bypassing the entire traditional directory lookup.
What WAFL provided was a mechanism for tracking all of the disk blocks. WAFL also managed the free space. WAFL was responsible for ensuring all of the data got written to disk reliably.
In fact, when we wanted to use WAFL as a standard file-system we discovered that WAFL did not provide the necessary interfaces. For example, WAFL had no mechanism for handling open and close operations. More annoyingly, WAFL was optimized, circa 2000, for 32kB and 64kB read and write operations not 12 Byte writes.
To build our product we ended up using a whole bunch of WAFL mechanism to build our own specialized data store optimized for streaming media. During the entire experience, WAFL never felt like a file system. A set of primitives that made it possible for us to build our own file-system like layer sure, but most definitely not a file-system.
So my personal experience was that building a high performance file system on top of WAFL was pretty easy, but WAFL was no file system.
However, given the fact that our founder, Dave Hitz, called WAFL a file-system, and our original patents and a lot of our historical literature call it a file system, I realize that my point of view may not be shared.
So over the next couple of days I will dig into this topic.
Comments