What is WAFL part III: An Answer

Let’s agree that a file system has the following elements:

1. Methods to allow operations on files
2. Methods to allow operations on directories
3. Methods for storing data on disk
4. Methods for retrieving data on data.
5. Data structures for representing data on disk
6. Rules about how files are found
7. Method for organizing disks into large physical pools
8. Methods of partitioning physical pools into smaller logical pools

Let’s focus a little bit on item #1 and #2. and #6

In the particular case of WAFL, what is a file or what is a directory is defined by the distributed file system WAFL is a part of. And very early on, and before I joined, and before I even knew what Data ONTAP was, smarter people than me figured out the following interesting layering:

What they did was create a common set of primitives on top of which the NFS and CIFS semantics could be commonly layered. The NFS and CIFS semantics, were part of the protocol stack, the shared primitives were part of WAFL. So even at this early stage, WAFL provided a mechanism for storing files and directories, without assuming the semantics of those operations.

In fact the rules about how  files are looked up is defined by how NFS clients choose to lookup files rather than how Data ONTAP works.

In this GX paper there is a discussion in section 5 of this separation in the context of the Spin NP protocol. Although the discussion is about GX, it supports much of what I am saying here.

So is just providing mechanisms sufficient to be called a file system? I’m not so sure.

At this point, my seems closed but ....

Now let’s focus on part 3 and 4.

It turns out that the layering of file-systems has evolved over the last 20 years. When I was in college, a file system was responsible for the disk layout, because the OS knew where the data blocks resided. Today, with LUN’s, many file systems persist in the illusion, but the reality is that the layout of blocks is a mystery. The genius of Hitz and Lau was that the designed WAFL around the explicit assumption that WAFL did not control exactly where disk blocks were allocated, but instead, a lower layer, RAID did that.

Ironically, ONTAP RAID is particularly tuned to work with WAFL. The two are very much tied together. So although today almost every file system lives on top of some RAID layer, only WAFL and ONTAP RAID know enough about each other to take advantage of each other.

But to the question of whether WAFL is a file system, well it does some of 3 and some of 4, but it very much relies on something outside of WAFL to do things that traditionally were the domain of the file-system.

And what about parts 7 and 8?

Today most file system vendors consider the volume manager (8) and the logical volume manager (7) to be external to the file-system. In fact, Symmantec clearly distinguishes between the volume manager, vxvm and the file system vxfs.

But at NetApp, WAFL is both the volume manager and logical volume manager.

Which leaves 5?

And yes, the on-disk data structures are all WAFL data structures. And this is an important point. Because WAFL has a single on-disk format, NFS and CIFS file operations can be performed on the same underlying on-disk object.

Well then what is WAFL?

Well it provides mechanisms for building file-system semantics, it manages the on-disk format, it manages the free and allocated space, and provides a logical and physical volume manager.

So in my mind, that’s not enough to call it a file system. WAFL is a necessary part of a file-system, but not sufficient.

In my next post I will explain how SAN fits into the picture and provides further evidence that WAFL is not a file system.

Comments

[Anil Gupta]

Kostadis,

You are missing two elements, IMO.

1. Deleting data
2. Reclaiming space.

Anil

[Martin G]

Kostadis,
I look forward to the next part because at the moment I'm leaning towards feeling what you describe is an incompletely implemented file-system. So really looking forward to the next part and the reveal.

[kostadis roussos]

Martin G,

I finished the series. Hope to see your reaction.

cheers,
kostadis

[kostadis roussos]

Anil,

Good point.

And actually the two use cases illustrate my point.

NFS provides a set of interfaces for deleting data. The ability to truncate a file using SETATTR deletes data. The ability to REMOVE a file also deletes data.

Internally WAFL has a set of mechanisms that allows both of those operations to be implemented.

However, the actual management of free space is done by WAFL itself.

cheers,
kostadis

[Shehjar Tikoo]

Hi

What they did was create a common set of primitives on top of which the NFS and CIFS semantics could be commonly layered. The NFS and CIFS semantics, were part of the protocol stack, the shared primitives were part of WAFL. So even at this early stage, WAFL provided a mechanism for storing files and directories, without assuming the semantics of those operations.

Arent those just the expectations from a virtual file system?

Well it provides mechanisms for building file-system semantics, it manages the on-disk format, it manages the free and allocated space, and provides a logical and physical volume manager.

That is a very interesting way to look at it. Perhaps, a question future file system developers can ask themselves: What if we leave out some aspects of a file system and only provide certain functionality? What kinds of layering can we get, by combining different subsets of functions?

[Shehjar Tikoo]

Oops, I quoted a few paras from your post within italics, but looks like the blog engine gobbled those up. :/

[kostadis roussos]

Shehjar,

Whether it's a VFS layer or not is an interesting question. VFS was intended to abstract out the details to the client application of whether the underlying file system was ufs or nfs or something else.

In WAFL's case WAFL sits below the software that makes the file system NFS.

But I think you get the broader point, that file systems assume a specific kind of layering.

What WAFL demonstrates is that within the set of layers of a file system there are software layers that are not file systems but that add tremendous value.

Thanks for your post.

kostadis