My Storage Systems TextBook

Three years ago I was asked to teach a graduate course on Storage Systems at University of California as a visiting faculty member. So, as I was preparing the curriculum I realized that there did not exist a good textbook on storage systems. By Storage Systems I mean the area below a database system. They have many textbooks on database systems, communication systems, programming languages and operating systems. But in my opinion there does not exist a good textbook on storage systems. So, I planned the lectures such that each week's lectures would contain the content present in one chapter of the textbook. Here is an outline that can be used by prospective authors of storage systems textbook:

Storage Architectures (block storage, file storage, object storage, SAN file systems, clustered systems): These lectures discussed the different types of storage architectures. We also discussed the trade-offs between these different types of architectures.

Storage Devices (Storage Controllers, Disks, Tapes, SSDs, Optical Devices): These lectures described the architectural and operational details of these different devices (and the variants in each device type). We also discussed the relative tradeoffs between these different device types.

File System: These lectures first dealt with the basics of a file system, and then discussed the design choices that were made by different types of file systems like GPFS, GFS, WAFL, EXT3 etc.

Storage Protocols (parallel SCSI, FCP, iSCSI, SATA, SAS, NFS, CIFS, pNFS, different RDMA protocols, WebDAV): These lectures dealt with different types of storage protocols. Once again we discussed the trade-offs between the different types of protocols.

Storage Protection Mechanisms (RAID algorithms, checksum techniques, Redundancy components in the controller, scrubbing techniques, DR services, long term data preservation): In these lectures we covered the different ways to protect oneself from disk failures, head failures, site failures, and also different ways to detect and correct data corruption.

Storage Efficiency Techniques (De-duplication, Compression, Thin Provisioning): I did not have time to cover this topic. However, storage efficiency has become an extremely important area of research.

Storage Management: In these lectures we covered the basic framework for monitoring, analyzing, planning and executing storage management tasks such as provisioning, backup/recovery, performance management etc. We also dealt with different types of storage virtualization techniques, and also storage management within the context of server virtualization at the host.

Storage Security/privacy: These lectures dealt with on-disk, on-wire, access control, authentication, provenance and trust issues. We also discussed the different types of possible threats and how to protect against them.

Storage Power Management: These lectures dealt with power management metrics, proactive (high density disks, efficient power sources, compression, de-dup) and reactive power management techniques (disks spin-down, shutdown), and also briefly dealt with data center cooling techniques.

Performance Enhancement (Caching, Pre-fetching, Log Structured file system, Data Layout, I/O scheduling): These lectures discuss different techniques for improving the overall I/O performance.

Workload Classification: I did not have time to cover this topic but the goal is to discuss the various types of workloads (HPC, DSS, OLTP, Archival etc) and their impact on the design of the storage systems with respect to performance.

So, as you can see a lot was covered in that class. Most importantly, the students enjoyed the class (they gave a positive evaluation) and many of them are doing very well with respect to working at good companies, and also in their PhD research with respect to publishing papers at reputed conferences. I am interested in getting feedback wrt what other topics should be covered if I write this book.

Comments

[Dirk Meister]

Are you planning to write this book? That would be great because such a textbook is really missing. We help us out with lots of research papers as reading material, but a textbook focused on that topic would be helpful.

[Sim]

Please write this book! There really is a dearth of information in this area.

You wouldn't be able to make your lecture notes, slides or presentations available in the interim? :)