ONTAP GX—Past and Future
This Monday we released ONTAP GX, which is a super-high performance clustered version of our software. GX combines Data ONTAP with the technology from our Spinnaker acquisition in 2003. Since you can read about how great GX is from our press release, web site, and in trade press (more press, and more), I'll talk about the big picture.
Origins of GX
GX descends from a distributed filesystem called AFS. AFS was developed at Carnegie Mellon University (CMU), spun out into Transarc Corporation, and eventually purchased by IBM. AFS didn't win in the market, but the customers who used it—like Wells Fargo and Intel—absolutely loved it. Much of the team that built AFS re-formed as Spinnaker in an attempt to apply their experience and do it better this time around.
A big advantage of AFS was that it used a powerful (but proprietary) protocol to connect the systems that were sharing data. It had excellent caching properties that allowed high performance over widely spread systems, and it had data management capabilities that most storage systems still don't have today. Unfortunately, the proprietary protocol was also a disadvantage because it required OS changes to every computer. This made AFS expensive to develop and maintain, and it was difficult for customers to manage, especially in environments with many different operating systems.
Spinnaker solved these problems by applying AFS-like techniques within a cluster of appliances, but using standard protocols to communicate with the outside world. In other words, the team created a powerful new architecture that combined their best ideas from AFS with the appliance philosophy pioneered by NetApp.
Acquiring Spinnaker and Merging the Technology
In some ways the acquisition felt very natural. Mike Kazar helped invent AFS at CMU, before co-founding Spinnaker, and I borrowed some of his ideas in developing WAFL. I knew his work and admired him. Likewise, the Spinnaker folks viewed NetApp as a source of inspiration on the appliance philosophy, and they respected us, despite the fact that their business plan was to attack us!
I believe the acquisition went so well in part because both teams had strong mutual respect. The beauty of GX is that it combines Spinnaker's overall architecture, including global namespace and clustering, with NetApp's crown jewels, including WAFL and RAID-DP. It runs on standard NetApp hardware, giving customers flexibility. The result is amazingly scalable and easy to manage. We just posted a million-ops-per-second SPEC SFS benchmark result that is triple the previous best result. It's easy to manage because it acts like a single system even as you add more compute nodes and storage.
I admit that merging Spinnaker and ONTAP took longer than we expected. In 2003 we hoped to ship by late 2005. We also changed our release strategy. Initially we planned to ship the merged software as the next major release of ONTAP. However, we realized that so much change would piss-off our enterprise customers. Instead, we will develop GX in parallel with ONTAP 7G for the next few years. Right now the interfaces between 7G and GX are not as similar as I'd like, but we'll enhance them both, and converge them over time, much as Microsoft converged Windows and NT.
I believe that GX puts us way ahead of large competitors like EMC, Hitachi, and HP. None of them have anything like this—certainly nothing that runs on their standard hardware. Our advantage over small competitors (like Isilon, 3Par, Ibrix, Panasas) is our rich data management features. These features mostly aren't turned on in this first release, but they are sitting there in the code—thousands of engineer-years worth of technology just waiting to be tested.
Future Strategy
The first market for GX is high performance computing (HPC)—chip simulation, giant software builds, oil and gas seismic processing, Hollywood special effects and the like. Here's a good rule of thumb: If you have hundreds of nodes of Linux, all working in parallel on the same task, then you are a great candidate for GX.
In the long run, ONTAP GX is for everyone. The benefits go way beyond speed and scalability. I think of GX as a whole new dimension of virtualization. (I define "virtualization" as making the physical reality that you are stuck with into a virtual reality that you like better.) With GX, we virtualize multiple systems so that they look like a single large system. This not only improves performance and simplifies management, but it enables many new opportunities for automation.
I believe that the combination of virtualization and automation creates a powerful foundation for innovation in storage.
Origins of GX
GX descends from a distributed filesystem called AFS. AFS was developed at Carnegie Mellon University (CMU), spun out into Transarc Corporation, and eventually purchased by IBM. AFS didn't win in the market, but the customers who used it—like Wells Fargo and Intel—absolutely loved it. Much of the team that built AFS re-formed as Spinnaker in an attempt to apply their experience and do it better this time around.
A big advantage of AFS was that it used a powerful (but proprietary) protocol to connect the systems that were sharing data. It had excellent caching properties that allowed high performance over widely spread systems, and it had data management capabilities that most storage systems still don't have today. Unfortunately, the proprietary protocol was also a disadvantage because it required OS changes to every computer. This made AFS expensive to develop and maintain, and it was difficult for customers to manage, especially in environments with many different operating systems.
Spinnaker solved these problems by applying AFS-like techniques within a cluster of appliances, but using standard protocols to communicate with the outside world. In other words, the team created a powerful new architecture that combined their best ideas from AFS with the appliance philosophy pioneered by NetApp.
Acquiring Spinnaker and Merging the Technology
In some ways the acquisition felt very natural. Mike Kazar helped invent AFS at CMU, before co-founding Spinnaker, and I borrowed some of his ideas in developing WAFL. I knew his work and admired him. Likewise, the Spinnaker folks viewed NetApp as a source of inspiration on the appliance philosophy, and they respected us, despite the fact that their business plan was to attack us!
I believe the acquisition went so well in part because both teams had strong mutual respect. The beauty of GX is that it combines Spinnaker's overall architecture, including global namespace and clustering, with NetApp's crown jewels, including WAFL and RAID-DP. It runs on standard NetApp hardware, giving customers flexibility. The result is amazingly scalable and easy to manage. We just posted a million-ops-per-second SPEC SFS benchmark result that is triple the previous best result. It's easy to manage because it acts like a single system even as you add more compute nodes and storage.
I admit that merging Spinnaker and ONTAP took longer than we expected. In 2003 we hoped to ship by late 2005. We also changed our release strategy. Initially we planned to ship the merged software as the next major release of ONTAP. However, we realized that so much change would piss-off our enterprise customers. Instead, we will develop GX in parallel with ONTAP 7G for the next few years. Right now the interfaces between 7G and GX are not as similar as I'd like, but we'll enhance them both, and converge them over time, much as Microsoft converged Windows and NT.
I believe that GX puts us way ahead of large competitors like EMC, Hitachi, and HP. None of them have anything like this—certainly nothing that runs on their standard hardware. Our advantage over small competitors (like Isilon, 3Par, Ibrix, Panasas) is our rich data management features. These features mostly aren't turned on in this first release, but they are sitting there in the code—thousands of engineer-years worth of technology just waiting to be tested.
Future Strategy
The first market for GX is high performance computing (HPC)—chip simulation, giant software builds, oil and gas seismic processing, Hollywood special effects and the like. Here's a good rule of thumb: If you have hundreds of nodes of Linux, all working in parallel on the same task, then you are a great candidate for GX.
In the long run, ONTAP GX is for everyone. The benefits go way beyond speed and scalability. I think of GX as a whole new dimension of virtualization. (I define "virtualization" as making the physical reality that you are stuck with into a virtual reality that you like better.) With GX, we virtualize multiple systems so that they look like a single large system. This not only improves performance and simplifies management, but it enables many new opportunities for automation.
I believe that the combination of virtualization and automation creates a powerful foundation for innovation in storage.
Comments