I worked for Brocade, a Fibre Channel switch company, from 2000 to 2005. About my third week on the job, Nishan ran full-page ads in the Wall Street Journal announcing Storage over IP (SoIP) and the imminent death of Fibre Channel. I suspect the ads cost Nishan more than the entire lifetime revenue of the company, but it did kick off a war that lasted 6 years - the iSCSI vs. Fibre Channel war.
In the beginning, the debate was like CNN Hardball - lots of dogmatic arguments and very little listening. When iSCSI products came to market and matured, it became clear that iSCSI would dominate the market for low-cost block network storage, and Fibre Channel would remain king in the high end. The problem with this for customers is that it forced them to make a major choice in storage switching infrastructure based on some pretty subtle differences. Lots of money was wasted putting low-end application servers on expensive Fibre Channel networks.
Never bet against Ethernet in the long run (remember token ring?). The Fibre Channel community - including Brocade, Emulex, Qlogic, and Cisco - has come together with the Ethernet community and defined two new standards that will allow a graceful migration of Fibre Channel networks to 10G Ethernet over the next several years. Data Center Bridging (DCB) is a set of extensions to 10G Ethernet that add the flow control and traffic prioritization that made Fibre Channel well suited to storage traffic. Fibre Channel over Ethernet (FCoE) makes the Fibre Channel framing and management protocols work over layer 2 Ethernet. It can't be directly routed over WANs since it does not use the IP layer, but then neither could Fibre Channel. FCoE also leverages much of the management tools and host-side driver work in the new Converged Network Adaptors (CNAs) that attach to the DCB 10GE network.
DCB is not just for FCoE. The “lossless” characteristics will also help other services such as NFS, CIFS, and iSCSI. All of these can run alongside each other on the same physical network. This is the real win for end users.
It will take a little while for all the standards to settle out. FCoE should be final by the middle of 2009 and DCB by the end of 2009, but there are first generation products out now. NetApp will ship FCoE target connectivity in our FAS systems around the end of 2008.
The net effect is twofold: Customers looking at building new Data Centers in the 2010 and after timeframe can choose to use a unified fabric technology - 10GE with DCB - for all of their server-server and server-storage needs. This kind of volume adoption will drive cost and prices down - something which the duoplistic nature of the Fibre Channel industry could never achieve.
In the near term, Fibre Channel customers can extend their fabrics using switches that bridge 10GE to Fibre Channel like the Cisco Nexus systems. New servers can be attached to 10GE using CNAs and access the Fibre Channel attached storage already in place. So customer can migrate gradually, or do it all at once with a new facility.
So does this mean the death of Fibre Channel? Not any time soon since there is so much of it out there. But I would bet that the generation beyond 8G FC will never see much adoption. By the time it might be available, 10GE adoption will be well along and 40Gbit Ethernet will be on the horizon.
I have also been asked if this means the death of iSCSI? Absolutely not. First, customers can run iSCSI and FCoE over the same 10G Ethernet DCB fabric. Some servers using iSCSI, some using FCoE depending on their needs and past. The physical network - the real investment - is the same. iSCSI also will continue to be the only block data protocol running over 1G Ethernet which will be around in Data Center for a decade or more.
The future is set. The only question is how fast it gets here. I believe that half the applications using Fibre Channel attachment today will be migrated to Ethernet within 5 years - the end of 2013. Virtualization will lower the absolute number of servers and ports, but by that time the trend will be unstoppable.
So what should IT managers do? If you are not planning a new storage fabric, there is no rush. If you are adding to your Fibre Channel fabric a few ports at a time, keep doing that since it works. Let the early adopters get some experience with the FCoE adaptors and DCB switches over the next year. If you are building a new Data Center or storage fabric for deployment in 2010 or later, you need to understand the 10GE option. It will most likely save you money. It is definitely the way the industry will go in the long run. I would hate to be the guy who put in the LAST new Fibre Channel network.
