The Pain of Tape-Based Backup—Disruptive Technology and the Red Queen
I love Clayton Christensen's book, The Innovator's Dilemma. I discussed his theory of "disruptive technology" in a blog entry about whether iSCSI is disruptive. (It isn't now but could be eventually.)
Anyway, I recently noticed an odd sort of exception to some of Christensen's rules in the backup market.
Christensen's first rule is that in most markets, user requirements go up a bit every year, but not too fast. If you buy a new car, you hope it's a bit better than your current one—maybe a bit faster, a bit better mileage, or more airbags to make it safer. But mostly you buy a new car to solve the same problems as the old car, so the requirements are about the same.
Christensen's second rule is that technology improves faster than customer requirements, especially in high tech. Ten years ago, my PC was slow and the disk was always full. Today my laptop has CPU to spare and 33 GB of free space. Christensen calls this a goodness oversupply. There's nothing wrong with a goodness oversupply except that customers won't pay for it. I don't want a faster laptop. I'd rather have it lighter and more efficient so that the batteries last longer and it doesn't light my lap on fire.
Summary: (1) User requirements go up slowly; (2) Technology improves quickly.
Together these rules set the stage for a disruption. That's when a low-end technology gets good enough to attack the higher-end: UNIX computers attack mainframes or PCs attack UNIX. I first became interested in Christensen's theory when I saw that it applied to NAS and SAN. Since then, his observations have become a key foundation for my strategic thinking.
What's so interesting about the backup market is that tape-based backup technology is not keeping pace with customer requirements. People struggle to meet backup windows they used to hit. Why are Christensen's rules failing?
The trick is that human behavior usually drives requirements, and humans just don't change that fast. But sometimes technology trends drive requirements, and then requirements rise just as fast as technology. In the case of storage, the capacity you get keeps doubling even if the corporate budget for storage remains flat. That, in turn, doubles backup requirements—twice as much data to move in the same backup window.
You could argue that human behavior must be changing in order to fill all that new capacity, but I believe technology trends are the ultimate driver. Faster computers generate more data. High-res cameras generate more data. Faster networks carry bigger e-mail attachments. Storage also has the full closet problem. My closet—however large or small—is always full. I never bought a new house because of closet space, but when I get a bigger one, it quickly fills. My behavior didn't change at all—I throw stuff away when my closet gets full—but now I have more stuff. Disks are the same. If disks stopped growing, corporate storage budgets would not double every year—not for long. So you can see, technology is what's changing, not behavior.
This is my update to Christensen's rule: When human behavior drives requirements, then technology improves faster than requirements. But when technology itself drives requirements, then you only keep even.
This reminds me of the Red Queen in Alice in Wonderland:
I think these observations explain why backup and data protection have been such hot areas for innovation. Based on the observations in this blog, I expect that to continue for the foreseeable future.
Anyway, I recently noticed an odd sort of exception to some of Christensen's rules in the backup market.
Christensen's first rule is that in most markets, user requirements go up a bit every year, but not too fast. If you buy a new car, you hope it's a bit better than your current one—maybe a bit faster, a bit better mileage, or more airbags to make it safer. But mostly you buy a new car to solve the same problems as the old car, so the requirements are about the same.
Christensen's second rule is that technology improves faster than customer requirements, especially in high tech. Ten years ago, my PC was slow and the disk was always full. Today my laptop has CPU to spare and 33 GB of free space. Christensen calls this a goodness oversupply. There's nothing wrong with a goodness oversupply except that customers won't pay for it. I don't want a faster laptop. I'd rather have it lighter and more efficient so that the batteries last longer and it doesn't light my lap on fire.
Summary: (1) User requirements go up slowly; (2) Technology improves quickly.
Together these rules set the stage for a disruption. That's when a low-end technology gets good enough to attack the higher-end: UNIX computers attack mainframes or PCs attack UNIX. I first became interested in Christensen's theory when I saw that it applied to NAS and SAN. Since then, his observations have become a key foundation for my strategic thinking.
What's so interesting about the backup market is that tape-based backup technology is not keeping pace with customer requirements. People struggle to meet backup windows they used to hit. Why are Christensen's rules failing?
The trick is that human behavior usually drives requirements, and humans just don't change that fast. But sometimes technology trends drive requirements, and then requirements rise just as fast as technology. In the case of storage, the capacity you get keeps doubling even if the corporate budget for storage remains flat. That, in turn, doubles backup requirements—twice as much data to move in the same backup window.
You could argue that human behavior must be changing in order to fill all that new capacity, but I believe technology trends are the ultimate driver. Faster computers generate more data. High-res cameras generate more data. Faster networks carry bigger e-mail attachments. Storage also has the full closet problem. My closet—however large or small—is always full. I never bought a new house because of closet space, but when I get a bigger one, it quickly fills. My behavior didn't change at all—I throw stuff away when my closet gets full—but now I have more stuff. Disks are the same. If disks stopped growing, corporate storage budgets would not double every year—not for long. So you can see, technology is what's changing, not behavior.
This is my update to Christensen's rule: When human behavior drives requirements, then technology improves faster than requirements. But when technology itself drives requirements, then you only keep even.
This reminds me of the Red Queen in Alice in Wonderland:
"In our country," said Alice, still panting a little, "you'd generally get to somewhere else—if you run very fast for a long time, as we've been doing."The strange country of tape-based backup is even worse; you lose ground while running as fast as you can. That's because disks get bigger faster than tapes get faster. Switching to disk-based backup helps because now you stay even rather than losing ground, but to actually make forward progress requires serious innovation.
"A slow sort of country!" said the Queen. "Now, here, you see, it takes all the running you can do, to keep in the same place. If you want to get somewhere else, you must run at least twice as fast as that!"
I think these observations explain why backup and data protection have been such hot areas for innovation. Based on the observations in this blog, I expect that to continue for the foreseeable future.
Comments