Why does the storage industry continue to make the same mistakes over again and continue to take bold statements as fact?
Catchy title, right? Here's where it came from?
A few times a year, the editorial team and I have a conference call to discuss future articles, brainstorm ideas and try to address industry trends. A few months ago we had such a call and I was ranting about a variety of topics that I was dealing with from customers and vendors. Many of the things I was ranting about I had dealt with more than 20 years ago. Dan Muse, Internet.com's editor in chief, was on the call and asked regarding one of the topics, "haven't we evolved beyond that?" A two-part article was born.
It continues to amaze me that we repeat the same errors over and over again. Stupid might not be the best word -- or a fair term -- but I do love to rant. As George Santayana said, "Those who cannot remember the past are condemned to repeat it." This time I want to look at some of the evolution of thought, or lack thereof, on storage and storage technology. Why do we want to believe this stuff?
Time Machine: 1991
For those of you who do not remember, or are not old enough to remember, when Seagate introduced its first SCSI drive it had .5 GB of storage capacity. It could read and write about 4 MB/sec and the drive had a seek time of 14 ms and latency time of 6.80 ms. Really nice technology for 1991. Late that year Seagate introduced a 1 GB drive. Wow, now we have two data points for increased density in a year. In 1992, we get a Seagate 2 GB drive with 7 MB/sec of transfer with a seek time of 8 ms and latency time of 4.17 ms. Now we're up to three data points.
Then it was time for analysts and various large organization that look at technology (and you know who I mean) to draw a straight line to show that the growth rate of storage is now on a path to double say every nine or 10 months. I saw those charts back in 1992 and laughed as I had talked to Seagate, knowing the facts. The first 4 GB drive from Seagate back then -- who led the industry -- was in 1994 with 9 GB in 1996. So many people in the industry read the analysts chart and believed that disk density would double based on the straight line drawn in 1992. I did not believe it then and I don't believe it now. But here we go again. Remember the expression, fool me once shame on you, fool me twice shame on me.
Time Machine: 2008
This claim is easily searchable: Flash density will continue to double every 12 to 18 months and flash technology in 2012 to 2014 will be less expensive and higher density than rotating storage (see Why Solid State Drives Wont Replace Spinning Disk ). Both claims I believe were based on charts I have seen come from drawing the same straight lines. Well, we're more than half way through 2011 and I have seen no signs of flash density or price getting close to disk storage even when comparing low-end MLC to SAS enterprise drives. So why would anyone say such a thing back in 2007 and 2008? Why did we believe it then and not run them out of town? Are we stupid or is it that we want to believe because it fits the model we want? Or is it because it is a simple model?
What? Tape is still alive?
The first time I heard that tape was dead was in 1994, but some others have told me they had heard it much earlier -- as early as the 1970s. Some vendors and some analysts have been predicting the death of tape for a long while. These people seem to be declaring the death of tape in a vacuum, not considering other factors such as power and cooling, reliability compared to disk drives, capability to ship tape media (sneakernet is so much faster and more cost effective than buying large amount of network bandwidth) and compression.
Recently, EMC -- and I do not usually or like to mention vendors -- posted a video and a slogan used at a meeting that "Tape Sucks" So for almost 20 years that I am aware of, and many more if you talk with people older than I am, tape has been a dead technology. Why do some storage vendors keep saying that tape is dead when clearly it is alive and well and is actually growing significantly for archival data? Again, this goes back to my question: Do they think we are stupid, do they think if they say it enough, we will believe it? Maybe do they think we have not evolved enough to remember what happened last time or that information is not passed down from generation to generation?
Holographic Storage and other tape and optical replacements
In the early 1990s millions of dollars were put into holographic storage research and the technology was touted as being five years away. In the late 1990s, a company called Terastor said that it was going to replace tape, CD-ROM and lots of other storage media. I visited with Terastor two times as part of a technical evaluation we were doing for a customer. I thought that the technology was interesting and was impressed with their people.
My understanding is that they got a few of writer/reader and media pieces working, and took them to shows but could never manufacture them. From reading the press reports back when they went out of business, they burned through more $800 million dollar with very little to show except some patents. Honestly, it was very cool stuff if it had worked. The same goes for about six years later and InPhase. They were going to make a holographic disk. This, of course, was different technology than Terastor, but from the point of view of replacing tape, and other optical they had the same goals. I visited InPhase for two customers. They had a great story of how they were going to make the product and lots of smart people, but they did not ever get the product into production and I never thought they would or thought they could be competitive. Everyone that attended the meeting really wanted them to succeed and some were hopeful. I was not (a curmudgeon before my time). I said to myself been there, done that.
Tell me the point of this series again?
I have been thinking a lot about why we seemingly cannot learn from the past. Is it because we think someone will be successful where others have failed, or is it that we beat our chests because we think it will change the reality that we are facing? Technology changes require that many things be considered for that change to be successful. The tape example is a good one. Up until very recently with the release of new enterprise tape from IBM and Oracle disk drives had greater density than tape, yet when these high-density products came out it made no difference to the mantra from parts of industry that tape is dead. This sounds like politics somewhat: If you misstate the truth often enough, people will believe you on the right or the left. Long term, we need to get out of the mode of not questioning bold statements without understanding the details and information behind those statements.
I have a saying: Minutia matters. It does not matter all of the time, but matters more often than people seem willing to believe. I rarely accept things at face value and always try and analyze and understand why people take a position or make a statement. I have argued with the tape-is-dead crowd for almost 20 years. When I provide all the facts, they never seem to want to address all the facts, but only the facts that they want to pick and choose. You can pick and choose facts to come to a conclusion, but the market choose that way. Technology, with rare exceptions, does not follow straight lines for long (Moore's Law being the rare exception). We live in an ever-changing and complex world, a world in people seemingly do not want the facts to get in the way of their opinions.
Next time, I will address the same issues for the server and file system side of the equation. Remember the minutia matters (at least a good percentage of the time).