Many of us remember those old computer messages that no more storage space was available on the hard drive. The user had no choice but to offload some data or spend some hours going through the files finding material that could be deleted. Enterprise users will also recall the fondness some storage administrators had for sending out those “You have exceeded your storage quota” notices. Fortunately, the size of today’s hard disk drives (HDDs) seems to have brought about a virtual disappearance of such unwelcome communications.
But what about the storage world as a whole? We are merrily merely packing information onto digital storage from every conceivable angle. Mobile phones, big data, the cloud, tablets, the internet of things (IoT), analytics and more are gobbling up available storage capacity at an alarming rate.
Could we reach the point where there is simply nowhere left to put all this data? In other words, could we arrive at Storage Armageddon or Stor-mageddon?
Over the next few weeks in this series of articles, Enterprise Storage Forum will investigate the known and unknown digital universe in terms of its current size as well as future capacity requirements. In addition, we will look at developments in technologies such as tape, disk, flash and storage management to see what solutions could help us avoid a potential storage end of days.
The Digital Universe
The size of the digital universe is staggering. People have gotten so used to bandying around storage capacity acronyms that they may have lost sight of what they really mean. So let’s begin by covering some basics. Terabyte (TB) drives have become quite common – a TB is a trillion bytes. Now multiply a TB by a thousand and you have a petabyte (PB), another thousand is an exabyte (EB) and another 1000 gives you a zettabyte (ZB). According to International Data Corp.( IDC), 4.4 ZB of digital data had been created by 2013. That number is predicted to reach 44 ZB by 2020.
IDC’s Digital Universe report provides many interesting tidbits on this problem. According to Vernon Turner, an analyst at IDC, data growth is moving at a rapid clip – about 40 percent per year.
“The world’s available storage capacity is growing slower than the digital universe,” said Turner.
At the start of last year, IDC estimated that available storage capacity could hold 33 percent of the total. Within five years, the sheer volume of data created means that less than 15 percent can be stored. There are many facets to this. Certainly, the fact that much of this data is transient (streamed video, phone and text traffic, etc.) serves to ease the pressure somewhat.
But even that transient data still gobbles up an awful lot of capacity (at least temporarily). Then there is the metadata aspect and the data transmission side. Just how much storage and bandwidth get used up by such viral hits as an Ellen DeGeneres/Bradley Cooper celeb selfie at the Oscars (26 million views in 12 hours)?
Cisco’s Global Mobile Data Traffic Forecast presents a frightening prospect. Let’s take up just a handful of its data points: Mobile data grew 69 percent last year, reaching 2 EB per month. The total worldwide mobile traffic for 2014 alone exceeded the entire extent of the internet in 2000. Mobile video traffic now accounts for more than half the overall volume. With about 7.5 billion mobile devices in operation and many more of these being converted to smartphones (a smartphone generates 22 times more traffic than a non-smart device), the mobile traffic burden is mushrooming. Factor in the fact that connection rates are growing at around 20 percent per year. As 4G service expands, it should be noted that 4G devices churn through 10 times more data than those of earlier generations.