Harness that Ever Growing Dat

We are currently in the midst of a digital revolution that is changing the way we communicate, work and live; which in turn has had significant ramifications on how we produce, store, and recall information-almost every bit of which is becoming digitized. Collectively, we are on pace to churn out 1.2 zettabytes (1021 bytes) of digital information in 2010, according to IDC. This is expected to grow to 35 zettabytes or 35 tn gigabytes by 2020. To put this in perspective, if each piece of digital information is a DVD, the stack of DVDs would now reach halfway to Mars. Given this reality, it is critical that we have a reliable, cost effective, and scalable storage solution in the form of an easily managed unified storage platform.

Drowning

The increase in data creation is taking place at personal and enterprise levels. At a personal level, the rise of personal digital devices speaks volumes about the way we create and share personal data. Photos have moved from physical photo albums to things like smartphones and photo sharing websites. On Facebook alone more than 25 bn pieces of content are shared each month, requiring over 60,000 servers to handle storage and transactions.

At an enterprise level, there is an unprecedented growth in online transactions and services needing to be stored, secured and managed. For companies like Taobao, an e-commerce site in China comparable to eBay, reliable data storage is critical. Consider that in one day in September 2009, the company sold 4.561 mn items and processed 64,808 transactions. With each item holding an accompanying description, photos, and customer data, there is a burgeoning need for robust and scalable storage solutions.

Insufficient Storage

We are creating so much digital content that we have already exceeded the amount of digital storage space available. In IDC's estimates, we would have a shortfall of around 35% if we try to store every single piece of digital data created in 2009. This gap will only increase with time.

Growing Need

Traditional servers have become too expensive as a long term solution, especially after factoring in the costs of equipments, software licenses, and also the management costs. In the recent years, a new class of storage solutions which consolidates network-attached storage (NAS) appliances and storage area networks (SAN) has become readily available. Instead of multiple servers running on various platforms, these new storage appliances run on a single storage platform and provide enterprises with reliability, scalability, and ease-of-use tools to effectively manage their storage needs.

High Reliability

Given the growing dependence on digital information for business transactions, a single minute of downtime has direct revenue impact for a company. For a business like Taobao that processes all its transactions via its website, losing one minute of uptime can cost the company $36,000, or roughly $2 mn per hour.
Taobao selected Huawei Symantec's Oceanspace N8000 unified storage platform because of its active-active node configuration which can survive single or multiple node failure through workload redistribution across a clustered solution. In addition, N8000 can recover data almost instantaneously through data snapshots. With these safeguards in place, companies can enjoy an assurance of 99.9% availability.

High Scalability

Enterprises also need to consider future business and data growth in their selection of storage solutions. For example, Shanghai Ocean University was looking to manage 100 terabytes of e-books and videos within its digital library on campus, while also planning ahead for its role as a supercomputing center. With scalability and flexibility in mind, it chose a unified storage platform that is not only a high performance file serving solution, but is also capable of scalability of upto 15 petabytes in the future.

Dynamic storage tiering (DST) within this platform also ensured the most efficient use of storage locations for the university's data. DST enables data to dynamically move to different storage tiers depending on usage needs. For example, frequently accessed data would be stored on tier-1 servers which consist of faster disks, while less volatile data resides on tier-2 or tier-3 servers. Because most organizations access only 10% of their data frequently, 90% of the university's library data would be redirected to the less expensive storage using DST technology, which is expected to bring more than $7,00,000 as savings.

High Performance

The constant demand for data is putting a strain on the storage system performance. To meet this demand, there is a need for even utilization across multiple nodes within a single storage system. A unified storage system addresses this by enabling every NAS engine node to share the same storage pool simultaneously. In addition, systems like N8500 make it possible for multiple NAS engines to read and write the same document simultaneously, thus improving response time and performance in cases where many users are trying to access a single document.

Management Simplicity

While technologies are becoming more sophisticated, management of these technologies has to be simple and intuitive. Unified storage systems offer centralized management tools that are easy-to-use and access, whether through a GUI application or a web interface. System administrators can use these management tools to set notification alerts via multiple communication mechanisms in the case of a system failure. Data backups can be configured and monitored for success and failures.

Conclusion

The digital revolution continues unabated, and we fully expect that more changes will occur in data creation, security, storage and management as we move into a digital universe of unfathomable amounts of content. For enterprises, the demand for innovative storage solutions will only increase with time and it will be increasingly important for these solutions to be highly reliable, scalable, and simple to manage. Unified storage platforms can adapt to an enterprise's changing business needs and provide it with peace of mind to focus on improving the business, as in the case of Taobao and Shanghai Ocean University.

Tad Lebeck
The author is CTO,
Huawei Symantec Technologies
vadmail@cybermedia.co.in