Trends in storage and its impact on the Cloud and data center

We talk to Don Foster, Senior Director of Product Management at CommVault on the value of data and how businesses can make best use of it.

1.    As businesses grow, how long can the Cloud remain an option? Or will going hybrid be the eventual goal?

I agree, cloud is not new. It has been around for a number of years now. However, only recently have organizations really started to understand how to leverage it in a way that makes sense for their business. They have started to understand the potential impact that cloud can have on all priorities of IT, and how users access applications, information and business services. As a result, we are seeing a number of organizations investing resources into implementing guardrails around their cloud environments so that developers can consume cloud services in a secure and compliant manner.

I really think cloud is a key driver for organizations to mature and innovate. That is why we will continue to see organizations leverage cloud more than ever.

The discussions we typically have with our customers and prospects are not centered on whether they should be on hybrid, public or private cloud. It is more around their need for flexibility. They want to be able to choose whatever best suits their business needs. We do, however, see companies consolidating their IT infrastructures to hybrid clouds, which can not only help simplify the interaction between public and private cloud but also enable better management and control.

As users gradually gain confidence in cloud security, privacy and reliability capabilities, IDC expects the global cloud market, including private, public and hybrid clouds, to hit $118 billion in 2015 and crest at $200 billion by 2018 .

Hybrid cloud environments will transform the entire computing industry, as well as the way businesses are able to leverage technology to innovate. One clear benefit of a hybrid cloud model is having on-premises, private infrastructure directly accessible — in other words, not having to go via the public internet for everything. This greatly reduces access time and latency in comparison to public cloud services. The hybrid cloud model offers companies the ability to have on-premises computational and storage infrastructure for processing data that requires extra speed or high availability, while retaining the ability to leverage the public cloud for failover circumstances or when the workload exceeds the computational power of the private cloud component.

Building out the private end of a hybrid cloud also allows for flexibility in virtual server design. Organizations can automate the entire virtual machine lifecycle as well as archive older VMs to the cloud. Some organizations are looking at private compute clouds to provide disaster recovery and business resumption services for virtual machines that would traditionally run in the datacenter.

Using a hybrid cloud can greatly facilitate employee connectivity in the workplace. In addition to employees sharing files for collaboration, companies should integrate remote workers with core business processes, such as internal messaging, scheduling, edge protection (laptops, tablets, etc), business intelligence and analytics.

With the hybrid cloud model, CIOs also have more control over where to process data and how much to spend on that effort with regards to value for the business.

2.    People talk of the Cloud offering speed, agility, cost efficiencies etc., but what are current key drivers and what keeps them in the Cloud? 

In a recent report, IDC predicted that the public cloud model will grow about six times faster than the overall IT market through 2018. This is a faster rate of adoption relative to the growth of the overall IT market in a five-year period. This report also points that spending on public IT cloud services is likely to grow to more than $127 billion in 2018.  Which means spending on public IT cloud services will more than double in four years.

Businesses are dependent on speed, and public clouds offer them the agility and faster deployment times – this is where most customers see the value of cloud. Cloud services provide greater mobility and the ability to perform tasks on smartphones and mobile devices, complementing the consumerization of IT which is also blurring the lines between the use of personal and work devices.

In addition, security barriers to cloud adoption are breaking down. With enhanced datacenter capabilities, more comprehensive compliance and reporting, better tools, and growing administrator experience with the platforms, the security barriers are being addressed one by one. Basic familiarity with the cloud platform’s security capabilities can go a long way in preparing for the transition as well.

What we see with our customers is that most of them don’t want to be bothered with managing too many moving components of their IT infrastructure. Rather than continuing to spend capital on their own datacenters, businesses will become increasingly comfortable utilizing big public cloud providers, such as Amazon Web Services, Microsoft Azure, Dimension Data and Rackspace.

As most organizations become more and more familiar with all of the different cloud service models (Infrastructure IaaS, Platform PaaS, and Software SaaS), it is apparent that there will be a greater focus on the application layer. In response, providers will likely grow their application-as-a-service offerings, providing a broader range of alternatives to the traditional on-premises infrastructure.

3.    Can the Cloud really deliver on all IT needs? How “off the shelf” can businesses be in the Cloud? Can it deliver results without extensive customization? 

Will the cloud take over enterprise IT? I think the answer partially depends on the size of organizations.

Small and medium-sized businesses (SMBs), for instance, have the potential to work with a provider and do everything in the cloud. Using the cloud makes things incredibly easier from an IT standpoint. SMBs are playing a growing role in the rise of the market for cloud services. More and more they are shifting their workloads to the cloud to take advantage of greater productivity, simplicity, elasticity and cost savings.

By 2015, SMBs’ share in the market is expected to reach USD 28 billion – with an expected profit pool of USD 3.5 billion – representing 40 to 50 percent of the total cloud market opportunity .

Enterprises today have no lack of cloud-based productivity applications at their disposal. Aside from Google Apps and Microsoft Office 365, cloud-based versions of accounting software, social media management tools and collaboration programs offer nearly all the features of their client-server counterparts – but without the messy installations, making them ideal for small to midsized businesses. While collaboration proves to be the breakthrough feature, the inability to customize the offering to fit a business’ specific environment can hinder adoption in larger enterprises. As a result, we are starting to see more and more large organizations and even more providers offering first and second level customization retainers. If you do want to customize the cloud, providers will give you a template to ensure you can leverage the infrastructure the way your business requires it.

4.    How much of the data businesses collect is really unstructured? How should they prioritize tier 1 and other data? How much data is too much? How essential is it for queries to be settled in real time? How important does tagging, metadata become? What about being able to find old data after we move to data pools?

The amount of data organizations are collecting will continue to increase incredibly fast. Analysts estimate that the amount produced globally more than doubles every two years — 1.8 trillion gigabytes in 500 quadrillion files . I believe Asia Pacific does not stand as an exception and Singapore-based enterprises are twice as likely as the average organization in Asia Pacific to anticipate year-on-year data growth of more than 51 percent .

This directly impacts organizations’ relentless need for more storage and faster access to corporate information. A recurrent question we get from organizations is: where do I store my data?

Cloud services can definitely help businesses effectively and holistically manage their data and storage more intelligently.

From CommVault’s point of view, the more data you have, the more of a potential asset you might actually have to work with. Meta data is key and it is essential for organizations to evaluate meta data to identify data that should be stored or deleted.

Therefore, when setting up processes, it is important to identify the organization’s most valuable data and prioritize storage management resources appropriately. Developing and implementing the right data retention policy is a necessity for both internal data governance and legal compliance. Some data needs to be retained for many years, whereas there may be data that are crucial or required for just a few days, while others should not be retained at all.

Data classification is the first critical step in placing the right data into the appropriate storage tier. Keeping this in mind, organizations can then effectively determine which route to take – be it building on-premises solutions, moving to the cloud, or opting for a combination of the two.

5.    EMC is now talking about data lakes. How do you move from traditional storage to a data lake? What assumptions do we need to place on the data before we can move them to data pools? If all data will be dumped into a single lake, how do you ensure data quality? What controls are there for security and privacy? Would things be easier if businesses deployed a proper data warehouse?

Despite what I was just saying, most companies still don’t have a good data management plan. They tend to capture and store all the data they own, even if it’s redundant. Traditionally, high storage costs forced companies to be more careful about how and where they stored their data. However today, with the cost of storage decreasing, the tendency to store everything without differentiating what’s really important, creates its own set of challenges –where to store different types of data and how to retrieve that data when needed. This is what is termed a data ‘lake’.

In this ‘lake’ of uncategorized data, organizations lack visibility. They don’t know where specific pieces of data is stored at any given point in time, nor are they able to quickly pinpoint applications which are hogging storage or even those which have gone ‘rogue’. This lack of visibility over their data ‘lake’ means that these organizations have no clue about what and how much sensitive information is stored, moved or accessed and by whom. In short, they are blind to whatever is happening inside their storage arrays.

Information has the potential to be a great asset. However, in order to turn data into an asset, enterprises need to have full view of and insight on what is ‘residing’ in their data ‘lake’.

The ability to easily search electronically stored data and provide accurate data search results instantly is critical for organizations, for daily restore operations, as well as in urgent situations such as when specific data is needed for an audit or use in litigation.

In addition to internal organizational compliance requirements, enterprises must also contend with continuously evolving external regulatory requirements. Regardless of industry or sector, any corporate information generated within an organization can be subject to regulatory or litigation requests. The risks of non-compliance are significant, and can lead to legal, financial and reputational damage.

6.    How much DR and BCP has moved to the Cloud?

The situation is very different from one country to another. I was in India a few weeks ago and businesses there, I found, have a strong propensity for a disaster recovery (DR) plan in the cloud. In the United States and in Europe, on the other hand, the adoption rate is much slower, mainly because of cross-border concerns.

At Commvault, we see cloud-based DR as a fantastic option. Through the cloud, effective DR can be within the reach of just about any company, including SMBs. Cloud-based DR gives organizations more benefits and control at a lower price point than traditional disaster recovery solutions.

In addition, having DR sites in the cloud can actually reduce the need for data center space, IT infrastructure and resources, enabling smaller companies to deploy DR options that were previously only available to larger enterprises.

Cloud-based DR creates an opportunity for SMBs to improve their data retention and recovery strategy, thus limiting the risk of data loss. However, as with most cloud services, companies need to be clear about their overall data protection strategy and budget before even coming close to deciding on the right provider for the job.

7.    Is the Cloud replacing tape as the backup of choice? Are businesses moving to a D2D2C rather than the old D2D2T? But a D2D2C approach only really works in certain Cloud environments, what other options are there for businesses? Will this see the end of tape? 

Cloud will probably not kill off tape. At the end of the day, tape is still the cheapest storage mechanism. Yet, as an operational recovery platform, we will see cloud overtake tape more and more.

I am not sure if cloud will ever become the main backup as a choice but it will definitely continue to be a complementary technology.

To my point of view, what is most crucial is that companies understand their data and classify it intelligently. At CommVault, we effectively help our customers classify the data they own. We are able to help customers understand what kind of data they have and where that data is located. We help them identify which data has not been touched in years or months or what has been looked at but not modified for a certain period of time. With this valuable information, CommVault can implement rules and decide which data should be stored where. For instance, what has not been touched in more than nine months should go off to tape, cloud or disk. CommVault will pick it up and move it for its customers. We will start managing where the data is located and we will know exactly what is there and if it has changed or if it has been corrupted.