August 07, 2009

Cloud: Then and Now: From Cool Technology to Economic Necessity

In this blog, I contrast the grid computing conversations I had with customers in 2003 to the Cloud (IT as a Service) conversations now.

Back in 2003, grid computing was an interesting concept. Customers agreed that that’s where they were heading. There were some leaders within the customer-base attempting to drive their enterprises towards grid computing. But there was resistance – resistance to change. We wondered and chatted about technology barriers and political barriers to grid adoption and how to stroll about addressing them. It was a great technology conversation.

Cloud conversations today couldn’t be more contrasting. IT budgets are flat if not decreased. Business is much more competitive and demands more flexibility and higher service-levels from IT.  Customers have no choice. They want Cloud. Question is about how fast they adopt Cloud and not if. There is urgency in the conversation. How to build cloud yesterday? What are the workarounds to existing technology gaps? Now, cloud is an economic necessity – an essential element for survival. On the technology front, Virtualization has played a big role in making IT more dynamic by breaking static and hard ties to physical hardware; and dramatically lowered the barrier of server hugging.

Now a great deal of customers are building internal clouds, i.e., gradually evolving towards IT as a Service; and increasing number of customers either use or are evaluating external clouds.

What’s so attractive about the cloud for business?

  • Instant delivery & capacity elasticity via self-service: Traditional IT took many months provision new business services while cloud providers (whether internal or external) offer to provision these business services in days if not hours. Traditional IT has been very brittle in growing or shrinking capacity while cloud providers offer capacity elasticity.  Guess what, subscribers can do this online via their self-service portal.
  •   Pay as you go: These service providers charge based on what you consume and does not incur heavy capital expenses as with traditional IT.
  •   Always on: Traditional IT designed business continuity of each enterprise application individually; delivering high-levels of business continuity was a costly enterprise. Given the nature of business, cloud providers have to deliver an ‘always on’ solution to their customers. A downtime at a cloud provider can be quite costly for the cloud provider.

In the next blog, I’ll expand on how NetApp can help with the above.

cheers,

Brajesh.


May 04, 2009

Internal clouds: The storm is coming, corporate IT be prepared….

If you read Dave Hitz’s recent blog, Three Ten-Year-Trends That Are Driving IT Change,  he believes that in 10 years time many medium-sized businesses and a few large ones will have cloud-outsourced pretty much all of their IT infrastructure.  I believe this too, and it may well happen faster.   So if you live in a corporate IT organization, responsible for IT infrastructure, where does this leave you? How will your role be changing and what will the infrastructure you still manage look like?

Let’s start briefly with the drivers for cloud computing – the CFO and CIO get to move capital expenses (capex) to operational expenses (opex) and “the business” (and application owners) can bring products to market faster.  The cloud computing model allows the end users to ramp up or ramp down capacity in response to market demand and therefore pay as they go.  Let’s face it, they never really know how successful their product or service is going to be – so today they over state the business case and ask for hardware for next three years; the last thing they want is to be constrained by led times on for additional capacity as their solution ramps up.  This practice leads to inefficiencies in the data center.  On average, the utilization of physical resources (including servers, network and storage) is has been abysmally low.  We frequently find customers with storage utilization in 30-40% range and lower wasting power, floorspace and hardware..

What are the smart corporate IT organizations doing?  They understand that the trend to cloud services is valid and requires i) governance for cloud-outsource applications and ii) the creation of internal cloud services to drive down costs and time to market for in house applications. The latter is built on a pooled dynamic infrastructure with utilization levels in excess of 75%.  This is achieved through thin provisioning, deduplication, and cloning technologies (which can raise utilization levels well in excess of 100%).  The bottom line is that this approach yields big cost savings.

Since there are applications (CRM, ERP, messaging and collaboration) that are common to every company, outsourcing to an external cloud provider that can do a better job managing the application at a lower cost structure makes sense.  Governance plays a central role in deciding which applications can be safely outsourced, and how to manage the processes.  Where to begin?  First, you will need to assess the applications and build policies based upon the type of data.  Factors to consider include: how it is accessed and by whom, security and compliance aspects, and the strategic importance or competitive advantage the application or data offers.  Second, you need to assess the cloud service provider’s service offerings.  Look at their capabilities, security, SLAs on availability and performance to see if they meet the levels required by the applications before agreeing to cloud-outsource the application.  Don’t forget to get the corporate IT governance and security group involved to help create policies that will protect corporate data and hence drive the decision on what can and cannot be cloud-outsourced.

What do we do with the rest of the applications that are not outsourced?  We (as in corporate IT and IT vendors) have a big opportunity, internal clouds will help the business launch applications faster and at much lower cost.  This is about building IT-as-a-Service capabilities in house.  Building shared infrastructure that is offered as service to the business.  We’ll need pooled infrastructure, policy based automation to simplify provisioning, metrics and charge backs, service assurance and conformance to SLAs, and capacity planning with a “just in time” inventory for the enterprise’s infrastructure resources with 3-6 months of headroom for the pool.  Add a self-service portal to your internal cloud and now the applications teams are happy they can deploy faster and lower cost and the corporate IT governance guys will be happy too.

Ok, so this space is evolving fast, so start with the basics; pool the infrastructure and use a vendor that offers dynamic virtualized infrastructure to quickly activate applications, or repurpose capacity and performance as loads from applications ramp up or down.  For this you need unified storage, network, and servers that can cater for wide range of applications requirements and choose highly efficient infrastructure. 

So now you have an internal cloud based on efficiency pooled infrastructure with drastically higher utilization level and big savings.  If clouds are politically sensitive then just call it IT-as-a-Service, that’s what it’s all about after all.

There are a few places to get started, one is to understand the technologies and the integration required to building a cloud infrastructure – such as NetApp with VMware vSphere cloud OS.  The second piece is to understand the service mentality that is required in transforming the data center and you can read about that journey and more on the dynamic data center here.

In future blogs from the dynamic data center team and we’ll tell you more about how NetApp is helping companies build internal cloud services.

Cheers,

Hamish

March 18, 2009

Cloud: Then and Now!

Well, it is my blog day. This is my 3rd blog in two 2 days and the first one on Cloud. The other two are on – “Say It’s Your Birthday! ”.

Last year, people put everything under the sun in the cloud umbrella. NetApp’s Chief Marketing Officer Jay Kidd’s blog on Scattered Clouds described the situation in 2008 extremely well. In the last 3-6 months, however, there is a consensus that’s emerging with regards to what’s cloud for the enterprise.

I am going to revisit a definition of Grid Computing from 2002 and propose a definition of Cloud Computing. I think that this could be a reason for Larry Ellison’s rant although Larry may have his own reasons. I believe this offers a vision and opportunity for the future.

In an Oracle Grid white paper from 2002 (also referred in my book: Enterprise Grid Computing with Oracle), Benny Souder and I proposed the definition of Grid Computing as:

“At the highest level, the central idea of grid computing is computing as a utility. The user of the grid should not care where the data resides or what computing resources process the requests. Instead, the user should be able to request information or computing from anywhere and have them delivered according to his or her needs and in a timely fashion.

From the perspective of a provider of the grid within an enterprise, grid computing is about providing a shared IT infrastructure that addresses the demands of business while utilizing the IT resources most efficiently and cost-effectively.”

Think about it.  I believe the concept of the cloud computing is exactly the same. I propose the following definition with some minor changes to the above definition:

 “At the highest level, the central idea of cloud computing is computing as a utility. The user of the cloud should not care where the data resides or what computing resources process the requests. Instead, the user should be able to request information or computing from anywhere and have them delivered according to his or her needs and in a timely fashion.

From the perspective of a provider of the cloud within an enterprise, cloud computing is about providing a self-service shared and dynamic IT infrastructure that addresses the demands of business (i.e., meet the demanded service-levels) while utilizing the IT resources most efficiently and cost-effectively. “

If you agree with the above definition, you can understand why Larry Ellison calls cloud computing another marketing propaganda for something that industry has been talking about for many years. I think this is also where the opportunity lies too. We are now at the inflection point for the adoption of the cloud. Virtualization played a strong role in creating this inflection point. The current economic climate only accelerates the pace of adoption of cloud computing.

If you notice, the second part of the definition talks about how the data center delivers cloud computing services. This service can be delivered internally within the same enterprise or delivered externally to the customers. I believe the management problems of measuring service-levels delivered and managing self-service driven shared and dynamic infrastructure share key common problems irrespective of the consumer. I call this the private cloud.

The first part of the definition is about the needs of the consumer. If the consumer is sitting outside, then this becomes the public cloud. The public cloud adds some more key requirements from security and mobility perspective. Consumers would like their environment to be isolated from other consumers and they would like their stuff to move seamlessly to and from provider and consumer premises.

There is a lot to be done to achieve this vision. Well, I work for a storage company – NetApp. So you can expect my friends and I to write more about what we are doing for our customers to realize this vision!! 

cheers,

Brajesh.

March 17, 2009

Welcome new bloggers - the Dynamic Data Center Team Blog

Over the last few months I've been adding here my thoughts on the industry and how the nature of the data center and data center management is changing. I'm really delighted to introduce some new bloggers who'll add their thoughts and opinions to this blog - welcome Brajesh, Jeff, Hamish and Assaf.

Dynamic Data Center Blogging Team

Team bloggers share their opinions and observations about the dynamic data center and integration with server virtualization.

Brajesh Goyal, Director, Manageability Customer & Partner Solutions

Brajesh leads the technical marketing and community efforts for Manageability solutions. He works closely with customers & partners to build their own and/or adopt NetApp's manageability solutions. He joined NetApp in Feb 2005 and previously worked for Oracle as the Product Manager for Grid Computing and was among the first few people to define Grid Computing for Oracle. He has written two books on Enterprise Grid Computing. He has Masters from University of Minnesota and Bachelors from Indian Institute of Technology, Mumbai in Computer Science.

Jeff O'Neal, Senior Director - Data Center Solutions

Jeff O'Neal manages the go-to-market strategy for Data Center Infrastructure solutions at NetApp, including the NetApp Dynamic Data Center Solution and the Unified Storage Platform, as well as the company's Storage Management Software portfolio. Jeff joined NetApp in 2007 and previously worked for Sun Microsystems as Senior Director - Professional Services - Global Practice Development. Jeff has an MBA degree from the Monterey Institute of International Studies (with distinction) and a BSEE from California State University Northridge.

Hamish McGovern, Senior Product Manager, Virtualization and Grid Infrastructure BU

Hamish is bringing to market the NetApp Dynamic Data Center Solution that enables organization to run IT-as-a-Service from multitenancy infrastructure. Hamish joined NetApp in 2000 in the sales organization and led the technical team for one of NetApp's largest customers. Prior to NetApp he led the systems integration team at Australia's largest ISP, and spent ten years in telco and online services industries. Hamish holds a MBA from Monash University, and a BSc. and B.E. (Elec) from University of Sydney.

Assaf Levy, Senior Technical Director, SW

Assaf Levy leads product management for the SANscreen suite. Assaf joined NetApp through the Onaro acquisitions, where he was the CTO, Vice President of Business Development and Co-Founder. Assaf was responsible for driving Onaro's future product vision as well as our partnering strategy and execution. He works closely with customers, partners and Onaro's sales and field operations to assure that the company's innovative roadmap for product expansion meets market demands. Assaf has 15 years of management experience in market-driven product and software development and holds a Bachelor's Degree in Computer Science from the IDC center in Herzlia

With their strong varied backgrounds and opinions I hope this Dynamic Data Center blog will be both interesting to you, the reader, and enlightening. Remember you're a part of this so don't keep your thoughts to yourself ... blog it.

Paul.

December 31, 2008

Happy New Year and to all

Just thought I'd wish everyone a very happy new year and successful 2009. This prior year of 2008 has certainly been one of dramatic changes but the prospects for 2009 are looking bright.

Wishing you and years a very happy new year.

Paul.

New Year Resolution - Get a Lasik for the Data Center

One of the biggest challenges in IT is visibility into how your assets are being utilized. This is true for many IT assets but particularly storage. However just like with your own eyesight without correct vision you will lack clarity and depth of vision into how storage is being consumed.

So treat yourself this New Year to vision correctness surgery for your Data Center. SANscreen 5.0 just released provides that level of visibility. With clarity comes solace and happiness in the datacenter. Treat yourself to a vision makeover this new year.

For example see the following chart which provides detailed chargeback reports based on the Business Unit consumers of the storage.

Cuserskristyappdatalocalmozillafi_2

SANscreen 5.0 benefits include:

  • Understanding storage capacity usage trends: By tracking usage trends over time, storage teams can quickly identify changes in storage usage
  • Generating usage forecasts: Projecting future usage on all storage entities enables storage teams to properly plan and justify future storage purchases
  • Planning future allocations: Based on projected storage usage, storage administrators can calculate when and how much storage needs to be allocated and/or purchased for each chargeback group
  • Developing chargeback reports: Storage administrators can generate detailed chargeback reports
  • Determining whether additional storage is really required: IT teams can avoid overprovisioning storage by relying on accurate forecasts of user demand
  • Identifying available configured storage: By identifying available configured storage, IT can easily allocate storage to the requesting groups
  • Reclaiming orphaned storage: Administrators can quickly reclaim unused storage that has been allocated to hosts but remains idle, and can easily reallocate it to other hosts as needed
  • Reclaiming overprovisioned storage: Storage administrators can identify where storage is currently underutilized (and forecasted to remain underutilized) and reclaim the unused storage

For more information on SANscreen benefits see the following paper http://media.netapp.com/documents/wp-7060-empowering-it.pdf

Oh and like Lasik this Data Center Vision Correctness with SANscreen 5.0 can be performed as simple outpatient surgery. It doesn't require agent installs, major system upgrades or is disruptive in any way. In less than a few weeks in most cases you'll have 20-20 datacenter vision.

New Year Resolution - Get a Lasik for the Data Center

One of the biggest challenges in IT is visibility into how your assets are being utilized. This is true for many IT assets but particularly storage. However just like with your own eyesight without correct vision you will lack clarity and depth of vision into how storage is being consumed.

So treat yourself this New Year to vision correctness surgery for your Data Center. SANscreen 5.0 just released provides that level of visibility. With clarity comes solace and happiness in the datacenter. Treat yourself to a vision makeover this new year.

For example see the following chart which provides detailed chargeback reports based on the Business Unit consumers of the storage.

image

SANscreen 5.0 benefits include:

  • Understanding storage capacity usage trends: By tracking usage trends over time, storage teams can quickly identify changes in storage usage
  • Generating usage forecasts: Projecting future usage on all storage entities enables storage teams to properly plan and justify future storage purchases
  • Planning future allocations: Based on projected storage usage, storage administrators can calculate when and how much storage needs to be allocated and/or purchased for each chargeback group
  • Developing chargeback reports: Storage administrators can generate detailed chargeback reports
  • Determining whether additional storage is really required: IT teams can avoid overprovisioning storage by relying on accurate forecasts of user demand
  • Identifying available configured storage: By identifying available configured storage, IT can easily allocate storage to the requesting groups
  • Reclaiming orphaned storage: Administrators can quickly reclaim unused storage that has been allocated to hosts but remains idle, and can easily reallocate it to other hosts as needed
  • Reclaiming overprovisioned storage: Storage administrators can identify where storage is currently underutilized (and forecasted to remain underutilized) and reclaim the unused storage

For more information on SANscreen benefits see the following paper http://media.netapp.com/documents/wp-7060-empowering-it.pdf

 

Oh and like Lasik this Data Center Vision Correctness with SANscreen 5.0 can be performed as simple outpatient surgery. It doesn't require agent installs, major system upgrades or is disruptive in any way. In less than a few weeks in most cases you'll have 20-20 datacenter vision.

November 28, 2008

All hell broke loose ... What happened ... Proctive alerting and diagnostics

Was just reading a response from Stephen Foskett to my last post on Storage Optimization and it made me think about alerting and diagnostics. How do you avoid the limit cases on systems to try to avoid problems and then to identify root cause on problems to avoid future cases.

I like to look at this from two aspects - performance and capacity management.

For the first Performance Advisor can be a great tool to help. It allows you to selectively choose from a large set of performance counters and the build thresholds or combination thresholds as needed. It also allows for good problem isolation later by allowing you reverse time and see former performance views of your storage system. You can then help isolate the problem with selective viewing of your data. For a good overview of the latest features see Raja's post on the NetApp Community http://communities.netapp.com/message/2874 . This will be a really interesting space to watch in the future as we look at add more proactive planning and predictive analysis to Perforamance Advisor.

As for capacity management - two products are there to help. The first is Provisioning Manager which I reviewed last time - the second which deserves a complete post to itself if NetApp Operations Manager. It provides space based reporting for primary, snapshot, reserve space allocations, shares, quotas, luns and helps you trend that space usage and report on it. You can use that for storage growth planning or also, if desired, for chargeback. It provides canned reports, custom reporting and also a recent addition was the export capability which allows you to feed your operational data into another reporting tool for more flexible analysis. We are also working on some very interesting capacity reporting in SANscreen which will provide full data hetrogeneous storage capacity planning ... more on that soon.

Hmmm... a few more follow on posts are needed on this topic - more soon.

November 18, 2008

Continuous Storage Optimization - how to safely provide more storage than you own

Storage systems recently have been adding some very cool capabilities like thin provisioning (over allocation of storage), auto-grow capabilities to dynamically add space and de-duplication of files or blocks. These are all aimed solidly at increasing storage efficiency and when used well can have a dramatic impact. One of NetApp's newest products Provisioning Manager helps you continuously manage your storage efficiency using these technologies.

Provisioning Manager is another of our policy based management products. The storage administrator sets a policy which controls the acceptable levels of storage allocation, backup space reservation (snap reserve), auto-grow triggers and alerting. Provisioning Manager then automatically manages the storage allocation to meet this policy allowing for safe and well managed over-allocation of storage.

image

What is really interesting about this is the separation between user demand and management of the physical assets - I call this Demand-Asset Isolation. It involves separating the users request for storage from the actual physical storage and helps to solve one of the key problems with storage efficiency today - IT teams unwittingly over planning for future growth in their storage demands.  With disks on an ever reducing in price curve,  deferring longer the purchase of these assets will directly affect your bottom line.

Demand-Asset Isolation is possible today and Provisioning Manager can help you achieve it. http://media.netapp.com/documents/ds_2742_provisioningmgr.pdf

October 15, 2008

To achieve real benefit from Storage Management – “Death to Agents!”

As our IT environments grow it is critical that we effectively manage and optimize our infrastructure – particularly in the storage domain where the historical doubling of capacity every 18 months has resulted in poor utilization of storage and has created islands of lost capacity.

Storage Resource Management (SRM) products aim to control this nightmare and are supposed to reduce capital costs through improved utilization as well as to reduce operating costs through automation and efficiencies.

However, deploying SRM products (such as EMC Control Center, Symantec Command Central Storage, IBM Tivoli Productivity Center, HP Storage Essentials, and others) has been littered with problems – especially in environments of any significant size and complexity. The primary reason is that SRM tools were originally architected to rely on host agents that must be installed on all servers (unix, linux, Windows) in the environment.

Why are SRM agents bad?

The first reason has little to do with technology and more to do with the way most companies have structured their IT organizations. If a storage team wants to deploy SRM agents, it needs to get permission and buy-in from the server and applications teams. Right out of the shoot, most SRM deployments are delayed by weeks or months because the server and application team wants to understand the ramification of putting an intrusive agent into their environment. What effect will it have on the performance and availability of their critical applications? How much effort and money will it cost them to support? For this reason alone, agent-based architectures are often dead on arrival.

The second reason agents fail is the high cost and complexity of deployment. Before you deploy agents across 100s or 1,000s of systems, you need a plan – and you need people to implement and manage that plan. The next time you run across a “successful” SRM deployment, try to find out how many professional services consultants where on-site and for how long – odds are that more money was spent on implementing the SRM product then was spent on buying it in the first place.

So why bother having agents in the first place? You can recognize the vast majority of storage management benefits and value without agents – in far less time and with far fewer professional services costs. NetApp SANscreen was designed and built from the beginning with an agentless architecture. This fast, flexible architecture allows SANscreen to discover large environments – even 10s of 1,000s of switch ports – within a matter of hours-days. If you could accomplish most of the value of storage management in only a matter of days without agents, why would you ever take the risk of spending months or years only to fail with an agent-based architecture?

And if very detailed host information is really required, there are plenty of industry standard technologies such as WMI (Windows) or Secure Shell (Unix) that can be used to collect host data “agentlessly.” Also, you can remotely collect data from databases like Oracle and SQL Server or even better just interface with their management tools for that data.

So if you are really interested in recognizing value from a storage management project – don’t bother with products that were originally designed to use agents. It is time to call out “Death to Agents!” – their time and their failures have come and gone.

Adios,

Paul.

© NetApp, Inc.  |  "Safe Harbor" Statement  |  Privacy Policy