Just thought I'd wish everyone a very happy new year and successful 2009. This prior year of 2008 has certainly been one of dramatic changes but the prospects for 2009 are looking bright.
Wishing you and years a very happy new year.
Paul.
One of the biggest challenges in IT is visibility into how your assets are being utilized. This is true for many IT assets but particularly storage. However just like with your own eyesight without correct vision you will lack clarity and depth of vision into how storage is being consumed.
So treat yourself this New Year to vision correctness surgery for your Data Center. SANscreen 5.0 just released provides that level of visibility. With clarity comes solace and happiness in the datacenter. Treat yourself to a vision makeover this new year.
For example see the following chart which provides detailed chargeback reports based on the Business Unit consumers of the storage.
SANscreen 5.0 benefits include:
For more information on SANscreen benefits see the following paper http://media.netapp.com/documents/wp-7060-empowering-it.pdf
Oh and like Lasik this Data Center Vision Correctness with SANscreen 5.0 can be performed as simple outpatient surgery. It doesn't require agent installs, major system upgrades or is disruptive in any way. In less than a few weeks in most cases you'll have 20-20 datacenter vision.
Was just reading a response from Stephen Foskett to my last post on Storage Optimization and it made me think about alerting and diagnostics. How do you avoid the limit cases on systems to try to avoid problems and then to identify root cause on problems to avoid future cases.
I like to look at this from two aspects - performance and capacity management.
For the first Performance Advisor can be a great tool to help. It allows you to selectively choose from a large set of performance counters and the build thresholds or combination thresholds as needed. It also allows for good problem isolation later by allowing you reverse time and see former performance views of your storage system. You can then help isolate the problem with selective viewing of your data. For a good overview of the latest features see Raja's post on the NetApp Community http://communities.netapp.com/message/2874 . This will be a really interesting space to watch in the future as we look at add more proactive planning and predictive analysis to Perforamance Advisor.
As for capacity management - two products are there to help. The first is Provisioning Manager which I reviewed last time - the second which deserves a complete post to itself if NetApp Operations Manager. It provides space based reporting for primary, snapshot, reserve space allocations, shares, quotas, luns and helps you trend that space usage and report on it. You can use that for storage growth planning or also, if desired, for chargeback. It provides canned reports, custom reporting and also a recent addition was the export capability which allows you to feed your operational data into another reporting tool for more flexible analysis. We are also working on some very interesting capacity reporting in SANscreen which will provide full data hetrogeneous storage capacity planning ... more on that soon.
Hmmm... a few more follow on posts are needed on this topic - more soon.
Storage systems recently have been adding some very cool capabilities like thin provisioning (over allocation of storage), auto-grow capabilities to dynamically add space and de-duplication of files or blocks. These are all aimed solidly at increasing storage efficiency and when used well can have a dramatic impact. One of NetApp's newest products Provisioning Manager helps you continuously manage your storage efficiency using these technologies.
Provisioning Manager is another of our policy based management products. The storage administrator sets a policy which controls the acceptable levels of storage allocation, backup space reservation (snap reserve), auto-grow triggers and alerting. Provisioning Manager then automatically manages the storage allocation to meet this policy allowing for safe and well managed over-allocation of storage.
What is really interesting about this is the separation between user demand and management of the physical assets - I call this Demand-Asset Isolation. It involves separating the users request for storage from the actual physical storage and helps to solve one of the key problems with storage efficiency today - IT teams unwittingly over planning for future growth in their storage demands. With disks on an ever reducing in price curve, deferring longer the purchase of these assets will directly affect your bottom line.
Demand-Asset Isolation is possible today and Provisioning Manager can help you achieve it. http://media.netapp.com/documents/ds_2742_provisioningmgr.pdf
As our IT environments grow it is critical that we effectively manage and optimize our infrastructure – particularly in the storage domain where the historical doubling of capacity every 18 months has resulted in poor utilization of storage and has created islands of lost capacity.
Storage Resource Management (SRM) products aim to control this nightmare and are supposed to reduce capital costs through improved utilization as well as to reduce operating costs through automation and efficiencies.
However, deploying SRM products (such as EMC Control Center, Symantec Command Central Storage, IBM Tivoli Productivity Center, HP Storage Essentials, and others) has been littered with problems – especially in environments of any significant size and complexity. The primary reason is that SRM tools were originally architected to rely on host agents that must be installed on all servers (unix, linux, Windows) in the environment.
Why are SRM agents bad?
The first reason has little to do with technology and more to do with the way most companies have structured their IT organizations. If a storage team wants to deploy SRM agents, it needs to get permission and buy-in from the server and applications teams. Right out of the shoot, most SRM deployments are delayed by weeks or months because the server and application team wants to understand the ramification of putting an intrusive agent into their environment. What effect will it have on the performance and availability of their critical applications? How much effort and money will it cost them to support? For this reason alone, agent-based architectures are often dead on arrival.
The second reason agents fail is the high cost and complexity of deployment. Before you deploy agents across 100s or 1,000s of systems, you need a plan – and you need people to implement and manage that plan. The next time you run across a “successful” SRM deployment, try to find out how many professional services consultants where on-site and for how long – odds are that more money was spent on implementing the SRM product then was spent on buying it in the first place.
So why bother having agents in the first place? You can recognize the vast majority of storage management benefits and value without agents – in far less time and with far fewer professional services costs. NetApp SANscreen was designed and built from the beginning with an agentless architecture. This fast, flexible architecture allows SANscreen to discover large environments – even 10s of 1,000s of switch ports – within a matter of hours-days. If you could accomplish most of the value of storage management in only a matter of days without agents, why would you ever take the risk of spending months or years only to fail with an agent-based architecture?
And if very detailed host information is really required, there are plenty of industry standard technologies such as WMI (Windows) or Secure Shell (Unix) that can be used to collect host data “agentlessly.” Also, you can remotely collect data from databases like Oracle and SQL Server or even better just interface with their management tools for that data.
So if you are really interested in recognizing value from a storage management project – don’t bother with products that were originally designed to use agents. It is time to call out “Death to Agents!” – their time and their failures have come and gone.
Adios,
Paul.
Sometimes you're so deep into the planning, development and implementation of products that you don't get enough opportunity to see the noticeable impact those products will have on your customers and the industry.
I was delighted to see the recent blog post from W. Curtis Preston, Mr Backup to you and me - http://www.backupcentral.com/content/view/193/47/ . His blog provides a glowing testimony of how Protection Manager dramatically changes the approach to backup and replication management.
One of our design criteria was how to change the management of scale. This required a fundamentally different approach and we created Policy-based Management. Central to that is the ability to separate the business logic (the policies) from the implementation (eg use of SnapMirror). It also then grouped and pooled objects into Datasets and Resource Pools so that you can manage at scale - for an overview see the Integrated Data Management overview paper. http://media.netapp.com/documents/IDMOVERVIEWPAPER.pdf and also the Protection Manager overview by Laura Dubois http://media.netapp.com/documents/ar1064.pdf
One follow up to Mr Backup's post - the application integration into Protection Manager is well under way. The first delivery was our recent (May 2008) release of SnapManager for Exchange which integrates into Protection Manager. This allows you to directly select and manager policy assignment from the SnapManager interface while delivering complete central administration of policies, monitoring and reporting from Protection Manager. We are adding SnapManager for SQLServer and SnapManager for Oracle support by the end of this calendar year.
Thanks Curtis for your note and thanks to the NetApp engineering team for the progress we have made so far...more please.
It's been quite a while since I last posted and during that time I've spent a lot of time of building out the best product roadmap and plans for the Dynamic Data Center. It has really made me think about the skills, personal attributes and values that make up great product managers.
Product Management is one of the most challenging role in any technology company. Many people have described the product management role as the CEO of their product. I don't particularly like that description but certainly they are the focal point of success for their product. They are the hub that keeps a successful product wheel moving.
So what are key success criteria for great product managers? In my mind the following are key themes that make up great product management.
Great product managers execute well against all of the prior themes but they bring more. There's certain personal attributes that are necessary for success - without which product managers can fail. These are soft factors, they're not measurable or predictive, process driven or analytic - they are about attitude.
Together the key themes and the personal attributes are what make great product managers. I am happy to say at NetApp and the team I lead we have these people and I'm impressed every day by their attitude and customer understanding.
Great Products come from Great Product Managers - I am thankful every day that my team are up to that task.
Adios,
Paul.
PS. A great resource for Product Management can be found at Pragmatic Marketing's site
As the scout's motto goes, "Always be prepared", it is best to prepare in advance, actively manage to avert disaster and be prepared in case a disaster does strike. This holds true in life and also for IT.
Your data is the most critical part of your IT environment and hence the storage infrastructure which supports it is critical. Today, 80% of all IT operational issues result from an unwitting change - these issues include application outages, performance problems, unutilized allocated storage, mis-configured systems, and non-compliant systems. Therefore managing the storage infrastructure to meet your required Performance, Security and Availability Service Levels becomes a key to success.
The challenge for IT however is that storage networks continue to grow and complexity spirals so actively managing the Storage Infrastructure becomes increasingly difficult. Storage Service and Change management becomes essential for a well management storage network - including management of Virtual Machine-to-host, host-to-storage, access paths, switch devices and heterogeneous storage arrays. Without tools to enable this it becomes an almost impossible task.
Today NetApp announced a definitive merger agreement to acquire Onaro a leader in Storage Service Management. Storage Service Management answers the challenges of managing large scale storage infrastructures to meet expected Service Levels - it increases data center and storage network efficiencies by proactively measuring and optimizing storage service levels for availability and performance in dynamic data center environments. The predictive nature of Onaro's SANscreen products also makes them ideally suited for modeling and forecasting data center and service level changes—capabilities that customers sorely need as they upgrade, rethink, and deploy their next-generation data centers.
Key capabilities of SANscreen includes
Over 100 large scale Enterprise customers, have found Onaro’s products valuable for ongoing storage service management and also to help storage consolidation, migration, upgrade and disaster recovery readiness. All delivered across the heterogeneous storage environments to meet the demands of large scale enterprise storage customers.
This is what "Being Prepared" is all about. Implementing effective Storage Service Management to your storage infrastructure will help will help your organization avoid unnecessary storage related disasters and in addition help plan for the future.
Adios,
Paul.
Well, it's been a busy week. Just been out on a series of customer visits in New York. Great time to be there - Lincoln Center skating, Holiday Fair in Grand Central, cold morning walks in Central Park - the place really has a great buzz at Christmas.
It was also a great opportunity to meet some of our customers there and talk about their NetApp deployments and plans for evolving data center management. These were all large New York Financial accounts and are some of the most demanding of customers. All of them are implementing more service driven ways to manage their IT environment - their administrative teams cannot scale to meet the data growth challenges and are getting asked for continued high (and higher) levels of service.
A great example of how Policy-based management can change the game for Data Centers came with our discussions with one of these accounts. They are implementing a building level DR solution - they want to have their entire data center mirrored to a different building to cope with disasters (sadly an all too common need and desire now in new post 9-11 New York). The challenge is how can they do this effectively and efficiently.
Traditional disaster recovery on a scale like this would be implemented at an using database inbuilt replication, host-based replication solutions like symantec's veritas volume replicator or storage level replication using EMC's SRDF or NetApp's SnapMirror. The storage level replication has great advantages as you don't have to affect host-side performance on data transfer and so is becoming a far more common approach. However in this customers case the number of objects they need to replicate is huge.
With NetApp Protection Manager in conjunction with SnapMirror a far easier implementation is possible.
The customer can define a single policy, their DR Policy, which governs the level of DR service which is expected - type of replication (single mirror or mirror then backup etc), how much lag time is allowed, max network utilization allowed, your recovery point objective defined in data movement frequency and alerts/warning thresholds. Then they can group entire storage systems at a time into a Dataset by just selecting them, we'll call it Building1 Dataset. (a dataset is a logical grouping of physical storage objects so that you can apply en mass data management techniques like mirror and backup - not requiring repetitive operations). At the destination side the customer also has a set of storage systems setup and grouped into a Building2 Resource Pool (again customers can just drag and drop entire systems in a resource pool). Now the act of setting up the building level replication is selecting the Building1 dataset, applying the DR Policy to it and giving it Building2 Resource Pool as the destination. Protection takes care of everything else, setting up any necessary destination volumes, qtrees, luns and of course the SnapMirror replication which is appropriate (qtree SnapMirror or volume SnapMirror). All this can be setup through a simple intuitive GUI or through CLI in a matter of minutes. Protection Manager lets administrators see what exactly is going to happen before applying and then afterwards it will continuously check for operational conformance and alert administrators to any possible error situations. It continuously enforces the Service Level defined in the policy. An even more powerful combination is possible if you use VMware or other server virtualization software. Then you can start managing the state of the server as data and replicate it in addition to your real data. Recovery becomes a matter of instantiating new VMs from the replicated OS images on the DR side and storage re-mapping the mirrored data (note. NetApp is working with VMware on Site Recovery Manager to make even this fully automated).
Policy-based management can implement the disaster recovery Service Levels required with greatly simplified management - who wouldn't want that.
Applying these same Policy-based management to Backup, Disaster Recovery, Compliance, Provisioning has the opportunity to change the way we manage data and data management operations.
Adios for now,
Paul.
So how did this all come about... A question I certainly ask myself often and those who know me probably more often again. It's just one of those things that bugged me.
There seem to be no really great forums for discussing a Dynamic grid model for Enterprise Data Centers.
The expectations on corporate data centers and the administrative staff who manage them are higher than ever. Business expects data centers to be predictable, deliver to service levels and be cost efficient. It requires a change in the way we manage from a physical system oriented view, as IT has in the past, to an Application Service-driven way to manage.
The Dynamic Data Center is an approach to designing and implementing a Service-driven data center.
Today, High Performance Computing Grids can certainly make statistical analysis scream but can they provide the dynamic grid that enables your Financial Database to meet your quarter end without incident. The fact is that the deploying Data Center applications onto a grid computing model requires more than what High Performance Computing Grids (HPC) needed in the past.
Data Center Applications can gain from the dynamic capabilities of compute and storage on demand just like HPC grids but in addition they require the specific levels of reliability, fault tolerance and security that enterprise applications demand.
A Dynamic Data Center has the opportunity to change not just the price (in a positive way) but also the entire IT management lifecycle - Installation, Updating and Patching, Operational management (including dynamic cpu and storage growth/reduce) and Final Obsolescence.
In this blog I'll attempt to raise discussion on this topic with my blog as a seed (with all of its and my imperfections).
Some likely upcoming topics which will hopefully provide some food for thought ...
Hope you can join me for some lively discussion on this topic,
Paul.
