NetApp - Storage Nuts & Bolts

Sneak Preview: SnapManager for Hyper-V (SMHV)

Server Virtualization is one of the most disruptive technologies that has hit the industry the past 20 years. Arguably, the only thing that comes to it, is the transition from Mainframes to Open Systems.

That said, server virtualization also presents some challenges that require customers to re-think their infrastructures because some things tend to become challenging. Backup and Recovery is an example of that.

Server Virtualization challenges that I/O limitations of Physical servers and can introduce new bottlenecks which can make it rather difficult to backup the large amounts of data now residing on the physical. Although traditional backup methods have served us well for years, unfortunately, in server virtualization environments don't scale particularly well, never mind that in some cases, they contradict the most basic values of server virtualization technology.

Continuing our integration theme with various server virtualization technologies, today i'm going to show you what we're doing in order to accelerate Backup and Recovery of Virtual Machines in Microsoft Hyper-V R2 environments and do so in manner where we do not impact the performance of the VM, the physical server host itself or the array. 

NetApp Collector & Analyzer for Virtual Environments

A little bit over a year ago we undertook a project to develop a NetApp Collector & Analyzer tool that would be used to collect all storage relevant performance data from physical hosts in order to properly size our disk arrays for customers embarking on server virtualization projects. 

Now, one would ask why would NetApp try to re-invent the wheel given there are a lot of tools that can provide this capability such as VMware's Capacity Planner, Microsoft's Assessment and Planning Toolkit, and Windows native tools like Perfmon and Logman. The reason we decide to proceed with a project like this, is because either the above tools do not collect all the performance counters we need, and/or the analysis phase can be quite complex.

Having to go over 1000s of CSV files, analyze them, plot the output and decide on what values to input into our Sizing Tool is no simple task and takes time and resources. The reality is, the entire process can be quite a pain in the a$$.

So we developed a NetApp Collector and Analyzer Tool that will...collect tall he relevant performance data we're interested in and will perform analysis of 1000s of CSV files produced with the push of a button.

  This is an agentless collection with the requirement being windows user with WMI access. You can set things as the:

• Collection Interval
• Collection Duration
• Output Directory to dump a zip file containing the CSVs
• Can add individual hosts
• Can import Hosts from a CVS
• Can export hosts to a CSV
• Can encrypt host credentials read from a CSV

  After you've collected the data, you'd fire up the NetApp Analyzer tool. Prior to doing that, you want to unzip the zip file containing the CSVs. Then you'd start the Analyzer and you will point to to the directory the CSVs are located.

Depending on the Number of Hosts, Collection intervals and Duration, the amount of the CSVs and the size of each CSV will vary.  For example, for a 7 day run at 5 minute interval (default) for 49 hosts, I ended up with 1500 CSV files to be analyzed each of which had a 32k size. 

Under normal circumstances, analysis of 1500 files, would be a huge task, however using the NetApp Analyzer the entire process took about 15-20 minutes.

So what were the results I got back?

The above shows an overview of the results in terms of AVG and MAX numbers as well as the different workload characteristics. We would then take these number and input then into our NetApp Application Sizer which will produce a configuration interms of the Disk Array required and necessary capacity to sustain this load.

If you look at the above screenshot, there's a substantial deviation between the AVG and the MAX. So what would you size for? AVG or MAX? It depends.

From the above graph we see there are some spikes so we need to figure out what causes them by showing the data to the customer and asking questions. In this particular case, the spikes occur every day at approximately 9:00pm. These spikes are most likely due to some form of a backup procedure.

After, you decide, based on customer input, as to what to size for, you would use the above data points as input into the NetApp's Custom Application Sizer which will produce the necessary array and relevant configurations for the customer.

The NetApp Collector and Analyzer is available to all NetApp SEs and NetApp partners to assist their customers in server virtualization projects and proper storage sizing. It can be download from Field Portal. If you have difficulty locating it, search for "Collector". 

The NetApp Custom Application sizer is a web based tool also accessible to partners thru Field Portal.

Cheers

 

Technorati Tags: VMware,Hyper-V-Citrix,Performance Sizing,NetApp Collector & Analyzer,Server Virtualization Storage Sizing

SnapDrive for Windows 6.1 Update

I've talked about Snapdrive at least once the past 2 years, here , but for those not familiar with it,  I will provide a high level overview.

SnapDrive (aka SDW) is part of NetApp's Server Suite Software that simplifies provisioning and increases availability, reliability of application data by tightly integrating with our SnapManager solutions part of Application Manageability SW Suite. 

• SDW Key Features
  • Simplified provisioning and robust Snapshot™ management
  • On-the-fly LUN creation
  • File system consistent Snapshot copies
  • Instantaneous Backups 
  • Rapid restores
• Increased Storage Utilization and Efficiency
  • Dynamic LUN Expansion
  • Dynamic LUN Shrink with Windows Server 2008 (New)
  • Immediate creation of space efficient clones of application data
  • NTFS Block Space Reclamation for Physical and Virtual environments

As it is typical with any software product out there, with every release there are bug fixes and new functionality. The 6.1 release follows that rule as well. So what's new?

• Server Virtualization
  • Support for Windows Server 2008 Hyper-V
    • Parent OS (Server Core, Full Install, Hyper-V Server)
  • VMware GuestOS
    • Allow creation of RDM LUN when OS vmdk resides in NFS Datastore
      • Drop down window lists Target VMFS Datastore(s) for Mapping file
• Infrastructure
  • IPv6 support
    • iSCSI over IPv6
    • HTTP/HTTPS over IPv6
    • SMTP notification over IPv6
  • FCoE support
• Security
  • Storage Access Control
• Storage Efficiency
  • Space Reclamation of Thin Provisioned volumes
  • Online shrink of NTFS and LUN

Hype-V Support Specifics

• LUN provisioning and Snapshot management
• Support for Windows Failover Clustering
• Support for Hyper-V quick Migration
• VSS auto-recovery
  • Backup and Restore
• Enables 3rd party VSS Requestors for Backup and Restore
• Integrates with NetApp's HyperVIBE solution
• MMC Changes
  • Displays details about Hyper-V VMs
  • Displays details about Physical Host

Storage Access Control

Without a doubt, one the areas that SDW needed to improve was Access Control. With previous releases, each volume, qtree, lun was available for SDW operations.

With the 6.1 release, this has been addressed, and now the functions of the Storage Admin can be separated from the functions of the Server admin for delegated environments. 

The new release uses an array base XML file to store access control information. Access control is applied at 3 levels, Volume, Qtree, LUN and there are 3 Access Control Types: Read, Write, and Delete.  

For every SnapDrive® operation, an Access Check is performed by reading the content of this file. Access Check verifies whether a SnapDrive user can perform an operation on a given storage system resource. This file is under the control of Storage Administrator.

Any attempt to create a LUN, a Snapshot, or restore without proper authorization  will fail and will be logged.

 

 

 

 

LUN Grow-Shrink

With this release we've combined the LUN Grow capability previously available with SnapDrive, with the new LUN Shrink operation now available with Windows Server 2008,  using a Single interface. Just drag the Slide Bar to either direction depending on what you want to accomplish.

 

 

 

 

 

 

Usability Improvements

• Select a Snapshot, click it, View the LUN(s) , and mount whichever one you'd like onto the host by selecting the "Connect Disk" option. There's no need to type the LUN Path.
• Add Volume Labels so LUNs can be identified in Disk Management and Explorer

• SnapDrive will now Auto-Populate all Microsoft Clustered Nodes

In summary, the new SDW release, enables backup and restore of Hyper-V VMs, improves upon existing VMware capabilities, addresses Storage Efficiencies by providing NTFS block reclamation for Thin Provisioned volume and LUN shrink, in Virtual and Physical environments. Additionally, it adds IPv6 support, introduces FCoE support and Storage Access Control. SDW 6.1 is scheduled to post on the NOW site within the few days.

Technorati Tags: SnapDrive,NetApp

Sneak Preview: NetApp System Manager (NSM)

For the past couple of months I've been playing with NetApp's upcoming Element Manager which is a complete re-write and eventually will replace FilerView. Since Val let the cat out of the bag, that opens the door for a blog entry.

Update - 03/06/2009: There has been an article recently posted by the Register stating that NetApp dumps FilerView in favor of the upcoming NetApp System Manager. That statement is not true. FilerView will continue to exist and be fully supported in the forseeable future.

The basic idea behind the NetApp System Manager (NSM) is to provide comprehensive management and capability of one of more arrays via a simple, easy to use, intuitive GUI.

NSM is a Windows/MMC 3.0 application that supports Discovery, Setup, FCP, iSCSI, CIFS, NFS, Deduplication, provisioning, Thin Provisioning, Snapshot and configuration management of multiple NetApp Storage Systems from a single pane of glass.

Storage Configuration Wizards

New wizards allow Admins to create quickly and easily create Aggregates, Volumes, LUNs, shares as well as a new Server Virtualization wizard that supports provisioning for VMware implementations.

Aggregate Wizard

Aggregate creation is literally a 3 click process. You can let the system select the appropriate disks based on the disk type you choose automatically, or you can select them manually. The Automatic process makes sure all disks in the aggregate are balanced across multiple back-end loops.

Volume Wizard

Creating a volume is similarly a no brainer process. Pick the aggregate you want, select the Volume size, snapshot space, %, enable thin provisioning if you'd like at the Volume level and/or enable deduplication under the Space settings tab and that's it. In there you can set the different Space Guarantees provided depending on what it is you want to accomplish.

LUN Wizard

The process of creating a LUN is pretty much out of the "How to Create a LUN for Dummies" book. Select a name for the LUN, it's size, the LUN type which corresponds to the OS type the LUN will be used by, click next, select an existing NetApp Aggregate/Volume that will contain the LUN or create a new Aggregate/Volume, select the initiators (FCP or iSCSI) that will have access to the LUN and you're done. Given the entire process is wizard driven I have personally found no way to mess up.

Server Virtualization Wizard

Given that we're seeing a large number of customer requests for NFS implementations for VMware, the NSM provides a Wizard for creating NFS Datastores. The process is similar to creating a Volume, however, the wizard will also ask for the VMkernel IP address of the ESX host(s) so the NFS volume gets exported. The dedup option is also there should you choose to take advantage of our NetApp level dedup capability.

Dashboard/System Tools

NSM provides a dashboard with notifications that are integrated with the Windows System Tray so you receive updated notifications regarding the system's health status. The Dashboard also includes it's own notification pane and performance pane. Autosupport, licensing, SNMP, and syslog can also be configured.

NSM will also be supported for the IBM N-Series system. The product will become available in the next few months, in the mean time it will soon become available to the NetApp Beta Community as a download for testing. We'd love to hear your feedback!!!

Technorati Tags: NetApp,NSM,NetApp System Manager

SPEC SFS 2008: The Benefit of the Performance Acceleration Module

Back in August I posted an entry regarding NetApp's Performance Acceleration Module (PAM). To summarize, the PAM card is a 16GB DRAM based read cache integrated with ONTAP that shipped around the middle of 2008. Depending on the platform, up to 5 modules can be installed for a maximum of 80GB of intelligent read cache.

The caching policies implemented as part of ONTAP in conjunction with the usage of the PAM(s) are designed to discern random from sequential reads and hold on to these blocks in cache in order to minimize back-end disk reads which can add latency.

In that article, I also said that an additional PAM benefit is that it can, potentially, provide significant reduction in terms of  the number of disks and shelves to be deployed in order to support the same application(s), thus reducing costs, while providing great response times.

Today, we announced results using the new SPEC SFS 2008 Benchmark  to test 3 configurations. For more on the Specifics of the SPEC SFS 2008 Benchmark, Mike Eisler has a couple of great posts here and here.

The individual benchmarks results can be be found below:

• FAS3140 + 224 15k RPM FC Disks
• FAS3140 + 112 15k RPM FC Disks + 2 PAM Modules (16GB each)
• FAS3140 + 112 7.2k RPM SATA Disks + 2 PAM Modules (16GB each)

However, thru the good work of a co-league of mine, the results from the 3 configurations have been summarized on the below chart:

 

Based on the specific results the conclusion one can easily arrive to is that:

• SATA is a viable option ,given that it is possible to increase storage capacity by 75%, reduce disks by 50% and still achieve highly acceptable levels of performance. 
• Improve response times, by 35% using FC drives while reducing the number of drives by 50%

Starting with DataONTAP 7.2.6.1, we've provided the ability thru the collection of Predictive Cache Statistics (PCS) to determine whether specific workloads will benefit from the deployment of PAM modules and with what significance.

 

Technorati Tags: Performance Acceleration Module,PAM,NetApp SPEC SFS 2008

Windows 2008 Hyper-V and SCVMM 2008 Part III

I didn't have a chance to continue writing last week on my Hyper-V series post because I had to deal with Domain issues that prevented me from adding my second Hyper-V node to the domain. Finally, I was able to figure out the root cause which was duplicate SPNs and fix it last night.

In the last two posts I talked about some of the Hyper-V requirements, and dealing with Server Core installations. In this post I'd like to talk a bit about SCVMM (System Center Virtual Machine Manager).

Hyper-V: Released To Manufacturing

Microsoft has made it official today. Hyper-V is RTM. The built will be available at http://www.Microsoft.com/Hyper-V today at 12:00 pm PDT. It will also be available via a Windows Update around July 8th.

More info from John Howard's, Sr.Product Manager,  Hyper-V Blog

In order to manager Hyper-V you want to download SCVMM RC1 from here. Access is open to the public after you register.