June 08, 2009

Virtual Storage Console

Back in March, I posted a blog and a video that was shown at VMworld Europe 2009 regarding the upcoming vCenter Host Utilities Kit. Since then I've gotten several inquires about it.  So let me dig a little deeper and provide a bit more insight.

First of all, the ESX Host Utilities Kit has been renamed to Virtual Storage Console (VCS). In fact, the last standalone ESX Host Utilities Kit that supports vSphere is the 5.1 version that's already posted on the NOW site. The next release will be the VSC 1.0.

What is it?

VSC is NetApp specific software that enables administrators to manage and monitor NetApp specific storage side attributes of ESX-ESXi hosts. VSC functions as a plugin to vCenter and uses NetApp specific APIs to set and retrieve information from the array .

By default VSC uses SSL and SSL signed certificates. Furthermore any array credentials and passwords are stored using 128bit encryption.

What does it do? 

VSC adds a tab into vCenter and enables the following:

  • View Status of Storage Controllers
  • View Status of physical hosts, including versions and overall status

 esx_ctlr_status1

 

 

 

 

 

 

 

 

 

  •   Check for the proper configuration of ESX settings as it applies to:
    • Multipathing Settings
    • NFS timeouts
    • HBA driver timeouts
  • Provide the ability to set the appropriate to set the appropriate timeouts on multiple ESX hosts simultaneously with a single mouse click 

 

settings2

  • Launch FilerView from within VSC  for storage provisioning
filerview

 

  • Provides access to mbrtools (mbrscan, mbralign, mbrcreate) to identify and correct partition alignment issues

mbrtools

 

  • Ability to set credentials to access storage controllers
  • Ability to collect diagnostics from the ESX hosts, FC switches and Storage controllers
diags
  • Capacity Reports on:
    • Datastore
    • LUN
    • Volume
    • Aggregate
  • Deduplication Reports
    • Deduplication State
    • Deduplication Status
    • Space Savings
  • LUN Status
    • Pathname
    • NAA ID
    • Status - Online/Offline
    • Protocol
    • ALUA - Enable/Disable

storage_properties

Requirements:

  • DataONTAP 7.3.1.1 or later
  • VCenter Server 4.0
  • ESX/ESXi 4.0

How about ESX 3.5?

In order to use all of the VSC functionality, ESX 4.0 is needed and vCenter 4.0. However, at a minimum, vCenter 4.0 is required. ESX 3.5 hosts will be discovered however the appropriate timeouts can not be modified from within the console.

VSC Status

VSC is currently under Beta and we believe we'll have it available as soon as the Beta process is complete. I can not comment any further on this without getting into NDA specifics. 

May 27, 2009

mbraling in ESX Host Utilities 5.1

First of all, I'd like to apologize since I haven't posted in over a month but it's been a pretty tough couple of months for me and my family.

My mother was diagnosed with NHL lymphoma on April 18 and then within 30 days, she passed away. Very rapid and shocking developments that forced me to fly back and forth some 8,000 miles away. What makes it even harder was the fact that she was only 69 years old. 

On a happier note, over the past 4-5 mos I've gotten more that 40 requests from partners and customers asking for access to the mbralign utility. For those who don't know what it does, you may want to take a look at  this post.

As of May 21, 2009, NetApp released a new version of the ESX Host Utilities (HUK), 5.1, which now contains both the mbrscan and mbraling utilities. Both utilities are now fully supported by NetApp and documentation is provided as part of the ESX HUK 5.1 Install and Configuration Guide available in the Download section of the Now Site

New Additions to the HUK

  • Support for vSphere.
  • Support for role-based access control (RBAC) user names so that root access to storage controllers is not required.
  • The mbralign program that fixes misaligned VMDK partitions.
  • New timeout values added to config_hba to support software iSCSI initiators using 10 Gb Ethernet. 
  • Updated SNIA API libraries for Emulex and QLogic adapters.
  • An -expert option for the Host Utilities install command that installs the Host Utilities components but does not set timeouts or paths.
  •  

    April 03, 2009

    SnapDrive for Windows 6.1 Update

    I've talked about Snapdrive at least once the past 2 years, here , but for those not familiar with it,  I will provide a high level overview.

    SnapDrive (aka SDW) is part of NetApp's Server Suite Software that simplifies provisioning and increases availability, reliability of application data by tightly integrating with our SnapManager solutions part of Application Manageability SW Suite. 

    NMS

    • SDW Key FeaturesSDW1
      • Simplified provisioning and robust Snapshot™ management
      • On-the-fly LUN creation
      • File system consistent Snapshot copies
      • Instantaneous Backups 
      • Rapid restores
    • Increased Storage Utilization and Efficiency
      • Dynamic LUN Expansion
      • Dynamic LUN Shrink with Windows Server 2008 (New)
      • Immediate creation of space efficient clones of application data
      • NTFS Block Space Reclamation for Physical and Virtual environments

    As it is typical with any software product out there, with every release there are bug fixes and new functionality. The 6.1 release follows that rule as well. So what's new?

    • Server Virtualization
      • Support for Windows Server 2008 Hyper-V
        • Parent OS (Server Core, Full Install, Hyper-V Server)
      • VMware GuestOSSDW11
        • Allow creation of RDM LUN when OS vmdk resides in NFS Datastore
          • Drop down window lists Target VMFS Datastore(s) for Mapping file
    • Infrastructure
      • IPv6 support
        • iSCSI over IPv6
        • HTTP/HTTPS over IPv6
        • SMTP notification over IPv6
      • FCoE support
    • Security
      • Storage Access Control
    • Storage Efficiency
      • Space Reclamation of Thin Provisioned volumes
      • Online shrink of NTFS and LUN

    Hype-V Support Specifics

      • LUN provisioning and Snapshot managementSDW 2
      • Support for Windows Failover Clustering
      • Support for Hyper-V quick Migration
      • VSS auto-recovery
        • Backup and Restore
      • Enables 3rd party VSS Requestors for Backup and Restore
      • Integrates with NetApp's HyperVIBE solution
      • MMC Changes
        • Displays details about Hyper-V VMs
        • Displays details about Physical Host

    Storage Access Control

    Without a doubt, one the areas that SDW needed to improve was Access Control. With previous releases, each volume, qtree, lun was available for SDW operations. Pre-SnapDrive 6.1SDW4

    With the 6.1 release, this has been addressed, and now the functions of the Storage Admin can be separated from the functions of the Server admin for delegated environments. 

    The new release uses an array base XML file to store access control information. Access control is applied at 3 levels, Volume, Qtree, LUN and there are 3 Access Control Types: Read, Write, and Delete.  

    For every SnapDrive® operation, an Access SDW5Check is performed by reading the content of this file. Access Check verifies whether a SnapDrive user can perform an operation on a given storage system resource. This file is under the control of Storage Administrator.

    Any attempt to create a LUN, a Snapshot, or restore without proper authorization  will fail and will be logged.

    SDW6

     

     

     

     

    LUN Grow-Shrink

    With this release we've combined the LUN Grow capability previously available with SnapDrive, with the new LUN Shrink operation now available with Windows Server 2008,  using a Single interface. Just drag the Slide Bar to either direction depending on what you want to accomplish. SDW7

     

     

     

     

     

     

    Usability Improvements

    • Select a Snapshot, click it, View the LUN(s) , and mount whichever one you'd like onto the host by selecting the "Connect Disk" option. There's no need to type the LUN Path. SDW8
    • Add Volume Labels so LUNs can be identified in Disk Management and Explorer

    SDW9

    • SnapDrive will now Auto-Populate all Microsoft Clustered Nodes

    SDW10

    In summary, the new SDW release, enables backup and restore of Hyper-V VMs, improves upon existing VMware capabilities, addresses Storage Efficiencies by providing NTFS block reclamation for Thin Provisioned volume and LUN shrink, in Virtual and Physical environments. Additionally, it adds IPv6 support, introduces FCoE support and Storage Access Control. SDW 6.1 is scheduled to post on the NOW site within the few days.

    Technorati Tags: ,

    March 20, 2009

    Virtual Desktop Ask the Expert Session

    Given the strong trend towards returning to thin computing by leveraging various Virtual Desktop technologies, NetApp will be hosting a weeklong "VDI Ask the Expert" discussion in our community forums. Ask the Expert

    We have identified several internal and well known external experts and bloggers with large scale Virtual Desktop implementation deployment experience that will help answer technical and non-technical questions around Virtual Desktop. If you follow the server virtualization blog-o-sphere, you will know these folks. If you don't, then here's your chance to meet them and ask questions...virtually.

    We'll also have a commitment from Citrix and we're in talks with VMware to secure their highly respected resource(s) as well. banner

    Registration

    There is no registration for the event and all content can be viewed by logging in as a "guest". However, for those who would like to actively participate by asking specific questions you will need to use your NOW login. If you don't have a NOW login you can request one by following the instructions below. 

    Viewing Content using as a Guest

    Navigate to the Community page below. By default you will have Guest access be able to view all contents in the forum. http://communities.netapp.com/community/technet/ask_the_expert

    For Active Participants

    For those of you that would like to actively participate, and ask questions, you will need to use you NOW account, if you have one. If you don't, we ask that you obtain one. NOW accounts are available to anyone.

    How do I get a NOW account?

    Follow the above link and click on "Login" listed in the Upper Left corner of the screen.  You will then be prompted to enter your NOW account Credentials or Register for a new NOW account. 

    Picture1NOw


    Hope to see you there!!


    March 05, 2009

    The NetApp Experience

    First of all, I'd like to apologize to those who have attempted posting comments on my latest 3 entries. For some reason comments on these entries are Closed and it is a TypePad issue with those who are using 3rd party editors. In fact, TypePad does not even list my latest 3 entries as present although there are there. That means I can not fix the comments issue unless I delete the blogs and re-post them using the TypePad editor...How's that for an inconvinient workaround? Again, my apologies.

    Anyway, right after I got out of college I was hired as a Tech Support Engineer by a small Supecomputing company called Convex Computer Corp. For me, Convex served as a great learning institution, not only from a technological standpoint but also from a cultural perspective. The company was not just product focus as many small companies are today, but it also paid particular attention to Customer satisfaction pre and post sales. In fact, if you were to walk into our Support Center in Richardson, Texas, there was a big sign posted saying "Take Care of YOUR Customer or Someone else Will". Convex meant it. So does NetApp and that's one of the reasons a lot of us love to work here.

    I've removed the Company names as well as the names of the individuals involved to protect their privacy and competitive advantage...as well as ours.

    The NetApp Experience - Are you experienced?

    In the summer of 2005 1 began a search to find and implement a distance DR replication solution for our data center. The solution we need would have to be able to replicate SQLServer 2005, Exchange, and millions of standard document files (Flat Text, Word, Excel, PowerPoint, etc.).

    We needed to have the data at the backup site in such a manner that we could achieve a maximum 2 hour RTO and RPO. We wanted to do this with as close to zero foot print on the servers hosting the data as possible.

    I spent many long hours with various storage and software vendors over several years trying to find the solution that would fit our stringent criteria. Many said they could but when we put their solutions to the test they failed to deliver.

    In the winter of 2007 1 was introduced to NetApp storage. At first I wasn't interested because NetApp didn't do fiber channel based storage solutions, only iSCSl and NAS. I am happy to say my presumption was wrong.

    XXXX did an excellent job of introducing me not only to the products that NetApp has to offer but how the solutions work in painstaking detail.

    He obtained time in your XXXX lab to setup a model of our full database environment and perform some preliminary real world tests of the products we were going to use. During our time in the lab there was no limit to the technical resources that were made available to us to make sure that things were working properly.

    I can remember spending hours working with XXXX to understand how Snap Manager for SQL actually works under the covers. Most of this discussion happened after 5PM and he stayed with us until 8:30PM in the lab. He seemed almost as excited as we were to get it working.

    After the preliminary tests proved promising, we proceeded into the next phase of the project, an onsite trial. We began to interact with several more people in the engineering department. In particular, we worked closely with XXXX and XXXX to develop a test plan and a realistic timeline to conduct the tests over.

    The help didn't stop there. XXXX also worked with us to make sure we had the tools necessary to perform the tests and record the results. XXXX made himself available to us to do the initial consult on disk layout. He was instrumental in guiding the initial installation of our environment. XXXX and XXXX implemented Don's design in short order.

    The other important thing to understand during this process is the engineers didn't just do the work. They took the time to explain what they were doing and why. We take a high level of ownership in all hardware in our datacenter. The NetApp SANs were to be no different. This knowledge transfer was critical to make this a success. While they were teaching us, we created much of the documentation that we use now to guide our day to day operations.

    As we worked through the agreed upon tests XXXX and XXXX checked in frequently to make sure things were on track. If we encountered any problems they made themselves available to solve the problem and make sure we understood why the problem happened. A fair number of these calls happened after normal business hours, in the very early morning, and on the weekend. They were always there for us. Ultimately we found that unlike previous experiences NetApp delivered largely what they promised.

    There were some tweaks to the plan but overall our expectations were met and we are very happy. We are within tolerance on our DR replication which gives us a huge competitive advantage over our peers. Not only that but it helped us change how we provide our report, test, and development database environment.

    Previously, this process would take at approximately 4.5 hours to complete. Now it takes

    0.5 hours. Now that we have completed all phases of this project I have a new perspective on hardware acquisition. It's not only important to weigh the quality of the hardware but also the mettle of the people servicing the hardware. Without both of these you're not really solving the problem but introducing new issues to your critical operations.

    XXXXXXXXXXXXXX

    Director of Infrastructure

    XXXXXX Company

    March 03, 2009

    NetApp ESX Host Utilities vCenter Plugin

    Last week we showed a NetApp demo at VMworld Europe of the NetApp ESX Host Utilities Kit (HUK) running as a vCenter Plugin.

    Thus far, the Host Utilities, including the current 5.0R2 release needed to be installed at the console of each ESX host. With the 6.0 release this will changes and a single vCenter plugin instance will manage all the ESX hosts in the Datacenter from a centralized management location.

    It will support both ESX and ESXi implementations as well as all protocols, FC, iSCSI, FCoE, NFS and you will be able to set all the necessary parameters for every ESX host in the cluster regardless of protocol implementation.

    All wizards, and panels have similar look at feel to vCenter. Access to NetApp Arrays is provided via NetApp Zephyr APIs (ZAPI) over SSL  but if SSL has not been configured, or it's not required, a non-secure authentication method is available.

    Additionally, you will be able to view the configuration details such as like Total capacity, Total used, Total allocated, System status,  IP information, and alerts that pertain to possible pathing problems, or any misconfigured driver parameters that NetApp recommends when connecting to ESX hosts. You will be able to address any of these issues from vCenter with the use of the Plugin.

    In fact, we've demo the upcoming ESX Host Utilities Kit 6.0 at VMworld Europe last week. I apologize for the absence of sound on the video however, the Callouts makes it easy to follow.

    The release of the new ESX Host Utilities Kit is dependent upon the official release of VMware's vSphere.

     

    March 02, 2009

    Sneak Preview: NetApp System Manager (NSM)

    For the past couple of months I've been playing with NetApp's upcoming Element Manager which is a complete re-write and eventually will replace FilerView. Since Val let the cat out of the bag, that opens the door for a blog entry.

    Update - 03/06/2009: There has been an article recently posted by the Register stating that NetApp dumps FilerView in favor of the upcoming NetApp System Manager. That statement is not true. FilerView will continue to exist and be fully supported in the forseeable future.

    The basic idea behind the NetApp System Manager (NSM) is to provide comprehensive management and capability of one of more arrays via a simple, easy to use, intuitive GUI.

    NSM1 NSM is a Windows/MMC 3.0 application that supports Discovery, Setup, FCP, iSCSI, CIFS, NFS, Deduplication, provisioning, Thin Provisioning, Snapshot and configuration management of multiple NetApp Storage Systems from a single pane of glass.

    Storage Configuration Wizards

    New wizards allow Admins to create quickly and easily create Aggregates, Volumes, LUNs, shares as well as a new Server Virtualization wizard that supports provisioning for VMware implementations.

    Aggregate Wizard

    Aggregate creation is literally a 3 click process. You can let the system select the appropriate disks based on the disk type you choose automatically, or you can select them manually. The Automatic process makes sure all disks in the aggregate are balanced across multiple back-end loops.

    NSM3 NSM4

    Volume Wizard

    Creating a volume is similarly a no brainer process. NSM5 Pick the aggregate you want, select the Volume size, snapshot space, %, enable thin provisioning if you'd like at the Volume level and/or enable deduplication under the Space settings tab and that's it. In there you can set the different Space Guarantees provided depending on what it is you want to accomplish.

    LUN Wizard

    The process of creating a LUN is pretty much out of the "How to Create a LUN for Dummies" book. NSM8 Select a name for the LUN, it's size, the LUN type which corresponds to the OS type the LUN will be used by, click next, select an existing NetApp Aggregate/Volume that will contain the LUN or create a new Aggregate/Volume, select the initiators (FCP or iSCSI) that will have access to the LUN and you're done. Given the entire process is wizard driven I have personally found no way to mess up.

    Server Virtualization Wizard

    Given that we're seeing a large number of customer requests for NFS implementations for VMware, the NSM provides a Wizard for creating NFS Datastores. The process is similar to creating a Volume, however, the wizard will also ask for the VMkernel IP address of the ESX host(s) so the NFS volume gets exported. The dedup option is also there should you choose to take advantage of our NetApp level dedup capability.

    NSM10

    NSM9

    Dashboard/System Tools

    NSM provides a dashboard with notifications that are integrated with the Windows System Tray so you receive updated notifications regarding the system's health status. The Dashboard also includes it's own notification pane and performance pane. Autosupport, licensing, SNMP, and syslog can also be configured.

    NSM12

    NSM will also be supported for the IBM N-Series system. The product will become available in the next few months, in the mean time it will soon become available to the NetApp Beta Community as a download for testing. We'd love to hear your feedback!!!

    February 03, 2009

    SPEC SFS 2008: The Benefit of the Performance Acceleration Module

    Back in August I posted an entry regarding NetApp's Performance Acceleration Module (PAM). To summarize, the PAM card is a 16GB DRAM based read cache integrated with ONTAP that shipped around the middle of 2008. Depending on the platform, up to 5 modules can be installed for a maximum of 80GB of intelligent read cache.

    The caching policies implemented as part of ONTAP in conjunction with the usage of the PAM(s) are designed to discern random from sequential reads and hold on to these blocks in cache in order to minimize back-end disk reads which can add latency.

    In that article, I also said that an additional PAM benefit is that it can, potentially, provide significant reduction in terms of  the number of disks and shelves to be deployed in order to support the same application(s), thus reducing costs, while providing great response times.

    Today, we announced results using the new SPEC SFS 2008 Benchmark  to test 3 configurations. For more on the Specifics of the SPEC SFS 2008 Benchmark, Mike Eisler has a couple of great posts here and here.

    The individual benchmarks results can be be found below:

    However, thru the good work of a co-league of mine, the results from the 3 configurations have been summarized on the below chart:

     SPECSFS

    Based on the specific results the conclusion one can easily arrive to is that:

    • SATA is a viable option ,given that it is possible to increase storage capacity by 75%, reduce disks by 50% and still achieve highly acceptable levels of performance. 
    • Improve response times, by 35% using FC drives while reducing the number of drives by 50%

    Starting with DataONTAP 7.2.6.1, we've provided the ability thru the collection of Predictive Cache Statistics (PCS) to determine whether specific workloads will benefit from the deployment of PAM modules and with what significance.

     

    January 19, 2009

    mbrscan/mbralign

    I figured I'd take a que from Duncan's recent post over at Yellow Bricks titled "Check your VM's Alignment" to write about a couple of tools not widely advertised called "mbrscan" and "mbralign".

    As you already aware partition alignment can become a performance inhibitor. The Partition alignment process is not a new concept and has existed in the industry for years for Windows and Linux system way before the introduction of server virtualization.

    Continue reading "mbrscan/mbralign" »

    December 22, 2008

    Unified Host Utilities Kit 5.0 for ESX Server is Released

    Last week we released a Unified version, 5.0, of the Host Utilities Kit for ESX server. There are some changes, as well as some new additions to the Kit.

    First of all, a new script has been added called config_nfs which sets the appropriate NFS Heartbeat parameters.

    The filer_info script has been renamed to controller_info and it now supports SSL. In fact, SSL is also supported for the config_hba and config_mpath scripts.

    UPDATE 01/19/2009: Unfortunately, we have temporarily pulled the ESX 5.0 Host Utilities Kit due to a bug found that causes the config_mpath utility to go into an infinite Loop. This issue does not affect all systems nonetheless we have seen 2 field occurrences with this behavior. A patch has been issued and is currently under QA. As soon as the QA process is complete the 5.0 HUK will be available for download again, most likely as 5.0.1. In the mean time time, continue to deploy the ESX 3.1 Host Utilities Kit. I will post a notice here as soon as the 5.0.1 kit is available

    Update 03/02/2009: The ESX Host Utilities Kit v. 5.0R2 is available for download from the NOW site.

    There's a new utility that has been added, called mbrscan. The purpose of mbrscan is to identify wheather or not a VM has properly aligned partitions. The script will run successfully under two conditions: a) The VM is powered off b) the VM has VMsnaps. The script will error out if none of the above two conditions are satisfied simply because the VMDK(s) are locked.

    There are new utilities added that will you help set GuestOS timeouts. That includes Windows, RHEL, SUSE and Solaris.

    Here's a list of the all the scripts included as part of the Unified Host Utilities Kit v5.0:

  • install - Install script for the EHU
  • brocade_info - Collects configuration information about Brocade FC switches
  • cisco_info - Collects configuration information about Cisco FC switches
  • config_hba - Utility used to set HBA parameters for communicating with NetApp storage devices. This script will get run as part of the installation and can be executed subsequently at any time.  Support was added for 8GB FC HBAs and well as FCoE CNAs.
  • config_mpath - Utility used to determine which of the available paths are primary paths and to set primary paths
  • config_nfs - Utility used to set the NetApp recommended NFS Heartbeat settings.
  • controller_info - Collects configuration information about NetApp storage devices.
  • mbrscan - Utilty used to check vmdk files for proper alignment from the ESX console (for VMFS and NFS datastores), and from unix/linux (NFS datatstores).
  • mcdata_info - Collects configuration information about McData FC switches
  • qlogic_info - Collects configuration information about QLogic FC switches
  • san_version - Prints the EHU version
  • sanlun - collects information about the LUNs currently mapped to your host
  • uninstall - Uninstall script for the EHU
  • That said, I'd like to wish Happy Holidays to everyone.

    © NetApp, Inc.  |  "Safe Harbor" Statement