In the world of Enterprise Storage, the target audience for any published benchmark should go a long way towards determining a vendor’s results and subsequent marketing spin. There are three common types of benchmarks I see storage vendors publish regularly. I’ve categorized them as follows:
- Bench-Marketing
- Imagineering
- Bench-Sizing
I’ll expose each of them starting with the first one today.
Bench-Marketing
This term is actually becoming quite common in the IT industry. This type of result is often marketed with a testosterone-heavy “mine’s bigger” approach where a giant number is published which is somehow bigger than the one your rival published in the recent past. Attention is usually lavished on some gleaming new hardware architecture which has just been introduced. The benchmark report itself often resembles a tome the size of the Yellow Pages. Not because of any elaborate requirement for complete transparency or detailed configuration disclosure. Nope, the actual reason is that these systems must list customizations so extensive, that reading the entire IRS Tax Code seems simple in comparison.
As a customer, VAR or consultant though, one thing is for certain – you will never configure said system in the same manner for a production environment. Based on that sobering truism, these benchmarks have little value in the real world of storage deployments. Therefore it’s clear that the corporate ego-driven target audience for a Bench-Marketing report is essentially the vendor’s direct competitors. Despite that, I actually consider this somewhat admirable, because the focus is still on improving your own product instead of tearing down your competitor’s – the subject of my next blog post.
