Nimbus Data: Blog

EMC/XtremIO Details Emerge, Fail to Impress

December 5, 2012
By Thomas Isakovich, CEO & Founder

StorageNewsletter today captured some details revealed by EMC's Chuck Hollis (Global Marketing CTO) about the unreleased XtremIO all-SSD array. Robin Harris of StorageMojo issued a less-than-positive analysis in response. To recap, EMC acquired pre-revenue XtremIO in May 2012 for a reported $430M.

Architecture of XtremIO

According to the report, XtremIO uses a scale-out architecture consisting of an Infiniband backend and iSCSI or Fibre Channel front-end ports. Each XtremIO brick consists of 7 TB of usable capacity (10 TB raw) in a 3U shelf. The brick appears to be an off-the-shelf general purpose server, namely a SuperMicro 6036ST-6LR according to this Google search image. Each brick supports up to 250,000 IOps.

Using a scale-out architecture in this manner is similar to the design employed by EMC Isilon: treat storage as "bricks" consisting of capacity and compute resources in one box, and add more bricks as you need more storage. In theory, such an architecture enables much higher capacities and much higher performance than scale-up arrays. Surprisingly, though, XtremIO's offering seems to provide the exact opposite: much less scalability and much less performance. Let's take a closer look.

Limitations of the XtremIO architecture

There are numerous limitations with the XtremIO design:

Scalability is surprisingly limited: According to the article, the XtremIO design only supports 8 nodes in a system. At 7 TB usable per node, that's 56 TB in one system, a pretty modest amount of capacity for an enterprise storage array. In comparison, Nimbus systems scale to 500 TB (400 TB usable), a better than 7x scalability advantage.
Density is subpar: At 10 TB per 3U brick, that's 3.3 TB per U. That's less rack density than most 15K rpm disk arrays. Nimbus' Gemini platform on the other hand delivers 48 TB per system in just 2U, or 24 TB per U. That's a 7x density advantage for the Nimbus system.
Performance per brick is low: Because XtremIO is based on a commodity server, it inherits the architectural limitations of off-the-shelf servers, limiting performance to 250,000 IOps. In comparison, Nimbus' Gemini system offers up to 1,200,000 IOps per system, a 5x performance advantage that demonstrates the strength of Nimbus' patent-pending purpose-built hardware. Even if one assumes XtremIO scales linearly (which is questionable -- see below), it would take five XtremIO bricks (at perhaps 5x the cost) to do what just one Nimbus system can do.
Power consumption seems high: The article reports 700W per 10 TB XtremIO brick, which works out to 70W per raw TB. This is comparable to 15K rpm disk arrays. Nimbus' fully-redundant Gemini system on the other hand draws about 600W at 48 TB, a 6x power efficiency advantage.
Reliability is undisclosed: A critical requirement with flash is endurance optimization. EMC is quiet on this subject, but I suspect that they are using third-party SSDs in the SuperMicro box. Most third-party SSDs suffer from write performance degradation over time and limited warranties. By comparison, Nimbus offers an available 10-year warranty and utilizes hardware-offloaded wear-leveling technology to ensure consistent performance over time.
Latency penalty: Any scale-out architecture relies on multiple hops along the backend fabric to service an IO. While bandwidth improves with scale-out, latency increases as more bricks are added. Latency is so high on some scale-out systems that they are specifically targeted towards content storage where latency is less of a concern. But this is primary storage, and latency matters. The latency penalty of scale-out may explain why XtremIO's scalability is limited -- as more bricks are added, hops increase and latency rises.
Full-time dedupe penalty: XtremIO appears to require full-time inline deduplication, which adds hashes and lookups to the IO path, increasing latency. The thinking here is that flash memory is too expensive, and deduplication is required to make it affordable. Since deduplication cannot be disabled, though, performance-critical applications like databases and analytics are unfairly penalized. Nimbus Data, on the other hand, gives users flexibility to enable deduplication for portions of the system that will benefit from it (VDI, etc.) while leaving it off for latency-sensitive applications that demand the highest possible performance.
No integration: EMC's marketing is being very careful about positioning XtremIO for a very narrow use case. According to the article, XtremIO will not integrate with other EMC products nor EMC FAST. Because of this, there is no inherent "one platform" advantage in buying EMC, making it just as easy for existing EMC customers to switch to a new vendor for their all-flash infrastructure.

What comes next?

EMC must protect its base of business from the all-flash onslaught being waged by Nimbus Data and others. XtremIO gives EMC an offering in the all-flash category. That may be satisfactory for now, but I am doubtful that this solution can compete effectively outside of EMC's base of existing customers. With EMC's worldwide storage systems marketshare at 30% according to IDC, that leaves plenty of opportunity for new leadership in next-generation flash-based primary storage.

What EMC's Acquisition of XtremIO Means

May 10, 2012
By Thomas Isakovich, CEO & Founder

Those who know me best say that I am a fairly private person. They can sense thoughts cranking away in my head, but I do not share them until well after they are refined and tangible. But this is a contradiction. One of my first public speaking opportunities was guest-lecturing an undergraduate class on entrepreneurship at Princeton in 1999. I was invited to tell a story – starting my first company in a dorm room at Stanford, building data storage arrays for the multimedia market, and doing so without venture backing at the time. I spoke about the highs – winning that first customer, hand-building their storage array, and biking (yes, biking) it down to the local FedEx station for shipment. And I also shared with them the challenges -- the enormous pressure of building a business from scratch from an idea, recruiting top talent that shares your passion and vision, and expanding sales and marketing without compromising customer service, innovation, or quality.

The talk was raw, but it resonated with the students, all eager to launch their own companies and focus their incredible passion behind their own inventions. Giving that talk was one of the most rewarding opportunities in my life. Even though I had little preparation and little time to refine my thoughts, the professor (long-time data storage veteran Ed Zschau) loved the story so much that he led an investment round that helped launch my company. Sometimes, I learned, free-flowing ideas are the best ones. So in that vein, this marks my first blog ever. I look forward to sharing my views on the solid state storage market and more. Thank you very much for tuning in!

Today the big news of course is EMC's acquisition of stealthy flash memory startup XtremIO.

What does this mean for the networked flash memory market?

It shows that EMC sees a huge flash storage market opportunity ahead. According to IDC, $12 billion is spent on primary storage, and Nimbus believes this will aggressively shift away from traditional 15K rpm disk arrays to flash memory-optimized systems. The acquisition is also clear confirmation that existing storage architectures are poorly suited for all-flash. Repurposing VMAX or VNX for all-flash was a no-go; a fresh approach was needed. EMC so firmly believes this that they were willing to make a $430M acquisition of a pre-product, pre-revenue company – the type of acquisition EMC has never made before.

What does this mean for the PCIe flash memory market?

I believe that the PCIe flash industry's most profitable days are behind it. EMC's actions confirm this theory. PCIe flash surely is not replacing SAN and NAS-attached flash as some vendors have predicted. In fact, flash will reside at the host side (edge) in addition to the SAN and NAS side (core). It’s interesting to note that EMC chose to OEM a PCIe card (from Micron and LSI) for its Lightning VFcache product, but acquire a flash systems company. EMC's actions demonstrate that it believes PCIe cards are commodities that can be sourced, whereas flash systems are strategic assets that must be owned.

What happens to the competitive landscape?

EMC's acquisition demonstrates their commitment to flash and throws down the gauntlet to datacenter rivals NetApp, IBM, Dell, HP, Oracle, HDS, FusionIO, and maybe Cisco to get competitive offerings ready or risk missing another huge market. EMC already dominates other once-emerging markets thanks to its aggressive acquisition strategy, such as backup deduplication (Data Domain) and scale-out NAS (Isilon).

Are there any negatives for EMC on this deal?

Yes, I believe there are risks with this deal. EMC continues to divide and complicate its enterprise storage product line. They now have VMAX, VNX, XtremIO, Isilon, and Atmos, not to mention Lightning and Thunder coming later, all targeted to enterprises and cloud providers. EMC is banking on tiering and EMC-specific management software to tie all these pieces together, but Nimbus believes this is a mistake. EMC is assembling another monolith that will drive up end-user hardware, software, and support costs. The datacenter market is actually trending away from monoliths to virtualized open systems. Virtualization of servers, desktops, and networks are tearing down monolithic offerings and replacing them with best-in-class multi-vendor solutions. As tiering functionality migrates to the hypervisor layer, customers gain more efficiency and flexibility to drive down costs and complexity, while also resisting vendor lock-in. Nimbus supports this vision: less hardware, fewer tiers, less complexity, and less cost.

Are there any positives for competitors?

EMC has shown its cards but cannot yet play its hand since XtremIO is not yet shipping a product. The other datacenter vendors have an opportunity to steal EMC’s thunder (no pun intended).

What does this mean for Nimbus Data?

All goodness. EMC's actions reinforce what we have believed for some time. When it comes to flash memory systems, software is key. Nimbus' HALO operating system continues to lead the market in this respect, providing the widest storage protocol support (iSCSI, NFS, SMB, FC, SRP), the most scalable flash-optimized file system (scalable to 4 exabytes), the most efficient data management with thin-provisioning, reclamation, and selectable deduplication technology, and unified storage. All this is provided in a solution that takes minutes to install and has two years of production deployments under its belt, including customers like DIA, eBay, and Mitsubishi.

What comes next?

It will be interesting to watch for sure. At Nimbus Data, we believe that the flash memory storage market is just getting started. We have a full pipeline of new products coming that will continue to set new standards for storage performance, efficiency, simplicity, and value. Watch this space!