File Virtualization … remember then?
Back in 2006/2007, file (NAS) virtualization was the hottest buzz world in the storage world. Remember the giant stack of “File Virtualization for Dummies” book that Acopia gave away at all the IT events ?
At the peak of the hype cycle, a good number of product were competing for customers, but very few survived.
- NuView StorageX (acquired by Brocade, OEM’ed to NetApp and IBM, EOL’ed this year)
- Acopia ARX (acquired by F5)
- Neopath File Director (Acquired by Cisco, EOL’ed soon after)
- Rainfinity(?) (Acquired by EMC, sales force seems to have stopped pushing it, hinting the pending FAST for Celerra release)
Out of the above 4 major players, 1 survived, (1.5 if counting Rainfinity)
What happened ?
NAS virtualization solves the scalability problem of NAS devices and provides an easy way to migrate files and/or volumes between dissimilar NAS devices. Most NAS devices back then were simple X86 file servers running code optimized for file serving. Although highly tuned, most of these NAS devices only scaled up to a active/passive fail-over cluster for HA, which improves the availability, but does nothing for scalability. The 32bit filesystems used in these NAS devices limited the volume size to ~ 16TB and can only hold less than a hundred K’s of inodes (file/directories). While the limitation is adequate for most organizations, it is a nightmare for those with large data sets (ie. HPC), or large number of small files (ie. Web based image serving sites). NAS virtualization broke the limitation by creating virtual volumes that span multiple NAS systems, effectively raising the capacity/inode ceiling.
The migration service lets customers move volumes around NAS devices from same or different vendors. This alleviates the pain of running long robocopy or rsync sessions and reduces the cutover window. Some of the migration services also comes with policy based file movements that moves files around automatically based on user defined terms (such as file age, size, last access time, file type) between faster ($$) tier and slower ($) tier.
(The net result is a relatively easy way to implement something like looks like distributed filesystem of the early late 90s … think IBM DCE/DFS, AFS and HPSS, minus the tape part and all the nerd quotient mandatory for implementation/support)
NAS virtualization hype seems to have completely deflated, considering that major NAS players are now selling devices with either built-in single name-space capability (NetApp), online migration mechanism (EMC, NetApp) and massive scalability (NetApp, Pillar). The only buyers of NAS virtualization would be organizations that have not standardize their NAS on a single platform and wants to move files around easily – A pretty small crowd.
However, NAS virtualization still has a chance, and this life line is Open Source – Cloud Storage.
Similar to the way SAN virtualization controllers (IBM SVC, HDS USP-V) improves the performance of the physical arrays by adding more cache and spreading out volumes across more spindles, NAS virtaulization device can create high performance virtual NAS devices by distributing a virtual volume onto a number of inexpensive Open Source file servers. The NAS virtualization in this sense then provides replica service to allow users to access their files from any of the replicated filesets that is either less busy or closer to the user. Since the filesets are replicated can also exist in multiple sites, all the data is always available – instant BC/DR function. Imagine – an easy and inexpensive way to build your own gfs (google, not RH’s). Slapping in a Amazon S3 or EMC Atmos interface, you now have both internal and public cloud all in a nifty device.
Maybe it’s time for a major linux vendor to venture into storage business ?
Comments
I ghost authored the intro to the dummies book – have distributed about 30,000 of them – just ordered another 10,000 – so much for deflation. So I speak as a bit of an expert on this subject.
The fact that some vendors solutions failed or were acquired is not indicative of issues with the solution. If that was true the acquistion of Arrowpoint by Cisco, and the failure of Nortel in the L4-7 business would mean the ADC market was doomed – just dumb.
The reason other vendor’s attempts at file virtualization failed is because the approach they took was flawed. Instead of building a solution appropriate for the real estate this solution occupies, they cobbled together PC software that was neither effective or resilient. Their solutions were disruptive to users, presented steep performance and availability risks, and simply did not provide business benefit.
ARX (ex-Acopia) was built from the ground up to live in the data path – like most any effective storage networking component. It has a range of uses – from consolidating name spaces across diseperate vendors and protocols (which is minor frankly)to fully automated and transparent storage tiering – between same or competing storage vendor (which is huge).
The business benefits of tiering are by now so completely ratified that I am not going to expound upon them again here, but urge you and your readers to explore the 3rd party validated responses of actual customers at http://www.techvalidate.com/product-research/f5-arx
Also note the partnership announced last week with Netapp, and similar cooperation with DataDomain and many other storage vendors. Clearly the industry and customers see value in File Virtualization.
And yes – acting as a gateway to cloud storage is a very, very good use for file virtualization – stay tuned
AceSage
January 26th, 2010
Tiering is good, but to make a huge difference, moving between FC and SATA is not enough. Seeing a good number of my clients going toward SATA-only NAS, the only way to make NAS tiering work would be allowing data to move between FC and SATA and Cloud. It’s good to hear about the upcoming Acopia cloud gateway. it’s a lonely world to be the only viable vendor in a market, I hope you guys do well.
(ps. please figure out a way to do global quota control. If I have to give up quota control (or having to do it form the backend per physical volume), my user’s growth of mp3 collection will soon eclipse any meaningful saving from tiering.)
(pps. what I am hoping for here is more than a mere cloud gateway. the file virtualization technology itself has a great potential as a cloud building device.)
tlin
January 26th, 2010