https://slotsdad.com/ - casino online slots

Your Next NAS Should Be A Tape Library or Optical Juke

by Administrator on July 2, 2008

GCN just reported on a presentation at USENIX in which NetApp and University of California researchers showed the preliminary results of an inspection of data access patterns on NAS gear.  Their key finding from 90 days of examination: nobody is referencing the old files stored on their NAS heads.

I’m not making this up.  From the piece:

During the three-month period that the network was under scrutiny, more than 90 percent of the material on the servers was never accessed. The researchers captured packets encoded using the Common Internet File System protocol, which Microsoft Windows applications use to save data via a network. About 1.5T of data was transferred.

“Compared to the full amount of allocated storage on the file servers, this represents only 10 percent of data,” Leung said. “[This] means that 90 percent of the data is untouched during this three-month period.”

Moreover, among the files that were opened, 65 percent were only opened once. And most of the rest were opened five or fewer times, though about a dozen files were open 100,000 times or more.

“What this suggests, in general, is that files are infrequently re-accessed,” Leung said.

The team also observed that the ratio of data being read from storage versus the amount of data written to storage had changed from what had been seen in previous studies. Bytes written compared to bytes read by a ratio of about 2-1. “Past read-write ratios saw read-to-write ratios of 4-1 or higher,” Leung added.  

So, why buy NAS with disk at all?  Why not spin the data out to greener media like optical or even tape?

Think about the configuration of the NetApp Filer (or any other high end NAS head):  you are already losing space to RAID, snapshots, OS overhead, value-add software, etc.  On the remaining space, you are storing bits that aren’t re-referenced at all frequently.  It’s like paying for the high end sports car sitting in your driveway: you never drive it, but the payments are roughly equal to your home mortgage and you spend additional money annually on insurance, tag renewal, etc.  For what?  The privilege of showing off to neighbors, the occasional (fe)male, car thieves, etc.?

Let the warfare begin.

{ 4 comments… read them below or add one }

Administrator July 2, 2008 at 9:19 am

If NetApp really wanted to do us a favor, they should put some software on box that identifies all files stored there with zero access and offer to send them to tape using NDMP or some other modality automatically. This research seems to show a willingness of the company to self immolate their value proposition generally. So, why not go with the flow?

zax218 July 2, 2008 at 9:57 am

This concept is on the minds of most enterprise shops, however the secret sauce is missing (at least AFAIK). The products in which I am familiar with are file system based and somewhat complicated to manage (SAMFS, Stornext). Each also has some architectural challenges (issues). Proprietary file system drivers, potential metadata corruption issues, performance overhead for small files…etc

Migrating files VIA NDMP doesn’t really cut it for me. The number of restore actions which will need to occur is high and will increase operational resources (users want their data now, not after logging a ticket to request a restore). I realize this data might not have been used for 3 months (or whatever rule you set up for migration) however that usage pattern is not predictable to assume it will not get used tomorrow. Also add to the fact that many users and applications need search/indexing capabilities and you complicate the issue even more. Enterprise vault type applications do some of this with expensive proprietary interfaces for accessing and searching data, but that won’t work for file based application data (digital assets, document imaging..etc).

EMC, HP, and others can serve up files from different Tiers of NAS through filers, however it is still spinning disk. I have yet to see a vendor simplify the process of migrating files to different tiers which include tape without impacting application and user access. This type of solution should be unknown to the user and application which uses the data, except for the possible variation in retrieval times.

I am curious to understand your thoughts on this issue. In this space, I believe, eventually F5 could add NAS like optical devices to the list of supported hardware and perform seamless file migration within their global namespace. What other vendors do you see with this capability in the near future.

Administrator July 10, 2008 at 10:28 am

You are reading my mind, Zax. I am off to Boston in a week or so to meet with F5/Acopia and Tervela (a start-up there) to address just the issues you are describing. I hope to integrate their ideas around data load balancing into my C-4 Summit, together with other notions of service hosting presented by companies like Crossroads Systems, Caringo, Gear6, and others.

The time has come to think about building infrastructure based on what business requirements and application requirements we can identify. The application folks are already dropping WC3 standard hooks to request services and resources. The network guys are beginning to use these hooks to route data. Some storage guys, notably Xiotech with ISE, are enabling their wares with WC3 hooks to expose the resources and services they can provide and to facilitate end-to-end policy based data routing.

This bodes well for driving cost out of infrastructure, providing a coherent management story, and refocusing IT efforts on data management rather than stovepipe platform management.

blargh July 13, 2008 at 5:27 pm

The annoying thing is that this problem was solved years ago by platforms like SGI’s Data Migration Facility (DMF).

http://www.sgi.com/products/storage/software/dmf.html

If you combine DMF with a suitable front-end, then it becomes completely transparent to the users.

Too bad SGI isn’t able to sell water in the desert.

Previous post:

Next post: