See this chart?
If you are a DD reader, you might want to get used to seeing it. (I welcome feedback on what I have designed, by the way.) I call it a sitrep, or situation report, showing some operational and managerial aspects of the typical enterprise IT shop. I hope to do a series of posts, supplemented by articles in the trade press, chapters in my blook projects, and occasional whitepapers, that deep dive into the dynamics of the various objects that are illustrated on the diagram to bring a bit more order and context to what have become a hodgepodge of posts to this blog in the second half of the year.
The dialog with Mike Linett over at Zerowait, posted previously, got me thinking about this. As I survey the landscape of IT, we are doing a lot of things at each layer of the operational environment — from a technology standpoint and from a human standpoint — that require us to engage in a more serious examination of strategy and management than perhaps is the case in many firms today.
Put another way, the challenges we face in 2010 and beyond have little to do with the tactical changes in hardware feature/function sets of specific platforms anymore. They have to do with a broader context, business impact if you will. Effective planning and decision-making must go hand in hand with the harnessing of technology in order to effect any positive business outcome from IT investments.
Clouds are not a silver bullet. VMware is not a strategy. Thin provisioning encourages waste and overspending on commodity gear. I have made these statements in the past, earning the ire of some vendors and the caustic response of fellow bloggers and pundits for being such a luddite. I will continue poking fun where deserved and dabbling in sarcasm to make a point here and there, but for 2010, I believe this blog needs to encourage intelligent debate on bigger issues. Self-servingly, this will also help me explore ideas and get feedback that will shape my blook projects around IT strategy, disaster recovery planning and the data democracy.
You are invited to participate and to offer views that are counter to mine. I would like to publish many dialogs like the one with Mike. Provided you avoid ad hominem attacks, I welcome your input.
Good luck to us all in the new decade.
{ 2 comments… read them below or add one }
Hi Jon,
You asked for it, so here is my take on storage matters in 2010 and beyond. By way of introduction, I’m the guy that writes one of the other columns in the Dutch Storage Magazine. My day job is consulting of the in dependent kind: I don’t sell stuff, I don’t tell people what to buy, but I do tell them to find out what they really need. RTO en RPO, IOPS and TB’s. Often they only know they need more of the last one. I do agree with lots of what you write and then I don’t with some. Shall we concentrate on the latter?
The “80% is junk but I don’t know which 80%” problem. A major newspaper publisher had trouble containing his email storage. Users had 20 MB mailboxes and 20 MB personal storage. While this is many years ago, it was small even then. Their storage was mainframe class, mainframe priced and they thought they couldn’t afford more. Every Friday users would sift through their mailboxes, pondering which messages to delete and what to retain. So we tried to calculate how much MB a user can delete in an hour. Multiply by 40 weeks and 3 years and you see what the price is of saving that much space. Well, mainframe storage was very expensive, but saving mainframe space this way proved to be thousand times more expensive. You simply can’t make a business case for user-powered data management if it must be done at the level of individual messages or files. Only if you can delete stuff with broad strokes, like “everything older than 7 year”, you can save some money. The problem is “everything older than 7 years” is only a tiny fraction of today’s’ storage, given our exponential growth rate (less than 1% at 50% growth).
Is it hopeless? Rather, but not entirely. Firstly, the problem is of course the unstructured data. Databases are easy, most are smallish and hardly grow. Today your best bet is to build database servers with internal SSD storage. Great performance (IOPS and latency), affordable and if you worry about the endurance of SSD’s, even if they would wear out in 12 months (they don’t), you will buy bigger, faster replacements for half the money or less. The hot tables go on the SSDs, logs and dumps to the SAN. Meanwhile, this takes a bite out of your SAN IOPS requirements, moving away from hot 15k disk to cool 7200 rpm 2TB drives. It beats SSD-based LUNs.
So, what with the unstructured data? It grows and grows. In fact, I suspect that it grows as fast as technology and economics allow. Here is an analogy: when Internet was all new, I often wondered why, with all the exponential growth in users and exponential build out in capacity, the net always seemed to keep its balance. Keeping two unrelated exponentials in sync is a small miracle, so there had to be a mechanism that made it happen. My explanation was that the net always is as overloaded as the users (grudgingly) accept. If the users get ahead of the infrastructure, their waiting time increases and in response they click a bit less, think a fraction of a second longer whether to download that song, etc. Then, if the network (and servers, etc.) are beefed up, the users experience better response and become a bit more click-happy. So we have a force (user behavior) that drives the system to an equilibrium state – unless the King of Pop dies of course.
Could it be that a similar mechanism is at work in our storage systems? Here too we have exponential growth of supply and demand. If so, storage administrators can’t win. The more you give, the more they take; always complaining that it is not quit enough. If like many admins you are trying to please the users, you might find this a depressing thought, but hey, relax; there is nothing you can do about it!
There may be some relief in document management systems – enterprise management systems as the mot du jour is. I have seen users struggle and fight these systems. One of the underlying problems seems to be that if you share a space, whether it is a file folder or a physical desk, when other users add or reorder things, we get the feeling that we are no longer in control. Did I read that already or is it new? Was that pile really in this order? I thought that there was a reddish thing on top that I should take care of first, where is it gone now? So a better DMS should give each user a private view of the shared stuff, with a virtual ‘inbox’ for stuff that others have added. These are of course just pointers, no copies. But wait, if my traditional file system has real-time de-duplication (ZFS, anyone?), regular file shares offer the same storage efficiency without the DMS hoopla. (Ok, there’s more to DMSses)
Since, *not* by accident, storage capacity and price tracks user demand, the real problem is not storage but backup/restore. It takes most companies up to a week to restore everything, partly because writing small files is a pain but mainly because capacity has been growing a lot faster than transfer speed for two decades. The real challenge in storage systems is in data continuity design. Most of the existing backup applications are just horrible, unreliable, complex, expensive, unsafe at any speed. Storage-based snapshots are not the answer, at least as long as there is no solid, open standard for interfacing with systems and applications. Unfortunately, such standardization is not on the horizon.
Jon, on to your “sitrep”. Nice picture, but I’m afraid it describes a model that will disappear over the next couple of years. To explain why, I need to go back to the early nineties. At least here in Holland, the dominant word processor was WordPerfect, we did our spreadsheets in Supercalc or Lotus 123 and there were various programs to draw our overhead sheets (this of course predates affordable beamers). Then things started to heat up. I’m not sure who fired the first shot, but suddenly the goal posts had been moved and the battle was no longer about individual applications, but about “office suites”. Initially this was little more than putting several programs in a single cardboard box, later came a measure of functional integration. The single-program companies dropped like flies and Microsoft soon was the winner that took all of the market (to be challenged only two decades later by an open source alternative).
Back to storage, or rather: data center infrastructure. With HP buying Compaq, with the remains of Digital and Tandem in its belly, competitors saw a new IBM in the making and rallied to beat it to the market with an “infrastructure suite”. This is not about technology: IBM, HP and others have been talking about On Demand computing, etc. since the start of this century, but the technology wasn’t there. It helps that server virtualization is maturing (a king and at least two contenders), but nobody has really solved the technology puzzle. The pitch is very well received in the board room: you buy our integrated, totally reliable, totally flexible server/network/storage black box and let go most of your architects, engineers and admins. There’s a webpage with two buttons: “more” and “less” and the rest is automatic. You can even use our data centers as overflow for peaks in your demand. Just sign on the dotted line, no need to ask the techies, they are against it for their own reasons.
Ok, I’m a techy, so I’m against it, but that won’t stop HP, EMC-Cisco-VMware, IBM and Dell from tying to make the kill. Software vendors will offer a VM-image with application and OS included and will propose to maintain the OS as part of their service contract. They will point out that they test the OS patches thoroughly and efficiently for all their customers at once. Sounds good!
The result will be a new lock-in, erasing two decades of struggle by the market to obtain open systems, standard interfaces and protocols. Remember the days that every computer manufacturer had their own processor, OS, network protocol, printers and terminals down to cables and connectors? When they own the whole hardware stack, the vendors will continue to use the ill-standardized FC out of shear inertia – they customer pays the bill. EMC will continue to sell their “open cookie jars”, systems with excess disk drives that you pay when you start to use them (doesn’t their success almost prove my theory of entwined demand and supply?). Again, it sounds great, but economically it is absurd for a commodity that gets cheaper 50% each year. The CFOs that sign these POs should know better.
We have a whole new battle ahead of us that will keep us busy for the next decade. Seven days to chill before it starts. Cheers!
Wow! A lot to think about and I really appreciate the effort that went into the post.
I am not entirely sure that we are at odds, though. The only difference is that I will not acquiesce to what you describe as inevitable. You might well be right and I may just be tilting at windmills, but hey I liked Mr. Smith Goes to Washington and I believe that we should be striving toward an open and standards-based approach to doing things. Lost cause or not, it is a battle worth joining.
You must log in to post a comment.