Recently in London I was talking to the CIO of one of the larger Brokerage firms. He noted that they currently have 750 terabytes of storage and that he expects by this time next year to have twice as much 1.5 petabytes.

How do you backup 1 petabyte of storage? The answer is: you don’t. At least not in any conventional way… there is no technology on the market that can write a petabyte of storage to off-line media in any reasonable amount of time. Size changes everything.

Now if you’re a Global 2000 multi-national like this brokerage firm, you may be able to skip backups altogether. (Backups, not archiving) If you have a world-wide distributed database, say in London, New York, and LA, and all of London, New York, and LA have ceased to exist, you probably don’t care very much about those very safe tapes in Iron Mountain… There would be no place to restore them to.

This is a turning point in our industry. Conventional backups must now give way to on-line backups, tiered storage and archiving. Don’t go buying stock in any of the tape companies…

Tuesday, February 14, 2006 4:06:58 PM (Eastern Standard Time, UTC-05:00)      Comments [0]  Observations | Storage Management

We all probably remember the parable about the emperor who had no clothes, riding around naked while his whole kingdom pretended that he was wearing the most beautiful set of robes... Today we have about as much data security as he had clothes. Early last year Mastercard admitted that a backup tape, written in clear text, with about 2 million customer records was lost. Last month (December '05), Marriott revealed that a tape with a couple hundred thousand customer records is missing.

Who among us thinks that these are isolated, once in a life time events? Not me! My organization has lost tapes before. Humans make mistakes. You have to assume anything that gets handled by humans can be lost or damaged. And the more humans that handle it, the more likely it is to be lost or damaged.

It's time for the industry to get it. There is little possibility of real data security until all data is encrypted. Until then, we're all just pretending...

Thursday, January 12, 2006 12:12:44 PM (Eastern Standard Time, UTC-05:00)      Comments [0]  Management | Storage Management

The Consumer Electronics Show in Vegas always provides plenty to think about. One of the more interesting discussions concerned putting hard drives in cell phones.

I remember the day - years ago - when I figured out that my laptop would eventually have a gig of memory. I thought that was pretty wild. Never did I consider that my camera would have a gig of memory. (It does - about 400 pictures worth.) Also never did I consider that my phone might have a hard drive... This changes everything. What would it mean to have - and I mean all - of your data in your cell phone? What will it mean to Corporate America that everyone will be coming to work with the ability to download and transmit pretty much any and all information they can access?

And how are you going to backup your cell phone? The data there won't be mirrored on network servers. What happens if it goes away?

To date the phone / PDA has been a pretty klunky device and not all that useful. This is about to change, and when it does, a lot of other things will change.

Wednesday, January 11, 2006 6:15:31 PM (Eastern Standard Time, UTC-05:00)      Comments [0]  Observations | Storage Management

As technology strategists, it is our job to select the right technologies that will accelerate our companies’ success in their chosen businesses. Often a single poor choice will cause our whole project to fail. Sometimes even whole companies fail over the technologies they chose to invest in. (Where is IBM in the PC and networking business now?)

Why, then, do so many companies use a second rate process over and over again to make their choices? Why does senior management encourage it to happen? It seems to me that the right way to make choices isn’t all that hard to find. And, by the way, this isn’t limited to picking technology; it applies to most decision making.

Let’s look at what generally happens. Most organizations build a matrix of features, recruit a few vendors, fill out the matrix, give a high weight to cost, and make their decision. Sound familiar? How well does this correlate to what most companies state as their mission? How many companies do you know whose mission statement is: “We want to be as good as we can be provided doesn’t cost too much”? Or, “Our commitment is to excellence, unless it’s expensive, in which case we might not do it at all”? Is this the way we really run our businesses? In some cases, I believe that it is.

Where I work, we run the business differently. We use a sequential process designed to produce the best possible solutions (and the best possible business) every time. Here are the steps:

Step 1: Does this problem really need to be solved? There is no lack of things for us to do. If the need isn’t compelling, then we shouldn’t work on it at all. But if we resolve to work on an issue, then it becomes policy. The problem will be addressed.

Step 2: Is this the most important thing for us to do next? Look at the government. The reason things are such a mess is that there is no rational order to the way things get addressed. Time and resources – not money – are our most precious possessions. If you allow them to be spent in the wrong order, you can get lots done and still fail.

Step 3: What is the best possible solution? Here is where everyone needs to bear down. Why is this the best possible solution? What alternatives did you consider? What values are you focusing on when you choose this one? Most people do a weak job here. Management doesn’t want to press too hard (and alienate their staff). Staff members are often reluctant to take clear cut positions and be right or be wrong. In our organization we simply force people to do this. It’s a necessary and required ingredient for success.

Step 4: Can we afford it? Which one of us comes to work intending to be mediocre? Who among us plans to be second rate? If you really have identified the best possible solution, why would you want to do anything else?

Everyone knows how to spend money. If you start with a budget you get a plan to spend it, not a solution, and guaranteed, not the best solution. If the way you handle cost is other than “Can we afford the best possible solution”, then, by design, you have committed to build a mediocre institution. Is this what you really want to do? Would the CEO give you a raise if you told him this was your plan?

World-class organizations are the best at what they do because they engineer mediocrity out of their institution. What does your company want to be?

Wednesday, December 21, 2005 2:21:56 PM (Eastern Standard Time, UTC-05:00)      Comments [1]  Management | Storage Management

Remarkably, as we work with clients we discover that they rarely analyze the component costs of their operations. So let me make this easy: the bulk of the money is in whatever you do for data protection. But even if you know this, have you looked at the details of what you’re protecting and how it flows through you systems? What components make up the bits you write? How often do you write them and why?

Last year we worked with a client who had done this analysis. They discovered that 40% of what they were writing each day was user PST files in home directories, independent of server backups. (This is a financial institution whose corporate policy requires the preservation of email. Each user needs and wants their own record.)

Outlook, as you may know, has the annoying feature of updating your PST file each time you open it whether you have changed its contents or not. This means that if you do daily incremental backups every PST file for every user is written every day. And, if you’re keeping most or all of your emails, and you have lots of users, this is lots of data. Can this be made less burdensome and less expensive?

We worked with the client to set a system-enforced limit on the size of an individual PST file. This means that each user now has a series of smaller files. Outlook only attaches and updates the current PST file; the others are left as they are. Enterprise-wide, this eliminated 100 tapes from a year’s worth of incrementals. 3M is commonly cited as stating that the cost of maintaining writing and maintaining one tape in a typical backup scenario is about $3,500 a year. One hundred tapes times $3,500 is a lot of money – a third of a million dollars a year. For the sake of argument, let’s assume that your operations are less expensive – only $1,500 per tape per year. This is still $150,000 a year.

We would all agree that every user’s archival mail is something we have to protect. But, maybe it’s worth drilling deeper into how you do it…

Wednesday, December 14, 2005 9:54:45 AM (Eastern Standard Time, UTC-05:00)      Comments [0]  Observations | Storage Management

We have all watched the price of on-line storage decrease year after year. While it is still neither infinte nor free (and it's certainly not free to manage), it is pretty inexpensive these days. So inexpensive, in fact, that tape is now dramatically more expensive. This cost inversion (tape used to be much cheaper than on-line storage) should change the way we manage our storage.

If you are a large corporation with sites all around the world, you need to replicate your data to these sites anyway. If London, New York and LA have all ceased to exist, you probably won't care much about the tapes at Iron Mountain... Trust me, your concerns will be elsewhere. As a result, there is really no need to have any of these tapes.It's a costly security blanket that really accomplishes nothing.

A smaller company needs tapes for disaster recovery, (although on-line vendor-based alternatives exist) but daily backups should be on-line. Writing tapes everyday is error-prone and costly. You just shouldn't do it anymore.

Thursday, December 01, 2005 5:16:11 AM (Eastern Standard Time, UTC-05:00)      Comments [0]  Storage Management

Listening to all the hardware vendors, you would think that virtualization is the greatest thing since sliced bread... It is true that it solves a number of problems, but it also comes at a price. For example, many organizations maintain historical backups. Suppose I use the facilites of my virtual storage to do some reoganizing. How do I then trace the files that I have - in their new locations - back to their ealier backups? You can't. At least you can't without NTP Software's technology. Does it matter? It matters to a lot of our customers. In fact, we work with clients who refuse to reorganize their storage for just this reason. The bottom line? Everything comes at a price...

Thursday, December 01, 2005 5:03:55 AM (Eastern Standard Time, UTC-05:00)      Comments [0]  Storage Management

I'm just back from watching the Red Sox get spanked by the Yankees... Normally I don't rant, but today I have to. How many times do they have to play out the same script. Pedro starts coming apart. They go out to the mound. Pedro says something that must translate to: “I'm fine, I can handle it.” Everyone leaves; Pedro does worse; the Sox lose the game... It cost us a Series, a pitching coach, and yesterday's game.

What's that old definition of insanity: do something once and get a result. Do it again and get the same result. Do it a third time and nothing changes. Do it again, expecting a different result. That's insane. So is asking Pedro whether or not he should come out. We know how this works - every time!

I see the same behavior in business all the time. People - managers - want to believe the promise rather than deal with the reality. Don't by the promise! Deal based on what happens, not on what gets said. 

Monday, September 20, 2004 11:15:29 AM (Eastern Daylight Time, UTC-04:00)      Comments [0]  Management

In case you haven't noticed, the operating system wars are pretty much over. Going forward, there will be Windows and UNIX. This means the interesting play has moved somewhere else - to the man-machine interface.

Having recognized this, last week, IBM put some of its speech technology out into the open source marketplace. This counters Microsoft's Speech Server, which up until now has been the only real pure infrastructure play in the space.

I've been a Dragon user for a few months now, and the technology is way beyond where most people think it is. The paradigm for speech interaction with your computer still needs some work, but the underlying technology probably recognizes what i'm saying better than most humans.

With digital photography it is already possible to create man-made images that cannot be distinguished from naturally occuring scenes. Pretty soon we will be able to create man-made speech that's indistinguishable as well. When that happens, it's a whole new world... Even if you don't call your bank on the phone, voice I/O is in your future. Trust me.

Monday, September 13, 2004 10:24:22 PM (Eastern Daylight Time, UTC-04:00)      Comments [0]  News

Recently I have been reading articles on Apple vs. MSN and the downloadable music wars. More interesting (at least to me) than who will win this battle is what it all means. In the last 25-30 years (one generation), we have gone from phonograph records to half inch square memory chips (which now, by the way, come in gigabyte sizes). Let's look at the trends here:

Music goes from tape, to optical (CD), to bits. Does anyone doubt that video will go down the same path?

Displays go from CRTs at 640x480 to flat screens at 1280x1024.

Why is a music DVD more appealing than it's companion CD? Part of it is the video, but the DVD also has 3-4 times the audio density. The music sounds better because it is better.

Ok. So what does this all mean?

If it were me, I wouldn't buy Blockbuster stock or glass company stock (although Corning is a player in flat screen technology as well). I figure that in the next 2 years I'll have all my music as bits and the CDs will be in the bottom of some closet. How long before my movies get there?

Well... Hitachi now sells a 400 gig hard drive. That's about 100 movies, which is more than I have. But the key for movies is still bandwidth. We are now at the point (just this year) where more than half of all the homes that have Internet connectivity have high-speed Internet. The problem with Movies is that we don't yet have the right software and appliances (something we're working on a bit...). Give it another couple of years.

Oh, by the way, did you know you can produce a pretty good display using laser interference patterns... something to think about!

Friday, September 10, 2004 3:41:45 PM (Eastern Daylight Time, UTC-04:00)      Comments [0]  Observations