One of the hottest topics in the storage industry today is deduplication. Deduplication is this year’s fancy way to do data compression, something that has been available for decades. If you were a fan of compression in years past, then you will probably be a fan of deduplication. But it’s not all upside.

Whereas most of the older compression formats have become standards (Winzip, for example) and thereby readable and writeable by most programs, all of the deduplication formats are proprietary. This means that the only thing that can read the deduped data is the hardware or software that wrote it in the first place. This, of course, is what all the platform manufacturers want – to lock you into their platform.

Dedupe is smarter than ‘ordinary’ compression, which does allow it to compress more, on average. These results, however, come at a price – processing time. Many of the dedupe solutions that perform brilliantly in demo fail when the volume being compressed grows beyond a few terabytes. Their algorithms become unwieldy and system performance deteriorates.

The bottom line for dedupe is the same as it is for most technologies… Under the right circumstances, it offers a lot of value. But dedupe is not the silver bullet. It is not a solution to every problem, and it may or may not be an answer to the challenges confronting you today.

Monday, July 27, 2009 3:43:06 PM (Eastern Daylight Time, UTC-04:00)      Comments [0]  Observations | Storage Management