“With every sale, Permabit promises to provide an expert witness should any litigation arise around the use of our technology.”  Add it to your sales contract and its golden.

Your whitepaper on compliance would make an interesting read.  How does it measure up to EMC’s paid for whitepaper saying that Centera is compliant.  As previously blogged here, there is a page and a half disclaimer by the company that wrote that one saying that the paper does not replace competent legal counsel and that your experience with the courts may be quite different than what is represented in the paper.

By the way, I agree that RAID 5 alone is not sufficient to protect data in the manner required by many regs.

Happy 4th to you and yours.

[quote] Deduplication has NOT yet been subjected to the acid test of litigation. Nor has lossy compression to my knowledge. [/quote]

Permabit’s systems are used in the field for the archive of critical data, including data subject to the SEC 17a-4 rule. While I don’t know if any specific cases, I’m quite certain data stored on our storage, incorporating deduplication technology, has been subject to litigation discovery. The integrity of the deduplicated data has never been called into question.

Also, deduplication is more along the lines of lossless, not lossy, compression. But, even for lossy compression, audio recorded onto portable devices that record in the MP3 format has definitely been used without problem in court.

[quote] Does Permabit provide any assurances to its customers that legal hassles will not result from deduplicating files? [/quote]

For our customers with SEC 17a-4 requirements, we assist in the process of filing a letter describing the archiving system and technologies; in doing so the customer can receive a “no action” letter indicating that the SEC does not believe they are in violation of the rules. (This is not quite the same as saying they are not in violation of the rules, a matter on which the SEC does not opine.)

In the event of litigation, I would be happy to be an expert witness to explain the integrity of the recording process. We also have an independent report from Compliant Systems Consulting LLC, finding that Permabit Enterprise Archive satisfies regulatory compliance requirements.

[quote] Call it FUD, but I wonder if this is simply just a discussion that you would prefer not to be had at this juncture. I welcome a legal opinion on this matter [/quote]

You could get an attorney’s opinion, if you really value that. You could probably get contradictory ones for that matter. What does matter is if deduplication is even considered an area to try to attack in a legal discovery case, and evidence is that it is not.

There are far more dangerous areas we should be concerned about if we’re looking for legal opinions. For example, should it be acceptable for companies to use RAID 5 to protect important data, given how vulnerable that is to data loss with modern, high-capacity drives?

[quote] If the question exists, and to many of my financial clients it most certainly does, why not get it out in the open and address it? [/quote]

Consider it addressed. Have a great Fourth of July!

Regards,
Jered Floyd
CTO, Permabit Technology Corp.

I am not disagreeing with any of your points from a technical standpoint. I don’t think that storage admins should decide what is important from a business perspective, deleting as they see fit electronic files of the company. But, you have made my point as well. Letting the IT guy field technology like de-dupe and expose all files to it, without a full understanding of the legal ramifications of doing so, is equally foolhardy.

Deduplication has NOT yet been subjected to the acid test of litigation. Nor has lossy compression to my knowledge.

What you claim about dedupe may be 100% true technically. That doesn’t have anything to do with the law, however.

How many people turn to the courts for justice? Their interpretation of fair and just is often quite different from the interpretation applied by the court based on case law and precedents.

Does Permabit provide any assurances to its customers that legal hassles will not result from deduplicating files? Are you offering your services as an expert witness or proffering your corporate attorneys to catch the legal bullet aimed at your customer if the need arises?  Will you pay whatever penalties or fines that may accrue?  Does your warranty or maintenance agreement so state?

Call it FUD, but I wonder if this is simply just a discussion that you would prefer not to be had at this juncture. I welcome a legal opinion on this matter and think that someone in the dedupe business ought to get just such an opinion in writing from a judge or the American Bar Association or somebody authoritative. Until that happens, we are all engaging in speculation that probably goes well beyond our areas of expertise.

If the question exists, and to many of my financial clients it most certainly does, why not get it out in the open and address it?

The point you ought to be focusing on is “trusted operators can manage files and space as required”. This is 100% in violation of rules like SEC 17a-4. A trusted admin can’t be allowed to delete the files, period — retained is retained.

As for whether or not deduplication affects data integrity for things like discovery, claiming it does is FUD plain and simple. As I responded in your dedupe questionnaire:

Dedupe does not change data any more than compression changes data, or traditional file systems change data. Plain old LZW compression gives you a different output bitstream than what went in, with redundant parts removed. Conventional file systems break up files into blocks and scatter those blocks across one or more disks, requiring complicated algorithms to retrieve and return the data. Dedupe is no different. Nonrepudiation requirements are satisfied by the reliability and immutability of the system as a whole, deduplicating or not.

Jered Floyd
CTO, Permabit Technology Corp.

When EMC said that Centera was compliance certified, I asked the same question you are asking. Who certified it? I was told that they simply sent a letter to the SEC describing the product and did not hear back for 30 days (probably because the letter went to a dead letter office and there was no one there who certifies anything). Their contention was that silence equals consent, which is not a valid legal tenet from what I have been told. So, bottom line, it is bullshit.

Now, we have de-duplication providers arguing, in this case, that they have locks to hold data for specified intervals, wrapping themselves in compliance, when it remains to be seen whether de-duplicated files will pass muster with the Man in any case.

We can’t just throw a blanket over the issue and hope it will go away. It is important to repeat that many financials are going to great pains to segregate data from de-dupe that fall under the aegis of regulations. That seems, to some degree, to defeat the purpose of de-dupe altogether.

That ain’t FUD, it’s just what folks are doing.

The plaintiff will throw out all the points they can to try to discredit the storage device and win the case. In the case of the DD, they will use the argument that the technology is based on hashing which has a mathematical probability of hash collision and data corruption. Yes, the likelihood may be small, but it is still there and they will use it.

The more complex the technology gets the more difficult it can be to prove the reliability which is one of the reasons why there may be more more concerns about subfile dedupe as compared to Single Instance Storage or CAS devices.

While many of the plaintiffs points may be misleading or just plain wrong, it will cost substantial time and money to defend the claims. This is why Jon is suggesting that a legal opinion be considered. If IT saves $ by implementing the solution, but opens themselves to these legal challenges, does the purchase make sense? There is no clearcut answer and it depends on the individual company and industry.

To close the loop on the case, if the case is won by the defendant then the court must have acknowledged that the storage system was accurate and the device then has court precedence for its accuracy. This means then that future lawsuits will have a difficult arguing the same point because the court will now have case law supporting the storage solution.

In the end, end users should understand the implication of this technology before purchasing and specifically the legal ramifications. The best place to get feedback on the latter point is someone with legal expertise.

What I’m trying to figure out is WHO gets to say what is or isn’t “compliant”? And WHY?

I’m not all knowing but I’ve never been made aware of some overarching entity that specifically tests and certifies technology for “compliance”. Without this, as I asked above, isn’t everything left up to broad interpretation (or you could call it ambiguity).

Does such an entity exist?

EMC calls CENTERA “compliance certified” — which it obviously is not. I have ranted about this point here, even challenging their paid analyst defense of the product, and have incurred the wrath of Hopkinton on exactly the same grounds as you are now criticizing my attack on de-dupe.

We should at least try to have legal provide an opinion before we deploy stuff that has specific legal ramifications. As long as de-dupe is just being used with backups and an original version of data is hanging around, I couldn’t care less what you put in the de-dupe repository. However, when the vendor represents the solution as a compliance solution, I have to ask about its compliance features, don’t I?

What are the compliance requirements that people should be worried about? And which “regulatory” agency is responsible for testing the technologies and certifiying them? It seems to me that a “requirement” that doesn’t have some “regulated” methodology established for “certifying” acceptable technologies is at best a hollow document that allows vendors to twist and manipulate their interpretation for their purposes and at worse a scare tactic re: LEGAL RAMIFICATIONS!!!

I would guess that without the above in place a good lawyer could just as easily rip to shreds the “requirements document”.

Haven’t these “compliance” issues been what EMC has used for years to convince companies to lock themselves and their data into the Centerra Platform and it’s WORM? Doesn’t this technology offer a “single instancing”? Why hasn’t the issue been raised before now?

Thoughts?