Courts Will Tolerate Search Inaccuracies

Today, courts sometimes ask parties to a suit to justify the relevance and accuracy of their search algorithms. That's a problem:

  • You can't explain the algorithm, because the search vendor doesn't reveal its proprietary secrets.
  • Even if the vendor were to explain its algorithms, chances are a cocktail of approaches are used with varying weightings assigned to different filter criteria. Explanations of such algorithms are of questionable value.

Long term, this will be a less pressing issue. Search engines aren't perfect. They often list material that's irrelevant, and they often miss material that is relevant. There's a trade-off: The less information that's overlooked, the more irrelevant material you'll have to sift through.

However, human beings have similar problems. They are subject to not understanding, or having slept poorly the night before and thus doing a slipshod job.

As time goes by, courts will get more and more comfortable with the fact that search technology isn't perfect--in just the same way as they recognize that humans can make errors.

... David Ferris

2 Trackbacks

  1. […] Recently, I ran that theory by David Ferris, whose firm Ferris Research has long been a/the leading small analyst firm covering e-mail and related technologies. He wasn’t buying. David believes courts are getting more sophisticated in their understanding of search technology. Even more to the point, David cited several other buying motivations that would lead enterprises to want best-available rather than just-good-enough e-discovery search technology, such as: […]

  2. By Quick Bites: E-Discovery and Transparency on September 19, 2008 at 8:01 AM

    […] I just read an article on how courts will tolerate search inaccuracies in e-Discovery by way of Curt Monash. It reminded me of our recent discussion of transparency in information retrieval. I agree that “explanations of [search] algorithms are of questionable value” for convincing a court of the relevance and accuracy of the results. But that’s because those algorithms aren’t sufficiently intuitive for those explanations to be meaningful except in a theoretical sense to an information retreival researcher. […]

Post a comment

You must be logged in to post a comment. To comment, first join our community.