Our client—a large energy company—was involved in litigation over an environmental matter. Only one lawyer was assigned to the case, overseen by one independent consultant—an especially small team, given the 778 GB of data collected for the case. In an effort to save costs, both sides agreed on a list of keywords to help identify relevant documents.
De-duplication culled the client's original count of 6.6 million documents down to 3 million documents. The list of keywords culled the document count significantly further, down to only 157,000 documents. However, with overly inclusive keywords like "environment" being used, it was evident there were still many non-responsive documents left in the collection to sort out—and that meant a lot of work remained for only one person to review.
Law In Order proposed using Relativity Assisted Review.
Murali Baddula, Head of eDiscovery at Law In Order, oversees the data processing services and assisting legal departments by offering technical solutions and workflows. Murali has over 14 years experience in the areas of dispute resolution, e-discovery, project management, and software development. He understands the complexities of unstructured data and offers solutions to make electronically stored information available for search, retrieval and review in a legally defensible manner.
"I am a great advocate of Assisted Review. We always put this option forward for any matter that has a high volume of documents," said Murali. "For this matter, there were a number of reasons it was a great option for our client."
Most notably, Law In Order calculated that if the one lawyer had to review all 157,000 documents linearly— strictly to identify relevant documents—the review would take 1,570 hours to complete. With the lawyer open to alternative processes, the decision was made. The team would use Technology Assisted Review.
The Technology Assisted Review Process
To begin the workflow, Our team used the overly inclusive keyword-responsive document set of 157,000 documents.
"We generally don't use keywords in addition to Technology Assisted Review, but in this scenario, it actually fit the bill. It's important to have a decent amount of both non-relevant and relevant documents to train the software—and after applying the keywords, it still looked like we were left with a good mix of relevant and non-relevant documents," said Murali.
The lawyer completed three review rounds—reviewing a random sample of 1,000 documents during each round—to train the software on responsive versus non-responsive. For each round, six to seven percent of the documents were identified as responsive.
From there, the lawyer tested Relativity's accuracy by conducting a QC round. For this fourth round, Murali and his team took a statistical sample of documents based on a 95 percent confidence level and a two percent margin of error. This resulted in a sample of 2,226 documents—and the results were consistent, as six to seven percent of the documents in the sample were coded as responsive.
In addition to the consistent results, the lawyer overturned only 174 of the 2,226 documents in the QC round. In other words, he disagreed with the software's coding decisions only 7.8 percent of the time, further demonstrating the computer's accuracy. Moreover, he made an interesting discovery while reviewing the overturns.
"As he double-checked the overturns to see what led the computer to make an incorrect decision, he actually ended up agreeing with the software the majority of the time," said Murali. "He realized his initial decisions were wrong."
With this realization and the already low overturn rate in mind, the lawyer felt the computer had achieved a consistent level of accuracy for this case—even after just one QC round—and decided to stop review.
Using the logic it learned from reviewing nearly 5,000 documents—three training rounds and one QC round— Relativity categorized the remaining 152,000 documents. In the end, 27,122 documents were marked responsive, 117,635 were non-responsive, and 300 remained uncategorised.
"Because of the limited resources, the legal team had very little time to complete this review. But, they were able to get good results quickly using Assisted Review," said Murali.
Achieving the Results
Relativity Assisted Review saved our client a considerable amount of money. Without this, the client would have likely hired a junior lawyer—generally billing at $200 AUD/hour—to manually review all 157,000 keyword-responsive documents. Assuming the junior lawyer reviewed strictly for relevance at a rate of 100 documents per hour, it would have taken 1,570 hours to complete the first-pass review and would have cost the client $314,000 AUD.
However, by using Relativity Assisted Review, the client's senior lawyer was able to complete the first-pass review at an estimated cost of $29,000—a savings of approximately $285,000 AUD for our client.
"This matter is a great example of the efficiency and cost savings realised by the use of Assisted Review," said Murali. "Our client was very pleased with the results."
