The Challenge
Our client received a tranche of disclosure from another party totaling around 55,000 documents. Their aim was to review all 55,000 documents, but to prioritise the review to find documents similar in nature to their own key documents first.
The Solution
Relativity has various tools to identify similar documents. Documents can be grouped by near textual duplicates, clustered by concepts or fed into a Technology Assisted Review (TAR) workflow.
In this matter where the client wanted to review all documents, but prioritise the order of review, the best solution was to deploy Relativity’s active learning workflow.
Active learning learns from decisions made on documents as they are coded in the review workflow. It uses these decisions to continuously deliver documents to a review queue based on what the software believes to be the next most similar document to those already coded as relevant.
As the review progresses, the review queue is continually updated based on the decisions being made.
In this matter we were able to take the documents that the client had already identified as key documents in their own disclosure and use these as a pre-coded set to train the system and kick start the active learning review queue.
The Outcome
The active learning project showed its value from the start.
Using only a very small seed set of around 260 key documents, the project returned a mean relevance rate of 45% for the first 2,000 documents. By the second 2,000 documents, the mean relevance rate had fallen to 24% and by the third set of 2,000 documents the mean rate had fallen to just 12%. In overall terms, the mean relevance rate fell from an initial peak of 63% (for the first 200 documents) to 6.8% by the end of the project, with the last 20,000 documents having a mean rate of just 1.54%.
The below graph illustrates how using active learning enabled the review team to review the most relevant documents sooner.
