What is Technology Assisted Review and how should it be used in modern discovery?

What is AI and how can it be used in discovery?

Artificial Intelligence (AI) refers to the concept of the simulation of human intelligence through machine processes, and in particular, computer systems.

Over the past 15 years, the legal profession has embraced electronic discovery tools to assist with the process of discovery (often referred to as e-Discovery).

Development of e-Discovery tools have resulted in traditional discovery processes becoming increasingly automated through the use of computer systems. e-Discovery processes are multifaceted and include, amongst others, de-duping documents, running searches across inboxes, email threading and predictive coding, also known as Technology Assisted Review (TAR), which is the focus of this article.

In large litigation, the pool of potentially discoverable documents can sometimes be very significant (in the hundreds of thousands of documents or more). The need to review pools of this size manually (ie, by humans) can be very time consuming and costly. To tackle this issue, one option is to use TAR.

In general terms, TAR involves the use of a software platform to conduct an automated review of a large set of documents for the purpose of determining whether each document is discoverable. To achieve this, the program needs to be trained. The training process takes place by a person familiar with the scope of discovery (such as a lawyer) reviewing a sample set of documents and deciding whether or not each of the documents in that sample is relevant to the discovery. These decisions are then extrapolated by TAR across the dataset to identify documents which are relevant. The program may then present subsequent datasets that may be reviewed by the lawyer to further refine (that is, to "teach") the process of extrapolation. The results produced by the software are also validated through quality assurance processes such as statistical sampling, to ensure that the model is working effectively.

But what does the Court think about AI in discovery?

Following in the footsteps of international jurisdictions, Australian courts are now receptive to parties using AI in discovery, provided that it is justified in the circumstances of the proceedings.

The first decision to comprehensively consider the use of TAR in discovery was the United States District Court for the Southern District of New York case of Da Silva Moore v Publicis Groupe 11-civ-1279 (ALC) (AJP), US Dist LEXIS 23350 (S.D.N.Y. Feb 24, 2012). Here, the Court recognised at 28-9 that “statistics clearly show that computerized searches are at least as accurate, if not more so, than manual review”, and ultimately found that TAR was "an acceptable way to search for relevant [electronically stored information] in appropriate cases". The High Court of the United Kingdom soon followed suit by approving the use of TAR in certain circumstances for the first time in Pyrrho Investments Ltd v MWB Property Ltd [2016] EWHC 256 (Ch).

The use of TAR in discovery was first acknowledged in Australia by the Federal Court in Money Max Int Pty Ltd v QBE Insurance Group Ltd Unreported, Federal Court of Australia, Murphy J, 7 Nov 2016 (VID 513/2015), where Justice Murphy ordered the applicant to provide a report that outlined the manner in which TAR was used to give discovery. A month following this decision, the Victorian Supreme Court in McConnell Dowell Constructors (Aust) Pty Ltd v Santam Ltd (No 1) (2016) 51 VR 421 expressly endorsed the use of TAR in completing a large discovery. Central to the Court's reasoning was that the use of TAR in this case:

1. fell within the overarching purpose of the Civil Procedure Act 2010 (Vic), namely it assisted with the just, efficient, timely, and cost-effective resolution; and
2. was proportionate to the cost of the proceedings.

Since those decisions, the Victorian and Federal Courts have issued practice notes which endorse the use of TAR in large proceedings. Whilst other courts have not issued a practice direction with respect to the use of TAR, it is likely that they will adopt a similar position to the Victorian and Federal Courts.

When should a party use TAR?

Benefits

1. Cost: If a party has a vast document pool to review, TAR can help to reduce the cost associated with discovery. While there can still be very significant costs involved in the use of TAR, including costs associated with lawyers reviewing documents to train the model, those can possibly be less than the costs associated with having a number of lawyers and paralegals review the documents.
2. Time: TAR can be quicker than manual review. For matters with tight discovery deadlines and a large pool of documents, TAR can minimise the time taken to review those documents.
3. Developing technology: TAR, like any AI technology, is constantly evolving and is only likely to become more accurate and faster with time.

Difficulties

1. Misses the nuances: One of the difficulties with using TAR is that it determines whether a document is relevant, privileged or confidential, based on an algorithm. Although that algorithm is trained by lawyers with familiarity of confidentiality, privilege and the issues in the matter, by relying solely on an algorithm, TAR can miss the nuances and context of documents.
2. Training of the model: As noted above, TAR is trained in order to know what documents are relevant, confidential or privileged. One of the difficulties associated with large reviews is that the training set is often large and not realistically able to be reviewed by one person. The difficulty with using multiple reviewers to train the model is that it creates variation in the coding and so it may prove harder to validate. If validation cannot be readily achieved, there is a risk that the Court orders that the whole process (or part thereof) be re-done, which occurred in the United Kingdom High Court decision of Triumph Controls UK Ltd v Primus International Holding Co [2018] EWHC 176.
3. Not suited for all matters: TAR will generally only be suited to discovery process involving a vast number of documents given the costs and time associated with setting up and training the model, and undertaking the quality assurance tests may be more than, or similar to, the costs associated with a manual review process. TAR is also unlikely to be suited for discoveries with a number of hard copy documents which cannot reliably be made text searchable or where there are a number of complex and nuanced categories.
4. Impacts your legal team's familiarity with the documents: During a manual review, often you will have members of your legal team closely conduct a "second level review" of the documents to confirm relevance and check for privilege and confidentiality. That process allows members of the legal team to become familiar with documents which are to be produced to the other side. That helps with future steps in the proceedings, including the preparation of evidence, and can be more cost efficient in the long run.