Asia (Other)

Screenshot 2020-09-08 at 5.04.30 PMKeyword searches have been the norm for identifying potentially relevant documents for legal review. With the advances in Technology Assisted Review (TAR), it raises questions as to whether keywords still have a place in eDiscovery. Justice Peck said:

“In too many cases……the way lawyers choose keywords is the equivalent of the child’s game of “Go Fish”….  keyword searches usually are not very effective”.1

Justice Peck referenced an article by David L. Blair & M. E. Maron2 studying the effectiveness of experienced lawyers retrieving relevant documents using keywords and other review techniques. The finding was, on average, recall using these methods was only 20 percent.

This finding was supported by Jason R Baron, US National Archives Director, who discovered as much as 78 percent of relevant documents may be left behind if only Boolean searches are used.3

Keyword searches have certainly developed since Blair & Maron’s 1985 article. Most review tools now include methods for improving efficiency including dictionaries, keyword expansion and fuzzy searches.

However, even using these advanced methods, in most cases keywords will still not find all potentially relevant documents, but only a sample of them.

The Alternatives

The EDRM guidelines define TAR as “a review process in which humans work with software to train it to identify relevant documents”. Whilst many TAR reviews use sets already culled using keywords, nowadays TAR runs on larger data sets to improve recall and precision in finding relevant data.

Hypothetically, in a 100,000 document set in which 10,000 documents were relevant, if keyword searches were applied using the 20 percent recall rate, we could assume that 2,000 documents would be returned for review. If improvements in keyword searches advances this 100-fold, we would return 40 percent totaling 4,000 documents.

A well-run TAR may see around 12,000 to 15,000 documents reviewed. Initially, it looks like a bad result due to the time and cost to review an additional 6,000+ documents. However, even assuming that the TAR workflow only finds 95 percent of relevant documents, the results are much more accurate and defensible as against only finding 40 percent of relevant documents. To reach the same level of recall in a traditional linear review would statistically require review of at least 95,000 of the 100,000 documents.

These hypothetical numbers are supported by John Tredennick & Andrew Bye4 who reported the use of keywords only located 39 percent of potentially responsive documents in their matter.

The goal is finding all relevant documents within a proportionate time and cost boundary.

Keywords appear a quick way of culling data, but are based on the current understanding of a matter. If keywords are built on imperfect knowledge, the results will be imperfect.

Judge Lois Bloom described Abbott Laboratories, et al. v. Adelphia Supply USA, et al.,5 as “…a cautionary tale about how not to conduct discovery in federal court.” In addition to other discovery failings that led to a ruling against the Defendant, Judge Bloom emphasised the Plaintiff’s argument that the “…defendants purposely designed and ran the ‘extremely limited search’ which they knew would fail to capture responsive documents.”

If using Keywords, ensure they are carefully designed and tested so that their effectiveness can be defended.


Since the decision in Da Silva Moore, TAR has exploded. In a recent report by Research and Markets6, it was predicted that TAR spend would grow from $3.2b in 2019 to $37.8b in 2026.

In a recent US case Nuvasive, Inc. v. Alphatech Holdings, Inc.,7 the court ruled:

“…electronic discovery has moved well beyond search terms. While search terms have their place, they may not be suited to all productions. Technology has advanced and software tools have developed to the point where search terms are disfavored in many cases….”

Change is sometimes slow, however. Whilst we may not see the death of keyword searches immediately, advances in TAR may mean we see a decline in the reliance upon them.


1.     Da Silva Moore v. Publicis Groupe – 287 F.R.D. 182 (S.D.N.Y. 2012)
2.     An Evaluation of Retrieval Effectiveness for a Full-Text Document-Retrieval System, 28 Comm. ACM 289 (1985)
5.     No. 15 CV 5826 (CBA) (LB) (E.D.N.Y. May 2, 2019)
6.     LegalTech Artificial Intelligence Market by Application and by End-User: Global Industry Perspective, Comprehensive Analysis, and Forecast, 2018 – 2026
7.     No. 18-CV-0347 (S.D. Ca.) (10/7/2019)


Law In Order is a leading provider to the legal profession of eDiscovery and legal support services including forensic data collection, information governance, managed document review, and virtual arbitration or mediation services. We provide a secure, flexible and responsive outsourced service of unparalleled quality to law firms, government agencies and inhouse corporate legal teams. The Law In Order team is comprised of lawyers, paralegals, system operators, consultants and project managers, with unparalleled knowledge and experience in legal technology support services.


By Phillip Buglass



Screen Shot 2019-04-12 at 3.34.17 PME:

Related Articles by Firm
Data collection and early case assessment for investigations
Data collection and analysis for investigations is very different to collection for discovery or review. This article discusses the differences; how Early Case Assessment (ECA) can assist and the benefits of using review technology ...
Spotlight on eDiscovery
Many people are still confused about what electronic discovery encompasses ...
How to Make Data Collection More ‘Effective’
Data can be collected efficiently, but if relevant data is not collected then the case will not come together.
Information Governance: Preserving Data and Being Prepared for Investigation
Organisations need to ensure their rules around information governance are being enforced. Many US organisations have an information governance officer in their IT, legal ...
How to speak IT
Ideally, everyone should have a basic understanding of the company’s IT infrastructure, not least because as more companies digitise, the risk of cyber threats increases. A cyberattack can come from anywhere ...
Computer Forensics and the Rise of the Drone
With more people being confined to their homes in different parts of the globe, hobbyists will be finding new uses for their drones ...
Redactions? How to Ensure There are no Nasty Surprises
With recent headlines highlighting the damage and embarrassment that can be caused by poorly redacted documents, it is no wonder many firms and corporates are turning to legal document management specialists to secure their redactions ...
Self-Collection Risks
When digital evidence is required at the start of a matter, it’s easy to get swept up in the moment and start rushing to gather the evidence but this is a key time to step back, take a moment ...
What Happens After the Dawn Raid?
The dawn raid has led to the forensic collection of 100,000 documents, now safely secured on a hard drive. What is the process from here? It’s important to plan your strategy in advance to minimise downtime, extract relevant documents and ...
Bringing eDiscovery In-House? Four Tips to Get You Started
With an increase in litigation and in costs for document review, more and more companies are considering bringing parts, if not all, of the eDiscovery process in house ...
A Lawyer’s Future is Looking Sharp with Electronic Hearings
The benefits of running an efficient collection and forensic process extends all the way to the hearings room ...
Forensic investigations, the role of corporate counsel and the rise of information governance
Head of Forensics – Erick Gunawan, discusses the Role of Corporate Counsel in the context of litigation or investigation and the increasing importance of information governance ...
Forensic investigations and cross-border matters
What are the current trends in forensic investigations for cross-border matters? Head of Forensics - Erick Gunawan, looks at the constant evolution in data types and volumes, and the ever-tightening data privacy laws and regulatory intervention ...
Adopting eDiscovery for internal investigations
In-house counsel are often called on to manage an internal investigation. How can you effectively plan for and manage these investigations? We explore how electronic discovery (eDiscovery) tools help you mitigate risk and achieve your fact-finding mission.
The evidence collector that’s always with you
It is an integral part of our life these days and an item that is rarely further than arm’s reach ...
Related Articles
IHC Magazine: April 2024 issue featuring Firms of the Year 2023 Results
As we round off the first quarter of this year, we are proud to showcase our IHC Firms of the Year for 2023. These firms have been selected by our IHC community in recognition of their invaluable contribution to their clients’ ...
Related Articles by Jurisdiction
Kroll: Opaque ownership fastest-growing concern for compliance professionals
Less than 25 percent feel highly confident in their program’s ability to address these risks.
Avoiding data breaches and managing them if they happen
As the risk of a data breach has rapidly increased with constantly changing technology, businesses have to address the issue now.
Electronic signatures and virtual meetings — the Bermuda, British Virgin Islands and Cayman Islands legal framework
The introduction globally of travel restrictions and containment measures arising from Covid-19 has significantly disrupted business, including creating logistical issues in closing corporate or financing transactions or holding board and shareholders’ meetings ...
Latest Articles