Casetext’s CoCounsel through the lens of the Typology

Everyone is talking to OpenAI’s ChatGPT.¹ Meanwhile, OpenAI has been talking to developers of legal technologies.² In March, Casetext announced a collaboration with OpenAI and launched CoCounsel, Casetext’s latest product offering.³ CoCounsel is built on GPT-4,⁴ OpenAI’s most advanced Generative Pre-trained Transformer to date and the first large language model to be credited with a passing score in a simulated bar exam.⁵ Casetext describes CoCounsel as ‘groundbreaking’ and ‘[t]he legal AI you’ve been waiting for’.⁶ It invites lawyers to ‘delegate substantive work to … [CoCounsel] and trust the results.’⁷ Should they?

Casetext is no rookie start-up. The US-based company, which has just entered into an agreement to be acquired by Thomson Reuters for $650m,⁸ was an early adopter of advanced machine learning technologies. Its product offerings include Casetext Research, now simply ‘Research’ (a legal research platform), Compose (a document generation system) and AllSearch (a document search tool launched in June 2022). Compose and AllSearch leverage the functionality of Casetext’s Parallel Search, a transformer-based technology that enables concept-based searching using natural language processing as opposed to keyword search. Yet, according to Casetext, CoCounsel represents a ‘quantum leap in AI for the law’.⁹ What, precisely, does the system claim to offer, how does it work and what are the implications of use of the technology?

The Typology of Legal Technologies

We draw on our experience in developing the Typology of Legal Technologies to answer these questions. The Typology is an online interactive tool developed by our team at COHUBICOL. It features thirty examples of different types of legal technologies and provides a systematic analysis of each. Each analysis is intended to tease out the potential implications of a technology taking account of its intended users, claimed functionality, features, rationale and benefits, and design choices. The ultimate aim of the Typology is to contribute to – and provide a method for securing – a better understanding of the impact of legal technologies on legal outcomes, legal practice and, above all, the nature and quality of the protection afforded by law.¹⁰

CoCounsel – claimed functionality, rationale and benefits

Casetext claims that CoCounsel will allow lawyers to save time, cut costs,¹¹ mitigate risks,¹² achieve better outcomes¹³ and improve access to justice.¹⁴ These claims about the rationale for and the benefits of CoCounsel are unremarkable – many providers of legal technologies claim these or similar benefits. What is remarkable is that Casetext present CoCounsel as ‘the first AI legal assistant’.¹⁵ ‘For the first time’, they say, ‘lawyers can delegate substantive work to AI and trust the results’.¹⁶ Notice, however, that Casetext’s Terms¹⁷ are much more cautious, making it clear that output should be evaluated, including by way of human review.¹⁸ 

According to Casetext, CoCounsel can

• Review documents – it will ‘read them in full, and answer with citations to sources’¹⁹
• Prepare for a deposition – using a description of the deponent and what’s at issue in the case, CoCounsel will ‘[identify] multiple highly relevant topics … and draft questions for [these]’²⁰
• Search a database – CoCounsel will provide answers to questions about the content of the database
• Carry out legal research – CoCounsel will provide answers to research questions in the form of a memo, complete with supporting references
• Summarise documents including contracts or legal opinions
• Extract data from contracts – CoCounsel will answer questions about contracts and provide a ‘complete list of relevant clauses from contracts in a set’²¹
• Identify contractual clauses which do not conform to a specified policy, report on the risks of using non-compliant language and recommend revisions²²

The claimed functionality is impressive but as readers familiar with ChatGPT will know, the key question is not whether the system can provide answers, summaries, or citations but whether, as Casetext claims, users can trust these results. We can begin to explore this issue as well as the broader implications of CoCounsel for law and legal outcomes by examining what is known about its design.

Claimed design choices

CoCounsel is built on GPT-4, one of a series of Generative Pre-Trained Transformers developed by OpenAI.²³ However Casetext differentiates CoCounsel from ‘a generalized, publicly available AI’ in three respects.²⁴ First, CoCounsel integrates Casetext’s Parallel Search which is also built using Transformer models²⁵ and has access to Casetext’s own legal databases. According to Casetext,

[t]his integration enables CoCounsel to draw upon reliable data to produce answers with exceptional accuracy and speed, supported by linked citations, making it easy for lawyers to verify CoCounsel’s output.²⁶

Second, it is well-known that large language models tend to ‘hallucinate’ or make up facts.²⁷ Casetext claims that:

CoCounsel does not … “hallucinate,” because we’ve implemented controls to limit CoCounsel to answering from known, reliable data sources … or not to answer at all.²⁸

The ‘controls’ appear to consist in the integration of Parallel Search and Casetext’s legal databases with GPT-4, prompt engineering and continual review.²⁹

Finally, Casetext claims that CoCounsel is configured so as to ‘keep […] lawyers’ and their clients’ data private and secure.’³⁰ According to Casetext,

CoCounsel only accesses OpenAI’s model through dedicated servers – a “private entrance” –and through an interface between our technologies (an API) that never stores any content our customers upload to or enter into CoCounsel. This means none of the information used in CoCounsel is sent back to “train” OpenAI’s model.³¹

How, if at all, can we verify these claims? What issues – technical and legal – are potentially associated with the system in its context of use?

Substantiation and Potential Issues

Substantiation

Some information is available about GPT-4 and Parallel Search, the building blocks of CoCounsel.³² However, Casetext has not published details of the architecture of CoCounsel, nor about how, exactly, Parallel Search and GPT-4 are integrated into the system. In principle, large language models such as GPT-4 can be used in combination with other models and tools.³³ They can be combined with information retrieval components and the ability to access external databases.³⁴

Similarly, Casetext does not supply details about the technical means by which CoCounsel provides answers with linked citations. However, OpenAI suggests an approach to the use of large language models for question answering where the model answer must be supported by references.³⁵ DeepMind proposes a system where the output of a large language model is accompanied by citations to external sources.³⁶ The aim (as with CoCounsel) is to allow for independent verification of the model output.

Is it possible that Casetext has eliminated the possibility of hallucinations? Existing research suggests that developers can employ a range of methods to reduce hallucinations notably by constraining the system such that its answers are grounded in factual content.³⁷ It is unclear whether these methods are wholly effective. It is doubtful that prompt engineering alone can eliminate hallucinations;³⁸ sources of content can contain factual and other inaccuracies;³⁹ even if the source can be trusted the system might still produce output that is inconsistent with the source.⁴⁰

Casetext maintains that they access OpenAI’s servers through a ‘zero-retention API’.⁴¹ Without information about Casetext’s contract with OpenAI it is impossible to know what arrangements are in place about data sharing between the two. Ordinarily, OpenAI retain data submitted through the API ‘for abuse and misuse monitoring purposes for a maximum of 30 days, after which it will be deleted (unless otherwise required by law).’⁴²

However, OpenAI state that:

For trusted customers with sensitive applications, zero data retention may be available. With zero data retention, request and response bodies are not persisted to any logging mechanism and exist only in memory in order to serve the request.⁴³

Their standard API data usage policies also confirm that

OpenAI will not use data submitted by customers via our API to train or improve our models, unless you explicitly decide to share your data with us for this purpose.⁴⁴

Potential technical issues

CoCounsel’s main claim is that users can trust it. What technical issues might potentially impact the trustworthiness of the system or its outputs?

Performance outside the scope of the data used to fine-tune the system

One of Casetext’s key claims is that they do not collect customer data,⁴⁵ including prompts and responses generated by CoCounsel, and do not share that data with OpenAI. It is important for lawyers to be able to rely on these claims but this design choice imposes certain limitations. The GPT-4 model must be fine-tuned on prompt/response pairs in order to provide appropriate responses to very specific queries of the CoCounsel customers. Since Casetext claims to not be using customer data, they must test the system on prompts they expect customers to provide or data collected from customers earlier. They claim to have fine-tuned the system on 30,000 questions. Given the large amount it may cover quite a lot of issues, it is however unclear how well such a system would perform when having to deal with requests outside the scope of the training. If customers input prompts that are unclear, poorly framed or lack context, the system may also generate suboptimal responses.

Linked citations may be of limited utility

CoCounsel claims to provide links within generated text, so that the client may verify the output. However, needing to verify all the information might not save lawyers much time even if all the links to the sources are provided. Moreover, this approach doesn’t allow one to check whether any crucial information is missing from the output.

Hallucinations

Casetext claims to have implemented controls so that CoCounsel cannot hallucinate. We are sceptical about this claim though we accept that the controls Casetext describes may reduce hallucinations by not relying on false information from the internet, however since the response is automatically generated, it may still produce incorrect responses based on truthful information since it still generates new text rather than simply copy from the source. Hallucinations would impact on system performance and likely increase the time required for verification.

Evaluating and maintaining system performance

To our knowledge, Casetext does not publicly disclose how they evaluate performance of their system. One must also take into account that given the fast pace of the changes of the legal landscape, the system needs to be regularly updated or even retrained (at least the fine-tuning part) to be able to provide up-to-date information to the customer. This requires a lot of resources, and the system might not perform at the same level with each iteration. Given that GPT-4 model is not being retrained on new data, system performance may also be affected.

Potential legal impact

Casetext claims the system is very accurate. Ironically, the better the system performs, the greater the risk of over-reliance on the system;⁴⁶ lawyers may be tempted to rely on the outputs without verifying the contents. Such over-reliance might impact on the nature and quality of legal advice, engender a passive and unproductive approach in aspects of lawyering and amount to an abrogation of lawyers’ professional obligations. Should lawyers fully ‘delegate to’ CoCounsel? Of course not – indeed this framing, and Casetext’s description of CoCounsel as an ‘AI legal assistant’ plays into anthropomorphic tropes. Should lawyers ‘trust’ CoCounsel’s outputs? That depends on what one means by ‘trust’. However well the system performs, however useful it may be, lawyers should recall that at its heart is a model that is trained to ‘predict the next token in a document’.⁴⁷ It has no conception of the aims of law, no awareness of the context of its use; it can neither engage in legal reasoning nor offer legal advice. In Casetext’s own words, directed to lawyers:

You and your end users are responsible for all decisions made, advice given, actions taken, and failures to take action based on your use of AI Services.⁴⁸