Vol. 99, No. 1

Artificial Incompetence? Unpacking AI’s Shortcomings in Contract Drafting and Negotiation

Carly Snell*
AI, Artificial Intelligence, Contract Law, Contracts
Notes

INTRODUCTION

This Note was inspired by my time as a data center procurement contracts intern during the summer after my first year of law school. In this role, I assisted contract analysts and attorneys with their procurement of space in data center facilities by contracting with data center suppliers. I regularly reviewed contract redlines from suppliers, identified non-market or disadvantageous terms in those contracts, and suggested changes for the next “turn of the redlines,” or when the company would return the contract to the supplier with new edits to the document. An impactful conversation with my manager about artificial intelligence’s potential as a useful tool in a transactional lawyer’s toolbelt inspired a deeper dive into the benefits and drawbacks of applying artificial intelligence (“AI”) to the contract drafting, redlining, and negotiation space—ultimately leading to the development of this Note.

After the internship concluded, I began my second year of law school. While the most noticeable change upon my return was that I was no longer a first-year student, I also immediately observed a greater emphasis on AI in legal education than before. My law school offered a course on AI’s legal applications, peers used AI to supplement their studies, and professors emphasized the importance of mastering AI during law school, as it would be an essential tool in future legal practice. Similarly, students at other law schools honed their negotiation skills against AI chatbots¹ and even developed their own AI-driven case briefing technology.²

As with the implementation of any new technology, however, there are some points of contention that arise when applying AI to the law—especially in the context of contract drafting, formation, and negotiation. This Note covers four main challenges to applying AI to contract drafting: (1) contract law principles, (2) equity concerns, (3) accuracy issues, and (4) legal profession challenges. Additionally, this Note presents the results of a novel empirical study designed to test AI technology’s tendency to discriminate when tasked with negotiating a contract on behalf of different types of clients. Interestingly, ChatGPT, a popular AI chatbot,³ appears to favor corporations and nonprofit organizations over individuals when acting as a negotiation assistant.⁴ This finding suggests that the excitement surrounding AI’s potential uses in the legal field⁵ is premature, and professionals should hesitate to implement this technology in contract drafting and negotiation until algorithmic discrimination is adequately addressed.

Part I of this Note introduces the historical development of AI technology and its rise to stardom that began with the public release of ChatGPT in 2022.⁶ Part I then describes early applications of AI technology to the contracting space, such as Spellbook, Harvey, and LegalSifter.⁷ After that, Part I discusses fundamental contract law principles, such as mutual and constructive assent, that AI contract drafting may not readily align with.⁸ Finally, Part I concludes by orienting the reader with basic legal profession concepts, such as the lawyer’s duties of confidentiality, communication, competence, and diligence.⁹

Part II introduces several illustrative examples of AI in contract drafting and negotiation that pose unique questions about the key differences between human and AI-driven contracting. These differences make it difficult to apply existing contract law to AI and raise important concerns about AI’s potential to discriminate when contracting and negotiating on behalf of different clients.¹⁰ Part III of this Note expands upon AI’s usurpation of traditional contract law principles. Fundamental contract law concepts, such as the “meeting of the minds” required to form a valid contract, do not readily apply to wholly AI-driven contracting.¹¹ Principally, AI’s application in contract drafting and negotiation can present novel complications when determining whether or not the parties to a contract mutually agree on its terms. These issues persist regardless of whether a party performs some of its obligations under an AI-driven contract and despite the controversial doctrine of constructive assent.

Part IV covers the equity concerns that arise when applying AI technology to contracting. In general, applications of AI technology in the contracting space raise concerns about “algorithmic discrimination”—AI’s tendency to produce discriminatory outputs as a consequence of being trained on tainted data.¹² AI in contracting also raises ethical issues regarding enforcement of fully automated contracts. A pervasive issue in the AI space is ensuring proper alignment between an AI model’s goals and those of its operator.¹³ Given that AI technology regularly suffers from misalignment problems, would it be ethical and equitable to enforce contracts drafted by these models? Another ethical dilemma that arises in the AI contracting context concerns legal liability and accountability if a party is injured by an AI-formulated contract. If harm results from an AI-drafted contract, who should be held accountable for these harms? Between the AI model itself, its designer, its user, and other parties, there is no readily apparent answer. Finally, the implementation of AI in contracting—a setting that involves a plethora of sensitive information—presents serious data privacy and security concerns.¹⁴

In Part V, this Note reviews the accuracy issues apparent in current and potential applications of AI technology. Simply put, AI technology can behave unpredictably and output inaccurate results known as “hallucinations.”¹⁵ In the litigation context, several lawyers, including Michael Cohen’s attorney, have recently been sanctioned or publicly admonished for citing fabricated cases generated by ChatGPT in their filings.¹⁶ In the contracting space, in which exact language and minor details can govern the legal meaning of an agreement, AI’s tendency to hallucinate can cause major problems.

Part VI presents the challenges to the legal profession that arise when using AI technology in contract drafting and negotiation. For example, overreliance on AI technology to draft and negotiate contracts may violate an attorney’s professional duties of competence and diligence—much like the actions of the lawyers who cited fabricated cases in their court filings. Overreliance may also violate an attorney’s professional duty of communication if they cannot explain their reasoning for a recommended course of action to a client due to reliance on ChatGPT in their decision-making. Additionally, since AI models operate as “black boxes,” their use may raise concerns about duty of confidentiality violations if client information is input into these systems without proper safeguards.¹⁷

Part VII discusses the empirical findings that resulted when the author “hired” ChatGPT to assist various types of fictitious clients with negotiating a standard commercial real estate lease. These research findings suggest that ChatGPT discriminates against individual clients by tending to recommend renegotiation less often and to a smaller degree when advising individual clients than when assisting corporate or nonprofit clients. These findings have immense equity implications for contract drafting and negotiation in an AI-driven world, as AI models that disfavor individual clients may exacerbate existing market power or resource inequalities between individuals and more sophisticated corporate or nonprofit clients.¹⁸ Finally, Part VIII discusses some strengths and potentially useful applications of AI technology in legal work in light of this Note’s theoretical discussion and empirical findings. Part VIII posits that, although AI technology excels at summarization,¹⁹ concerns about its ability to exercise discretion and judgment suggest that it may be best suited for administrative tasks.

I. A CRASH COURSE IN AI AND RELEVANT LEGAL THOUGHT

A. What Is Artificial Intelligence and How Can It Contract?

There is no widely accepted definition of what constitutes artificial intelligence, which is partially a byproduct of how technological capabilities have rapidly improved in recent years.²⁰ To oversimplify, computer programs were historically classified as artificial intelligence if they successfully mimicked human rational thought.²¹ An early example of this concept is the Turing test for artificial intelligence, which was developed by the “father of modern computer science,” mathematician Alan Turing.²² The Turing test assesses how well a machine can imitate human thought and behavior via a competition that Turing called the “Imitation Game.”²³ In the game, a machine and human compete by answering questions asked by a human interrogator; at the end of the game, the interrogator must identify which competitor is a human and which is a machine.²⁴ If the interrogator gets it wrong—i.e., says that the machine is the human—then the machine is thought to demonstrate human-level thought and intelligence.²⁵

This Note utilizes a relatively expansive definition of artificial intelligence that is reminiscent of the Turing test. For the purposes of this Note, artificial intelligence is any computer software program that demonstrates human-like behavior or intelligence. As discussed below, the focal point of artificial intelligence in this Note is large language models, which are some of the best modern examples of AI that would likely pass Turing’s test for artificial intelligence, given their language-based design and applications.²⁶

Artificial Intelligence’s Rise to Prominence: The “AI Boom”²⁷

Artificial intelligence has taken the public consciousness by storm since the release of ChatGPT, OpenAI’s text-generating chatbot, in November 2022.²⁸ ChatGPT is an AI model trained to engage in natural language conversations, which means that when users interact with ChatGPT, it converses with them by generating textual responses comparable to that of a human.²⁹ The model’s successful imitation of human-sounding speech captured the public’s imagination,³⁰ prompting increased interest in potential applications of AI technologies from the general public³¹ and software developers³² alike.

ChatGPT can complete a variety of academic tasks in a matter of seconds, such as writing essays, generating ideas, and answering mathematical problems.³³ It is no surprise, then, that students from primary school to collegiate grade levels were some of the model’s most prevalent initial users, asking ChatGPT to write papers and complete homework assignments on their behalf.³⁴ Students’ widespread use of ChatGPT to complete assignments led many schools and universities to initially ban the AI model altogether,³⁵ although it was difficult, if not impossible, to enforce AI bans—especially outside of the classroom.³⁶ A new industry of tools meant to detect the use of AI in students’ writing emerged to combat this issue, but their accuracy remains widely disputed.³⁷

Although initial widespread applications of ChatGPT were somewhat rudimentary in nature, such as students’ use of the tool to complete assignments,³⁸ OpenAI’s introduction of the model to the public sphere was instrumental in prompting other AI developers to invest in the creation and public release of their own large language models (“LLMs”).³⁹ After witnessing OpenAI’s successful launch of ChatGPT, prominent tech industry leaders such as Google and Meta immediately sought to turn AI technologies into tangible, profitable products that they could sell to individuals and companies.⁴⁰ Although these major technology companies had already been developing (and, in some cases, even released, to little success⁴¹) their own AI technologies before November 2022, ChatGPT’s successful public launch prompted an expansion of the AI industry like never before.⁴² By the following spring, a flurry of new LLMs had emerged on the market: Meta’s LLaMA model, Google’s PaLM-E, and even OpenAI’s newest iteration of its LLM: GPT-4.⁴³

In essence, large language models are AI models designed to interact with and produce language.⁴⁴ “Large” refers to the increasing trend to train these models on large quantities of data stored in massive data sets that are usually housed in collocated data centers.⁴⁵ While ChatGPT, LLaMA, PaLM-E, and GPT-4 are all generally considered LLMs, much like AI more broadly, a concrete definition of what constitutes a large language model remains an open question.⁴⁶ There are no exact parameters for how large an AI model must be or how it must interact with language in order to be categorized as an LLM.⁴⁷

On the other hand, LLMs are generally considered to be a subset of generative AI.⁴⁸ Generative AI is defined as artificial intelligence capable of producing new creations, such as graphic images, text, and audio, based on training data inputted into the model.⁴⁹ Therefore, generative AI enables a user to generate substantial quantities of work product with minimal effort by prompting a generative AI model and letting it create content for them based on the query. This is partly why ChatGPT became wildly popular in a short period of time⁵⁰—and why the generative model caused concerns about students using it to complete homework and other assignments on their behalf.

Beyond their avocational applications as homework helpers⁵¹ and joke writers,⁵² LLMs are being increasingly used by industry professionals to improve and expand the potential of their products and services.⁵³ For instance, Amazon Web Services implemented an externally facing AI chatbot on its Amazon.com retail site designed to handle returns, provide shipment tracking information, and generally improve the site’s customer service capabilities⁵⁴ (albeit the chatbot has garnered mixed reviews⁵⁵). Similarly, in 2024, Target Corporation launched an internally facing generative AI model, called Store Companion, to assist with employee training, store operations management, and general problem-solving tasks.⁵⁶ Meanwhile, social media platforms such as Instagram use AI models to filter content and craft feeds that are better personalized to users’ individual preferences.⁵⁷

Early Applications of Artificial Intelligence to Legal Contracting

Naturally, the ever-increasing implementation of LLMs in a variety of businesses, industries, and settings includes applications in the legal field as well.⁵⁸ For example, AI has already been used to create legal workflow companions with suites of legal skills,⁵⁹ contract lifecycle management software programs,⁶⁰ and contract redlining and drafting assistants.⁶¹ A simple Google search for AI contracting services yields a plethora of (interestingly named) AI-powered software programs that purport to assist an attorney with redlining (e.g., Harvey,⁶² Lawgeex,⁶³ Superlegal,⁶⁴ Ivo,⁶⁵ Screens,⁶⁶ and Spellbook⁶⁷) or managing (e.g., Evisort,⁶⁸ Ironclad,⁶⁹ Sirion,⁷⁰ and LegalSifter⁷¹) their legal contracts. Even companies that operate widely used legal research databases, such as LexisNexis and Thomson Reuters, have created and marketed their own generative AI-powered legal assistants.⁷²

Legal professionals are generally excited about new and potential future applications of AI to the legal world.⁷³ Many believe the technology will increase efficiency in a time-intensive industry by synthesizing documents and reducing the time a human attorney needs in order to perform certain legal tasks.⁷⁴ Some hopefuls even view AI as infallible—capable of outperforming humans, whose work is prone to errors, because AI can craft perfectly completed and accurate work product.⁷⁵ Finally, AI is thought by some to make legal services more affordable and accessible to the general public⁷⁶ by reducing the number of billable hours an attorney must dedicate to any given task, enabling individuals to access legal services without hiring a human attorney, or both. In fact, Utah and Arizona have already implemented pilot programs that allow non-lawyer entities, such as AI chatbots, to provide legal services, and Washington may be the next state to institute such a program.⁷⁷

Despite this enthusiasm about AI, the immediate application of LLMs to the legal space has not been without its challenges. Some attorneys have wrongfully used LLMs to shirk their responsibilities by asking AI models to conduct legal research or write briefs on their behalf.⁷⁸ This practice has resulted in massive sanctions and fines for attorneys who cited “bogus” cases that were fabricated by ChatGPT in documents that they later submitted to a judge.⁷⁹ Furthermore, as discussed later in this Note, issues regarding lawyers’ ethical and professional duties, algorithmic discrimination, AI’s inaccuracies, and the subversion of traditional contract law principles also arise when large language models are applied to the legal field.

B. A “Meeting of the Minds” Regarding Contract Law Theory

An orientation into the foundational principles underlying contract law theory is needed before one can take a proper deep dive into the applications of AI in contracting. A great place to start is the traditional contractual theory of mutual assent, colloquially known as the “meeting of the minds.”⁸⁰ Mutual assent is one of many requirements that must be demonstrated for a court to hold that a given contract is legally valid and enforceable.⁸¹ “Meeting of the minds” refers to the idea that both parties must mutually agree to the terms of a contract in order for the agreement to be legally binding.⁸² That is, the parties’ minds must, in a sense, “meet in the middle” at the moment when the contract is formed. For that reason, mutual assent may not be found when one or both of the parties to a contract entered into the agreement based on a misunderstanding or a mistake of law or fact.⁸³ Intuitively, this makes sense; it would not be good public policy to bind people to a contractual agreement if they did not fully understand the obligations and consequences they allegedly agreed to when the agreement was executed. Beyond equity justifications, it may also be inefficient to hold a party accountable for obligations that they did not intend to undertake and may not be equipped to fulfill. Relatedly, to create a binding agreement, the parties to the contract must specifically mutually assent to the material terms of the contract.⁸⁴ Without a “meeting of the minds” between the parties to any given contract regarding the essential provisions of the agreement, the contract is invalid and not legally binding on the parties.

In some instances, courts have imputed assent to a party based on their conduct even if they did not explicitly agree to or approve of the terms of an agreement.⁸⁵ This doctrine is known as “constructive assent,”⁸⁶ and it is common among online transactions.⁸⁷ For example, if a user of an online webpage affirmatively acknowledges the page’s terms of use by clicking an “I accept” or “I agree” button without actually reading the agreement, the user is usually found to have constructively assented to the terms of the agreement despite not actually being aware of its contents.⁸⁸

Although many people make light of the fact that nobody ever reads various websites’ terms of use or, more notably, Apple’s Terms and Conditions,⁸⁹ constructive assent is no laughing matter. In these types of situations, constructive assent can be used to essentially waive the traditional contract theory requirement of a “meeting of the minds,” instead holding individuals accountable for the contracts that they sign even if they do not fully understand or have knowledge of the terms that they allegedly agreed to.⁹⁰ Unsurprisingly, the doctrine of constructive assent is controversial—especially its application to consumer contracts⁹¹ and form contracts more broadly.⁹² Further, the ethics of constructive assent are hotly debated among scholars, with some arguing that applying constructive assent to a contested contract unfairly disadvantages the weaker party (e.g., the consumer) to the benefit of the dominant party (e.g., the retailer) whose greater market power enables them to force the weaker party to consent to the dominant party’s preferred terms.⁹³

C. Attorneys as Ethical and Professional Fiduciaries

Another important factor to consider when analyzing the potential applications of AI to the contracting space is the ethical and professional complications that arise due to attorneys’ special fiduciary duties to their clients. In general, attorneys are held to a higher standard than those who work in many other professions.⁹⁴ Specifically, attorney conduct is governed by each state’s bar association, many of which have adopted the Model Rules of Professional Conduct—the generic rules promulgated by the American Bar Association.⁹⁵ The Model Rules serve as a fundamental guideline for attorney conduct by prescribing various professional and fiduciary duties to attorneys, such as client confidentiality, competence, diligence, and communication.⁹⁶ The Model Rules also address various topics relating to an attorney’s practice—like conflicts of interest, the formation of an attorney-client relationship, the scope of one’s representation, and how to interact with unrepresented persons⁹⁷—and explain how model attorneys should approach these issues. Importantly, the Model Rules detail practices that constitute misconduct, like engaging in dishonesty or fraud, violating the Model Rules of Professional Conduct, or committing a criminal act.⁹⁸ For the purposes of this Note, it is important for one to keep the Model Rules of Professional Conduct in mind when considering how an attorney may use AI technology in drafting or negotiating contracts, as certain applications of AI may subvert the underlying goals that the Model Rules were designed to support in more traditional applications.

II. ILLUSTRATIVE EXAMPLES

Several ethical, practical, and theoretical questions arise when one considers various applications of AI to contract drafting, formation, and negotiation. To better illustrate the issues that arise from applying AI to the contracting space, consider the following numbered examples and the questions they raise regarding their implications for the contract law principles and legal profession concepts that we have discussed:

Example #1: Laypeople Using AI to Draft a Contract⁹⁹

Two laypeople (i.e., not attorneys) are doing business together. Interested in summarizing their deal in a written form, they “draft” a contract by asking ChatGPT to do so for them. Once ChatGPT has drafted the contract, the two parties both read and sign the contract, despite not understanding the agreement’s legalese or terms. Later, something goes wrong, and the contract’s validity and enforceability are disputed.

Was there a “meeting of the minds,” or mutual assent, here?

Is this a case of AI-assisted human contracting, or was this effectively an entirely AI-created contract?

Is the contract enforceable?

Should society want the contract to be enforceable?

Example #2: AI as a Contract Drafting Tool for Attorneys¹⁰⁰

As is industry practice, a lawyer in a corporate law firm normally uses a standard form contract from prior deals as a starting point when drafting new contracts. However, for a particular deal, she decides to use ChatGPT to draft the initial form contract instead.

Is this an example of AI as a tool that assists humans in contract drafting, or is this a wholly AI-drafted agreement?

Does this distinction have important implications for the contract’s validity and enforceability?

Is there any significant difference between this attorney using AI to create a form contract or pulling a precedent contract out of her firm’s database?

Would this amount to a breach of the attorney’s professional duties of competence, diligence, or anything else?

Example #3: Human Error Versus AI-Drafted Terms

Overwhelmed with his busy workload, a lawyer mistakenly inserts a clause in a contract he is drafting for his client. Both his client and the other party to the contract sign the agreement; neither party nor the attorney knows at the time the agreement is executed that the accidental provision is included in the contract.

Is the extra provision in the agreement enforceable (i.e., did the parties mutually assent to the term)?

Is this scenario any different from if AI completely drafts and executes a contract without humans involved in the contracting process?

How are these two examples reconciled in terms of mutual assent? Are they the same, or fundamentally different in any way?

Example #4: AI Automatically “Agreeing” to Online Terms

Annoyed with websites’ many Terms of Service and Cookies pop-ups, an inventor creates an AI-driven “ad blocker” software that automatically clicks through and “agrees” to these pop-ups on the software user’s behalf so that they never have to see them again.

Would this constitute the user’s assent to various websites’ Terms of Service?

Does the answer to this question depend on how long the user has had the software, or whether they knew or reasonably should have known that specific websites had Terms of Service or Cookies pop-ups?

* * *

There are two possibilities when applying AI technology to contract drafting and negotiation: (1) AI effectively functions as an assistant, aiding humans with their contracting, and (2) fully automated decision-making, in which AI completely takes over contracting, from start to finish, with no humans involved in the process. Under either scenario, four categories of problems arise when implementing AI in contract drafting and negotiation: the subversion of contract law principles, equity concerns, accuracy issues, and legal profession challenges.

III. AI’S SUBVERSION OF CONTRACT LAW PRINCIPLES

If AI functions as a mere contract drafting and negotiation assistant, mutual assent concepts would apply in the same manner that they do for purely human-conducted contracting. An underlying principle of the mutual assent requirement for a valid contract is the notion that the parties to a given contract must understand the terms of the agreement and have a “meeting of the minds,” or mutual agreement, that they find the terms acceptable.¹⁰¹ If AI technology merely assists an attorney with drafting or negotiating a contract, this does not affect the portion of the dealmaking process that mutual assent concerns. The only point in time that is relevant for mutual assent is when the parties come to a consensus that the contract’s terms are agreeable and subsequently execute the agreement.¹⁰² By that point in time, the drafting and negotiating phases of the process are complete (and, truthfully, long gone)—the agreement is in its final drafted form and will not undergo further redlines or revisions. Thus, the implementation of AI as a mere assistant in the contracting and negotiation process is not within the timeline or contextual scope that mutual assent concerns. AI’s use as a contracting assistant is therefore akin to any personal opinions the drafting attorney may have (outside of their thoughts and duties as a fiduciary of their client) regarding the deal at hand—i.e., irrelevant to questions about mutual assent.

While some may argue that the cyclical drafting, redlining, and negotiation process drives the parties to a contract toward the ultimate goal of mutual assent at the end of the contracting cycle, it is not a necessary component of mutual assent that agreements are modified and negotiated by the parties. If one party presents a complete agreement to another party, who signs it without criticizing its contents or insisting on revisions, it is still a valid contract. Furthermore, in many instances, an attorney drafts and negotiates on behalf of their client, who signs the final contract without a comprehensive legal understanding of the negotiations and redlines that were made during the dealmaking process. This is arguably like Example #1 in Part II, in which the two laypeople used AI to draft a contract that they then signed. Although the individuals did not negotiate between themselves, mutual assent was arguably satisfied because the humans—not ChatGPT—assented to the agreement at the end of the contracting process.

On the other hand, if contracting is entirely managed by AI—without humans involved in the process—then the contract law requirement of mutual assent is not satisfied. Arguably, if the laypeople in Example #1 did not understand the contract because ChatGPT performed a substantial portion of the legal lift for them (which is possible, considering that they did not understand the AI-drafted agreement’s legalese or terms), then the mutual assent requirement may not be satisfied because the contracting process was effectively completed without human involvement. Example #4 details a more abstract example of this concept. In Example #4, the inventor’s software “agrees” to websites’ terms of use on its users’ behalf. In this situation, the human user never sees, let alone reads, the terms of service that they allegedly agreed to through the AI-driven software. Although some might argue that there is mutual assent because a person who installs the software knows that it will “agree to” the terms on any site that the person visits, this argument does not hold up to pragmatic scrutiny. Given how often and extensively people surf the Internet, it is highly likely that, over time, the person would not know which websites had pop-up advertisements or terms of use that the AI bot “agreed” to on their behalf, let alone the content of those agreements.

Therefore, the contract law requirement of mutual assent goes unsatisfied when AI fully takes over the contracting process. This flaw in solely AI-executed contracting becomes even more apparent when considering contracts that involve multimillion- or multibillion-dollar transactions, fundamental changes in a company’s structure or dealings, or changing the client’s financial or business practices in any substantial way. Without providing notice of these changes to the client and securing their informed assent to new and material contractual terms, solely AI-driven contracting is unlikely to satisfy traditional contract law principles.

Some might argue that a party’s performance of its obligations under a fully AI-driven contract would justify its validity and waive the mutual assent requirement, much like the traditional contract law enforcement principles surrounding the Statute of Frauds.¹⁰³ However, a fully automated contracting process differs from classic applications of the Statute of Frauds—such as when a party denies a prior verbal agreement, claiming that they never agreed to the deal because no written proof of it exists.¹⁰⁴ Rather, if AI completely drives the contracting process, then the parties to a contract would likely never be aware of, let alone read, the AI-drafted and executed agreement. Due to this disconnection, it is highly unlikely that the parties would completely perform their obligations under the agreement—simply because they would not know what their obligations are. Even if the parties were generally aware of their performance obligations (e.g., because the AI model contracted an extension of an existing purchase agreement between a purchaser and supplier), they would still not know the specifications of the agreement to a high enough degree for public policy to justify holding them to the transaction.

Furthermore, although some may argue that the doctrine of constructive assent can waive the mutual assent requirement in the purely AI-driven contracting setting, this argument is specious. Constructive assent is a highly controversial doctrine in its current limited uses, such as form contracts.¹⁰⁵ Scholars have raised particular concerns about constructive assent eliminating the need for mutual assent in online transactions, such as clickwrap agreements,¹⁰⁶ because the doctrine can infer an Internet user’s assent from their decision to click “I agree”—regardless of how “ill-informed and not well considered” that decision might have been.¹⁰⁷ Therefore, because constructive assent is thought by many to subvert traditional contract law theory, especially in online transactions, it provides a weak justification for waiving the mutual assent requirement in a purely AI-driven contracting setting.

Therefore, the distinction between AI as a contracting assistant and wholly AI-driven contracting carries significant contract law implications. In Example #2 in Part II, the legal difference between an attorney using a precedent contract from prior deals and relying on an AI-generated form contract is crucial, even though practicing attorneys may see little to no practical difference between the two. As AI technology continues to advance, the line between human-driven and AI-driven contracting will increasingly blur, raising questions about contract validity, enforceability, and an attorney’s professional obligations. Whether AI serves merely as a drafting tool or takes on a more autonomous role could have far-reaching legal consequences.

IV. EQUITY CONCERNS

A. Algorithmic Discrimination

Algorithmic discrimination occurs when ostensibly impartial AI technology produces discriminatory results because it was trained on tainted inputs.¹⁰⁸ Put more simply, algorithmic discrimination is a perfect example of “Garbage In, Garbage Out.”¹⁰⁹ Proponents of AI argue that even if algorithmic discrimination occurs, automated decision-making is preferable to human decision-making because humans are biased.¹¹⁰ However, algorithmic discrimination can perpetuate and amplify existing biases or stereotypes in an AI model’s training data, with the dangerous added implication that the tainted model appears facially objective and neutral.¹¹¹ Furthermore, because of their reliance on human inputs, algorithms will arguably never be fully bias-free and nondiscriminatory, but perpetually flawed as “partially human.”¹¹² Additionally, due to its highly advanced pattern-detection abilities, AI technology has the potential to develop new forms of discrimination by extracting patterns from its inputted data that humans alone would not have been able to detect.¹¹³

Algorithmic discrimination is also concerning because current legal theories do not supply satisfactory remedies for discrimination by AI systems.¹¹⁴ For example, imagine that an online job search site, such as LinkedIn, uses an AI-driven algorithm to “match” employers with potential interview candidates on the site by recommending certain user profiles to employers.¹¹⁵ If a user believed that the algorithm discriminated against them in choosing not to suggest their profile to employers, they would have limited options to seek legal redress. In the employment space, discrimination claims are separated into two categories: (1) disparate treatment and (2) disparate impact.¹¹⁶ Disparate treatment is focused on combating explicit discrimination, which requires a finding of intent.¹¹⁷ In a traditional, non-AI setting, explicit discrimination may be demonstrated by a qualified job candidate who was denied employment by a firm that refused to hire her by proving that the refusal was based on one of the candidate’s protected characteristics, such as race or gender.¹¹⁸ Conversely, to claim disparate treatment in the case of an AI algorithm, the disgruntled LinkedIn user would have to demonstrate that the algorithm had the intent to discriminate, which may be incredibly difficult, if not impossible, to prove in the case of a nonhuman entity. Thus, algorithmic discrimination is likely thought to be a product of unintentional or incidental discrimination.

Alternatively, disparate impact claims do not require the plaintiff to prove discriminatory intent;¹¹⁹ rather, the doctrine considers whether there is a disparate impact on members of a protected class, any business necessity for the impact, and a less discriminatory alternative means of achieving the same result.¹²⁰ Therefore, given the aforementioned difficulty of ascribing any particular cognitive motivations to an AI model, disparate impact discrimination is the only potential mode of existing discrimination law that

might provide legal redress for members of protected classes who experience algorithmic discrimination in the employment context.

In the contracting space, algorithmic discrimination has the potential to create disastrous consequences. If an AI model is trained on discriminatory data or its algorithm is improperly weighted by its human developers, it may tend to favor one type of party over another, such as men over women.¹²¹ This bias may then prompt the AI model to negotiate more favorable deals for certain parties than it would for others. This potential for AI to act as a discriminatory advocate may exacerbate existing inequalities, especially if the model’s reliance on tainted training data causes it to reinforce biases that disproportionately harm certain groups. Particularly sensitive communities include women, racial or ethnic minorities, and people who are socioeconomically disadvantaged. In the contracting setting, where every word in a contract has an important implication for the meaning of the agreement, a tainted AI model could selectively include unfavorable terms—or simply choose terms that are not the most favorable—in an agreement when “hired” by a party that the model’s data disfavors. The individual who experiences discrimination by receiving the “short end of the stick,” or undesirable contract terms, would likely never know that they were discriminated against by the model they used to contract. Even if the disadvantaged individual later became aware of the discriminatory term selection, it is likely that they would not have the ability or resources to advocate for themselves.

Furthermore, the contracting setting presents a multitude of consequential and important situations in which a person’s livelihood depends on the degree of favorability they are able to negotiate for themselves in a given contract. For example, in an employment contract, the starting salary, amount of paid family leave, and inclusion of any noncompete provisions may have huge implications for a prospective employee’s financial stability and future wellbeing. If an AI model poorly negotiates on a potential employee’s behalf, that potential employee may experience a lower quality of life than they would have otherwise—and if the reason for AI’s poor performance is discriminatory conduct, these disadvantaged outcomes will only exacerbate existing inequalities in our society.

B. Ethics of Enforcing Automated Deals

Another serious concern that arises when using AI in contracting is the ethical dilemma of deciding when to enforce completely automated deals. If we get to the point in which contracting is an entirely AI-driven task, do we feel comfortable holding humans accountable for the deals that an AI model entered into on their behalf?

A critical consideration when determining accountability in this circumstance is AI (mis)alignment. Broadly speaking, direct alignment refers to the ability to program an AI system so that it pursues goals consistent with the goals of its operator.¹²² There are a plethora of difficulties in ensuring proper direct alignment, including (1) determining the operator’s goals, (2) conveying those goals to the AI software, and (3) getting the AI model to correctly translate those goals into actions.¹²³ It is often incredibly difficult for an AI user to overcome these challenges, and efforts to do so sometimes cause AI programs to take unexpected actions that result in adverse consequences.¹²⁴

In the contracting context, holding the user of an AI contracting software to an agreement that the AI model drafted on their behalf can have especially inequitable consequences. Much like Example #3 in Part II, in which the human attorney mistakenly added language to the contract he was drafting, if an AI program is misaligned with its user’s goals, then it may draft contracts that do not reflect those goals. Both general intuition and contract law theory suggest that in a scenario like Example #3, the parties to the contract should not be bound by terms to which they did not assent. Similarly, in the case of misaligned AI contracting software, intuition suggests that it would be unethical to bind a party to an agreement if the AI model that contracted on their behalf did so in a manner that did not align with the user’s intentions.

C. Who Is Liable or Accountable?

If and when AI-assisted or wholly automated contracting goes wrong, who should we hold liable for breached contracts? Would we want to differentiate between the AI developer, the human who “hired” the AI to contract on their behalf or otherwise used the model to contract, and the AI model itself?

These questions are especially difficult to answer because traditional liability frameworks are designed with an inherent assumption that a human decisionmaker caused the alleged harm.¹²⁵ In the contracting setting, we would hold this human decisionmaker accountable for their breach of a contractual promise. If AI functions as a contracting agent, however, a human may not have made decisions that directly caused the complaining party’s harm. If an AI contracting program enters into agreements on a human’s behalf, that may not be enough under traditional liability frameworks to justifiably say that the human caused the alleged harm and hold them liable for it.

For similar reasons, it also appears unreasonable to hold an AI developer liable for breaches of contracts that its AI contracting software simply aided in drafting. To oversimplify, in order to prove causation of harm due to a breached contract, a plaintiff must demonstrate that the defendant’s breach was more than just an actual cause of the plaintiff’s harm.¹²⁶ Rather, the plaintiff has a higher burden: they must prove that the defendant’s act was the proximate cause of their harm.¹²⁷ To demonstrate proximate cause, the plaintiff must show that the harm was a foreseeable consequence of the defendant’s breach of contract.¹²⁸ In the AI context, a developer and its AI software may be actual, or but-for, causes of the harm suffered by a party who contracts with the software. However, the broad applicability of AI contracting software and its limitless potential uses suggest that, in many cases, the developer’s creation of the software would not be the legal, or proximate, cause of the injury because the alleged harm was not foreseeable.

Given these uncertainties about holding either the user or developer of AI-driven contracting software accountable, a plaintiff’s final potential avenue in a breach of contract claim might involve asserting that the AI program itself is liable for the harm. However, while holding the contracting algorithm liable may initially appear to be a plausible approach, it poses two serious concerns.

First, there is no legal precedent for holding a completely nonhuman entity liable for a person’s harm. Although corporations have been found liable for various harms, they are not analogous to AI-powered software programs. As “legal fictions,” corporations achieve legal personhood by “acting” through the actions of their human agents (that is, their officers, directors, promoters, and employees).¹²⁹ AI contractors differ significantly from corporations and operate in an almost entirely opposite manner. Instead of operating through human agents, AI software operates on behalf of humans. As a result, efforts to attribute liability to AI software by drawing analogies to corporate liability may be both inaccurate and misguided.

Second, if an AI model is held liable for contract breaches and required to pay damages to compensate for the resulting harms, this could expose AI software developers to above average or substantial levels of risk.¹³⁰ This increased risk may discourage AI developers from investing in further innovation, fearing that their investments could be lost to breach of contract, product liability, or other lawsuits. Additionally, if AI companies or algorithms were exposed to liability in this way, potential entrants to the AI contracting industry might hesitate, hindering further technological advancements. This suppression of innovation could cause greater harm to society than that posed by the inability of those alleging harm from breached contracts to obtain damages.

Thus, preserving innovation and investment into AI technology and its legal applications may involve specially protecting AI software, its users, and its developers from liability for harm-causing AI contracts—or, at the very minimum, maintaining existing standards of proof that prevent plaintiff-victims with lower socioeconomic statuses from securing damages in these types of cases.¹³¹ Under the current legal framework, only those individuals with higher socioeconomic statuses would be able to secure the costly expert testimony needed to demonstrate that an AI’s contract drafting did not satisfy the standard cost-benefit analysis used in determining liability in product warning, instruction, or design liability cases.¹³² Lowering the burden of proof would combat this issue, but such a change is unlikely to occur as it would expose AI software, its developers, and its users to substantial liability due to the highly unpredictable nature of AI-created risks.¹³³ Although there are numerous instances in recent history when the American public has accepted negative consequences for a minority group to achieve broader benefits for society as a whole,¹³⁴ the benefits of AI contracting do not outweigh its disproportionate harms.

Another issue in the context of assigning liability for AI contracting-related harms is allocating fault between the multiple parties that were involved in the contract’s creation and implementation. Parsing out which party should be held liable—whether it be the AI software itself, its designer, seller, or user, or another party altogether—inherently includes a significant policy decision as to how society chooses to (dis)incentivize AI technology’s development, usage, and applications.¹³⁵

D. Data Privacy and Security Concerns

When you log into ChatGPT to ask it a question, the prompt that you send the model does not stay on your laptop. It does not even stay on ChatGPT’s webpage.¹³⁶ By the time your query has been answered by the LLM (which is within seconds), your information is long gone—out into the ether of wherever OpenAI stores the many gigabytes of data it uses to train its AI models.¹³⁷ In reality, the information likely ends up in a remotely located and highly classified data center, where it sits on a server until OpenAI uses it to train its next LLM.¹³⁸

The average person may not care that their question asking ChatGPT to craft a new diet for them may get stored somewhere.¹³⁹ However, sophisticated legal clients commonly include their proprietary information—such as property addresses, purchase prices, and highly technical engineering or software information—in high-level contracts. Thus, legal clients are typically very protective of the private information in their contracts and subsequently include confidentiality clauses in their agreements to safeguard against disclosure to third parties.¹⁴⁰

For cases in which legal clients have highly sensitive information, AI’s “black box” can become a major issue. The “black box” problem refers to the fact that we are unable to see how LLMs make their decisions.¹⁴¹ Although the inputs and outputs of LLMs are observable, given the algorithms’ ever-evolving nature, their internal workings are a mystery—including what input data they retain.¹⁴² AI models’ mysterious inner workings may interfere with the efficacy and implementation of AI in the contract redlining and negotiation space because legal clients who are protective of their proprietary information may object to an AI model’s use in the contracting process. Even if a law firm used an “internal” AI software program, clients with sensitive information may not be comfortable with such a program because their information would be stored within the firm’s model for perpetuity.

There is an inherent tension between training an LLM and protecting clients’ confidential information. LLM models are trained on inputted data—and they improve if provided with greater quantities of training data.¹⁴³ Therefore, without clients who are willing for their information to be input into an LLM, the model’s efficacy will not improve. This may create problematic incentives for law firms to encourage their clients to commingle their sensitive information with that of other clients in the firm’s AI model in order to produce a better-quality software program for the firm.

Finally, LLMs’ greatest skill is their ability to recognize patterns in data. With more and more sensitive client information inputted into and stored by an LLM, the potential for an AI model to identify connections between data increases. In the case of an outsourced AI model not owned by a law firm, these recognized patterns may be disclosed to third parties for nefarious purposes. For instance, an LLM may analyze contracting patterns to determine which companies are economically successful, leading a third party to misappropriate this information and engage in fraudulent or deceptive dealings. In a more alarming scenario, third parties who gain access to confidential company addresses or security details that an LLM extracted from contracts—such as the location of a technology company’s classified data center—could use this information to break into the facility and steal servers.

V. AI: ARTIFICIAL INTELLIGENCE OR ACCURACY ISSUES?

Artificial intelligence is widely known to “hallucinate,” or misinterpret patterns in its data and create inaccurate or nonsensical outputs.¹⁴⁴ When an LLM hallucinates, it can fabricate legal cases, contradict itself, or provide outright wrong answers to questions.¹⁴⁵ In the contracting space, minute missteps when negotiating or redlining an agreement can have enormous consequences.¹⁴⁶ Therefore, AI’s tendency to hallucinate presents a major barrier to its successful implementation as a contractor. Given its pattern recognition functionality, AI is also known to provide different answers to the same question if it is asked multiple times, with slightly different wording, or by different people. These inaccuracies and inconsistencies are unacceptable in a detail-oriented field such as contract law, where “the devil is in the details.”

Furthermore, there are currently no regulatory compliance standards that would require AI models to be regularly updated with new case law, statutes, and other sources of law. On the other hand, state bar associations require attorneys to remain knowledgeable about updates in the law and complete continuing legal education (“CLE”) courses.¹⁴⁷ The nonexistence of regulation that would mandate AI models to remain up to date on new laws presents major challenges in the contracting space. Just like an attorney who refuses to complete their CLEs, an AI model that is not fully updated on what the current law is cannot adequately contract or negotiate for a client. Even if regulations were eventually implemented that required regular updates to AI models so that they included new case law, statutes, and other laws, this would be difficult to administer. Since it would be incredibly difficult, if not impossible, for an AI model to be instantaneously updated as new laws came into effect, this time lapse means that these models will always be somewhat out of date and not fully updated on the newest laws. Additionally, such regulations, if they came into effect, would place immense compliance costs on AI developers to continually update their models and may even discourage certain developers from entering the legal contracting space altogether.

Finally, LLMs are not sufficiently accurate to be used in contracting because of their technical limitations. AI technology lacks the ability to exercise judgment and is known to struggle with customization, context, and complexity (“CCC”)¹⁴⁸—all of which are highly relevant aspects of contracting. In fact, CCC is a major reason in-house counsel as a general concept exists; businesses that are highly technical or complex in nature often prefer to have their own attorneys who are better suited than outside counsel to understand the company’s unique situation and needs. Thus, AI would not serve well as a legal assistant because it would not understand the context or complexity of a prospective client’s specific contracting needs.

VI. LEGAL PROFESSION CHALLENGES

As fiduciaries for their clients, lawyers are held to a high professional standard. Subsequently, lawyers’ use of AI technology poses unique challenges to the legal profession, particularly in the context of contract drafting and negotiation.

A compelling argument can be made that an attorney who relies on AI technology to draft contracts violates their professional duties of competence and diligence.¹⁴⁹ Although the AI-toting attorney may argue that an LLM is a tool that they use to aid their work, much like Microsoft Word or Excel, such an analogy is misplaced. Generative AI differs from these types of technologies because it allows lawyers to create substantive work product with minimal effort.¹⁵⁰ Therefore, relying on ChatGPT for contract drafting may undermine an attorney’s obligation to provide competent and diligent representation for their client.

Furthermore, an attorney’s reliance on AI technology to draft and negotiate contracts may create communication gaps between the attorney and their clients. If an attorney blindly accepts an LLM’s output as the best possible redline or negotiation strategy in a given situation, the attorney may be incapable of explaining to their client why they undertook the AI-suggested action.¹⁵¹ This blind acceptance of an AI model’s output is very likely if an attorney uses an AI model to contract because we often cannot look into an LLM’s inner workings or see why they generate the outputs that they do.¹⁵² The black box problem exacerbates this duty of communication issue if an AI model executes contracts without humans involved in the contract drafting and negotiation process, as the model would provide little to no legal reasoning to its client to explain its outputted action.

As mentioned in Section IV.D, serious duty of confidentiality concerns arise when clients’ data is input into an LLM.¹⁵³ Even if placeholder information is used in an effort to protect confidential client data, an AI model may be able to use its ability to detect patterns to extract confidential information from the provisions and context that are inputted into it. This is especially possible if an attorney or law firm inputs substantial amounts of client data into an AI model, as in the case of AI-driven contract lifecycle management programs or internal AI programs more broadly.

Finally, AI is not suited for the ethical and emotional dilemmas that are inherent in legal contracting and negotiation. Attorneys regularly encounter ethically and emotionally intense situations when negotiating and contracting for their clients. If an AI model is tasked with contracting in an ethically ambiguous situation, it would lack the human touch necessary to appropriately respond. Even if the model was trained to provide canned outputs in specific scenarios, it would be impossible for the model’s programmers to predict all potential ethical dilemmas that the AI model may encounter in practice. Additionally, in emotionally intense contracting settings, such as mergers and acquisitions, partnership agreements, or certain real estate transactions, clients are likely to value the human touch of an attorney over the detached and indifferent nature of an AI model.

VII. EMPIRICAL RESEARCH: “HIRING” CHATGPT IN A CONTRACT NEGOTIATION

To test AI’s current capabilities in the contract drafting and negotiation space, the author conducted novel empirical research using OpenAI’s Application Programming Interface (“API”). The experiment was designed to imitate “hiring” ChatGPT¹⁵⁴ as a legal assistant by tasking it to assist with a client’s negotiation of a commercial real estate lease. To investigate whether ChatGPT suggests different negotiation recommendations depending on its type of client, the author selected four general client types for this experiment: (1) an individual; (2) a small, privately held corporation; (3) a large, publicly held corporation; and (4) a nonprofit organization. ChatGPT was not provided with additional information about each client, and the rest of the experiment—including the exact prompt language, base contract structure, and output scale—was held constant across all client types in order to control for differences in the AI model’s responses.

A commercial real estate lease was selected for this experiment because all four of the selected client types could plausibly negotiate and enter into a commercial real estate lease as a tenant. To simulate a real-world commercial real estate contract, the author provided ChatGPT with thirty generic boilerplate provisions typically found in a commercial real estate lease, such as assignment, security deposit, renewal option, and maintenance provisions.¹⁵⁵ For each provision, the AI software was asked whether it would recommend renegotiation to its client. To facilitate objective comparisons between ChatGPT’s responses for different client types, the query solicited numerical responses by specifically asking ChatGPT to output its response on a scale from 0 to 100. On this scale, 0 indicated that ChatGPT would recommend to the client that the language was acceptable and should not be renegotiated, while 100 signified that ChatGPT would recommend that the language was unacceptable and the client should renegotiate the provision.¹⁵⁶ The 0 to 100 scale was chosen to prevent ChatGPT from outputting renegotiation advice in plain English. With numeric outputs, the author did not need to make subjective judgments about the quality of ChatGPT’s negotiation recommendations—which would have been necessary if they were in plain English—in order to compare the outputs across client types.

ChatGPT was selected as the AI chatbot for this experiment due to its popularity.¹⁵⁷ Because ChatGPT is pervasive, the results of an experiment utilizing it are more easily generalized to real-world applications and settings than the results of an experiment conducted with a less popular AI program. Put simply, the author chose to use ChatGPT for this research because this experiment seeks to replicate laypeople’s use of AI to negotiate contracts and laypeople are more likely to use ChatGPT than other AI programs.

The author also selected OpenAI’s API to conduct this experiment rather than prompting ChatGPT manually because the API provided an efficient and cost-effective method of testing the author’s algorithmic discrimination hypothesis.¹⁵⁸ In general, an API is a set of protocols that connects software programs, devices such as computers, and applications by enabling them to more easily communicate with each other.¹⁵⁹ APIs are useful because they enable a researcher to automate repetitive tasks such as scraping information from webpages or, in this case, prompting ChatGPT repetitively.¹⁶⁰

To conduct this experiment, the author drafted Python code that prompted ChatGPT for each client-provision pairing through its API and saved the AI model’s outputted numbers in an Excel file. Notably, iterating prompts through OpenAI’s API enabled the use of its log probabilities (“logprobs”) feature to construct more accurate data as compared with the data that would result from manual prompting.¹⁶¹ Logprobs is a feature in OpenAI’s API that responds to a particular prompt with both ChatGPT’s most likely outputs and the corresponding log probabilities for those responses.¹⁶² In essence, the logprobs feature enables a researcher to determine the estimated probability that ChatGPT would respond to any given prompt with particular responses.¹⁶³ For instance, in the context of this experiment, when ChatGPT is tasked with advising an individual client about whether to renegotiate the “Premises” provision of the provided lease agreement, the AI program is 78.629% likely to output “25,” 11.181% likely to output “50,” and 6.966% likely to output “75” on the 0 to 100 scale.¹⁶⁴

The logprobs feature allowed the author to construct a weighted response output for each inputted client-provision pairing that represents ChatGPT’s landscape of potential responses in a single number. The author created each client-provision prompt’s corresponding weighted response by utilizing the five most common responses for each prompt. For example, the mathematics behind the average weighted response when ChatGPT advises an individual client about the “Premises” provision of the lease is shown in Figure 1 and described below.

Figure 1. Weighted Response Calculation for Individual Client “Premises” Provision

First, each of the top five response values were multiplied by their corresponding probabilities, which were extracted from the log probabilities provided by OpenAI’s API. Then, these individually weighted values (shown in Figure 1 under the “Response × Probability” column) were summed. For the “Premises” provision and individual client prompt in Figure 1, this sum totaled approximately 31.095. Then, the individual probabilities of the five most likely outputs were summed; in Figure 1’s example, that total equaled approximately 0.9798, or 97.98%. This total conveys that approximately 97.98% of ChatGPT’s responses to this particular client-provision prompt were either 25, 50, 75, 20, or 85. Finally, the “Response × Probability” sum (approximately 31.095) was divided by the probability sum (approximately 0.9798) to calculate the weighted average response for this particular client-provision combination, or 31.73. Therefore, when ChatGPT is tasked with assisting an individual client and the provided provision of the lease agreement is the “Premises” provision, the AI program’s weighted average response is 31.73. Qualitatively, a result of 31.73 on the 0 to 100 scale facially suggests that ChatGPT may not be highly likely or enthusiastic to recommend to the individual that they should renegotiate this provision. However, the nature of this experiment was to derive comparisons between client types, so although the 31.73 value might suggest that ChatGPT is unlikely to be a zealous advocate,¹⁶⁵ this value must be compared with the AI program’s average weighted responses for other client types with the same “Premises” provision to be able to draw substantive conclusions about ChatGPT’s propensity to discriminate against certain types of legal clients.

As demonstrated above, this math derived a single numerical response for each client-provision pairing, facilitating objective comparisons between ChatGPT’s outputs when it is “hired” by different clients. The individual client’s average weighted response was used as a baseline measure by taking each non-individual client response and subtracting the corresponding individual response for the same lease provision to calculate a difference between the two values for each provision. Then, these difference calculations (one value for each provision of the lease agreement) were plotted. The visual representations of the differences between the average weighted responses for an individual client and a small corporation, large corporation, and nonprofit organization were constructed by plotting these differences on the following histogram plots.¹⁶⁶

Small Corporation Versus an Individual as a Client

Figure 2. Histogram of Differences in Average Weighted Responses Between a Small Corporation and an Individual Client

The histogram of differences between ChatGPT’s average weighted responses for a small corporation and those of an individual client demonstrates a few takeaways. First, the differences are clustered around zero, where zero indicates no numerical difference between ChatGPT’s responses when hired by either an individual or a small corporation. This finding suggests that, for the most part, ChatGPT treats individual and small

corporate clients similarly when tasked with advising them in a contract negotiation.

However, the histogram includes some instances of large differences between individual and small corporate responses, such as one provision where ChatGPT output a renegotiation suggestion for a small corporation that was over thirty points larger than the recommendation it provided the individual client. Notably, there were no instances of ChatGPT outputting a weighted response for the individual client that was greater than or equal to ten points higher than its corresponding small corporate output. On the other hand, there were multiple provisions where ChatGPT output renegotiation suggestions for small corporate clients that were ten or twenty points higher than the provision’s corresponding individual-client responses. These provisions, in addition to the rightward-skewed shape of the histogram in Figure 2, suggest that ChatGPT tends to recommend renegotiation for small corporate clients more often and to a greater extent than it does for individual clients.

Large Corporation Versus an Individual as a Client

Figure 3. Histogram of Differences in Average Weighted Responses Between a Large Corporation and an Individual

Figure 3, which shows the differences between ChatGPT’s responses for large corporate clients and individual clients, demonstrates similar patterns. Much like the small corporate client example in Figure 2, Figure 3 includes clustering around zero. This suggests that for a variety of provisions, ChatGPT will provide similar renegotiation recommendations for both individual and large corporate clients.

However, Figure 3 also includes the most dispersed results of the three client comparisons conducted in this experiment. The histogram includes a wide variety of difference values, most of which are relatively numerically different from one another—so different, in fact, that they fall into individual difference bins in Figure 3’s histogram. The dispersed nature of these results suggests that, while there is some clustering around zero, ChatGPT provides a wider range of negotiation recommendations when advising large corporate clients compared with other client types. This variability may indicate that ChatGPT’s training data assumes that large public corporations are more varied and complex than smaller, privately held corporations¹⁶⁷ and subsequently require a broader variety of negotiation advice or have greater market power to exert its will in a contract negotiation.¹⁶⁸ Additionally, the broader spread of the differences in responses for large corporate clients as compared with individual clients might also suggest that ChatGPT views large corporate clients as having more nuanced or varied negotiation capabilities and needs compared with individual clients.

Nonprofit Organization Versus an Individual as a Client

Figure 4. Histogram of Differences in Average Weighted Responses Between a Nonprofit Organization and Individual Client

Figure 4 visualizes the difference in weighted responses for a nonprofit organization as ChatGPT’s client as compared with an individual as its client. Here, we see the strongest clustering of results around zero of the three client comparisons studied in this experiment.¹⁶⁹ This suggests that, between corporations and nonprofit organizations, ChatGPT considers a nonprofit to be most analogous to an individual in the contracting space. This makes some intuitive sense if ChatGPT assumes that both individuals and nonprofit organizations tend to have less financial and political resources, market power, and influence over negotiations than large public or small private corporations.¹⁷⁰

However, despite this stronger clustering of differences around zero for nonprofit organizations, the histogram in Figure 4 continues to demonstrate the same trend seen for both corporation types: a rightward shift. This again suggests that ChatGPT favors nonprofit organizations over individuals in the negotiation space by more strongly or commonly recommending renegotiation to them, potentially because the model perceives individuals as having less power than nonprofit organizations to effectively negotiate for favorable provisions.

D. Overall Trends and Conclusions

Figure 5. Histogram of Differences in Average Weighted Responses Across All Four Client Types

Figure 5 is an overlay of the results from Figures 2, 3, and 4. Taken as a whole, while there is some clustering around zero, the rightward shift in the data demonstrates that ChatGPT tends to recommend renegotiation to (1) large, public corporations; (2) small, privately held corporations; and (3) nonprofit organizations more often and to a greater extent than it does when its client is an individual. Additionally, there are few occurrences of negative values on the combined histogram, which represent when ChatGPT outputted an individual client renegotiation value that was higher than the value outputted for any of the other client types for a given provision. Collectively, these trends suggest that ChatGPT may discriminate against individuals when “hired” to consult a contract negotiation by recommending

less favorable terms or negotiation strategies to an individual than it would to other types of clients.¹⁷¹

Interestingly, the minimum differences for the small corporation, large corporation, and nonprofit organization clients were -5.82, -8.42, and -5.36, respectively. These values represent the provisions for which ChatGPT most strongly recommended negotiation to an individual client as compared with other client types. Conversely, the maximum differences, which represent the instances when ChatGPT most strongly recommended the small corporation, large corporation, and nonprofit organization to negotiate as compared with an individual client, are significantly larger than the minimum differences. The maximum differences for the small corporation, large corporation, and nonprofit organization were 39.28, 22.68, and 29.43, respectively. Taken together with each client type’s mean differences (3.98, 2.99, and 3.71, respectively), this data demonstrates the systematic disadvantage in negotiation advising that individual clients experience compared with their corporate or nonprofit counterparts when using ChatGPT to assist in a contract negotiation.

E. Shortcomings

Although the findings of this empirical study are intriguing, there are some important caveats to note as well. First, the author chose to specifically use OpenAI’s GPT-4 Turbo model for this experiment, meaning that its results may not be readily generalizable to other OpenAI or AI models. Additionally, to best balance creativity with coherence, the author set the API’s temperature to 0.7. Temperature is a parameter value that controls how often ChatGPT outputs a less likely response; in essence, it is a measure of how random or creative the model’s responses are.¹⁷² The author initially tested the experiment with GPT-4 Turbo’s default temperature of 1 but ultimately tamped the parameter down to 0.7 in an effort to replicate the deterministic nature of legal advising.¹⁷³ The author also decided to use only the top five logprobs, rather than more, in conducting this analysis.¹⁷⁴ Both the temperature and top logprob decisions were made in an effort to replicate an individual user’s experience on ChatGPT while maintaining consistency across various API code executions.¹⁷⁵

Unfortunately, while these decisions were necessary to conduct the research, they also inherently shaped its results. Any modification of the temperature or number of requested logprobs alters ChatGPT’s renegotiation recommendations. Furthermore, this style of research does not easily facilitate demonstrating statistically significant findings—such as with a p-value used in traditional statistical analyses—because the model generates different outputs each time the code is run. As a result, these findings are not readily replicable, which is an unfortunate nature of conducting social science experimentation with the black boxes that are AI models.¹⁷⁶

Beyond technical limitations, other factors may impact the generalizability of this study’s findings. Only one type of contract, a lease agreement with thirty boilerplate provisions, was used in this research. Future scholars can expand upon this work by incorporating new and additional types of contracts and more detailed or varied provisions into this study’s framework to investigate if AI models discriminate against individuals when contracting in different contexts or with multiple types of contracts. Additionally, given that ChatGPT is a large language model, it is likely that the exact phrasing of the prompts used in this research impacted the model’s recommendations. Therefore, future scholarship can include a greater diversity of prompt language to determine if these findings hold across different prompting styles and approaches.

Similarly, additional research can incorporate more specific details about the AI model’s client when soliciting negotiation advice, whether in the contract itself or by expanding on the details included when contextualizing the prompt for the AI model. Inclusion of greater detail in a future study may determine if the use of specific company or individual names or other information results in similar algorithmic discrimination patterns. Greater contextualization is also more likely to align with real-world uses of AI modeling in contract negotiation, as the user would probably provide information about themself, the other party, and the deal at hand while soliciting assistance from an AI model.

Additionally, another version of this research might request AI’s assistance in renegotiating a contract that initially includes blatantly favorable (or unfavorable) provisions for the client. This arrangement may demonstrate different findings than an experiment conducted with relatively neutral starter provisions would, like those used here. The author intentionally used neutral lease provisions in this case to facilitate easier comparisons between client types and force ChatGPT to rely on its training data in making renegotiation recommendations rather than following an implicit suggestion to renegotiate provisions that are blatantly unfavorable (or vice versa).

Another alternative experiment design might use iterative follow-up prompts, rather than a single prompt, to solicit advice from the AI model because the language and structure of the prompt used to solicit advice may influence the AI model’s recommendations. For example, uploading a contract to ChatGPT and asking it a leading question such as “Should I negotiate Provision A?” may result in the AI model suggesting renegotiation more often or to a stronger degree than a broadly phrased prompt that asks ChatGPT what it thinks about the provision. Furthermore, this experiment used a numeric scale to gather ChatGPT’s outputs in a form that was easily and objectively comparable across client types. The 0 to 100 scale used in this Note’s empirical framework inherently assumes that this continuum is representative of the quality and strength of the renegotiation advice that ChatGPT would output in plain English to a real-world client. In real life, an AI model’s output would be substantive—it would tell the user in plain English what it thinks of the provision, whether or not to renegotiate it, and why. Therefore, it may be worthwhile for future research to solicit and examine substantive outputs and assess whether those outputs are equally clear, definite, and confident across different client types.

Although this study’s findings have limitations that are common to empirical research, this Note offers novel insights into algorithmic discrimination in the contracting space. Plausibly, ChatGPT discriminates against individuals when tasked with advising them in a contract negotiation—as evidenced by the AI model suggesting renegotiation to individual clients less often and to a smaller degree than it does when advising other types of clients.

As noted above, additional scholarship can expand upon the research implemented in this Note to strengthen this conclusion. If future research confirms algorithmic discrimination in the contracting space, then AI models must be retrained to prevent further exacerbation of existing inequalities. If AI models discriminate against individuals as their contracting client, this behavior may worsen inequities between those who have the resources to renegotiate favorable contract terms (such as corporate firms) and those who do not (individuals, for example) and are therefore more likely to rely on AI as an accessible contract negotiation tool.¹⁷⁷

VIII. ENOUGH NEGATIVITY—WHAT IS AI GOOD AT?

While AI has a plethora of disadvantages that hinder its applicability to contract drafting and negotiation, it does have advantages in limited legal applications. For instance, given its ability to summarize information quickly and accurately, AI is a prime candidate for administrative, clerical, or other summary tasks. A number of these types of AI applications already exist, such as Evisort,¹⁷⁸ a contract workflow management program. AI can also streamline a law firm’s tracking of its billable hours (e.g., Clio AI¹⁷⁹). Furthermore, AI technology can prove useful in speeding up legal research by summarizing documents, as seen with LexisNexis’s Protégé.¹⁸⁰ As a rule of thumb, AI is best suited for tasks that do not require judgment. Unlike billing or other administrative tasks, contract drafting and negotiation requires immense judgment, which is why AI technology is better suited for legal uses other than contracting.

CONCLUSION

Artificial intelligence technology has taken the world by storm in recent years. Nearly every industry has experimented with new and innovative applications of AI technology, and the legal profession is no exception. Despite this enthusiasm, transactional attorneys should pause and seriously consider the negative implications and serious challenges involved when applying AI technology to the contracting space before they attempt to implement AI models into their practice. At the same time, it is important to remain mindful of the distinction between the “practice of the . . . [law]” and the “business of . . . [a law] firm[].”¹⁸¹ Given the contract law issues, equity concerns, legal profession challenges, and accuracy problems that abound when AI models draft and negotiate legal contracts, AI may be better suited to assist attorneys with administrative business tasks rather than the practice of law itself. This limitation on the use of AI in the contracting space is further underscored by ChatGPT’s tendency to discriminate against individuals when asked to assist them in contract negotiations, as demonstrated by the empirical research presented in this Note.

On the other hand, those determined to use AI in the contracting space may find it more useful in an in-house setting than in a traditional law firm. The typical in-house counsel functions as a “jack-of-all-trades” for their employer, managing multiple projects and legal practice areas simultaneously. Additionally, in-house counsel usually manages standard form contracts, particularly in cases when their business holds significant market power in negotiations with other parties. Maintaining a consistent client (i.e., the business) and contractual structure over multiple contract cycles would allow an AI program to detect familiar patterns and better understand the context and complexity needed to tailor contracts to the business’s needs. Furthermore, an experienced human in-house attorney may be able to manually adjust for any discriminatory patterns in an AI model’s outputted negotiation suggestions and provisions. Finally, the research presented in this Note indicates that large public and small private corporations face a lower risk of AI-driven discrimination in contract drafting and negotiation compared with other clients, such as individuals. Therefore, in an in-house attorney’s busy, consistent, and controlled setting, AI models may prove to have some utility.

However, technological innovation has its limits, and AI models are not yet suited for broad applications in legal contracting and negotiation. While this author is eager to see how AI developers and legal professionals address the current challenges of applying AI to contract drafting and negotiation—particularly, AI’s discriminatory tendencies—she is also reassured that transactional attorneys still enjoy some level of job security, at least for now.

Attachment A: Commercial Real Estate Lease Provisions

PREMISES

Landlord hereby leases to Tenant and Tenant hereby leases from Landlord those certain premises (the ‘Premises’) consisting of approximately _______ square feet located at _______________________, as more particularly described in Exhibit A attached hereto and incorporated herein by reference.

TERM.

The term of this Lease shall be for a period of ______ years, commencing on ____________, 20___ (the ‘Commencement Date’) and ending on ____________, 20___ (the ‘Expiration Date’), unless sooner terminated as provided herein.

BASE RENT.

Tenant shall pay to Landlord as Base Rent for the Premises, without any setoff or deduction, the annual sum of $_______________ payable in equal monthly installments of $_______________ in advance on the first day of each month during the Term.

SECURITY DEPOSIT.

Upon execution of this Lease, Tenant shall deposit with Landlord the sum of $_______________ as security for the faithful performance by Tenant of all terms, covenants, and conditions of this Lease. If Tenant fails to pay rent or other charges due hereunder, or otherwise defaults with respect to any provision of this Lease, Landlord may use, apply or retain all or any portion of the Security Deposit to cure such default or to compensate Landlord for any loss or damage resulting from such default.

PERMITTED USE.

Tenant shall use and occupy the Premises solely for _______________________ and for no other purpose without the prior written consent of Landlord.

OPERATING EXPENSES.

In addition to Base Rent, Tenant shall pay as Additional Rent Tenant’s proportionate share of all Operating Expenses. ‘Operating Expenses’ shall mean all costs and expenses incurred by Landlord in connection with the ownership, management, operation, maintenance, repair, and replacement of the Building and Property, including but not limited to: property taxes and assessments, insurance premiums, utilities, management fees, common area

maintenance, landscaping, and repairs and maintenance not required to be performed by Tenant.

MAINTENANCE AND REPAIRS.

Landlord shall maintain in good repair the structural portions of the Building, including the foundation, exterior walls, structural portions of the roof, and common areas. Tenant shall, at Tenant’s sole cost and expense, maintain the Premises in good condition and repair, including all interior non-structural portions of the Premises, such as doors, windows, glass, and utility systems exclusively serving the Premises.

ALTERATIONS AND IMPROVEMENTS.

Tenant shall not make any alterations, additions, or improvements to the Premises without the prior written consent of Landlord, which consent shall not be unreasonably withheld for non-structural alterations costing less than $____________. All alterations shall be made at Tenant’s sole cost and expense and shall become the property of Landlord upon the expiration or termination of this Lease.

INSURANCE REQUIREMENTS.

Tenant shall, at Tenant’s expense, obtain and keep in force during the Term of this Lease a policy of commercial general liability insurance with coverage of not less than $____________ per occurrence and $____________ general aggregate. Tenant shall also maintain property insurance covering Tenant’s personal property, fixtures, and equipment. Landlord shall be named as an additional insured on Tenant’s liability policies.

INDEMNIFICATION.

Tenant shall indemnify, defend, and hold Landlord harmless from any and all claims, damages, expenses, and liabilities arising from Tenant’s use of the Premises or from any activity permitted by Tenant in or about the Premises. Landlord shall indemnify, defend, and hold Tenant harmless from any and all claims, damages, expenses, and liabilities arising from Landlord’s negligence or willful misconduct.

ASSIGNMENT AND SUBLETTING.

Tenant shall not assign this Lease or sublet all or any part of the Premises without the prior written consent of Landlord, which consent shall not be unreasonably withheld. Any assignment or subletting without such consent shall be void and shall constitute a default under this Lease.

DEFAULT AND REMEDIES.

The occurrence of any of the following shall constitute a material default and breach of this Lease by Tenant: (a) failure to pay rent when due if the failure continues for ____ days after written notice has been given to Tenant, (b) abandonment of the Premises, or (c) failure to perform any other provision of this Lease if the failure is not cured within ____ days after written notice has been given to Tenant. Upon any default, Landlord shall have all remedies available under applicable law.

QUIET ENJOYMENT.

Landlord covenants that Tenant, upon paying the rent and performing the covenants herein, shall peacefully and quietly have, hold, and enjoy the Premises during the Term hereof.

ENTRY BY LANDLORD.

Landlord reserves the right to enter the Premises at reasonable times to inspect the same, to show the Premises to prospective purchasers, lenders, or tenants, and to make necessary repairs. Except in cases of emergency, Landlord shall give Tenant reasonable notice prior to entry.

SIGNAGE.

Tenant shall not place any sign upon the Premises without Landlord’s prior written consent. All signs shall comply with applicable laws and ordinances.

COMPLIANCE WITH LAWS.

Tenant shall comply with all laws, orders, ordinances, and other public requirements now or hereafter affecting the Premises or the use thereof. Landlord shall comply with all laws, orders, ordinances, and other public requirements relating to the Building and common areas.

ENVIRONMENTAL PROVISIONS.

Tenant shall not cause or permit any Hazardous Materials to be brought upon, kept, or used in or about the Premises by Tenant without the prior written consent of Landlord. Tenant shall indemnify, defend, and hold Landlord harmless from any and all claims, judgments, damages, penalties, fines, costs, liabilities, or losses arising from the presence of Hazardous Materials on the Premises which are brought upon, kept, or used by Tenant.

SUBORDINATION.

This Lease is and shall be subordinate to all existing and future mortgages and deeds of trust on the property. Tenant agrees to execute any subordination, non-disturbance and attornment agreements required by any lender, provided that such lender agrees not to disturb Tenant’s possession of the Premises so long as Tenant is not in default under this Lease.

FORCE MAJEURE.

Neither party shall be deemed in default hereof nor liable for damages arising from its failure to perform its duties or obligations hereunder if such failure is due to causes beyond its reasonable control, including, but not limited to, acts of God, acts of civil or military authority, fires, floods, earthquakes, strikes, lockouts, epidemics, or pandemics.

HOLDOVER.

If Tenant remains in possession of the Premises after the expiration or termination of the Term without Landlord’s written consent, Tenant shall be deemed a tenant at sufferance and shall pay rent at _____ times the rate in effect immediately prior to such expiration or termination for the entire holdover period.

SURRENDER OF PREMISES.

Upon expiration or earlier termination of this Lease, Tenant shall surrender the Premises to Landlord in good condition, ordinary wear and tear and damage by fire or other casualty excepted. All alterations, additions, and improvements made to the Premises by Tenant shall remain and become the property of Landlord, unless Landlord requires their removal.

DISPUTE RESOLUTION.

Any dispute arising under this Lease shall be first submitted to mediation, and if mediation is unsuccessful, then to binding arbitration in accordance with the rules of the American Arbitration Association. The costs of mediation and arbitration shall be shared equally by the parties.

NOTICES.

All notices required or permitted hereunder shall be in writing and may be delivered in person (by hand or by courier) or sent by registered or certified mail, postage prepaid, return receipt requested, or by overnight courier, and shall be deemed given when received at the addresses specified in this Lease, or at such other address as may be specified in writing by either party.

OPTION TO RENEW.

Provided Tenant is not in default hereunder, Tenant shall have the option to renew this Lease for ____ additional period(s) of ____ years each on the same terms and conditions as set forth herein, except that the Base Rent shall be adjusted to the then-prevailing market rate. Tenant shall exercise this option by giving Landlord written notice at least ____ days prior to the expiration of the then-current term.

OPTION TO EXPAND.

Subject to availability, Tenant shall have the right of first offer to lease additional space in the Building that becomes available during the Term. Landlord shall notify Tenant in writing of the availability of such space and the terms upon which Landlord is willing to lease such space. Tenant shall have ____ days from receipt of such notice to accept or reject such offer.

RELOCATION.

Landlord reserves the right, upon providing Tenant with not less than ____ days’ prior written notice, to relocate Tenant to other premises within the Building or Project that are comparable in size, utility, and condition to the Premises. In the event of such relocation, Landlord shall pay all reasonable costs of moving Tenant’s property and improving the new premises to substantially the same standard as the Premises.

PARKING AND TRANSPORTATION.

Tenant shall be entitled to use ____ parking spaces in the Building’s parking facility on a non-exclusive basis. Landlord reserves the right to designate parking areas for Tenant and Tenant’s agents and employees.

BUILDING RULES AND REGULATIONS.

Tenant shall comply with the rules and regulations of the Building adopted and altered by Landlord from time to time, a copy of which is attached hereto as Exhibit B. Landlord shall not be responsible to Tenant for the non-performance of any of said rules and regulations by any other tenants or occupants of the Building.

GOVERNING LAW.

This Lease shall be governed by and construed in accordance with the laws of the State of ______________. If any provision of this Lease is found to be invalid or unenforceable, the remainder of this Lease shall not be affected thereby.

ENTIRE AGREEMENT.

This Lease contains the entire agreement between the parties and supersedes all prior agreements, whether written or oral, with respect to the subject matter hereof. This Lease may not be modified except by a written instrument executed by both parties.

Attachment B: Excel Spreadsheet & Python Code

The Excel spreadsheet of OpenAI’s API outputs and the Python code used to obtain this data is on file with the author and available upon request.

99 S. Cal. L. Rev. 239

Download

Carly Snell*

*Executive Articles Editor, Southern California Law Review, Volume 99; J.D. Candidate 2026, University of Southern California Gould School of Law; Master of Public Policy Candidate 2027, University of Southern California Sol Price School of Public Policy; B.S., Mathematics, 2023, University of Arizona; B.A., Political Science, 2023, University of Arizona. I extend my sincere gratitude to Professor Jonathan H. Choi for his invaluable guidance, my friends and family for their unwavering support, and the editors of the Southern California Law Review for their hard work and dedication in preparing my Note for publication.