Stepping Up to the Plate: Understanding the “Nuts and Bolts” of AI

From late 2023 through early 2024, we published a series of blog posts on the “nuts and bolts” of artificial intelligence (AI) for legal professionals. In that series, we provided key definitions for AI terms – as defined by ChatGPT-4, which was the latest release of the product at that time (as of this writing, the current release is up to GPT-5.5).

We also discussed topics such as the types of bias in AI algorithms, privacy considerations associated with the use of AI, considerations related to transparency, explainability and interpretability of AI models, guidance on the use of AI from the American Bar Association (ABA), the current state of AI regulations, proven AI legal use cases, and the future of AI for legal. These were important topics to cover to help you understand the “nuts and bolts” of how AI works and how it impacts legal professionals.

Trends in AI move quickly, so it’s good to revisit them periodically to discuss new or evolved considerations. In fact, we’ve already done that – twice – for AI regulations with updates in 2025 and again earlier this year and will continue to do so. But regulations aren’t the only changes to know about – the considerations associated with the use of AI in the legal world continue to evolve. Many of those considerations relate to the ethical use of AI and ensuring defensibility of its use in legal practice.

So, we’re starting a new blog series for “stepping up to the plate” through best practices for AI ethics and defensibility in legal practice. In this initial post in the series, we will discuss the differentiation between AI overall and generative AI, and we’ll identify some of the current considerations impacting legal professionals in today’s landscape.

AI vs. Generative AI

When discussing the current AI landscape for the legal community, people tend to use the catch-all term of “AI”; however, many of the considerations and challenges legal professionals face today are related specifically to the use of generative AI technology.

Artificial intelligence (AI) is a broad term that refers to computer systems designed to perform tasks that typically require human intelligence, including pattern recognition, prediction, classification, automation, and decision-making. The term “artificial intelligence” was first formally coined in 1956 – 70 years ago! – during the Dartmouth Summer Research Project on Artificial Intelligence. Various forms and implementations of AI have literally been around for decades.

Generative AI is a specific subset of AI that creates new content such as text, images, audio, code, summaries, or analyses based on patterns learnt from large datasets. While GenAI as a concept dates also back several decades, its modern form emerged gradually through advances in machine learning and neural networks. The modern GenAI era accelerated after the introduction of the “transformer” architecture in 2017 through the landmark paper Attention Is All You Need by researchers at Google. Transformers enabled large language models to process and generate human-like text at unprecedented scale. This ultimately led to systems such as GPT-3 in 2020 and ChatGPT in 2022, which brought GenAI into mainstream public and enterprise use.

Why is the distinction important? Because traditional AI tools may operate within more predictable and rule-based parameters, whereas generative AI systems are probabilistic and capable of producing inaccurate or fabricated information that appears credible. Understanding the distinction is important for legal professionals because the risks, governance requirements, and reliability considerations can differ significantly between the two. Legal professionals tend to use the catch-all term of “AI” (we catch ourselves doing it too) when discussing these challenges when many of them are related specifically to GenAI.

Generative AI Considerations and Challenges for the Legal Industry

In today’s landscape, there are several considerations for legal professionals in the ethical use of generative AI, including:

AI Hallucinations in Case Filings

One of the most visible risks associated with generative AI in the legal profession is the potential for AI “hallucinations,” where an AI system generates inaccurate facts, fictitious case citations, or unsupported legal analysis that appears convincing on its face. Courts across the U.S. have increasingly confronted filings containing nonexistent cases, misstated holdings, and fabricated quotations generated by public AI tools.

These incidents raise concerns not only about attorney competence and ethical obligations, but also about the reliability of AI-assisted legal work product. Legal professionals must now consider how to supervise AI-generated content, implement verification procedures, and ensure that all citations, authorities, and factual assertions are independently validated before submission to a court or regulator.

Uploading Discovery ESI into Public LLMs

Another growing concern involves the possibility that opposing parties, outside counsel, experts, or vendors may upload produced discovery materials into public large language models for analysis or summarization. Discovery datasets often contain confidential business information, trade secrets, personally identifiable information, privileged communications, or regulated data.

Once uploaded into a public AI platform, organisations may lose visibility into how the data is stored, processed, retained, or potentially used for future model training. This risk has prompted increasing discussion about whether protective orders, clawback agreements, or discovery protocols should specifically address the use of generative AI platforms and restrict parties from loading produced ESI into public LLM environments without consent.

Defensible Validation of Generative AI Outputs

GenAI tools can produce useful summaries, chronologies, privilege descriptions, and issue analyses, but the outputs are probabilistic rather than deterministic. As a result, legal professionals face challenges in determining how to validate AI-generated work in a defensible manner.

Unlike traditional keyword searches or technology-assisted review workflows that may have established validation methodologies, there is still limited consensus around acceptable standards for testing and documenting generative AI accuracy. Organisations must consider issues such as human review requirements, sampling methodologies, audit trails, prompt documentation, and quality control procedures to demonstrate that AI-assisted legal work was appropriately supervised and reasonably reliable.

Disclosing the Use of AI

Legal professionals are also wrestling with whether, when, and how to disclose their use of GenAI tools in litigation and investigations. Some courts have adopted standing orders requiring attorneys to certify that AI-generated filings were reviewed for accuracy, while other jurisdictions have not imposed formal disclosure obligations. This creates uncertainty regarding whether AI usage should be affirmatively disclosed to opposing counsel, clients, regulators, or the court itself.

Attorneys must balance transparency considerations against concerns about revealing litigation strategy, work product, or internal workflows. The lack of uniform rules governing AI disclosure continues to create inconsistency and uncertainty across jurisdictions and practice areas.

The Rise of “Shadow AI”

The rapid adoption of GenAI has also contributed to the rise of “Shadow AI,” where employees use unsanctioned public AI tools outside approved governance and security frameworks. Lawyers, paralegals, investigators, and legal operations professionals may independently experiment with AI tools to summarise documents, draft communications, or analyse contracts without organizational approval or oversight.

This creates significant challenges for law firms and corporate legal departments attempting to maintain confidentiality, data security, compliance, and defensibility. In many organisations, governance policies, monitoring capabilities, and AI usage guidelines have not kept pace with employee adoption, increasing the risk of inconsistent practices and inadvertent exposure of sensitive legal information.

Discoverability of AI Prompt Histories

As GenAI becomes more integrated into legal workflows, questions are emerging regarding whether AI prompt histories and interactions may themselves become discoverable in litigation. Prompts may reveal attorney thought processes, legal strategy, factual assumptions, mental impressions, or investigative direction, raising complex issues related to work product protection and privilege.

At the same time, opposing parties may argue that prompts and AI interaction logs are relevant to understanding how legal conclusions, document categorizations, or investigation findings were developed. Because many AI platforms retain prompt histories and metadata, legal professionals must now consider retention policies, logging practices, privilege implications, and whether AI interactions could later become subject to discovery requests or regulatory scrutiny.

Conclusion

For legal professionals, the “nuts and bolts” of GenAI aren’t limited to defining it and understanding how it works – it’s also important to understand the ethical considerations for using it as well. As the series progresses, we will discuss each of them in more depth and how to address them.

Next time, we’ll discuss how ethics rules requiring technological competence apply to generative AI tools, including obligations to understand risks, limitations, and validation requirements.

For more regarding Cimplifi expertise in AI & enablement and consulting, click here.