As tech companies rapidly develop and deploy generative AI products, litigators and enforcers seek to understand how these companies make decisions about the safety of their products. Yet litigators aren’t necessarily equipped with technical knowledge about how these products are built, how safety evaluations are conducted, and where responsibility lies across the companies involved. In order to pursue accountability for harms, they need to know what kinds of questions to ask.
To meet this need, this summer Data & Society’s Public Technology Leadership Collaborative and Tech Justice Law co-organized a three-part series of workshops aimed at training enforcers and litigators who are building cases around generative AI products. We designed the series to follow an arc: The first session introduced a technical map of the AI supply chain and key AI safety concepts, while the second gave an overview of where safety evaluations happen in the AI product lifecycle. The third session applied those learnings to a hands-on exercise, where we walked through hypothetical scenarios based on real cases.
Six guest experts helped participants understand the broad landscape of the generative AI tech stack. Their presentations covered how AI safety decisions are implemented and distributed at various points during AI product development, and the kinds of questions litigators and enforcers should be asking of tech companies. Their key points included:
The generative AI tech stack is increasingly financially concentrated.
It’s important that litigators have an understanding of how the generative AI tech stack is organized. One common version consists of four key layers: chips, cloud computing, foundation models, and applications. The application layer, at the top, is what consumers and businesses actually use; the layers below it are the underlying infrastructure required to build and run AI.
Right now, a small number of companies dominate each of the infrastructure layers. NVIDIA makes the vast majority of chips used to build AI and charges very high prices for them. A handful of tech giants (namely Amazon, Microsoft, and Google) control most of the cloud computing power that runs AI systems. And just three companies — OpenAI, Anthropic, and Google — make most of the AI models that enable chatbots and tools. Because AI infrastructure is so costly and the stack is highly concentrated, there has been a flurry of circular financing and vertical integration across the market. NVIDIA, for example, is using its massive profits from chip sales to invest tens of billions of dollars directly into AI companies like OpenAI, as well as into the neocloud companies that run AI systems. This creates a tight web of financial relationships where the same handful of companies are supplying each other, investing in one another, and depending on each other for revenue.
When it comes to accountability, this means that responsibility for a specific incident or harm rarely lies with one single company. To determine which players made key decisions and who had the ability to prevent harm, enforcers should understand how the full stack operates.
Safety decisions are spread across different stages of the AI product lifecycle. Each stage raises foundational questions about safety design.
Litigators should have a basic technical understanding of how safety is implemented across different stages of the generative AI product lifecycle and know where to probe further. Our panel of experts provided an overview of the product lifecycle, and a checklist of questions that auditors or litigators should ask of the process.
When investigating a company’s AI safety systems, it is essential to ask questions about the data and training process. The examples below represent just a starting point for inquiry:
- What is the source of the training data, and on what basis were those sources selected or excluded?
- Which third-party vendors, if any, were involved in acquiring or processing the data?
- What automated or human review processes screened the training corpus for harmful, illegal, or biased content?
- Were bias audits conducted, and if so, by whom, when, and what did they find?
- What documentation exists describing the dataset’s composition, provenance, and licensing status?
The science behind AI evaluation and benchmarks is immature, but still being developed.
AI benchmarks are standardized tests that measure specific characteristics of an AI system. Benchmarks appear at multiple points in the AI safety lifecycle, but there is currently little consensus on how evaluations should be conducted or which benchmarks are most valid. Litigators should think carefully about how they craft arguments around what a company should have been doing, especially when benchmarking standards are limited.
In a report that surveys the spectrum of approaches to AI auditing, the Center for Democracy & Technology notes that there are few standardized methods for testing AI systems. Most AI labs, for instance, design and validate their own safety tests in-house, and the results of those tests are reported selectively. Even when the tests are public, some of the widely-used measurement tools have validity problems, and the implications are not always clear: a high score on a safety evaluation, for example, doesn’t necessarily correlate with the existence or extent of real-world harm. This means that litigators should question what it means when companies say their models passed safety evaluations — but it also means that a single failed benchmark or set of benchmarks might not be evidence of negligence. A company’s choice and justification for a particular benchmark or measurement tool often tells us more than the actual result of that testing.
Litigators can use an “upstream-downstream” framework to approach discovery, targeting specialized information and data.
As courts increasingly allow technology cases to move past initial motion-to-dismiss stages into full fact-finding, discovery provides an important mechanism for unraveling the complexities of digital platforms and AI systems, and establishing harm. Pursuing discovery against big tech companies presents formidable challenges. Based on Tech Justice Law’s analysis of recent tech litigation, it is clear that companies tend to oppose data requests broadly, raising objections grounded in undue burden or lack of ability to access data, trade secret protection, and legal privilege. AI companies also invoke statutory and constitutional defenses, such as Section 230 or First Amendment protections, to argue that certain products or features of products are shielded from discovery.
Litigators can overcome these challenges through thoughtful probes. Upstream discovery looks at internal company behavior like internal testing, the source code, and design rationale. Downstream discovery focuses on information pertaining to users, including user dependence, behavior, and data. Understanding the basic functionality of the technology and engaging technologists early on is key to forming the most effective requests for production and interrogatories, and to moving litigation forward.
The series reaffirmed the importance of bringing AI safety and development experts into direct conversation with litigators, enforcers, and other investigators. Across three sessions, we brought together 142 participants, many of them staff from state AG offices representing more than 24 states. It was a signal that enforcers are looking for this kind of technical expertise.
Participants told us that they left with a stronger technical grounding in the issues and a clearer understanding of where AI safety decisions are being made. At the same time, these sessions underscored how much work remains to be done in the field. The science behind AI benchmarking is still young, the field hasn’t settled on a shared technical vocabulary, and legal frameworks are catching up. That means there’s an opening for further exploration.
If you are interested in learning more about the PTLC, reach out to [email protected] for more information. To learn more about TJL’s work, reach out to [email protected].
Becca Ricks is the project director of Data & Society’s Public Technology Leadership Collaborative.
An attorney with a background in healthcare, civil rights, and emerging technology, Tiffany Gillis Brown is litigation counsel at Tech Justice Law.