The language of human oversight gives us a familiar way to talk about mitigating AI failures. Oversight organizes the promise of accountability through the presence of a person who reviews, approves, pauses, or corrects what the system does when needed. This picture is reassuring because it assigns responsibility to an informed user. It suggests that autonomous delegation can expand effectively as long as someone is nearby, ready to intervene when the system begins to fail.

AI agents make this picture difficult to sustain. Their value comes from the very thing that complicates oversight: they turn high-level goals into plans and autonomously carry those plans into action across tools and digital environments. Users delegate more than a discrete step in a workflow. They also delegate decisions about how the work should proceed. Even small divergences between the goal the user gives, the plan the agent forms, and the execution it carries out can change how work is done. Some of these divergences are obvious. Others appear as plausible outputs produced through wrong methods or shortcuts that preserve the look of success.

Our recently released primer The Oversight Fallacy: Why AI Agents Require More than Humans-in-the-Loop takes this problem as its point of departure. In it, we argue that the human-in-the-loop frame is insufficient, because it assigns responsibility without explaining the conditions that make oversight possible. What must be made visible for a person to notice a consequential deviation in an agent’s working? What organizational conditions allow that person to intervene before a mistake cascades through the agentic workflow? These are sociotechnical questions about how agentic work reorganizes human expertise, professional judgment, and the allocation of responsibility.

In the primer, we argue that effective oversight requires knowledge, observation, control, and intervention. This argument unsettles the assumption that intervention is the only means of oversight and that it is always possible for a user to intervene. A pause button, for example, matters only if the user can recognize the need to pause, observe the relevant part of the agent’s work, and redirect what happens next. In practice, many oversight arrangements provide moments of formal intervention while leaving the conditions for meaningful intervention underdeveloped and ignoring emergent oversight practices and heuristics. The user is invited to supervise without being given a workable view of what an agent is doing.

For researchers, this makes observability a first-order object of study. Transparency alone does not resolve this problem. A long action trace may show every tool call while leaving a user unable to tell why an agent changed strategy midway through execution. Observability is situated. It depends on the task, the domain, the user’s expertise, the relevant standard of success, and the point at which intervention remains possible.

This is why we believe that the design of agent interfaces deserves close attention. Interfaces help determine what appears by default, when confirmation is required, and when the agent can continue. These choices reveal how agent builders imagine oversight and configure the user by teaching people what deserves attention and where their authority begins or ends. As these conventions stabilize, they shape the common sense of agentic autonomy in practice. 

Oversight as Everyday Work

If a user has to watch every step, delegation loses much of its value. If the user sees too little, oversight becomes nominal. The practical problem is selective attention: how people and organizations determine which uncertainties can be tolerated before trust gives way to inspection. A sociotechnical research agenda should follow how people learn to oversee agents that are built to reduce the need for human intervention.

Selective attention depends on judgment, and judgment is learned through repeated work. Competent practice develops through doing a task and correcting it. Over time, people learn to distinguish normal messiness from consequential anomalies. When routine tasks are delegated away, users may lose access to the repetition through which judgment develops. Agentic workflows also create new routines that must be learned. The organizational and pedagogical problem is how teams preserve judgment about delegated processes while cultivating judgment for workflows in which supervision may begin after action.     

Failures make this problem visible retrospectively by revealing whether an oversight arrangement held or broke down. To repair a failure, people have to reconstruct what happened and decide which moment was critical for intervention. Postmortems and arguments about acceptable delegation, therefore, are consequential beyond the immediate context of an agentic failure. For sociotechnical researchers, troubleshooting reveals how teams reattach agentic action to an acceptable standard and decide what should be noticed, interrupted, documented, or repaired next time.  

Scientific settings make this especially vivid because disciplinary oversight extends beyond the final output of an AI system to the method that other scientists can inspect, contest, reproduce, or recognize as valid. In the computational biology cases discussed in the primer, failures appeared through this gap between output and method. Oversight here means preserving the evidentiary chain through which an output becomes a claim.

Oversight as Distributed Institutional Responsibility

Oversight cannot be treated as the responsibility of an individual user alone. If an output remains accountable only when its path can be reconstructed, then oversight depends on arrangements that precede the moment of review. Agents enter organizations through procurement, default settings, documentation, and internal policy. These arrangements distribute responsibility before any user begins supervising the system and shape what counts as appropriate access or sufficient review. Over time, they produce an organizational sense of what oversight requires.  

The organizational processes through which this sense develops are often slow and implicit. A vendor frames oversight as user control, and the organization absorbs that expectation into training and review. When a failure occurs, the question becomes whether the user should have noticed. Yet the conditions that made noticing possible were set elsewhere, in the interface and log, as well as the user’s workload and authority. The user becomes responsible at the point of failure for a system of delegation operationalized upstream.  

The moral crumple zone problem takes a distinct form in agentic systems. The person nearest the system is treated as responsible even when they lack the practical capacity to identify shortcomings in and redirect the system’s work. Human oversight can protect institutions from accountability while offering users only a narrow channel for action. Training and troubleshooting materials intensify this dynamic when they document what users should have done. They create a record of obligation without creating the conditions for agency.  

Studying oversight as an organizational accomplishment means following the relocations of responsibility as they take shape in practice. Which failures become incidents, and which are treated as the ordinary noise of delegation? These distinctions reveal how accountability travels from builders to deployers and from those who set agentic goals to those who inherit the consequences of their execution. 

Friction also needs to be reclaimed as an analytic term. Agentic systems are often sold through the promise of reducing friction, even as oversight depends on preserving it where judgment is critical. Review points and escalation paths slow action where consequences can cascade. The oversight of AI agents therefore offers a way to study delegation as it becomes infrastructure. Agents distribute consequential decisions across plans, tools, traces, and routines. Oversight is the work of reconnecting these dispersed elements so that action remains accountable to the setting in which it matters. If this reconnection defines oversight, the next research problem is how it can scale. 

What Would It Take to Build Scalable Oversight?

In technical AI research, scalable oversight describes methods for evaluating agent performance on tasks that exceed unaided human judgment. Many of these are “hard-to-supervise fuzzy tasks.” Their criteria for success are difficult to specify and human judgment about their quality may be systematically unreliable. Proposals for scalable oversight often respond by decomposing the larger task into parts that people can evaluate more reliably, then using those judgments to assess or train the agent. However, solving oversight at scale first requires unpacking oversight in practice.

Scaling oversight begins with an underlying sociotechnical problem. Before a task can be decomposed, someone must determine which parts of the work are consequential and what evidence must be preserved to judge whether the task has been carried out according to the expected method. Decomposition carries judgment into the delegation protocol. It also creates a problem of recombination. Each subtask may pass its local check while the assembled workflow still departs from the expected method. For example, an agent may accurately summarize multiple studies that support the same finding while missing that all of them draw on the same dataset. Each summary is correct, but treating the studies as independent evidence exaggerates confidence in the result. As agentic outputs multiply, shared assumptions can make repetition look like corroboration. Scaling oversight therefore depends on how situated judgments are translated into proxies and how dependencies remain visible across a workflow.

This is where scalable oversight becomes a research direction. How do practitioners recognize competent oversight before it is formalized? Which proxies remain coupled to the goal an agent was given and the method expected of it? When does decomposition make review easier, and when does it relocate ambiguity into the boundaries between subtasks? How should separate judgments be recombined when outputs inherit common assumptions? Answering these questions requires following teams as they build oversight protocols and revise them after failure. Scalable oversight, on this account, is the work of carrying situated judgment across a larger field of delegated action. 

AI agents introduce a denser form of delegation into organizational life. The challenge is how organizations can preserve the capacity for judgment when no person can follow the entire course of action. Sociotechnical research can trace how oversight is organized under these conditions. It can show whether delegated action remains accountable to the settings in which it matters or whether oversight has become the language of displaced responsibility.