AI use has grown since 2023, bringing benefits and risks such as hallucinations and harmful outcomes that require human oversight in AI systems.

Governance is taking shape in practice, from NYC’s Local 144 audit requirements for automated hiring tools to EU-style standards that emphasize monitoring, understanding limits, and the ability to override AI outputs.

Meaningful human oversight means a decision-making human who can identify problems, exercise independent judgment, and intervene before harm, not merely a token presence.

Strategies to bolster oversight include restricting disruptive smartphone use by backup drivers in autonomous vehicles and requiring attorneys to verify AI-generated inputs and citations in filings.

A set of governance questions is proposed to help legal and compliance teams assess oversight effectiveness, accountability, expertise, authority, and bias mitigation before deploying AI systems.

Notable oversight failures illustrate the stakes, such as AI-generated, hallucinated legal citations in Michael Cohen’s work and the Uber self-driving car crash, underscoring cognitive and automation biases and the need for robust checks.

AI incident trends show increasing harm reports, while regulatory response lags due to the speed of adoption, diversity of AI types, and fragmented governance.

Regulations and frameworks increasingly require human-in-the-loop oversight, with examples from the EU AI Act and NIST, though implementing these standards remains challenging.

A practical, ethical test for meaningful oversight proposes that if potential harm is material, systems should involve proportional and empowered human involvement.