What does human in the loop mean for AI in GMP, and is it enough?

1Where the term comes from in GMP.

The draft Annex 22 on AI, which the European Commission and PIC/S published for consultation in July 2025, uses “human-in-the-loop (HITL)” in two ways.¹ For generative AI and large language models, which it says should not be used in critical GMP applications, it is the condition for any non-critical use: “personnel with adequate qualification and training should always be responsible for ensuring that the outputs from such models are suitable for the intended use”. For other AI models, it describes the case “where a model is used to give an input to a decision made by a human operator”, and the model’s testing has been reduced on that basis.¹

Regulators use the idea outside the draft too. An HPRA inspector said in May 2026 that “testing / confirming the outputs is key, e.g. through verification of a ‘Human in the loop’ or using alternative guardrails”.² EMA’s reflection paper asks for a system risk management plan for every model, “especially those where there is no human-in-the-loop”.³ FDA’s April 2026 warning letter told Purolea Cosmetics Lab that if it resumed production, AI output must be “reviewed and cleared by an authorized human representative of your firm’s QU”.⁴

2What the draft expects of the loop.

The draft spreads its expectations across several sections.¹

Behind all of it is the point an HPRA inspector made in May 2026: the company and the personnel responsible for the GMP activity “are responsible for the decisions made based on the outputs of those models”.²

3Why a human check is not enough on its own.

A reviewer who approves every output adds little if they defer to it. NIST’s Generative AI Profile calls this automation bias, “excessive deference to automated systems”, and notes that it can make confabulation worse; generated text can include invented logic or citations that make a wrong answer look justified.⁵ For high-risk AI systems, the AI Act requires providers to supply them in a way that lets the people overseeing them “remain aware of the possible tendency of automatically relying or over-relying on the output”, understand the system’s limits, and “disregard, override or reverse” its output.⁶ Deployers must assign that oversight to people with “the necessary competence, training and authority”.⁷ Those duties apply only to high-risk systems, but they describe what a working loop needs.

The loop has to be tested as a whole. The FDA and EMA guiding principles of January 2026 say that risk-based performance assessments “evaluate the complete system including human-AI interactions”.⁸ FDA’s draft guidance on AI for regulatory decision-making treats independent evidence as lowering a model’s influence: in its example, release testing of a sample from each batch means an AI fill-volume inspection “will not be the sole determinant for the release of product”. With a high decision consequence and low influence, the model risk is still medium, and the credibility work stays “commensurate with the AI model risk”.⁹ Annex 11 sets the floor for any computerised system that replaces a manual operation: “there should be no resultant decrease in product quality, process control or quality assurance.”¹⁰

4Where human review cannot carry the decision.

In the draft Annex 22, a human in the loop does not make a large language model acceptable in a critical GMP application; such models should not be used there at all. Critical applications call for static, deterministic models tested against pre-set acceptance criteria on independent test data, with explainability, confidence thresholds and ongoing monitoring. Human review can reduce the testing of those models, but only with the reviewer’s role defined in the intended use and records of the review kept.¹

5How it runs in Seal.

In Seal, the decision is a person’s. Neil drafts and proposes; it signs, approves and releases nothing. Each finding links to the record, version and field behind it, Neil names what the records don’t show, and it can change a process only by proposing a change whose tests run before your team releases it.

The validation and AI assurance guide asks you to define, for each GxP use, the acceptable outputs, the reviewer’s responsibilities and the actions Neil may propose, and to retain the request, the source records and their versions, the output and the reviewer’s decision. It notes that a citation helps you inspect the evidence but does not prove the interpretation. Each of the jobs Neil does shows the review a person makes before anything changes.