AI in GxP Systems

The introduction of artificial intelligence into GxP systems is profoundly changing the way we approach Quality, CSV, and Data Integrity. For many years, we have worked with deterministic systems based on fixed and predictable logic. AI follows a different path: it learns from data, adapts to the operational context, and can change its behavior over time. Consequently, reliability no longer depends solely on the software but also on the quality of the dataset, the process used to train the model, and the conditions under which it will be used.

As artificial intelligence finds its way into a growing number of activities within the pharmaceutical sector, the European regulatory framework is also taking on more defined contours. The EMA has introduced ten principles that outline a broad and structured approach to using AI throughout the medicinal product lifecycle. These guidelines highlight aspects well known to those working in the quality domain: attention to data provenance and management, transparency in model operation, the ability to interpret results, and processes ensuring human oversight during all critical phases. In practice, AI is treated as an element of the quality system rather than an isolated component, and organizations are expected to integrate this technology with clear roles, defined responsibilities, and a coherent governance model.

Complementing this general framework, the draft Annex 22 provides more operational guidance specifically oriented toward GMP processes. While not yet binding, it reflects a set of expectations already influencing how companies approach AI adoption. The document emphasizes static and deterministic models, the quality and representativeness of training and test datasets, and the need to maintain a strict separation between the two. Particular importance is given to the definition of intended use, result explainability, and controlled change management, i.e. elements requiring a detailed understanding of the manufacturing process and ongoing supervision over time. The goal is to ensure that the model behaves reliably and predictably, at least as well as the process it may replace.

This shift in nature requires broadening the traditional validation approach. Guidelines published in recent years, i.e. GAMP 5 Second Edition and the 2025 GAMP AI Guide, clarify that the evaluation of an AI-based system must also include the model, the training data, the training techniques, the hyperparameters, and the way model stability is monitored over time. It is therefore not about adding yet another document to the validation package, but about expanding the scope to include elements previously considered external to the concept of "software".

The underlying regulations remain unchanged. 21 CFR Part 11 and Annex 11 continue to require complete audit trails, access controls, data-security mechanisms, and record-integrity safeguards. However, AI introduces additional risks, i.e. dataset bias, performance degradation during real-world use, or excessive dependence on specific operating conditions. All these aspects require new controls.

To better understand this transformation, a concrete example is helpful.
Consider a visual inspection system for vials or bottles. Before AI can be used in production, it must be trained on a large number of images.

In a GxP environment, dataset versions must be recorded, any transformations applied to data must be tracked, real-world model performance must be evaluated, and any increase in false rejects or false accepts must be controlled. If the production process changes, the model may react differently, requiring retraining or updating. This dynamic aspect is very different from what we are used to with traditional software.

At the European level, the 2024 AI Act adds another layer of responsibility. The regulation introduces a risk-based classification and requires high-risk systems, into which many GxP applications fall, to comply with stricter rules regarding data governance, robustness, documentation, and human oversight. This approach integrates well with validation logic and pushes companies toward more mature processes.

For those working in CSV, this does not mean reinventing everything, but shifting the focus. The familiar activities remain, but they must be applied to a different object: no longer only deterministic software, but models that evolve over time, depend on data, and require continuous monitoring. As a result, it will be necessary to evaluate dataset representativeness, control data transformations, define objective acceptance criteria for models, manage behavior changes, and implement continuous performance monitoring. Data Integrity also expands: beyond classical principles, the traceability of the training dataset, the explainability of model decisions, and the verification that data are fit for purpose all become essential. These aspects require an organization with well-defined roles and responsibilities and a "2.0" data-driven culture.

AI offers many opportunities and can make processes more robust, faster, and more controllable. But it is not a shortcut. Without high-quality data and rigorous governance, AI risks becoming ineffective. If, however, it is embedded in a prepared environment with clear processes and responsibilities, it can become a powerful ally for Quality and the entire organization. Ultimately, it is not the model itself that makes the difference, but how the organization chooses to manage it. This is also the convergence point between the EMA principles and Annex 22, both highlighting the importance of a quality system capable of supporting data, processes, and models coherently. AI becomes truly useful only when it is rooted within a solid structure, with clear roles and adequate data governance.

Article authored by Andrea Bussi - CSV Business Unit Manager, S.T.B. Valitech S.r.l.