Artifacts: AI/ML Lifecycle Management

Formal documentation of the data strategy covering data acquisition, understanding, processing, and quality assessment for AI model development.

Documentation of the AI model deployment process and continuous monitoring strategy. Covers deployment to GxP environment, integration specifications, and monitoring metrics with alert thresholds.

Documentation of the AI model design, development, and evaluation strategy ensuring model reproducibility. Reviewed by Sub-System Owner and approved by Product Owner and QA.

Living document capturing AI-specific risks in a structured manner across all AI lifecycle phases with mitigation actions. Must be updated throughout the lifecycle and approved before production deployment.

Multidisciplinary team assessment evaluating each agent across seven capability dimensions (Agent Autonomy, Adaptability, Actioning, Knowledge, Coordination, Perception, Internal Planning) at four levels to determine Agent Complexity and GxP Risk Level.

Matrix identifying potential risks for each agent capability dimension and defining minimum controls to be implemented and tested based on assessed capability levels and GxP risk level. Contains 60 Agent Controls (AGCs) across Agent Autonomy, Adaptability, Actioning, Knowledge, Coordination, Percept

Documented evidence from agent testing that quantitatively and qualitatively measures agent performance. Contains results from unit testing, SAT (integration, functional, red-teaming), LLM-as-a-Judge validation, and UAT with structured human acceptance judgments.

Document derived from the regulatory impact and intended use outlined in the System Risk Assessment that defines the specific context in which a GenAI model will be used. Required for determining testing rigor, evaluation metrics, and performance thresholds.

Procedure defining tools, roles, and responsibilities for continuous monitoring and maintenance of the AI sub-system. Complements the AI Model Deployment Approach and Monitoring Plan.

Operational deliverable defining ongoing monitoring procedures for GenAI sub-systems including operational thresholds, sampling strategy, and knowledge database maintenance procedures during the operation and maintenance phase.

Procedure document defining monitoring objectives, metrics, AGCs to monitor, and response framework for the deployed agentic sub-system. Must include automated and manual monitoring modes, structured human acceptance judgment collection, alert thresholds, and HITL effectiveness verification.

Formal assessment identifying and managing risks associated with data quality to ensure fitness of data for AI modelling. Reviewed and approved by the Process Owner.

Structured evaluation instrument for scoring GenAI outputs on a numbered scale (e.g., 1-7) with clearly defined criteria per level, as described in GD-0304459. Must have at least 7 distinct scoring levels to support parametric t-test analysis and requires validation and acceptance by relevant SMEs b

Prioritized list of items for AI Product delivery including AI-related User Stories with performance metric acceptance criteria and thresholds.

Outlines the delivery and maintenance approach across the lifespan of the AI Product including processes and controls to meet CSV requirements. Fulfils the requirements of the Validation Plan within the Agile framework.

Human-readable, time-stamped record capturing the step-by-step internal logic of an AI agent during operation. Includes chain-of-thought reasoning, tool invocations, tool responses, data retrieval steps, and rationale for chosen actions. Must be maintained in searchable and queryable format.

Confirms all activities defined in the PDS related to the AI sub-system and overarching system have been completed and all DoD criteria for Release have been met. Fulfils the Validation Report requirements in the Agile framework.

Statistical configuration specifying sample size, acceptance criteria, and significance level for GenAI sub-system testing, based on risk-based test rigor as described in GD-0304459. Derived from reference tables for binary, proportional, and ordinal outcomes and must be endorsed by an approved stat