CERT-In Issues AI Security Audit Guidelines, Introduces Four-Tier Risk Classification Framework

India's cybersecurity agency, the Indian Computer Emergency Response Team (CERT-In), has released new guidelines for auditing artificial intelligence (AI) systems, introducing a four-tier risk classification framework to strengthen security and accountability in the country's rapidly expanding AI ecosystem. The 107-page document, titled Artificial Intelligence (AI) Security Auditing Guidelines, outlines security assessment requirements for organisations developing or deploying AI systems in India. It also covers overseas entities providing AI services to Indian users or handling Indian data.

The framework classifies AI applications into four categories — low, moderate, high and critical — based on their potential impact, the sensitivity of the information they process, their level of autonomy and the possible consequences of system failures. Under the proposed classification, low-risk AI systems may undergo self-assessments or basic internal security reviews. Moderate-risk applications require annual independent audits, while high-risk and critical-risk systems face more stringent independent security assessments and monitoring requirements.

Critical-risk AI systems will also be required to undergo annual red-team exercises to identify vulnerabilities and assess their ability to withstand cyberattacks. These exercises involve simulated attacks designed to expose weaknesses before malicious actors can exploit them. The guidelines come amid growing concerns over cybersecurity threats associated with AI adoption, including data breaches, manipulated content, prompt injection attacks, algorithmic vulnerabilities and unauthorised access to sensitive information.

CERT-In has also outlined specific requirements for reporting AI-related cybersecurity incidents. Organisations covered by the applicable reporting requirements must notify the agency within six hours of identifying a reportable incident. They are also expected to retain relevant system logs for at least 180 days to support cybersecurity investigations and forensic analysis. The framework places additional emphasis on AI governance, data protection and transparency. Organisations are expected to maintain proper documentation of their AI systems, including information about their architecture, datasets, models, software components and associated security risks.

It also recommends maintaining an AI Bill of Materials (AI-BOM), which provides an inventory of the different components used in an AI system. Such records can help organisations identify vulnerable dependencies and investigate security incidents more effectively. Generative AI and autonomous AI agents have also received special attention in the guidelines. Organisations using generative AI are expected to introduce safeguards against the creation of unauthorised deepfakes involving real individuals and adopt content provenance mechanisms to help establish the origin and authenticity of AI-generated media.

For agentic AI systems capable of independently executing tasks and interacting with external applications, the guidelines highlight additional risks, including prompt injection and the misuse of connected tools. The framework covers organisations of different sizes, including startups, and does not provide relaxed security requirements for high-risk and critical-risk systems simply because they are operated by smaller businesses.

The guidelines are not legally binding on all organisations developing or deploying AI systems, according to media reports. They establish an audit framework, with requirements applying to CERT-In-empanelled cybersecurity auditors. The introduction of the guidelines marks another step in India's efforts to strengthen cybersecurity standards as artificial intelligence becomes increasingly integrated into business operations, financial services, healthcare, education and government functions. With the new risk-based approach, CERT-In aims to encourage organisations to identify vulnerabilities early, improve oversight and ensure that AI technologies are developed and deployed with stronger security safeguards.

Disclaimer: This image is taken from Unplash.