AI Security

Active learning is a training approach where the model flags the examples it is least sure about so humans label those first, cutting how much labelled data is needed.

An AI agent is an AI system that plans and executes multi-step tasks on a user's behalf, calling tools such as browsers, code interpreters, and APIs rather than only generating text.

An AI ethics board is a cross-functional group that reviews proposed AI uses against the organization's values and risk appetite, especially where they affect people.

An AI fallback plan is the documented way of operating when an AI system must be turned off or bypassed: the manual or alternative process that keeps things running.

An AI gateway is a proxy that sits between an organization's users or applications and external AI models, enforcing policy on every prompt and response.

An AI governance charter is the founding document that sets who decides what about AI in an organization: the roles, the scope, the decision rights, and the escalation paths.

An AI impact assessment is a structured review of an AI use case's risks, benefits, and affected parties, run before deployment to inform the decision to proceed.

AI observability is the practice of monitoring an AI system in production: its inputs, outputs, performance, and drift, so you can tell when it starts behaving in ways you did not intend.

The AI supply chain is everything an AI system depends on that you did not build: base models, training data, fine-tuning datasets, open-source libraries, model hubs, plugins, and hosted APIs.

Artificial general intelligence (AGI) is a hypothetical AI system able to match or exceed human performance across most cognitive tasks, rather than excelling only in a narrow domain.

Artificial narrow intelligence (ANI) is AI built to do one kind of task well, such as spam filtering or image tagging, rather than reasoning across domains like a human.

Average precision is a single score that summarises how well a model ranks correct results ahead of wrong ones across all thresholds, used to compare detection and classification models.

Backpropagation is the core algorithm that trains a neural network by working backwards from its errors to adjust each internal weight so the next prediction is a little more accurate.

Bayes' theorem is a rule for updating the probability that something is true as new evidence arrives, combining prior belief with fresh data.

A Bayesian network is a model that maps how uncertain factors influence each other, so you can reason about the probability of one event given what you know about the others.

ChatGPT is OpenAI's conversational AI assistant, launched in November 2022, and the product that took large language models mainstream.

Claude is the AI assistant family built by Anthropic, an AI safety focused company founded in 2021 by former OpenAI researchers.

Cohere is an enterprise AI company, founded in 2019 in Toronto, that builds language models aimed at business use rather than consumer chat.

A context window is the maximum amount of text, measured in tokens, that a language model can consider at once when producing a response.

A neural network built for grid-shaped data such as images, which learns its own visual features by sliding small filters across the input.

Copilot is Microsoft's brand for the AI assistants embedded across Windows, Microsoft 365, and GitHub, largely powered by OpenAI models.

Data leakage, in an AI context, is the unintended exposure of sensitive information through an AI system: data pasted into prompts, revealed in model outputs, or absorbed into training sets.

Corrupting the data an AI model trains or retrieves from so that the model learns, retrieves, or repeats attacker-chosen behavior.

A model that reaches a decision by asking a sequence of yes/no questions about the data, splitting it at each step until it arrives at an answer.

A deepfake is synthetic audio, video, or imagery generated by AI to convincingly impersonate a real person.

An embedding is a numeric vector that represents the meaning of text, images, or other data, so that similar items sit close together in mathematical space.

Excessive agency is the failure mode where an AI system is granted more permissions, tools, or autonomy than its task requires, so a mistake or a hijacked prompt causes real-world damage.

Explainability is the ability to answer how an AI system arrived at a particular result, in terms the person receiving the answer can actually act on.

Feature attribution assigns each input a share of the responsibility for a particular model output. It is a reconstruction made after the fact, not a record of what the model did.

Fine-tuning is the process of further training an existing AI model on a smaller, specialized dataset so it performs better on a specific task or domain.

A frontier model is one of the most capable general-purpose AI models available at a given time, typically built by a small number of well-funded labs.

A fundamental rights impact assessment (FRIA) is an EU AI Act requirement for certain deployers of high-risk AI to assess the system's impact on people's rights before use.

Gemini is Google's family of multimodal AI models and assistants, developed by Google DeepMind and integrated across Google Search, Workspace, and Android.

Generative AI is a class of artificial intelligence that creates new content, including text, images, audio, video, and code, rather than only classifying or ranking existing data.

Guardrails are the technical controls wrapped around an AI model to constrain what it will accept and produce, separate from the model's own training.

A hallucination is an AI output that is fluent and confident but factually wrong or entirely invented.

Hugging Face is the largest open platform for sharing machine learning models and datasets, founded in 2016 and often described as the GitHub of AI.

Inference is the phase in which a trained AI model is actually used: it receives an input and produces an output, as opposed to the training phase in which the model learns.

ISO/IEC 38507 is an international standard on the governance implications of using AI, written for the board and governing body rather than the technical team.

A jailbreak is a prompt or technique that manipulates an AI model into ignoring its safety rules and producing output it was trained to refuse.

An unsupervised method that sorts data into a number of groups you choose in advance, assigning each record to whichever group centre it sits nearest.

A large language model (LLM) is an AI model trained on massive amounts of text to predict the next token, which enables it to generate and work with language fluently.

A model that predicts a number by fitting a straight-line relationship to the data, weighting each input by a coefficient you can read and check.

Building software that learns patterns from data rather than following only rules a person wrote, so its behaviour comes from the examples it was trained on.

Midjourney is an AI image generation service, run by a small self-funded research lab, known for producing high-quality stylized images from text prompts.

Mistral AI is a French AI company, founded in 2023, that builds high-performing language models and releases many of them with open weights.

The Model Context Protocol (MCP) is an open standard, introduced by Anthropic in 2024, that lets AI applications connect to external tools and data sources through a common interface.

Model extraction is an attack that reconstructs a proprietary AI model, or the sensitive data behind it, by systematically querying it and studying the responses.

A model owner is the named person accountable for a specific AI model in production: its performance, its risks, and the decision to retrain, roll back, or retire it.

Model poisoning is tampering with an AI model itself, during training, fine-tuning, or distribution, so that it carries hidden malicious behavior.

A model risk committee is the governance body that reviews and approves AI models against risk criteria before deployment and on a recurring basis afterwards.

Model rollback is reverting a production AI model to a previous version known to work, usually because the current one has failed quality, drift, or safety checks.

Multicollinearity is two or more model inputs carrying substantially the same information, leaving the model no basis for deciding which one earned the credit for a prediction.

Multimodal AI is an AI system that can process or generate more than one type of data, such as text, images, audio, and video, within a single model.

A model built from layers of simple connected units whose weights are adjusted during training, letting it learn relationships too complex to write down.

An open-weights model is an AI model whose trained parameters are published for anyone to download, run, and fine-tune, unlike closed models accessible only through a vendor's API.

The OWASP Top 10 for LLM Applications is a community-maintained list of the most critical security risks in applications built on large language models.

Perplexity is an AI answer engine, founded in 2022, that responds to questions with synthesized answers and cited sources rather than a list of links.

Post-market monitoring is the ongoing tracking of an AI system's performance and risks after deployment, an explicit obligation for high-risk systems under the EU AI Act.

Prompt engineering is the practice of writing and structuring inputs to an AI model to get reliably useful outputs.

An attack that smuggles instructions into an AI system's input so the model follows the attacker's intent instead of the operator's.

Provider vs deployer is the EU AI Act's split between the party that builds or substantially modifies an AI system and the party that puts it to use, each carrying different duties.

AI red teaming is the structured adversarial testing of an AI system to find harmful, insecure, or policy-violating behavior before attackers or users do.

Reinforcement learning from human feedback (RLHF) is a training technique that tunes an AI model toward outputs humans rate as helpful and safe.

Retrieval-augmented generation fetches relevant documents from a knowledge base and inserts them into the model's prompt, so answers are grounded in current, private, or authoritative data.

A serious AI incident is a malfunction or harm from an AI system that meets a defined severity bar, which the EU AI Act requires providers to report within strict deadlines.

Shadow AI is the use of AI tools by employees or teams without the knowledge or approval of IT and security.

Training a model on examples that have already been labelled with the right answer, so it learns to reproduce those labels on data it has not seen.

Synthetic data is artificially generated data that mimics the statistical properties of real data, used for training or testing when real data is scarce, sensitive, or regulated.

A system prompt is the hidden instruction block an operator gives an AI model to define its role, rules, and constraints before any user input arrives.

A token is the basic unit of text an AI language model reads and writes, typically a word fragment of about four characters in English.

Training data is the corpus of examples an AI model learns from, which determines what the model knows, how it behaves, and what biases and secrets it may carry.

The transformer is the neural network architecture, introduced by Google researchers in the 2017 paper 'Attention Is All You Need', that underlies virtually all modern large language models.

Finding structure in data that carries no labels, so the model groups or ranks what it sees without being told in advance what any of it means.