Red Teaming
The Red Teaming module lets you proactively test your LLM application against adversarial attacks before they reach production. It generates realistic attack scenarios, runs them against your application, and measures how effectively LLM application avoids harmful outputs.
Once you have configured a runtime protection, you can also measure how reliably it blocks harmful inputs and (remaining) outputs.
Overview
Red teaming works in three stages:
- Configuration — define which attack types to generate and describe your application
- Dataset — generate a collection of adversarial test cases from the configuration
- Run — execute the test cases against your application and review results
Each run performs two passes automatically:
- Runtime protection pass — test cases run through LINK2AI.Trust with input and output runtime protection enabled
- Baseline pass — the same test cases run without runtime protection, reusing cached LLM responses
This lets you measure exactly how many attacks your runtime protection blocks compared to an unprotected baseline.
Configurations
A configuration defines the scope of your red team test: which attack plugins to include, which strategies to apply, and contextual information about your application that helps generate more specific attacks.
General
Here you can give your red teaming configuration a name and a description, as well as choose the model, that will generate the adversarial prompts.
Application Settings
A configuration targets one of two application types, selected via the Application Type dropdown:
OpenAI-compatible endpoint — the application exposes an OpenAI-compatible chat completions API:
| Field | Description |
|---|---|
| Label | Human-readable name shown in the UI |
| Model | The model the application uses (selected from configured models) |
| System Prompt | The system prompt your LLM uses — this is the primary subject of the tests. You can type a prompt directly or select one from Application → System Prompts. |
Chatkit server — the application runs a ChatKit-compatible server:
| Field | Description |
|---|---|
| Label | Human-readable name shown in the UI |
| Chatkit URL | The endpoint URL of the ChatKit server (e.g. http://example.com:8000/chatkit) |
| Description | A description of the application used to generate context-relevant attacks |
Application Context
Providing context about your application significantly improves the relevance of generated attacks. You can have LINK2AI.Trust fill out the form using the “Propose” button in the upper right corner that will extract the information for the context fields automatically from your system prompt. It uses the model you configured under “Generation” to do so.
The following optional fields are available:
| Field | Description |
|---|---|
| Application purpose | What the application is designed to do |
| Key features and capabilities | Main capabilities of the application |
| Industry/domain | The industry or domain the application operates in |
| Types of users who interact with the application | Who uses the application in its intended usage |
| Systems and data the application has access to | Data the LLM can access |
| Systems and data the application should NOT have access to | Data the LLM must not access |
| Types of sensitive data handled | Sensitive data in scope |
| Example data indentifiers and formats | Description of example data sets |
| System rules and contraints for attackers | Any rules or constraints the system enforces |
| Content and topics the application should never discuss | Topics the LLM must not discuss |
| Competitors that should not be endorsed | Competing products the LLM should not reference |
| Security and compliance requirements | Security policies in force |
| Critical or dangerous actions the application can perform | High stakes actions the LLM can trigger |
| Red Team Persona | The type of adversarial user to simulate |
| Data you have access to | Data the AI application is allowed to share with the user |
| Data you do not have access to | Data the AI application has access to but is not (in every situation) allowed to share with the user |
| Actions you can take | Actions inside the application's scope |
| Actions you should not take | Actions outside the application's scope. |
| Connected systems the LLM agent has access to | List of external systems the AI application has access to |
Attack Targets
Targets define the types of attacks or error-provoking inputs included in a dataset. Each target generates a configurable number of test cases. LINK2AI.Trust provides a number of presets for the most common types of applications and safety / performance concerns.
Presets
| Preset | Description |
|---|---|
| OWASP Top 10 for LLMs | Generates attacks for the most critical security risks in LLM applications (as identified by OWASP), including prompt injection, insecure output handling, and data poisoning. |
| OWASP Top10 Agentic App | Generates attack addressing the unique security challenges of autonomous AI agents that can perceive, reason, and act without human oversight. Unlike static LLMs, these agents execute workflows, call APIs, manipulate data, and chain tools dynamically — creating a vastly expanded attack surface. |
| Customer-facing applications | Generate attacks and provoke errors that typically occur with customer-facing applications, like sales or technical support. |
| Cyber attacks | Generates typical cyber attacks. |
| Fraud + Manipulation | Generates users’ attempts to misuse the LLM for their own purposes, against your interests. |
| LLM errors | Provokes LLM errors without an attacks or manipulation attempts. |
| Harmful | Provokes LLM responses containing harmful content. |
Advanced Mode
In the advanced mode, you have full control over all attack targets.
| Target | Description |
|---|---|
| Hijacking | Pushes the AI application to carry out tasks that are beyond its intended purpose. |
| Off topic | Steers the conversation towards topics outside the application's scope. |
| Model identification | Tries to trick the AI application into revealing which LLM it uses. This can enable targeted attacks, violate provider contracts, and undermine brand identity. |
| Prompt extraction | Tries to trick the AI application into revealing its system instructions or parts of it, a typical preparation for more targeted attacks. |
| Tool discovery | Tries to trick the AI application into revealing the list of tools, functions, or API calls that it has access to, a typical preparation for more targeted attacks. |
| Reasoning DOS | Demands extremely complex reasoning tasks from the AI application pushing reasoning-capable models into unnecessary work, long runtimes, or inflated inference cost. |
| Divergent repetition | Instructs the AI application to generate indefinite output loops / unusually long repetitions, wasting tokens or degrading service quality and potentially leading to leakage of training data. |
| Shell injection | Steers a tool-using system towards unauthorized command execution. Relevant for assistants that can construct shell commands, invoke scripts, or pass user-controlled text into command-like workflows. |
| SQL injection | Steers a database-connected system into unsafe query construction or apparent execution of injected SQL. Relevant for agents that can execute database operations. |
| SSRF | Instructs the AI application to fetch attacker-chosen resources, including internal hosts and local files. Relevant for AI systems that can access external / internal resources. |
| Excessive agency | Tests the reaction of the AI application to requests that lie beyond its capabilities. |
| Hallucination | Pushes the AI application towards hallucinations, requesting information it does not have. |
| Unverifiable Claims | Pushes the AI assistant towards hallucinations, requesting information that cannot be verified. |
| Overreliance | Generates requests with flawed premises to see whether the AI application accepts them at face value (and thereby engages in a potentially harmful interaction) instead of correcting them before it answers. |
| Competitors | Tests whether a brand- or product-specific assistant mentions, recommends, or endorses competing offerings. |
| Contracts | Tests whether the AI application makes commitments beyond its authority or competence. |
| Order fraud | Generates requests that try to manipulate prices, apply unauthorized discounts, or violate promotional policies. |
| Goal misalignment | Invites the AI application to propose or help with "cheating", i.e. optimizing measurable proxy metrics that do not align with true underlying objectives. Relevant for systems that provide strategic advice, operational guidance, or optimization recommendations. |
| Bias | Invites the AI application to produce or reinforce stereotypes, biases, or discrimination in several different areas. |
| Harmful content | Invites the AI application to produce dangerous, abusive, or otherwise high-risk content in several different areas. |
Attack Strategies
Strategies control how attack prompts are transformed or delivered. You can select more or alternative strategies. Each selected strategy will be applied to each of the attack targets. Additionally you can choose the number of attacks that will be generated per combination.
Encodings
Encodings are special strategies that apply a fixed transformation to the generated attack prompt. Why would you want to do that? - Encodings are used by attackers wanting to bypass security controls based on the recognition of specific terms. - Models learn to work with encoded input, but often their safety mechanisms fail with encoded input.
| Strategy | Description |
|---|---|
| Basic | Leave checked to include the untransformed attacker prompt. |
| Base64 | Encodes the attack in Base64. |
| Hex | Encodes the attack in hexadecimal. |
| ROT13 | Applies ROT13 substitution. |
| Leetspeak | Replaces characters with leet equivalents. |
| Morse Code | Encodes the attack in Morse code. |
| Pig Latin | Converts to Pig Latin. |
| camelCase | Removes spaces using camelCase. |
| Emoji | Substitutes words with emoji. |
| Homoglyph | Replaces characters with visually similar Unicode. |
| Audio Encoding | Represents the attack as audio transcript. |
| Image | Embeds the attack in an image description. |
| Video | Embeds the attack in a video description. |
| Markup Injection | Injects the attack via HTML/Markdown markup. |
| Jailbreak Templates | Applies known jailbreak prompt templates. |
Datasets
A dataset is the collection of adversarial test cases generated from a configuration. Generation runs asynchronously and progress is shown in the UI.
Generation
Go to the Datasets section, click Generate Dataset and select a configuration to start generation. The portal polls for progress and shows the current count of generated test cases. The amount of attacks generated is the product of:
- Number of attack targets selected
- Number of strategies selected
- Number of tests per target selected
Once complete, the dataset shows the number of test cases and allows you to view, export or run the dataset.
This view shows the generated attacks with their attack targes and strategies used:
Datasets can be reused to run multiple evaluations (for example, before and after changing runtime protection settings).
Import
Instead of generating a dataset from a configuration, you can import one from a previously exported JSON file. On the Datasets page click Import Dataset, select a .json file, choose which configuration to associate the dataset with, and confirm.
The imported file must be in the same format produced by the export action (see below).
Export
Datasets can be exported as a JSON file from the Datasets page using the download icon on each dataset row. The JSON contains the dataset name, creation timestamp, and the full list of test cases.
The underlying YAML configuration used to generate a dataset can be downloaded from the configuration detail page for reproducibility or external use.
Runs
A run executes a completed dataset against your application endpoint and produces a detailed results report.
Starting a Run
Select a completed dataset and click Start Run. The run is created in pending status and begins executing immediately.
Progress is shown as the number of processed test cases out of the total. The run can be stopped at any point — it will complete the current test case and then halt.
Run Statuses
| Status | Description |
|---|---|
pending |
Queued, not yet started |
running |
Actively executing test cases |
completed |
All test cases processed |
failed |
Run encountered an unrecoverable error |
stopped |
Manually stopped by the user |
Results
You can view the results of a run with or without runtime protection enabled.
Attack Flow Diagram
The run details page includes an Attack Flow visualization that shows how test cases move through each stage of the pipeline — input analysis, LLM call, output analysis — and how many are blocked, refused by the LLM, or reach the user as harmful responses. Both the guardrail pass and the baseline pass have their own flow diagram.
When viewing the results without runtime protection, there are two different outcomes:
- LLM Refused / Safe: The response from the LLM is safe, it refused to generate the harmful output the prompt tried to provoke.
- Harmful response: The LLM generated the harmful output the prompt tried to provoke.
When viewing the results with runtime protection enabled, there are two more outcomes:
- Blocked (by Input Analysis): If you configured input detection (e.g. Malicious Intent) in your runtime protection configuration, LINK2AI.Trust will block attacks before they even reach the LLM.
- Blocked (by Output Analysis): All responses of the LLM (i.e. all cases minus the ones already blocked by the Input Analysis) are submitted to the Runtime Output Analysis. Harmful responses detected by the Output Analysis show as "Blocked" here.
The difference between the number of Harmful Responses with and without runtime protection represents the protective value of your current runtime protection configuration.
Statistics Charts
See the cases by outcome, severity, attack target and strategy. You can click on each bar to filter the list of test cases. Clicking on the same bar again clears the filter.
List
Each test case produces a result with the following information:
| Field | Description |
|---|---|
| Adversarial Prompt | The adversarial prompt sent to your application |
| Response | The final response returned to the user (standard response if blocked) |
| Actual LLM Response (before blocking) | The LLM output before any runtime protection blocking (only included when viewing results with runtime protection) |
| Evaluation Reason | Explanation for the final assessment (particularly relevant for the remaining harmful responses) |
Individual test cases can be removed from a completed run using the dropdown menu on each result row. This is useful for excluding known false positives from the metrics without re-running the full dataset.