Passing OSAI Blackbox – How I prepared for Offsec’s AI Red Team Exam

Disclosure: I I work at OffSec, but I took the OSAI exam in a personal capacity and prepared for it on my own time. This article reflects only my individual experience and opinions. It is not official OffSec guidance, and none of the tools, models, workflows, or time allocations mentioned below are required or endorsed for the exam. Candidates should rely on the current official OffSec documentation for exam rules, requirements, and policies. This article contains no exam targets, answers, flags, or other restricted content.

All references to the exam format and policies are based on the public OffSec sources listed at the end.

Why I wrote this

OSAI is still a new certification. It launched on 31 March 2026, so there is not yet the same volume of reliable community material that exists for established exams such as OSCP. Some third-party guides have also repeated an especially consequential claim: that ChatGPT, Claude, Gemini, and similar tools are prohibited during the exam.

That was not correct. OSAI was explicitly listed as an exception in OffSec’s AI usage policy, and the use of AI tools was permitted and encouraged.

This is the article I would have found useful before booking my attempt. It explains how I personally prepared, which parts of my offensive-security background transferred well, how I incorporated AI assistants into my workflow, where those assistants had limitations, and why I treated evidence collection and reporting as part of the technical engagement rather than as administrative work at the end.

Nothing below is intended as a prescribed preparation path.

The experience you should bring

AI-300 is a 300-level course, and that designation is meaningful. The published prerequisites describe a foundation that includes practical penetration-testing ability, familiarity with Linux and Windows, basic scripting, and OSCP-level or equivalent experience. Prior knowledge of large language models is described as helpful rather than mandatory.

That is reasonable as an entry requirement, but I did not want to encounter the fundamental concepts for the first time during a timed engagement. Before my attempt, I made sure I could explain what embeddings do, how retrieval changes a model’s context, how an agent decides to invoke a tool, and where trust boundaries appear in AI-enabled applications.

Four capabilities were particularly valuable in my own preparation.

First, I was comfortable compromising and navigating conventional systems and networks. AI red teaming did not replace infrastructure, web, identity, cloud, or application-security skills. It added a new attack surface to them.

Second, I could read unfamiliar Python reasonably quickly. Agent definitions, orchestration logic, tool schemas, retrieval code, and data flows often reveal where trust is misplaced. Being able to follow that logic helped me turn an unfamiliar application into a map of components, permissions, inputs, and possible escalation paths.

Third, I had built and attacked AI applications myself. Reading about prompt injection is useful, but building a small retrieval-augmented generation pipeline, manipulating its inputs, and observing the resulting behavior gave me a much more practical understanding of the problem.

Finally, I had practised documenting my work while testing. I knew from experience that evidence collected only after exploitation is often incomplete. Commands, intermediate responses, environmental conditions, and failed attempts can all matter when reconstructing a finding later.

For me, strengthening those foundations was more valuable than memorizing another list of vulnerability names. Other candidates may identify different gaps, but I found that the AI-specific material became easier to absorb once the underlying offensive-security workflow was already familiar.

How I used AI during preparation and the exam

OffSec’s general exam policy restricts AI assistants, but OSAI is an explicit exception. I therefore incorporated AI into my preparation instead of treating it as a last-minute convenience.

I found an assistant most useful when I gave it concrete context and a bounded task. Examples included summarizing unfamiliar orchestration code, mapping reachable tools, identifying trust boundaries, transforming raw observations into testable hypotheses, generating structured payload variations, drafting small scripts, and organizing notes into report-ready findings.

I also used AI as a challenge mechanism. When I had spent too long on one direction, I asked the assistant to question my assumptions, identify missing enumeration, and propose alternative explanations for what I was seeing. That did not make the assistant an authority. It gave me another way to test my own reasoning.

The operator still owns enumeration, validation, scope, and judgment. An assistant can produce a fluent and technically convincing explanation of the wrong environment. I treated every generated claim, payload, and proof of concept as untrusted until I had validated it against the actual target.

How I personally structured the 24 hours

My primary hosted assistant was Claude Code. I used it alongside `/red-run-ctf` and HexStrike AI. In my workflow, this combination was useful for reviewing code, preserving the context of attack hypotheses, iterating on payloads, and converting partial discoveries into reproducible tests. /red-run-ctf and HexStrike AI. That combination was useful for rapid code review, keeping attack hypotheses organized, iterating on payloads, and turning partial discoveries into reproducible tests.

Claude Code was most effective when the objective was specific: inspect this code, map these tool calls, explain this boundary, modify this proof of concept, or help me compare several hypotheses. `/red-run-ctf` and HexStrike AI helped me structure the interaction around an offensive-security engagement rather than presenting every problem as a generic programming question.

This was simply the setup that worked for me. OSAI does not require Claude Code, `/red-run-ctf`, HexStrike AI, or any other product mentioned in this article. Their inclusion is descriptive, not prescriptive. Another candidate may be more effective with a different assistant, a different collection of tools, or a substantially more manual workflow.

I also encountered a practical limitation with hosted assistants. Even when I was operating within an approved coordinated vulnerability disclosure program and had authorization to test the systems in scope, a hosted assistant sometimes declined to complete security-sensitive proof-of-concept code.

Authorization and a provider’s model policy are separate considerations. Permission to conduct a security test does not guarantee that a hosted assistant will help implement every part of it.

For some authorized work outside the exam, I had already experimented with a local stack based on Ollama and Hermes. That gave me a different development environment when a hosted service stopped short. It did not change the authorization, scope, data-handling requirements, or rules governing the engagement.

The local setup is not an OSAI requirement, and I do not consider it necessary exam preparation. I mention it only because it was part of my broader personal workflow and illustrates a distinction that matters in professional testing: removing assistant-side restrictions never removes the tester’s responsibilities.

My broader lesson was to understand the tools I intended to use before relying on them under time pressure. I wanted to know how they handled context, where they were likely to fail, and how I would continue working if one became unavailable or unsuitable. That principle is independent of any particular model or product.

A practical 24-hour plan

The following schedule describes how I approached my own attempt. It is not an official strategy, a recommendation, or a template that every candidate should follow. Individual working styles, findings, energy levels, and technical paths will differ.

Twenty-four hours initially sounded generous to me. Once I accounted for enumeration, testing, evidence collection, reporting notes, breaks, and sleep, it became clear that sustained judgment—not simply elapsed time—would be the resource I needed to protect.

Hours 0–4: broad enumeration

I used the opening phase to map the environment before committing deeply to the first promising idea. I looked for model endpoints, APIs, agent definitions, exposed tools, retrieval components, storage, cloud services, identity relationships, and conventional infrastructure.

My goal was to build a working model of the environment. I deliberately resisted spending the entire opening period perfecting the first obvious prompt injection or isolated behavior.

Hours 4–12: testing across the discovered surfaces

After the initial map was in place, I tested the surfaces I had identified and compared the emerging paths. I kept notes and captured evidence as I worked instead of relying on memory.

This period was broad by design. I wanted enough information to distinguish an interesting behavior from a path that could demonstrate meaningful impact.

Hours 12–16: rest

I reserved time for sleep. For me, a planned break was more valuable than continuing for several additional hours with deteriorating attention and decision-making.

The exact amount and timing of rest will be personal. My reason for planning it in advance was to avoid treating sleep as a failure that would occur only after exhaustion had already affected the work.

Hours 16–22: returning to the strongest paths

After resting, I returned to the most promising findings, developed them further, and focused on producing reproducible results. The earlier enumeration helped me choose where to spend this more concentrated period.

Hours 22–24: verification and consolidation

I used the final portion to repeat important steps, verify results, check that screenshots were readable, and organize my notes so the attack paths would still make sense after the exam session ended.

The precise schedule matters less than the principles behind it. In my case, those principles were to enumerate before narrowing the investigation, preserve evidence immediately, revisit assumptions, and protect enough attention to finish cleanly.

Mistakes I tried to avoid

The following points are personal observations about risks I considered during my preparation and attempt. They are not statements about OffSec’s internal scoring process, nor should they be interpreted as official exam guidance.

1. Treating the report as an afterthought

I treated the report as the place where the technical work had to become understandable and reproducible. A result that existed only in my terminal history or memory would be difficult to explain clearly later, so I began organizing the evidence during the engagement.

That’s why I created TLoggerV2; it keeps a history of each terminal, so I was safe when I wrote my report and could double-check everything.

2. Capturing only the final result

A final flag or successful response rarely tells the full story. I captured the relevant setup, input or payload, execution, and observed result. I tried to make each screenshot readable and to connect it to a written step rather than leaving the reviewer to infer what it demonstrated.

My personal note-taking workflow used Flameshot on Kali for screenshots and Notion for notes, code blocks, images, tables, and structure. These were personal preferences, not required tools.

3. Focusing only on the chatbot interface

An AI-enabled environment is still an enterprise environment. Identity, storage, APIs, network exposure, cloud permissions, deployment configuration, and traditional application weaknesses may be as important as the model-facing interface.

I tried to examine the complete system rather than treating the language model as an isolated target.

4. Stopping after a basic prompt injection

A successful prompt injection can be an entry point rather than the final impact. When I observed one, I continued asking what the model could retrieve, which tools an agent could invoke, what data or state it could influence, and whether another component trusted its output.

The important question for me was not merely whether instructions could be overridden, but what security boundary could be crossed as a result.

5. Identifying capabilities without exercising them

Knowing that a tool or ingestion path exists is not the same as understanding its behavior. I tried to enumerate agent tools and the routes by which content could enter a retrieval system, then test what those components could actually reach or influence within the authorized scope.

6. Hiding non-determinism

LLM behavior is probabilistic, so I did not want a single favorable output to create a misleading impression of reliability. Where relevant, I recorded the number of attempts, the observed success rate, and the conditions that appeared to influence the result.

For example, “the behavior occurred in four of ten trials under these conditions” communicates more than presenting one successful response without context.

7. Accepting assistant-generated work without validation

Code or analysis produced by Claude, Codex, Hermes, or any other model still required testing. Regardless of where an idea originated, I relied on the behavior I had personally reproduced in the target environment.

I treated AI output as a hypothesis generator and accelerator—not as evidence.

8. Relying on one tool without understanding its limitations

Any hosted assistant can change behavior, lose context, become unavailable, or decline a task. Rather than assuming a particular service would behave exactly as expected, I familiarized myself with its limitations and retained the ability to continue manually when necessary.

Challenges helped me a lot in testing and understanding the limitations of every AI used. Even being in the CVP program with raised limitations, I still encountered them.

That did not mean duplicating every tool or building a mandatory local stack. It meant ensuring that my workflow and understanding did not depend entirely on a single assistant.

What I liked about OSAI

OSAI is a practical exam; it asks candidates to interact with an environment, form hypotheses, test them, demonstrate impact, and communicate the results.

In my view, allowing AI tools was also an appropriate design decision. AI red teamers use models to read code, build test harnesses, generate variations, analyze behavior, and accelerate documentation. The ability to use those tools critically is more representative of professional practice than pretending they do not exist.

For an experienced penetration tester, I found the subject to be a natural extension of familiar work. The core craft remained recognizable: enumerate carefully, understand trust relationships, control inputs, cross boundaries, demonstrate impact, and document the complete path.

The model layer changes the syntax of some attacks and introduces probabilistic behavior, but it does not invalidate the fundamentals of offensive security.

Final thoughts

The most useful preparation for me combined conventional penetration-testing experience with practical exposure to AI-enabled applications. Building small systems helped me understand how models, retrieval, tools, identities, and infrastructure fit together—and therefore where their boundaries could fail.

I used AI assistants actively but critically. Claude Code, /red-run-ctf, and HexStrike AI accelerated parts of my personal workflow. Ollama and Hermes had a place in my broader testing environment. None of them was a requirement, and none replaced enumeration, technical understanding, authorization, or validation.

My main lessons were to enumerate before going too deep, record non-deterministic behavior honestly, preserve meaningful evidence while working, and protect enough rest to maintain sound judgment. I treated reporting as part of the engagement itself rather than something to begin after the technical work was finished.