Most AI assessments start at the chat box. That is the front door of a building nobody has drawn a plan of. Here is how I draw the plan, and what I refuse to conclude from each source along the way.
Someone finds a chat box and starts throwing prompts at it. That is how most AI security work begins, and you will get interesting output that way.
You will also miss the system around the model. A production AI application usually has a gateway, an application layer, an orchestration framework, retrieval logic, a vector store, one or more tools holding real permissions, an inference service, and then the model. The chat box is the front door.
So I map the stack first. Not because prompt testing is unimportant, but because a prompt test you cannot explain is not a finding. It is an anecdote.
Passive does not mean invisible
The phrase "passive reconnaissance" gets used loosely, and if you are new to this, that vagueness will eventually cause you a problem.
Strictly, passive means no traffic reaches the target. Public repositories, package manifests, documentation, job adverts, certificate transparency logs, archived pages, conference talks by the engineering team.
Fetching a page, or just its headers, is a different thing. Low impact, indistinguishable from ordinary traffic, and still recorded. A reverse proxy, CDN, WAF or SIEM can all log it.
I call that low interaction instead of pretending it leaves no trace. The distinction decides four practical things: what your authorisation has to cover, whether you can trip an alert, how you describe the evidence in the report, and the moment the defenders can see you working.
If a packet reaches the target, assume someone can log it.
Keep a register from the first minute
Before collecting anything, I open a table. It sounds bureaucratic for a one-person job. It is also the habit that separates a report which survives review from one that does not.
| Observation | Source | Hypothesis | Confidence | Validation needed |
|---|---|---|---|---|
| requirements.txt imports crewai | Public repository | Agent orchestration with tool calling | Medium | Confirm the orchestrator is on the request path at runtime |
| .env.example references a demo vector store | Public repository | Retrieval over an external index | Medium | Confirm the index is queried per request |
| Job advert mentions vLLM and Kubernetes | Careers page | Self-hosted inference on a cluster | Low | Corroborate with a second independent source |
| Authorised request returns x-app-version | Response headers | Version disclosed at the edge | High | Check whether the value changes per deploy |
Two of those columns do the real work. Observation is what you saw. Hypothesis is what you think it means. People new to this collapse the two, and by Thursday "the repository imports crewai" has quietly become "the application runs CrewAI." Nobody decided to make that leap. It just happens when the note-taking allows it.
Confidence rates the evidence, not how good the theory sounds. A stale README can be extremely specific and completely wrong. A live response can be current and still name an upstream service that was decommissioned last quarter.
Job adverts deserve particular suspicion. They describe what a team wanted to hire for at some past moment, which may be a system that was never built, or one that has since been replaced.
What you are actually looking for
I work through nine layers:
- User interface and public API
- Gateway, authentication and routing
- Application layer
- Orchestration framework
- Retrieval and vector storage
- Agents, tools and external integrations
- Inference service
- Underlying model or model family
- Deployment infrastructure, identities and logging
Naming the layers stops you writing "an LLM" in the scope column and leaving it there. Each one leaks different evidence and carries different trust assumptions. For the demonstration environment, the starting hypothesis looks like this.
The boundaries matter more than the brand names, and two examples show why.
An OpenAI-compatible response format does not mean OpenAI hosts the model. vLLM, Ollama, LiteLLM and several gateways speak the same dialect. Likewise a CrewAI dependency does not mean every production request passes through CrewAI. Both are leads.
Source one: public repositories
Repositories give up architecture because developers have to declare dependencies and configuration somewhere, and that somewhere is usually version control.
I start with the files that describe how the thing is built and deployed: requirements.txt, pyproject.toml and package.json; Dockerfile and docker-compose.yml; anything under helm/, k8s/ or terraform/; then .env.example, config/*.yaml, prompts/ and tools/.
I am not hunting for credentials. A live key in a public repository is a different kind of engagement and a different conversation with the client. What I want is the shape of the system.
The demonstration repository keeps configuration, prompts and tool definitions separate from the build files. Its Python dependencies:
crewai>=0.41
pinecone-client>=3.0
fastapi>=0.109
uvicorn>=0.27
pydantic>=2.0Read that and the working hypothesis writes itself: a FastAPI application using CrewAI to orchestrate tasks, with a managed vector service behind retrieval.
Now the discipline. Dependencies get left behind after migrations, pulled in indirectly by something else, installed for a feature nobody enabled, or used only in development. Any of those produce exactly the manifest above.
A declared dependency is not a confirmed runtime component.
Configuration is where it gets interesting
Manifests tell you what is installed. Configuration tells you what someone intended to build, which is far more useful.
I look for the embedding provider and model, the vector store type and namespace, chunk size and overlap, retrieval count and score threshold, reranking, the inference endpoint and model identifier, tool names and parameter schemas, approval requirements, logging destinations, feature flags, and anything describing tenant isolation.
The demonstration configuration:
chunking:
strategy: text
chunk_size: 512
chunk_overlap: 100
embeddings:
provider: demo
model: demo-embedding-v1
dimensions: 768
retrieval:
top_k: 5
score_threshold: 0.75
distance_metric: cosine
vector_store:
provider: demo-vector-store
namespace: synthetic-documents
security:
tenant_filtering: required
expose_retrieval_scores: falseLook at what that disagrees with. The manifest declares pinecone-client. The configuration names a provider called demo-vector-store. Both cannot describe the running system.
That contradiction is worth more than either file on its own. It usually means one of three things: a migration that left the old client installed, an abstraction layer that swaps providers by configuration, or a split where development and production genuinely differ. Each implies a different attack surface. Each is cheap to record now and expensive to guess at later, when you are trying to explain why a test behaved oddly.
The rest of the file sets up later work. top_k: 5 with a 0.75 threshold tells you roughly how much retrieved text reaches the model, which matters when you get to indirect injection. tenant_filtering: required names a security boundary somebody thought about. Whether the running system enforces it is a separate question, and that question is the whole reason this file is interesting.
Tool definitions show intent
Tool schemas are the most valuable file in most AI repositories, because they enumerate the actions an agent is expected to take.
Two tools here. search_docs reads, and takes a namespace argument restricted to two values. create_ticket writes, sets requires_human_approval to true, and carries a description saying create-only with no update or close.
That is a permission boundary written down in one place. It is also a test list. Every constraint in that schema is a claim the server either enforces or does not, and the schema cannot tell you which.
Reading the headers, and leaving your first trace
Everything so far was stage one. This is where you cross into stage two, so check the authorisation covers it before you run anything.
curl -sS -D - -o /dev/null https://ai-lab.example.test/-D - writes the response headers to stdout, -o /dev/null throws the body away, and -sS hides the progress meter while keeping errors visible. One request, no body downloaded.
In the lab that returns a version header, a backend hint and a retrieval provider hint, which is generous by design. Real targets are usually stingier: a generic server value, no custom headers, everything interesting stripped at the CDN. That absence is information too. A stack that scrubs its headers is a stack where somebody has thought about disclosure, and you should expect the same care further in.
Two things worth noting when headers do appear. A version header that changes between deploys gives you a rough release cadence. A request ID is worth recording verbatim, because if you are working alongside a blue team, that value is how they find your traffic in their logs afterwards.
Turning findings into a validation plan
By now the register has a dozen entries and none of them are proven. The plan converts each one into a test small enough to be safe and specific enough to settle the question.
Five fields per hypothesis. What the evidence is, how confident you are in it, what it means for security if it turns out true, the smallest first test, and a status that starts at not tested and does not move until you have run something.
The smallest test is the part people get wrong. For the ticket tool, the schema claims create-only. The test is not a fuzzing run against the ticketing API. It is one attempted update in the lab tenant, watching whether the server rejects it. That single request distinguishes "the schema says so" from "the server enforces it," which was the entire question.
Notice also what the plan does for your report. When a finding is challenged three months later, there is a real difference between "the application uses CrewAI" and "the public manifest declared CrewAI, and the orchestration behaviour observed on 12 March was consistent with it." One is an assertion. The other is evidence with a date on it.
Four things I see people new to this get wrong
Starting at the chat box. Prompt injection is the interesting part, so it gets attempted first, against a system nobody has mapped. When something odd comes back there is no way to explain it.
Promoting a dependency to a fact. Covered above, and it happens to experienced testers too. The fix is structural: keep the columns separate so the leap requires a deliberate edit.
Losing the source. An observation without a source is not evidence. Six weeks later you will not remember which of eleven repositories the config came from, and neither will anyone reviewing the report.
Forgetting that stage two is loggable. Header checks against production can land in a SOC queue as an unexplained scan. Tell the client's team you are doing it, or accept that someone will spend an afternoon investigating you.
What this actually buys you
Nothing here proves anything. What you have at the end is a ranked set of hypotheses, each tied to specific evidence, each with a confidence rating and a note on what would confirm it.
That is the point. Testing an AI system without the map means sending prompts and interpreting whatever comes back with no model of what produced it. Testing with the map means every request is aimed at a question you already wrote down, and every result lands somewhere.
The next step is deciding which hypotheses justify active testing, and designing the smallest experiment that answers each. That is a different discipline with its own authorisation requirements, and it is worth its own article.