# Blocking prompt injection with streaming AST filters

[Skip to content](#lm-inhoud)Network/[NL](/en/prompt-injection-blokkeren-via-streaming-ast-filters)EN[Hubhub.llmnet.nlCompare models on task, language, cost and license.](https://hub.llmnet.nl/en/)[Communitycommunity.llmnet.nlPrompt techniques, patterns and system prompts.](https://community.llmnet.nl/en/)[APIapi.llmnet.nlLLMs in production: rate limits, routing, structured output.](https://api.llmnet.nl/en/)[Consultancyconsultancy.llmnet.nlRolling out AI in an organization, pilot to production.](https://consultancy.llmnet.nl/en/)[Newsnieuws.llmnet.nlAI developments, explained for the Netherlands.](https://nieuws.llmnet.nl/en/)[Benchmarkbenchmark.llmnet.nlMeasure AI quality yourself, on your own tasks.](https://benchmark.llmnet.nl/en/)[Careersvacatures.llmnet.nlAI roles, salaries and career paths in the Netherlands.](https://vacatures.llmnet.nl/en/)[Learnleren.llmnet.nlAI concepts in plain language, beginner to builder.](https://leren.llmnet.nl/en/)[Guidegids.llmnet.nlRun AI privately on your own Mac, PC, NAS or home server.](https://gids.llmnet.nl/en/)[Directorydirectory.llmnet.nlMapping the AI ecosystem: tools, models, companies.](https://directory.llmnet.nl/en/)[Radarradar.llmnet.nlSignals from X, research and communities for indie developers.](https://radar.llmnet.nl/en/)[Appsapps.llmnet.nlReviews of AI apps and open-source repos, with tips for builders.](https://apps.llmnet.nl/en/)[llmnet.nl — main site](https://llmnet.nl/en/)[](https://x.com/intent/post?url=https%3A%2F%2Fradar.llmnet.nl%2Fen%2Fprompt-injection-blokkeren-via-streaming-ast-filters&text=Blocking%20prompt%20injection%20with%20streaming%20AST%20filters)[](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fradar.llmnet.nl%2Fen%2Fprompt-injection-blokkeren-via-streaming-ast-filters)[](https://www.reddit.com/submit?url=https%3A%2F%2Fradar.llmnet.nl%2Fen%2Fprompt-injection-blokkeren-via-streaming-ast-filters&title=Blocking%20prompt%20injection%20with%20streaming%20AST%20filters)[](#)[](https://x.com/intent/post?url=https%3A%2F%2Fradar.llmnet.nl%2Fen%2Fprompt-injection-blokkeren-via-streaming-ast-filters&text=Blocking%20prompt%20injection%20with%20streaming%20AST%20filters)[](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fradar.llmnet.nl%2Fen%2Fprompt-injection-blokkeren-via-streaming-ast-filters)[](https://www.reddit.com/submit?url=https%3A%2F%2Fradar.llmnet.nl%2Fen%2Fprompt-injection-blokkeren-via-streaming-ast-filters&title=Blocking%20prompt%20injection%20with%20streaming%20AST%20filters)[](#)

 
# Blocking prompt injection with streaming AST filters

 By Ivo Donker — compiled with AI assistance (Claude & Gemini)

 When building autonomous agents and integrations with large language models, prompt injection forms a persistent structural risk. As soon as a model processes external data via Retrieval-Augmented Generation or reads incoming payloads, malicious instructions can take control of the execution flow. Classic solutions such as static regex filters or post-generation evaluations fall short as soon as applications rely on streaming responses and direct tool execution.

 When an agent translates streamed tokens directly into function calls, a single malicious payload can already cause damage before the full API response has completed. In this article we examine the architecture of streaming Abstract Syntax Tree (AST) filters. We analyze how incremental parsers isolate suspicious syntax and payload hijacking during the generation process, without letting time-to-first-token increase unacceptably.

 
## The fundamental vulnerability of streaming tool execution

 Classic web applications maintain a strict separation between program code and user data. Within LLM-based systems, however, instructions and data flow through the same semantic context. To understand the theoretical foundations behind this separation problem, the article on [separating instructions from data as the foundation of prompt security](https://community.llmnet.nl/en/instructies-gescheiden-van-data-de-basis-van-promptbeveiliging) offers a thorough conceptual introduction. As soon as a model communicates with external systems via tool calling or structured JSON outputs, an indirect instruction can manipulate the structure of the response. If an attacker manages to force an escape from the data context via retrieved website data, the model generates valid syntax for a dangerous tool call, such as overwriting configuration files or sending internal session tokens to an external IP address.

 The defense problem escalates when systems are optimized for minimal latency. With streaming responses, JSON fragments are evaluated token by token and forwarded to the client or runtime. A traditional guardrail only checks the complete payload after the stop_token has been received. In streaming agent execution, this leads to an impossible trade-off: either latency doubles because the stream must be buffered, or the runtime already partially executes the tool call while the payload is still coming in. Anyone who wants to dive deeper into how applications falter when streams are interrupted midway can consult the analysis on [streaming with tool calls and interrupted API calls](https://api.llmnet.nl/en/streaming-met-tool-calls-gedeeltelijke-output-en-afgebroken-calls) is worth consulting.

 
 Signal: Increasing exploitation of parser mismatches between LLM tokenizers and backend JSON interpreters.
 Action: Implement incremental streaming validation directly at the protocol level of the model gateway.
 

 
## Why regular expressions and static guards fail

 Many development teams try to secure token streams with regular expressions on incoming chunks. This approach is fundamentally fragile. Regular expressions do not understand recursive grammars and are blind to semantic context. An attacker can split strings across multiple tokens, inject hexadecimal encodings, or nest JSON keys so that regex patterns fail to detect a match.

 Moreover, LLMs generate tokens in variable sizes. A regex that searches for rm -rf misses the injection when the tokenizer splits the input into r, m, - and rf across consecutive server-sent events. By the time a naive buffer recognizes the command, the argument buffer in the runtime is already filled. In the broader context of system integrations, we saw similar patterns in the dossier on [agent security incidents and lessons learned](/en/agent-security-incidenten-juli-2026), in which unvalidated argument streams led to unauthorized data extraction.

 
 
 
 
 Filter mechanism | 
 Processing type | 
 Latency overhead | 
 Resistance to splitting | 
 Grammatical context | 
 

 
 
 
 Regular Expression | 
 Chunk-based | 
 < 1 ms | 
 Very low (fails on token boundaries) | 
 None | 
 

 
 Secondary Guard Model | 
 Post-evaluation | 
 300 - 800 ms | 
 High | 
 Semantic, not syntactic | 
 

 
 Full JSON validation | 
 Buffered (end of stream) | 
 Equal to total generation duration | 
 High | 
 Fully static | 
 

 
 Streaming AST parser | 
 Incremental (per token) | 
 1-4 ms per token | 
 High (node-based validation) | 
 Fully deterministic | 
 

 
 
 

 
## Architecture of a streaming AST filter

 A streaming AST filter sits directly between the HTTP transport layer of the LLM provider and the runtime dispatcher of the agent. Instead of textual string matching, the filter incrementally builds a syntax tree as the tokens trickle in. As soon as a new token arrives, the lexer performs a state transition in a Pushdown Automaton (PDA). This allows the system to know at all times exactly which syntactic context the token falls into: within a function name, within a parameter key, or deeply nested in a string value.

 This architecture enables structural validation before the full tree is complete. If a model tries to call a function name that does not appear in the assigned JSON schema, or if a parameter value contains an escape sequence that tries to inject a nested expression, the parser immediately terminates the connection. The stream is cut off with a TCP RST or a controlled protocol message, before the application layer can even pass the payload through to a shell or database.

 For those running agent architectures on their own servers, this connects directly to the methods described in the overview on [agent runtime security for production environments](/en/agent-runtime-security-juli-2026), which centers on defensive isolation at the process level.

 
## Incremental JSON and BNF state machines in practice

 To understand how a streaming AST works, we look at the state transitions of an incremental JSON parser. When receiving streamed tool calls, the parser must handle incomplete states (such as unclosed quotation marks or missing closing braces) without throwing a fatal parsing error. The parser uses a transition table that tracks specific grammar states:

 // Pseudocode voor een incrementele streaming AST node inspector
interface ASTNode {
 type: 'Program' | 'FunctionCall' | 'Identifier' | 'Literal' | 'Object' | 'Array';
 name?: string;
 value?: any;
 parent?: ASTNode;
 children: ASTNode[];
 isComplete: boolean;
}

class StreamingASTFilter {
 private state: 'IDLE' | 'IN_FUNCTION' | 'IN_PARAMS' | 'IN_STRING' | 'BLOCKED' = 'IDLE';
 private currentPath: string[] = [];
 private tokenBuffer: string = '';

 public processChunk(tokenChunk: string): boolean {
 this.tokenBuffer += tokenChunk;
 
 // Voer lexicale tokenisatie uit op de openstaande buffer
 const transition = this.evaluateNextState(tokenChunk);
 
 if (!this.isValidTransition(transition)) {
 this.state = 'BLOCKED';
 return false; // Signaleer de gateway om de stream onmiddellijk af te breken
 }

 // Inspecteer structurele knooppunten
 if (this.detectInjectionPattern(this.currentPath, tokenChunk)) {
 this.state = 'BLOCKED';
 return false;
 }

 return true;
 }

 private isValidTransition(nextState: string): boolean {
 // Valideer of de transitie binnen de toegestane BNF-grammatica valt
 return nextState !== 'INVALID_SYNTAX';
 }

 private detectInjectionPattern(path: string[], rawToken: string): boolean {
 // Blokkeer pogingen om shell-karakters of controle-instructies in specifieke velden te stoppen
 if (path.includes('arguments') && path.includes('command')) {
 const dangerousPatterns = [';', '&&', '||', '`', '$(', '\n'];
 return dangerousPatterns.some(char => rawToken.includes(char));
 }
 return false;
 }
}

 The parser builds the tree recursively. As soon as the node FunctionCall.arguments.query is opened, the filter knows the security policy for that specific field. A search query may contain free text, but may not contain nested structures that simulate SQL tokens within a JSON value. This fine-grained per-node control prevents false positives in innocuous text streams.

 
## Detection mechanisms for syntax escape and context hijacking

 Attackers applying prompt injection to agents generally try to achieve two things: instruction hijacking (the model ignores the system prompt) or structure hijacking (the model is forced into a syntax escape from the data string). In structure hijacking, the payload injects specific sequence delimiters, such as unescaped quotes followed by JSON structures.

 A streaming AST filter recognizes this because the state machine registers an unauthorized state change. If a data field suddenly transitions into a new key-value structure without the correct escape characters being present in the lexer context, the filter flags an anomaly. The filter inspects not only the static string but also checks whether the depth of the AST matches the expected function schema.

 To measurably map such vulnerabilities and validate how effectively a filter performs against advanced attacks, it is advisable to study the methodology at [systematically measuring resilience against indirect prompt injection](https://benchmark.llmnet.nl/en/indirecte-prompt-injection-weerbaarheid-systematisch-meten). Without standardized evaluation sets, securing parser logic remains, after all, a guessing game.

 
## Formal grammars and determinism in agentic pipelines

 In addition to blocking known exploit patterns, an AST filter also helps prevent structural chaos during code generation. When an agent autonomously produces scripts or delivers patch files, a hallucinating or injected closing element often leads to corrupted files. By directly enforcing formal grammars (such as GBNF or JSON Schema regexes) on the logits during the inference stream, a mathematically airtight corset is created.

 How to set up such formal constraints to guarantee that generated code blocks remain syntactically flawless is explained in detail in the guide on [enforcing determinism in agentic code generation](/en/determinisme-afdwingen-bij-agentic-code-generatie). This determinism significantly reduces the chance that an injection attack lures the model into producing unexpected control commands.

 
## Latency, TTFT, and resource impact in edge environments

 One of the biggest objections to heavy security layers around LLMs is the increase in Time-To-First-Token (TTFT) and total CPU usage. Checking a streamed response must not become a bottleneck in the latency-sensitive inference chain. Fortunately, deterministic AST state machines are computationally extremely light compared to running secondary neural networks for content moderation.

 
 
 
 
 Infrastructure | 
 Memory per stream | 
 Parser overhead per token | 
 Max throughput per core | 
 

 
 
 
 Node.js Worker | 
 ~ 120 KB | 
 1.2-2.8 ms | 
 ~ 1,400 req/s | 
 

 
 Rust / WASM (Cloudflare Worker) | 
 ~ 18 KB | 
 0.08-0.25 ms | 
 ~ 18,000 req/s | 
 

 
 Go Reverse Proxy | 
 ~ 32 KB | 
 0.15-0.40 ms | 
 ~ 12,500 req/s | 
 

 
 Python (Asyncio ASGI) | 
 ~ 450 KB | 
 3.5-7.0 ms | 
 ~ 450 req/s | 
 

 
 
 

 The table shows that implementations in Rust or Go have a negligible impact on total response time. Because the parser works incrementally and holds state in memory via a compact pointer structure, no repeated allocation is needed. Tokens are forwarded to the client socket immediately after inspection, unless the filter triggers a block.

 
## Integration with runtime sandboxing and permission models

 A streaming AST filter forms the first line of defense, but does not solve every risk. If an attack generates syntactically valid arguments that fit within the schema but are semantically malicious (for example, reading a file that falls within the valid file structure but contains sensitive data), the defense must fall back on the application runtime.

 The most effective architecture combines the streaming filter with strict containerization and process isolation. As soon as the AST filter approves the syntax and validates the JSON payload, the tool is executed within a short-lived sandbox with minimal privileges. In the practical dossier on [sandboxing LLM tools with Docker isolation](/en/sandboxing-van-llm-tools-docker-isolatie-in-de-praktijk) it is worked out step by step how network restrictions and read-only mounts prevent a breached payload from still causing lateral damage to the host system.

 
## Implementation pitfalls and blind spots

 When rolling out streaming AST inspection, developers run into a number of specific technical challenges. We list the most important pitfalls below:

 First: Unicode and multibyte splitting. An LLM can split a multibyte UTF-8 character across two separate streaming chunks. If the lexer tries to decode this character at the buffer boundary, the parser can crash or misinterpret the token. A robust lexer must hold incomplete byte sequences in a sliding window until the code point is complete.

 Second: Syntactic ambiguity in JSON streaming. As long as a string is not closed with a quote, a parser cannot determine with certainty whether the string is complete. Attempts to perform intermediate evaluations on incomplete strings require heuristics that are tolerant of missing terminators, without undermining validation strength.

 Third: State exhaustion through memory attacks. If a model is forced by an attack to generate infinitely long JSON keys or deeply nested objects, the parser can succumb to a resource exhaustion attack. A streaming AST filter must enforce hard limits on tree depth (a maximum of 8 levels) and individual token length (a maximum of 4,096 bytes per string value).

 
## The balance between deterministic inspection and flexibility

 Streaming AST filters offer a mathematically verifiable and deterministic method for checking streamed payloads from language models. By moving control from slow, unpredictable LLM guardrails to fast, incremental parsers at the protocol level, the user experience remains fast while dangerous tool execution is nipped in the bud.

 For anyone developing serious agentic systems, filtering streams is no longer an optional luxury but a fundamental part of the gateway infrastructure. Combined with formal JSON schemas, runtime sandboxes, and strict permission models, the streaming AST filter forms the necessary buffer between the unpredictability of probabilistic models and the hard security requirements of modern production systems.