Blocking prompt injection with streaming AST filters

When building autonomous agents and integrations with large language models, prompt injection forms a persistent structural risk. As soon as a model processes external data via Retrieval-Augmented Generation or reads incoming payloads, malicious instructions can take control of the execution flow. Classic solutions such as static regex filters or post-generation evaluations fall short as soon as applications rely on streaming responses and direct tool execution.

When an agent translates streamed tokens directly into function calls, a single malicious payload can already cause damage before the full API response has completed. In this article we examine the architecture of streaming Abstract Syntax Tree (AST) filters. We analyze how incremental parsers isolate suspicious syntax and payload hijacking during the generation process, without letting time-to-first-token increase unacceptably.

The fundamental vulnerability of streaming tool execution

Classic web applications maintain a strict separation between program code and user data. Within LLM-based systems, however, instructions and data flow through the same semantic context. To understand the theoretical foundations behind this separation problem, the article on separating instructions from data as the foundation of prompt security offers a thorough conceptual introduction. As soon as a model communicates with external systems via tool calling or structured JSON outputs, an indirect instruction can manipulate the structure of the response. If an attacker manages to force an escape from the data context via retrieved website data, the model generates valid syntax for a dangerous tool call, such as overwriting configuration files or sending internal session tokens to an external IP address.

The defense problem escalates when systems are optimized for minimal latency. With streaming responses, JSON fragments are evaluated token by token and forwarded to the client or runtime. A traditional guardrail only checks the complete payload after the stop_token has been received. In streaming agent execution, this leads to an impossible trade-off: either latency doubles because the stream must be buffered, or the runtime already partially executes the tool call while the payload is still coming in. Anyone who wants to dive deeper into how applications falter when streams are interrupted midway can consult the analysis on streaming with tool calls and interrupted API calls is worth consulting.

Why regular expressions and static guards fail

Many development teams try to secure token streams with regular expressions on incoming chunks. This approach is fundamentally fragile. Regular expressions do not understand recursive grammars and are blind to semantic context. An attacker can split strings across multiple tokens, inject hexadecimal encodings, or nest JSON keys so that regex patterns fail to detect a match.

Moreover, LLMs generate tokens in variable sizes. A regex that searches for rm -rf misses the injection when the tokenizer splits the input into r, m, - and rf across consecutive server-sent events. By the time a naive buffer recognizes the command, the argument buffer in the runtime is already filled. In the broader context of system integrations, we saw similar patterns in the dossier on agent security incidents and lessons learned, in which unvalidated argument streams led to unauthorized data extraction.

Architecture of a streaming AST filter

A streaming AST filter sits directly between the HTTP transport layer of the LLM provider and the runtime dispatcher of the agent. Instead of textual string matching, the filter incrementally builds a syntax tree as the tokens trickle in. As soon as a new token arrives, the lexer performs a state transition in a Pushdown Automaton (PDA). This allows the system to know at all times exactly which syntactic context the token falls into: within a function name, within a parameter key, or deeply nested in a string value.

This architecture enables structural validation before the full tree is complete. If a model tries to call a function name that does not appear in the assigned JSON schema, or if a parameter value contains an escape sequence that tries to inject a nested expression, the parser immediately terminates the connection. The stream is cut off with a TCP RST or a controlled protocol message, before the application layer can even pass the payload through to a shell or database.

For those running agent architectures on their own servers, this connects directly to the methods described in the overview on agent runtime security for production environments, which centers on defensive isolation at the process level.

Incremental JSON and BNF state machines in practice

To understand how a streaming AST works, we look at the state transitions of an incremental JSON parser. When receiving streamed tool calls, the parser must handle incomplete states (such as unclosed quotation marks or missing closing braces) without throwing a fatal parsing error. The parser uses a transition table that tracks specific grammar states:

// Pseudocode voor een incrementele streaming AST node inspector
interface ASTNode {
  type: 'Program' | 'FunctionCall' | 'Identifier' | 'Literal' | 'Object' | 'Array';
  name?: string;
  value?: any;
  parent?: ASTNode;
  children: ASTNode[];
  isComplete: boolean;
}

class StreamingASTFilter {
  private state: 'IDLE' | 'IN_FUNCTION' | 'IN_PARAMS' | 'IN_STRING' | 'BLOCKED' = 'IDLE';
  private currentPath: string[] = [];
  private tokenBuffer: string = '';

  public processChunk(tokenChunk: string): boolean {
    this.tokenBuffer += tokenChunk;
    
    // Voer lexicale tokenisatie uit op de openstaande buffer
    const transition = this.evaluateNextState(tokenChunk);
    
    if (!this.isValidTransition(transition)) {
      this.state = 'BLOCKED';
      return false; // Signaleer de gateway om de stream onmiddellijk af te breken
    }

    // Inspecteer structurele knooppunten
    if (this.detectInjectionPattern(this.currentPath, tokenChunk)) {
      this.state = 'BLOCKED';
      return false;
    }

    return true;
  }

  private isValidTransition(nextState: string): boolean {
    // Valideer of de transitie binnen de toegestane BNF-grammatica valt
    return nextState !== 'INVALID_SYNTAX';
  }

  private detectInjectionPattern(path: string[], rawToken: string): boolean {
    // Blokkeer pogingen om shell-karakters of controle-instructies in specifieke velden te stoppen
    if (path.includes('arguments') && path.includes('command')) {
      const dangerousPatterns = [';', '&&', '||', '`', '$(', '\n'];
      return dangerousPatterns.some(char => rawToken.includes(char));
    }
    return false;
  }
}

The parser builds the tree recursively. As soon as the node FunctionCall.arguments.query is opened, the filter knows the security policy for that specific field. A search query may contain free text, but may not contain nested structures that simulate SQL tokens within a JSON value. This fine-grained per-node control prevents false positives in innocuous text streams.

Detection mechanisms for syntax escape and context hijacking

Attackers applying prompt injection to agents generally try to achieve two things: instruction hijacking (the model ignores the system prompt) or structure hijacking (the model is forced into a syntax escape from the data string). In structure hijacking, the payload injects specific sequence delimiters, such as unescaped quotes followed by JSON structures.

A streaming AST filter recognizes this because the state machine registers an unauthorized state change. If a data field suddenly transitions into a new key-value structure without the correct escape characters being present in the lexer context, the filter flags an anomaly. The filter inspects not only the static string but also checks whether the depth of the AST matches the expected function schema.

To measurably map such vulnerabilities and validate how effectively a filter performs against advanced attacks, it is advisable to study the methodology at systematically measuring resilience against indirect prompt injection. Without standardized evaluation sets, securing parser logic remains, after all, a guessing game.

Formal grammars and determinism in agentic pipelines

In addition to blocking known exploit patterns, an AST filter also helps prevent structural chaos during code generation. When an agent autonomously produces scripts or delivers patch files, a hallucinating or injected closing element often leads to corrupted files. By directly enforcing formal grammars (such as GBNF or JSON Schema regexes) on the logits during the inference stream, a mathematically airtight corset is created.

How to set up such formal constraints to guarantee that generated code blocks remain syntactically flawless is explained in detail in the guide on enforcing determinism in agentic code generation. This determinism significantly reduces the chance that an injection attack lures the model into producing unexpected control commands.

Latency, TTFT, and resource impact in edge environments

One of the biggest objections to heavy security layers around LLMs is the increase in Time-To-First-Token (TTFT) and total CPU usage. Checking a streamed response must not become a bottleneck in the latency-sensitive inference chain. Fortunately, deterministic AST state machines are computationally extremely light compared to running secondary neural networks for content moderation.

The table shows that implementations in Rust or Go have a negligible impact on total response time. Because the parser works incrementally and holds state in memory via a compact pointer structure, no repeated allocation is needed. Tokens are forwarded to the client socket immediately after inspection, unless the filter triggers a block.

Integration with runtime sandboxing and permission models

A streaming AST filter forms the first line of defense, but does not solve every risk. If an attack generates syntactically valid arguments that fit within the schema but are semantically malicious (for example, reading a file that falls within the valid file structure but contains sensitive data), the defense must fall back on the application runtime.

The most effective architecture combines the streaming filter with strict containerization and process isolation. As soon as the AST filter approves the syntax and validates the JSON payload, the tool is executed within a short-lived sandbox with minimal privileges. In the practical dossier on sandboxing LLM tools with Docker isolation it is worked out step by step how network restrictions and read-only mounts prevent a breached payload from still causing lateral damage to the host system.

Implementation pitfalls and blind spots

When rolling out streaming AST inspection, developers run into a number of specific technical challenges. We list the most important pitfalls below:

First: Unicode and multibyte splitting. An LLM can split a multibyte UTF-8 character across two separate streaming chunks. If the lexer tries to decode this character at the buffer boundary, the parser can crash or misinterpret the token. A robust lexer must hold incomplete byte sequences in a sliding window until the code point is complete.

Second: Syntactic ambiguity in JSON streaming. As long as a string is not closed with a quote, a parser cannot determine with certainty whether the string is complete. Attempts to perform intermediate evaluations on incomplete strings require heuristics that are tolerant of missing terminators, without undermining validation strength.

Third: State exhaustion through memory attacks. If a model is forced by an attack to generate infinitely long JSON keys or deeply nested objects, the parser can succumb to a resource exhaustion attack. A streaming AST filter must enforce hard limits on tree depth (a maximum of 8 levels) and individual token length (a maximum of 4,096 bytes per string value).

The balance between deterministic inspection and flexibility

Streaming AST filters offer a mathematically verifiable and deterministic method for checking streamed payloads from language models. By moving control from slow, unpredictable LLM guardrails to fast, incremental parsers at the protocol level, the user experience remains fast while dangerous tool execution is nipped in the bud.

For anyone developing serious agentic systems, filtering streams is no longer an optional luxury but a fundamental part of the gateway infrastructure. Combined with formal JSON schemas, runtime sandboxes, and strict permission models, the streaming AST filter forms the necessary buffer between the unpredictability of probabilistic models and the hard security requirements of modern production systems.