Generate valid JSON and code structures

Guide LLM sampling using formal grammars and control flow to ensure valid output

Batch generation where you need guaranteed valid JSON/SQL/code and sampling speed is critical.

• · LLM model
• · grammar or control flow spec
• · prompt

• · valid structured output

• · outlines
• · vLLM or other inference engine

Grammar is unambiguous and terminating; LLM tokenizer matches grammar expectations

Grammar mismatch with tokenizer causes infinite backtrack; ambiguous grammar produces slow generation

• · Used in Together.ai production
• · Supports vLLM backend