Structured Output Backends

Structured output backends ensure reliable data extraction from LLM responses. Cognee supports two frameworks that convert LLM text into structured Pydantic models for knowledge graph extraction and other tasks.

New to configuration?See the Setup Configuration Overview for the complete workflow:install extras → create .env → choose providers → handle pruning.

Supported Frameworks

Cognee supports two structured output approaches:

LiteLLM + Instructor — Provider-agnostic client with Pydantic coercion (default)
BAML — DSL-based framework with type registry and guardrails

Both frameworks produce the same Pydantic-validated outputs, so your application code remains unchanged regardless of which backend you choose.

How It Works

Cognee uses a unified interface that abstracts the underlying framework:

from cognee.infrastructure.llm.LLMGateway import LLMGateway
await LLMGateway.acreate_structured_output(text, system_prompt, response_model)

The STRUCTURED_OUTPUT_FRAMEWORK environment variable determines which backend processes your requests, but the API remains identical.

Configuration

LiteLLM + Instructor (Default)
BAML

The default framework — no extra install needed. Uses LiteLLM and the instructor library to coerce LLM responses into Pydantic models.

STRUCTURED_OUTPUT_FRAMEWORK=instructor

Optionally, control how the model is prompted for structured output:

# Override instructor mode (e.g. json_mode, tool_call, md_json)
LLM_INSTRUCTOR_MODE=json_schema_mode

BAML is an alternative structured output framework that uses a DSL-based type registry to extract data. It is particularly useful when small local models (such as Ollama models like llama3.1:8b or qwen3.5:0.8b) struggle to produce valid structured output with instructor, causing repeated InstructorRetryException errors.Installation: BAML requires a separate install:

pip install "cognee[baml]"

Configuration: BAML uses its own LLM settings, independent of the main LLM_* variables:

STRUCTURED_OUTPUT_FRAMEWORK=baml

# BAML-specific LLM settings (required)
BAML_LLM_PROVIDER=openai
BAML_LLM_MODEL=gpt-4o-mini
BAML_LLM_API_KEY=sk-...

# Optional BAML overrides
# BAML_LLM_ENDPOINT=https://api.openai.com/v1
# BAML_LLM_API_VERSION=
# BAML_LLM_TEMPERATURE=0.0

BAML_LLM_PROVIDER and BAML_LLM_MODEL accept the same provider names and model identifiers as the main LLM configuration. You can point BAML at a different model than your main LLM — for example, use a small Ollama model for general text generation while routing structured extraction through a cloud model.

If STRUCTURED_OUTPUT_FRAMEWORK=baml is set but the cognee[baml] extra is not installed, Cognee will raise an ImportError on startup. Run pip install "cognee[baml]" to resolve it.

Using BAML for Small Local Models

Small Ollama models (e.g. llama3.1:8b, qwen3.5:0.8b) often fail to produce valid JSON-structured output when using the default instructor backend, resulting in repeated InstructorRetryException errors during cognify for types like KnowledgeGraph or SummarizedContent.Switching to BAML bypasses instructor entirely and uses BAML’s own extraction pipeline, which is more forgiving with smaller models:

# Main LLM — small local model via Ollama
LLM_PROVIDER=ollama
LLM_MODEL=llama3.1:8b
LLM_ENDPOINT=http://localhost:11434/v1
LLM_API_KEY=ollama

# Use BAML for structured extraction (can point to a different, more capable model)
STRUCTURED_OUTPUT_FRAMEWORK=baml
BAML_LLM_PROVIDER=openai
BAML_LLM_MODEL=gpt-4o-mini
BAML_LLM_API_KEY=sk-...

See the Local Setup guide for a complete Ollama configuration including embeddings.

Important Notes

Unified Interface: Your application code uses the same acreate_structured_output() call regardless of framework
Provider Flexibility: Both frameworks support the same LLM providers
Output Consistency: Both produce identical Pydantic-validated results
Performance: Framework choice doesn’t significantly impact performance

LLM Providers

Configure LLM providers for text generation

Overview

Return to setup configuration overview

Custom Prompts

Learn about custom prompt configuration

Getting Started

Core Concepts

Setup Configuration

Guides

Examples

CLI

OSS

Supported Frameworks

How It Works

Configuration

Important Notes

LLM Providers

Overview

Custom Prompts

Getting Started

Core Concepts

Setup Configuration

Guides

Examples

CLI

OSS

​Supported Frameworks

​How It Works

​Configuration

​Important Notes

LLM Providers

Overview

Custom Prompts

Supported Frameworks

How It Works

Configuration

Important Notes