Low-Level LLM

• Complete Quickstart to understand basic operations
• Ensure you have LLM Providers configured
• Have some text to process

​

What It Is

• Single entrypoint: LLMGateway.acreate_structured_output(text, system_prompt, response_model)
• Returns an instance of your Pydantic response_model filled by the LLM
• Backend-agnostic: uses BAML or LiteLLM+Instructor under the hood based on config — your code doesn’t change

​

Code in Action

​

Step 1: Define Your Schema

class MiniEntity(BaseModel):
    name: str
    type: str

class MiniGraph(BaseModel):
    nodes: List[MiniEntity]

​

Step 2: Write a System Prompt

system_prompt = (
    "Extract entities as nodes with name and type. "
    "Use concise, literal values present in the text."
)

​

Step 3: Call the LLM

result = await LLMGateway.acreate_structured_output(text, system_prompt, MiniGraph)

​

Custom Tasks

​

Backend Doesn’t Matter

• STRUCTURED_OUTPUT_FRAMEWORK=instructor → LiteLLM + Instructor
• STRUCTURED_OUTPUT_FRAMEWORK=baml → BAML client/registry
• STRUCTURED_OUTPUT_FRAMEWORK=litellm_native → LiteLLM native structured output

​

Full Example

import asyncio

from typing import List
from pydantic import BaseModel
from cognee.infrastructure.llm.LLMGateway import LLMGateway


class MiniEntity(BaseModel):
    name: str
    type: str


class MiniGraph(BaseModel):
    nodes: List[MiniEntity]


async def main():
    system_prompt = (
        "Extract entities as nodes with name and type. "
        "Use concise, literal values present in the text."
    )

    text = "Apple develops iPhone; Audi produces the R8."

    result = await LLMGateway.acreate_structured_output(text, system_prompt, MiniGraph)
    print(result)
    # MiniGraph(nodes=[MiniEntity(name='Apple', type='Organization'), ...])

if __name__ == "__main__":
    asyncio.run(main())