Cognee’s LLM-agnostic architecture allows you to switch between providers or use multiple models for different tasks within the same application.

​
Usage

To use an LLM, you must provide a configuration to customize its usage. If no configuration is supplied, a default configuration will be applied, and OpenAI will be used as the LLM provider. For a comprehensive list of available parameters for LLM configuration, please refer to Configuration.

​
Basic Configuration

import os
import cognee

# Default OpenAI configuration
os.environ["LLM_API_KEY"] = "your-openai-api-key"

# Cognee will use GPT-4 by default
await cognee.add("Your data")
await cognee.cognify()

​
Supported LLMs

See the list of supported LLMs below. All LLMs are supported in Python:

OpenAI

GPT-4, GPT-3.5-turbo, GPT-4-turbo

Anthropic

Claude 3.5 Sonnet, Haiku, Opus

Azure OpenAI

Enterprise OpenAI models

Ollama

Llama, Mistral, CodeLlama

Together

Llama, Mixtral, Code models

Groq

Ultra-fast inference models

LiteLLM

100+ model providers

Mistral AI

Mistral 7B, 8x7B, Large

Google AI

Gemini Pro, Gemini Flash

AWS Bedrock

Claude, Llama via Bedrock

Hugging Face

Open source models

Local Models

Self-hosted inference

​
LLM Tasks in Cognee

Structured Data ExtractionLLMs analyze text to identify and extract structured information:
# Example entity extraction
text = "Apple Inc. was founded by Steve Jobs in Cupertino, California."

# Cognee extracts:
# - Entity: "Apple Inc." (Organization)
# - Entity: "Steve Jobs" (Person)  
# - Entity: "Cupertino, California" (Location)
# - Relationship: "Steve Jobs" -> "founded" -> "Apple Inc."

​
Structured vs Unstructured Outputs

Cognee supports different output formats depending on your needs:

Structured Outputs

​
Model Selection Guide

Production Applications
  • OpenAI GPT-4: Best overall performance and reliability
  • Claude 3.5 Sonnet: Excellent for analysis and long contexts
  • Azure OpenAI: Enterprise security and compliance
Development & Testing
  • GPT-3.5-turbo: Good balance of performance and cost
  • Claude Haiku: Fast and cost-effective
  • Local Ollama: Free and private
Specialized Tasks
  • Code Analysis: GPT-4, Claude 3.5 Sonnet
  • Multilingual: GPT-4, Gemini Pro
  • Long Documents: Claude 3.5 (200k context)

​
Configuration Examples

import os

# Use different models for different tasks
os.environ["LLM_PROVIDER"] = "openai"
os.environ["LLM_MODEL"] = "gpt-4"  # For complex reasoning

os.environ["EMBEDDING_PROVIDER"] = "openai" 
os.environ["EMBEDDING_MODEL"] = "text-embedding-3-small"  # For embeddings

# Alternative: Use local embeddings to reduce costs
# os.environ["EMBEDDING_PROVIDER"] = "fastembed"

​
Next Steps

OpenAI Setup

Configure OpenAISet up OpenAI models with API keys and advanced parameters.

Anthropic Setup

Configure ClaudeIntegrate Anthropic’s Claude models for advanced reasoning.

Azure OpenAI

Enterprise SetupConfigure Azure OpenAI for enterprise deployments.

Local Models

Privacy-FirstRun models locally with Ollama and other providers.