Ollama

Ollama enables you to run embedding models locally on your own infrastructure, providing complete control over your embedding generation process. This is ideal for privacy-sensitive applications, custom models, and environments requiring full data sovereignty.

Ollama embeddings provide self-hosted embedding generation with configurable models, complete privacy, and no external dependencies after initial setup.

Key Benefits

Complete Control

Self-HostedRun embedding models on your own infrastructure with full control over the process.

Custom Models

Model FlexibilityUse any Ollama-compatible embedding model or create your own custom models.

Data Sovereignty

Maximum PrivacyAll data processing happens on your infrastructure with no external API calls.

Configurable Endpoints

Flexible DeploymentConfigure custom endpoints, ports, and deployment architectures.

Default Configuration

Ollama embeddings in Cognee use the following defaults:

Default Settings

Model Details

Setup Guide

Install Ollama

Get Ollama RunningInstall Ollama on your system and start the server.

Download Model

Pull Embedding ModelDownload the embedding model you want to use.

Configure Cognee

Set Environment VariablesConfigure Cognee to use your Ollama instance.

Test Integration

Verify SetupTest that embeddings are working correctly.

Installation & Setup

System Installation

# Install Ollama (macOS)
brew install ollama

# Install Ollama (Linux)
curl -fsSL https://ollama.ai/install.sh | sh

# Start Ollama server
ollama serve

Verify Installation

# Check Ollama is running
curl http://localhost:11434/api/tags

# Should return JSON with available models

Advanced Configuration

Custom Endpoints

Flexible DeploymentConfigure Ollama for different deployment scenarios:

# Remote Ollama server
EMBEDDING_ENDPOINT=http://ollama-server:11434/api/embeddings

# Custom port
EMBEDDING_ENDPOINT=http://localhost:8080/api/embeddings

# HTTPS endpoint
EMBEDDING_ENDPOINT=https://secure-ollama.company.com/api/embeddings

Custom Models

Use Your Own Models

# Create custom Ollama model
# 1. Create Modelfile
cat > Modelfile << EOF
FROM your-embedding-model.gguf
PARAMETER embedding true
EOF

# 2. Build custom model
ollama create custom-embeddings -f Modelfile

# 3. Configure Cognee
export EMBEDDING_MODEL=custom-embeddings
export EMBEDDING_DIMENSIONS=your_model_dimensions

Performance Tuning

Optimize for Your Hardware

# GPU acceleration (if available)
OLLAMA_GPU_LAYERS=32

# Memory allocation
OLLAMA_HOST=0.0.0.0:11434
OLLAMA_ORIGINS=*

# Concurrent requests
OLLAMA_NUM_PARALLEL=4
OLLAMA_MAX_LOADED_MODELS=2

Integration Examples

import cognee
import os

# Configure Ollama embeddings
os.environ["EMBEDDING_PROVIDER"] = "ollama"
os.environ["EMBEDDING_MODEL"] = "avr/sfr-embedding-mistral:latest"
os.environ["EMBEDDING_DIMENSIONS"] = "1024"

# Use with Cognee
await cognee.add("Ollama provides local embedding generation")
await cognee.cognify()

# Search with local embeddings
results = await cognee.search("local embedding generation")
print(f"Found {len(results)} results using Ollama embeddings")

Troubleshooting

Connection Issues

Server ProblemsOllama server not running:

# Check if Ollama is running
curl http://localhost:11434/api/tags

# Start Ollama if not running
ollama serve

# Check process
ps aux | grep ollama

Model not available:

# List available models
ollama list

# Pull missing model
ollama pull avr/sfr-embedding-mistral:latest

# Verify model works
ollama run avr/sfr-embedding-mistral:latest "test"

Performance Issues

Configuration Issues

Setup ProblemsDimension mismatch:

# Verify model dimensions
model_dimensions = {
    "avr/sfr-embedding-mistral:latest": 1024,
    "nomic-embed-text": 768,
    "mxbai-embed-large": 1024
}

# Ensure EMBEDDING_DIMENSIONS matches your model

Tokenizer issues:

# Verify tokenizer configuration
export HUGGINGFACE_TOKENIZER=Salesforce/SFR-Embedding-Mistral

# For other models, use appropriate tokenizers

Best Practices

Model Selection

Choose the Right Model

Development: Use smaller, faster models
Production: Balance quality vs performance
Specialized: Choose domain-specific models
Resource-constrained: Use efficient models

Infrastructure

Deployment Tips

Run Ollama on dedicated hardware for consistency
Use SSD storage for better model loading
Monitor resource usage and scale accordingly
Set up health checks and monitoring

Next Steps

FastEmbed

Local AlternativeCompare with FastEmbed for local embedding generation without server setup.

OpenAI Embeddings

Cloud ComparisonCompare with OpenAI embeddings for quality vs control trade-offs.

Embedding Overview

Provider ComparisonCompare all embedding providers to choose the best fit for your needs.

Configuration

Advanced SetupConfigure advanced Ollama settings and performance optimizations.

Getting Started

Core Concepts

Examples

CLI

Python SDK

UI

Key Benefits

Complete Control

Custom Models

Data Sovereignty

Configurable Endpoints

Default Configuration

Setup Guide

Installation & Setup

Advanced Configuration

Integration Examples

Troubleshooting

Best Practices

Model Selection

Infrastructure

Next Steps

FastEmbed

OpenAI Embeddings

Embedding Overview

Configuration

Getting Started

Core Concepts

Examples

CLI

Python SDK

UI

​Key Benefits

Complete Control

Custom Models

Data Sovereignty

Configurable Endpoints

​Default Configuration

​Setup Guide

​Installation & Setup

​Advanced Configuration

​Integration Examples

​Troubleshooting

​Best Practices

Model Selection

Infrastructure

​Next Steps

FastEmbed

OpenAI Embeddings

Embedding Overview

Configuration

Key Benefits

Default Configuration

Setup Guide

Installation & Setup

Advanced Configuration

Integration Examples

Troubleshooting

Best Practices

Next Steps