Embedding Providers

• EMBEDDING_PROVIDER — The provider to use: openai, gemini, mistral, bedrock, ollama, fastembed, openai_compatible, custom. These are not the same values LLM_PROVIDER accepts — see Valid EMBEDDING_PROVIDER values and endpoint URL forms
• EMBEDDING_MODEL — The specific embedding model to use
• EMBEDDING_DIMENSIONS — The vector dimension size (must match your vector store)
• EMBEDDING_API_KEY — Your API key (falls back to LLM_API_KEY if not set — with one exception when LLM_PROVIDER="custom")
• EMBEDDING_ENDPOINT — Custom endpoint URL (for Azure, Ollama, or custom providers)
• EMBEDDING_API_VERSION — API version (for Azure OpenAI)
• EMBEDDING_MAX_COMPLETION_TOKENS — Maximum input tokens per embedded text; used for tokenizer-based chunk sizing (optional, default 8191). Set it to your embedding model’s real input limit — see Max Completion Tokens
• HUGGINGFACE_TOKENIZER — HuggingFace Hub model ID that overrides the tokenizer Cognee uses for token counting when the embedding model is not itself a HuggingFace repo. Commonly used with Ollama embeddings (for example, nomic-ai/nomic-embed-text-v1.5).

• EMBEDDING_PROVIDER="openai_compatible" has no LLM counterpart. Setting LLM_PROVIDER="openai_compatible" fails with ValueError: 'openai_compatible' is not a valid LLMProvider — use LLM_PROVIDER="custom" instead (LM Studio, vLLM).
• anthropic, llama_cpp, and mcp-sampling are LLM-only: they provide no embeddings, so pair them with one of the values above.

• fastembed: any model returned by TextEmbedding.list_supported_models(), such as sentence-transformers/all-MiniLM-L6-v2. See the Fastembed section for the common list.
• ollama / LM Studio: any embedding model loaded locally, such as bge-m3:latest or all-minilm:latest.
• openai_compatible: any model exposed by a local /v1/embeddings server, including llama.cpp, TEI, vLLM, LocalAI, and Infinity.
• openai / gemini / mistral / bedrock / custom: any model available through the configured provider API, using the corresponding LiteLLM prefix such as openai/ or gemini/.

1. Read the model card or provider docs — for example text-embedding-3-large → 3072, nomic-embed-text-v1.5 → 768, BAAI/bge-m3 → 1024. See the table in Which embedding models are supported? for more.
2. Measure it — embed one string and count the floats. This works for local aliases and self-hosted models that no registry knows:

import asyncio
from cognee.infrastructure.databases.vector.embeddings import get_embedding_engine

async def main():
    vectors = await get_embedding_engine().embed_text(["hello"])
    print(len(vectors[0]))  # -> use this as EMBEDDING_DIMENSIONS

asyncio.run(main())

Could not auto-derive embedding_dimensions for provider='custom' model='...'. Falling back to 3072.
If your embedder produces vectors of a different size, set EMBEDDING_DIMENSIONS explicitly —
otherwise the first write into the vector store will fail with a shape mismatch.

• Chunk size, cost and latency. Chunks are sized as min(EMBEDDING_MAX_COMPLETION_TOKENS, LLM_MAX_COMPLETION_TOKENS // 2). A lower value forces smaller chunks, so a document produces more chunks → more embedding requests (and more LLM extraction calls) → higher latency and cost. A higher value (within the model’s real limit) produces fewer, larger chunks. See Chunkers for how chunk size shapes the graph.
• Over-length requests. Setting this above the embedding model’s real input limit can make individual texts exceed the limit. Cognee no longer fails outright — it splits the batch (or mean-pools an over-length string, see Timeout and Retry Behavior) — but that recovery adds latency, so it is not free. “Higher” is only better up to the model’s actual limit.

​

It is a chunk-sizing hint, not an enforced cap

• Setting it too high silently produces over-length requests. The provider rejects them; whether Cognee recovers depends on the error message (see below).
• Setting it above LLM_MAX_COMPLETION_TOKENS // 2 has no effect. With the default LLM_MAX_COMPLETION_TOKENS=16384, chunks are capped at 8192 regardless — EMBEDDING_MAX_COMPLETION_TOKENS=131072 and =8192 behave identically until you also raise the LLM value.

​

Chunk size vs. embedding request size

​

Recovery only fires on recognized error messages

• A request that fails fast (connection refused, rate limit, 5xx) is retried repeatedly until the 128-second window is exhausted, exactly as before.
• A request that genuinely hangs consumes the full 300 seconds on its first attempt. By the time it is cancelled the 128-second window has already elapsed, so it is not retried — the EmbeddingException is raised directly. A single hung request therefore blocks its task for up to 5 minutes.

• 404 Not Found errors are raised immediately — they indicate a configuration problem (wrong model name or endpoint) rather than a transient failure.
• asyncio.CancelledError is treated as terminal and re-raised without retrying, so cancelled tasks unwind promptly instead of consuming the full retry window. The same exclusion applies to the FastembedEmbeddingEngine, OllamaEmbeddingEngine, and OpenAICompatibleEmbeddingEngine retry decorators.
• EmbeddingContextWindowTooSmallError is treated as terminal and raised immediately. It is thrown when a single embedding text still exceeds the model’s context window but can no longer be split (the string is too short to divide further), so retrying would deterministically fail again. This exclusion applies to the LiteLLMEmbeddingEngine, FastembedEmbeddingEngine, and OpenAICompatibleEmbeddingEngine retry decorators; the failure returns at once instead of consuming the full 128-second retry window. The exception subclasses EmbeddingException (default message Text is too short to split further but exceeds context window.), so code that already catches EmbeddingException continues to catch it — catch EmbeddingContextWindowTooSmallError specifically to distinguish this deterministic, non-retryable case and shorten or pre-split the offending input.

• If the batch contains more than one text, it is split in half and each half is embedded in parallel, then the results are concatenated.
• If a single over-length string is left, it is split into two overlapping segments (the first two-thirds and the last two-thirds of the string), each segment is embedded, and the two vectors are averaged (mean-pooled) into one.

• EmbeddingException: Embedding request timed out. Check EMBEDDING_ENDPOINT connectivity. — The attempt did not complete within 300 seconds. Verify that EMBEDDING_ENDPOINT is reachable from your network. Since the deadline also covers time spent queued client-side, check that your embedding concurrency is bounded before assuming the endpoint itself is at fault.
• EmbeddingException: Cannot connect to embedding endpoint. Check EMBEDDING_ENDPOINT. — TCP connection was refused or the server closed the connection before responding. Confirm the server is running and the URL is correct.
• EmbeddingException: Failed to index data points using model <model> — The provider returned a 404 Not Found. Common causes: wrong model name, missing hosted_vllm/ prefix for vLLM, or an unsupported model at that endpoint. Non-over-length 400 BadRequestError responses are re-raised unchanged.
• 400 invalid_value on encoding_format from OpenRouter — Older LiteLLM releases serialize an omitted encoding_format as JSON null, which OpenRouter rejects. Cognee now forces encoding_format="float" on detected OpenRouter routes (see the Custom Providers accordion under Provider Setup Guides), so this should no longer occur; if you still see it, upgrade Cognee to a version that includes this guard.

EMBEDDING_BATCH_SIZE="5"

HUGGINGFACE_TOKENIZER="nomic-ai/nomic-embed-text-v1.5"

​

Common model-to-tokenizer mappings

​

Finding the tokenizer for any model

1. Look up the model on huggingface.co/models.
2. The repository ID is the {organization}/{model-name} part of the URL (e.g., huggingface.co/BAAI/bge-m3 → BAAI/bge-m3).
3. Use the repository ID that corresponds to the model your Ollama tag is built from. The Ollama model page typically links to the original Hugging Face repository.

​

Troubleshooting dependency errors

pip install 'cognee[huggingface]'
# or, if you are using Ollama embeddings:
pip install 'cognee[ollama]'

pip install --upgrade "transformers>=4.46.3,<5"

​

Important Notes

• Dimension Consistency: EMBEDDING_DIMENSIONS must match your vector store collection schema
• API Key Fallback: If EMBEDDING_API_KEY is not set, Cognee uses LLM_API_KEY (except for custom providers)
• Tokenization: HUGGINGFACE_TOKENIZER is optional and no longer enforced by Cognee’s startup validation — but it is recommended for the Ollama provider; set it to the HuggingFace model repo ID that matches your embedding model for accurate token counting
• Performance: Local providers (Ollama, Fastembed) are slower but offer privacy and cost benefits

• openai — the model’s TikToken (BPE) encoding.
• gemini — the default TikToken encoding (Gemini has no local tokenizer, so counts are approximate).
• mistral — the Mistral tokenizer.
• fastembed — the model’s own HuggingFace (wordpiece) tokenizer, resolved from the fastembed model list.
• ollama — the tokenizer named by HUGGINGFACE_TOKENIZER.
• openai_compatible / custom — the embedding model id used as a HuggingFace repo (e.g. BAAI/bge-large-en-v1.5); when that is not a loadable repo — such as a local alias or a LiteLLM-prefixed id — Cognee falls back to TikToken.

​

The “could not load a matching tokenizer” warning is benign

Could not load a matching tokenizer for embedding model 'openai/qwen3-vl-embedding-2b'
(... is not a valid model identifier listed on 'https://huggingface.co/models' ...).
Falling back to TikToken, so token counts are approximate.

• ollama — set HUGGINGFACE_TOKENIZER to the HuggingFace repo your Ollama model is built from (see the mappings earlier in this section).
• openai_compatible — set EMBEDDING_MODEL to the served model’s real HuggingFace repo id (for example Qwen/Qwen3-Embedding-4B instead of default). This provider needs no LiteLLM prefix, so the id can be the repo id directly.
• custom with hosted_vllm/ — the prefix is stripped automatically before tokenizer resolution, so the warning does not appear as long as the model name after the prefix is a valid HuggingFace repo id (see the vLLM accordion).
• custom with any other prefix (openai/, lm_studio/, openrouter/, …) — the prefix is required for routing and is passed to tokenizer resolution whole, so the warning cannot be avoided; treat it as informational. If your endpoint is OpenAI-compatible, switching to openai_compatible with the plain repo id removes it.