​
🚀 How to Use Remote Models

First and foremost, make sure to keep your API keys secure and never commit them to version control.

​
OpenAI

This is the easiest setup, technically you can start with only one set variable. If your .env consists only of one LLM_API_KEY per default it is considered as an OpenAI key. 
LLM_API_KEY = "your_api_key"

​
Azure OpenAI

You can use Azure’s OpenAI models for both LLM and embeddings. Set the following environment variables in your .env file: 
LLM_PROVIDER=openai
LLM_MODEL=azure/gpt-4o-mini
LLM_ENDPOINT=https://...
LLM_API_KEY="your_api_key"
LLM_API_VERSION=2024-08 ## (different when not mini model)

EMBEDDING_PROVIDER=openai
EMBEDDING_MODEL=azure/text-embedding-3-large
EMBEDDING_ENDPOINT=https://...
EMBEDDING_API_KEY="your_api_key"
EMBEDDING_API_VERSION=2023-05-15
Your model endpoint should look like this for example: 
https://.......-swedencentral.cognitiveservices.azure.com/openai/deployments/gpt-4o-mini

​
Google Gemini

You can use Google’s Gemini models for both LLM and embeddings. Set the following environment variables in your .env file: For LLM: 
LLM_PROVIDER="gemini"
LLM_API_KEY="your_api_key"
LLM_MODEL="gemini/gemini-1.5-flash"
For embeddings: 
EMBEDDING_PROVIDER="gemini"
EMBEDDING_API_KEY="your_api_key"
EMBEDDING_MODEL="gemini/text-embedding-004"
EMBEDDING_DIMENSIONS=768
EMBEDDING_MAX_TOKENS=8076

​
Custom endpoints

​
Anyscale

LLM_PROVIDER="custom"
LLM_MODEL="anyscale/mistralai/Mixtral-8x7B-Instruct-v0.1"
LLM_ENDPOINT="https://api.endpoints.anyscale.com/v1"
LLM_API_KEY="your_api_key"

​
vLLM

LLM_PROVIDER="openai"
LLM_MODEL="openai/<model name>" #must start with openai/
LLM_ENDPOINT="https://vllm-host/v1" #must end with /v1
LLM_API_KEY="<key>"

​
Deep Infra

LLM_PROVIDER="custom"
LLM_API_KEY="your_api_key"
LLM_MODEL="deepinfra/meta-llama/Meta-Llama-3-8B-Instruct"
LLM_ENDPOINT="https://api.deepinfra.com/v1/openai"
As of writing this you cannot use embedding models from Deep Infra with cognee.

​
OpenRouter

You will need an API key set up with OpenRouter. 
LLM_PROVIDER="custom"
LLM_API_KEY="your_api_key"
LLM_MODEL="openrouter/x-ai/grok-code-fast-1"
LLM_ENDPOINT="https://openrouter.ai/api/v1"
As of writing this OpenRouter does not have embedding models. If you don’t set up EMBEDDING_API_KEY it might default to LLM_API_KEY and cause issues You can also use popular providers like DeepSeek, Groq and Anthropic with cognee as custom providers. For a full list of custom providers you can use, check out the openrouter documentation. Note that models change often.