> ## Documentation Index
> Fetch the complete documentation index at: https://docs.cognee.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Rust SDK LLM and Embedding Configuration

> cognee-rust env vars, config keys and defaults for the LLM and embedding providers.

Every value here resolves as `defaults < config.json < .env < env < code`
([precedence](https://docs.cognee.ai/rust/configuration.md#precedence)), and can be set from code with
the [config API](https://docs.cognee.ai/rust/configuration.md#runtime-configuration-api) or
[`cognee-cli config`](https://docs.cognee.ai/rust/configuration#cli-config-subcommand).

## LLM

The `llm` section, registered by the LLM kind and shared by every LLM adapter
(see [Component config sections](https://docs.cognee.ai/rust/configuration.md#component-config-sections)). Each key also
answers to its Python flat name, the alias shown in brackets. The deep reference is
[CLI: LLM retries](https://docs.cognee.ai/rust/tools/cli.md#llm-retries) for retries and the `cognee-llm`
rustdoc for the adapter.

| Env var (aliases) | Key (alias) | Default |
| - | - | - |
| `LLM_PROVIDER` | `llm.provider` (`llm_provider`) | `openai` |
| `LLM_MODEL` / `OPENAI_MODEL` | `llm.model` (`llm_model`) | `openai/gpt-5-mini` |
| `LLM_API_KEY` / `OPENAI_TOKEN` | `llm.api_key` (`llm_api_key`) | *(empty)* |
| `LLM_ENDPOINT` / `OPENAI_URL` | `llm.endpoint` (`llm_endpoint`) | *(empty)* |
| `LLM_API_VERSION` | `llm.api_version` (`llm_api_version`) | *(empty)* |
| `LLM_REASONING` | `llm.reasoning` (`llm_reasoning`) | `auto` |
| `LLM_STRUCTURED_OUTPUT_MODE` | `llm.structured_output_mode` (`llm_structured_output_mode`) | `auto` |
| `LLM_TEMPERATURE` | `llm.temperature` (`llm_temperature`) | *unset* (see below for what is sent) |
| `LLM_STREAMING` | `llm.streaming` (`llm_streaming`) | `false` (accepted for Python parity; no effect, see below) |
| `LLM_MAX_COMPLETION_TOKENS` / `LLM_MAX_TOKENS` | `llm.max_completion_tokens` (`llm_max_completion_tokens`) | `16384` |
| `LLM_ARGS` | `llm.args` (`llm_args`) | *(empty)* |
| `LLM_MAX_RETRIES` | `llm.max_retries` (`llm_max_retries`) | `3` |
| `LLM_NETWORK_RETRIES` | `llm.network_retries` (`llm_network_retries`) | `2` |
| `LLM_MIN_RETRY_SECONDS` | `llm.min_retry_seconds` (`llm_min_retry_seconds`) | `240` |
| `LLM_MAX_PARALLEL_REQUESTS` | `llm_max_parallel_requests` (a `Settings` field) | `1000` (`128` on Android/iOS) |
| `LLM_REQUEST_TIMEOUT_SECONDS` | `llm.request_timeout_seconds` (`llm_request_timeout_seconds`) | `600` |
| `LLM_CONNECT_TIMEOUT_SECONDS` | `llm.connect_timeout_seconds` (`llm_connect_timeout_seconds`) | `10` |
| `LLM_REQUEST_DEADLINE_SECONDS` | `llm.request_deadline_seconds` (`llm_request_deadline_seconds`) | `1200` (`0` disables) |
| `MOCK_LLM` | `llm.mock` (`llm_mock`) | `false` |
| `MOCK_LLM_CASSETTE` | `llm.cassette` (`llm_cassette`) | *(empty)* |
| `COGNEE_RECORD_LLM` | `llm.record_path` (`llm_record_path`) | *(empty)* |
| `LLM_RATE_LIMIT_ENABLED` / `_REQUESTS` / `_INTERVAL` | `llm.rate_limit_enabled` / `_requests` / `_interval` (`llm_rate_limit_*`) | `false` / `60` / `60` |
| `AUTO_RATE_LIMIT` | `llm.auto_rate_limit` (`auto_rate_limit`) | `true` |
| `COGNEE_DEBUG_LLM_REQUEST` | `llm.debug_requests` | `false` (dump every request/response body to stderr) |

Adapter sections, registered by their factory and present only when it is:

| Env var (aliases) | Key (alias) | Default |
| - | - | - |
| `LLM_VISION_MODEL` | `llm.openai_compatible.vision_model` (`llm_vision_model`) | *(unset: uses `llm.model`)* |
| `TRANSCRIPTION_MODEL` | `llm.openai_compatible.transcription_model` (`transcription_model`) | `whisper-1` |
| `ANTHROPIC_BASE_URL` / `ANTHROPIC_API_BASE` | `llm.anthropic.base_url` (`anthropic_base_url`) | *(unset: public API)* |
| `AWS_*` (see [Bedrock](#bedrock)) | `aws.*` (`aws_*`), shared by the Bedrock LLM and embedding adapters (`bedrock` feature) | *(unset)* |

`LLM_MAX_COMPLETION_TOKENS` is a *completion*-token budget, and it does double
duty: besides capping the option-less `generate` call, it is the LLM term in
cognify's chunk-size auto-calculation
(`min(<embedding engine limit>, llm_max_completion_tokens / 2)` — Python's
`get_max_chunk_tokens()`). Lowering it for a small-context endpoint therefore
shrinks the prompts cognify sends. On adapters that know their model's documented
output cap (Anthropic, Bedrock) the configured value is clamped to it first, as
Python does via litellm's `model_cost`.

> [!NOTE]
>
> Structured-output calls do **not** apply this ceiling — option-less ones send
> the built-in `16384` default. Cognify's extraction calls send no cap at all
> (Python parity).

`LLM_ARGS` is the escape hatch for that last case. It takes a JSON **object**
whose keys are merged into each chat and structured-output request body — the
port of Python's `llm_config.llm_args` and its `{**self.llm_args, **kwargs}`
merge, so an explicitly-passed option always wins over an `LLM_ARGS` key of the
same name. Its practical use is giving the extraction path more output headroom:
because those calls send no `max_tokens`, the provider's own default applies, and
a low one (Baseten's is 4096) truncates the structured JSON mid-object. The
adapter detects that case (`finish_reason: "length"`) and re-asks at a raised
budget, but starting from a sufficient budget avoids the wasted first attempt.

```bash theme={null}
LLM_ARGS='{"max_tokens": 16384}'
```

Three details worth knowing. A bare `max_tokens` is folded into
`max_completion_tokens` for reasoning models, and sampling params those models
reject (`temperature`, `top_p`, `frequency_penalty`, `presence_penalty`) are
dropped rather than passed through. A malformed value, or valid JSON that is not
an object, is ignored with a `warn!` instead of aborting startup. And the
vision/transcription path does not see it at all: it bypasses the merge.

`LLM_TEMPERATURE` reaches every built-in provider (previously only Bedrock
applied it). The OpenAI-compatible, Azure and Anthropic factories fold it into
`LLM_ARGS` as `temperature`, as Python's `fold_sampling_params_into_llm_args`
does, so a `temperature` in `LLM_ARGS` itself wins. It is sent on every call
that does not pick a temperature itself: requests whose options carry none
(graph extraction) and option-less calls, plain or structured (search
completions, feedback detection), which otherwise send `0.0`. A temperature a
call picks (summarization's `0.3`) wins, and the OpenAI-compatible and Azure
adapters never send one to a reasoning model, which rejects it. Bedrock sends it as
`inferenceConfig.temperature` on the same calls; an `LLM_ARGS` `temperature` is
not used for it there, but passed through unchanged in
`additionalModelRequestFields` like the rest of `LLM_ARGS`. Unset, these providers send a `temperature` from `LLM_ARGS`
if it has one, on option-less calls too (where the built-in `0.0` used to
shadow it). Otherwise extraction sends none and option-less calls send `0.0`.
Unlike Python, Rust does not pin `0.0` for local inference servers when the
variable is unset.

`LLM_STREAMING` has no effect. Python's `llm_streaming` is likewise unread by
its OpenAI and Azure adapters, and no Rust adapter streams: the `Llm` trait has
no token-streaming call for a stream to reach.

A fallback LLM (`llm.fallback_provider/_model/_endpoint/_api_key`, aliases
`llm_fallback_*`) is configurable programmatically (no env binding) but not used
yet. `MOCK_LLM` + cassettes power the offline
benchmark — see [performance/mock-benchmark.md](https://github.com/topoteretes/cognee-rs/blob/main/docs/performance/mock-benchmark.md).

### Supported LLM providers

Several providers are OpenAI-compatible HTTP endpoints, so they route through the
same adapter — differing only in base URL and litellm-style model-prefix stripping.
`LLM_API_KEY` is required for every provider **except `bedrock`** (matching the
Python SDK; for a local Ollama any non-empty value works). On `bedrock` an unset
or empty `LLM_API_KEY` is a supported configuration — it falls through to SigV4
signing rather than erroring (see [Bedrock](#bedrock) below). The OpenAI-only
request quirks are gated on the `api.openai.com` host, so they never fire against
another endpoint.

| `LLM_PROVIDER` | `LLM_API_KEY` | `LLM_ENDPOINT` default | Model prefix stripped |
| - | - | - | - |
| `openai` | required | `https://api.openai.com/v1` | `openai/` |
| `ollama` | required (any value for local) | `http://localhost:11434/v1` | `ollama/` |
| `mistral` | required | `https://api.mistral.ai/v1` | `mistral/` |
| `gemini` | required | `https://generativelanguage.googleapis.com/v1beta/openai/` | `gemini/` |
| `custom` / `openai_compatible` | required | **required** (no default) | *(none)* |
| `anthropic` | required | `https://api.anthropic.com/v1` (ignores `LLM_ENDPOINT`; see below) | `anthropic/` |
| `azure` | required | **deployment URL** (see below) | *(see below)* |
| `bedrock` | optional | ignores `LLM_ENDPOINT` (see [Bedrock](#bedrock)) | *(model id normalised; see below)* |
| `mock` | — | — | — (see `MOCK_LLM`) |

`LLM_ENDPOINT` always overrides the default when set on the OpenAI-compatible
providers. Audio transcription (Whisper) is wired only for `openai` and
`custom`/`openai_compatible` (which may expose `/audio/transcriptions`);
`ollama`/`mistral`/`gemini`/`bedrock` get graceful no-audio.

**`anthropic`** uses a **native Messages-API adapter** (not the OpenAI-compatible
factory): it authenticates with `x-api-key`, hoists the system prompt into the
top-level `system` field, and produces structured output via a forced `tool_use`.
It requires `LLM_API_KEY`, strips an `anthropic/` model prefix, honors `LLM_ARGS`,
and supports image description (vision) but not audio transcription. The endpoint
defaults to `https://api.anthropic.com/v1` and deliberately ignores
`LLM_ENDPOINT`/`OPENAI_URL` so a stray `OPENAI_URL` cannot misroute Anthropic
traffic (matching Python, which passes no base URL). To route Anthropic traffic
through a proxy / gateway, set the dedicated `ANTHROPIC_BASE_URL` (alias
`ANTHROPIC_API_BASE`); other providers ignore it.

**`azure`** reuses the OpenAI request path with Azure's auth and URL conventions:
it authenticates with the `api-key` header and appends an `?api-version=<v>`
query. Set `LLM_PROVIDER=azure`, `LLM_API_KEY`, `LLM_API_VERSION` (e.g.
`2024-12-01-preview`), and `LLM_ENDPOINT` to the **deployment** URL
(`https://<resource>.openai.azure.com/openai/deployments/<deployment>`). Azure
ignores the model in the request body for routing, but `LLM_MODEL` still selects
the reasoning-model parameter shape, so for an o-series/`gpt-5` deployment set
`LLM_MODEL` to the underlying model name (a `.../deployments/o3` style segment is
also treated as reasoning), or pin it with `LLM_REASONING`.

### Bedrock

`LLM_PROVIDER=bedrock` is a **native adapter over Amazon Bedrock's Converse API**
(`POST {endpoint}/model/{modelId}/converse`), not an OpenAI-compatible endpoint.
Its AWS settings live in the `aws` config section (registered by the Bedrock LLM
and embedding factories), so they can be set like any other section key: with
`set`, in a config file or through the bindings, as `aws.access_key_id` or its
flat alias `aws_access_key_id`, and so on. A field left unset there falls back to
the `AWS_*` environment variable of the same name inside the adapter. The region
has no env binding in the section (`aws.region`, alias `aws_region_name`); the
adapter applies the env rungs of the region chain itself. The env names mirror the
ones litellm reads, and several of them are *not* the boto3-standard spellings:

| Env var | Purpose |
| - | - |
| `AWS_ACCESS_KEY_ID` | Static access key |
| `AWS_SECRET_ACCESS_KEY` | Static secret key |
| `AWS_SESSION_TOKEN` | Session token accompanying temporary static credentials |
| `AWS_REGION_NAME` | Region — the first env rung of the region chain below |
| `AWS_REGION` | Region — one rung below `AWS_REGION_NAME` |
| `AWS_DEFAULT_REGION` | Not part of the region chain; read only when resolving the STS signing region |
| `AWS_PROFILE_NAME` | Shared-config profile. **Not** the boto3-standard `AWS_PROFILE`, which this path ignores |
| `AWS_ROLE_NAME` | Role ARN to assume via STS |
| `AWS_ROLE_ARN` | Ambient role ARN published by an IRSA / EKS pod identity |
| `AWS_SESSION_NAME` | STS session name |
| `AWS_WEB_IDENTITY_TOKEN` | **Path to** a readable OIDC web-identity token file. A literal token value is rejected with a config error (a documented divergence from litellm) |
| `AWS_WEB_IDENTITY_TOKEN_FILE` | Ambient web-identity token file published by an IRSA / EKS pod identity |
| `AWS_STS_ENDPOINT` | Override the STS endpoint |
| `AWS_EXTERNAL_ID` | `ExternalId` passed to `AssumeRole` |
| `AWS_BEDROCK_RUNTIME_ENDPOINT` | Override the Bedrock runtime host |
| `AWS_BEARER_TOKEN_BEDROCK` | Bedrock API key, used as a bearer token |

**Auth ladder.** `LLM_API_KEY`, when set, is sent verbatim as
`Authorization: Bearer <key>` and the request is **not** signed — no credential
lookup runs, so a bearer-only deployment needs no AWS credentials. Otherwise
`AWS_BEARER_TOKEN_BEDROCK` takes the same short-circuit. Only when both are absent
does the request fall through to SigV4 (signing service `bedrock`), whose
credentials resolve in this order:

1. web-identity token + role + session name → STS `AssumeRoleWithWebIdentity`;
2. role ARN → STS `AssumeRole` — **skipped when the process is already running
   as that role**, so an IRSA pod that already holds it does not assume itself;
3. profile name → the shared-config profile;
4. static access key + secret (plus session token when present);
5. otherwise the default chain (env / shared config / SSO / ECS / IMDS).

**Region and endpoint.** The region resolves as: an explicitly configured
region → a region embedded in a model ARN → `AWS_REGION_NAME` → `AWS_REGION` →
the profile / default chain → the hard default **`us-west-2`**. The endpoint
resolves as: an explicit API base → `AWS_BEDROCK_RUNTIME_ENDPOINT` →
`https://bedrock-runtime.{region}.amazonaws.com`. `LLM_ENDPOINT` is **not** that
API base: like the Anthropic path, the Bedrock factory deliberately passes no base
URL, so a stray `LLM_ENDPOINT` cannot misroute Bedrock traffic. Use
`AWS_BEDROCK_RUNTIME_ENDPOINT` for a custom runtime host.

**Converse only.** Every model cognee ships routes to Converse after the model id
is normalised (an ARN wrapper is unwrapped; cross-region inference prefixes such
as `eu.`/`us.`/`apac.` and provisioned-throughput / context-window suffixes are
stripped). Normalisation feeds routing and capability decisions only; the request
URL always carries the model id exactly as configured. A model that routes to
Bedrock's legacy `invoke` API for **chat** is unsupported: construction fails
with `LlmError::FeatureNotSupported`. (Embeddings are a separate path and *do* go
over InvokeModel — see [Embedding](#embedding).)

**Capabilities.** No streaming. No audio transcription. Image description
(vision) **is** implemented over Converse image blocks — deliberately exceeding
Python parity — but gated on a per-model capability table
([`caps.rs`](https://github.com/topoteretes/cognee-rs/blob/main/crates/llm/src/adapters/bedrock/caps.rs)):
an unlisted model is treated as text-only and returns
`LlmError::FeatureNotSupported`. Structured output is capability-gated from the
same table: models that advertise native structured output get Converse's
`outputConfig.textFormat` carrying the JSON schema, the rest get a synthetic
`json_tool_call` tool (with `toolChoice` forced only for models that support it).

**Feature gating.** The adapter and the embedding engine live behind the
`bedrock` Cargo feature. It is **non-default** on the two leaf crates that carry
the code (`cognee-llm`, `cognee-embedding`), but it **is** in the `default`
feature list of `cognee-components`, `cognee` and `cognee-cli` (and of
`cognee-http-server`), so every shipped binary has Bedrock available.
Building with `--no-default-features` (or without `bedrock`) drops the whole AWS
stack, and `LLM_PROVIDER=bedrock` then reports an unsupported provider.

### Reasoning-model parameters

OpenAI reasoning families (`o1*`, `o3*`, `o4*`, `gpt-5*`) require a different
request shape — `max_completion_tokens` instead of `max_tokens`, and no
`temperature`/`top_p`/penalties. The adapter auto-detects this from the model
name for any non-local endpoint (loopback and Ollama's port keep the legacy
shape). `LLM_REASONING` overrides that detection:

* `auto` (default) — name/host auto-detection as above.
* `always` — force the reasoning shape (e.g. a proxy that hides a reasoning model
  behind an opaque alias).
* `never` — force the legacy `max_tokens` + `temperature` shape (e.g. a gateway
  serving a reasoning-*named* model that only accepts the legacy parameters).

### Structured-output request shape

Nothing in the OpenAI-compatible protocol advertises which structured-output
shape a server understands, so by default the adapter cascades: native `tools`,
then legacy `functions`, then `response_format: {"type": "json_object"}`. The
first shape that yields a parseable payload wins. The adapter keeps a per-mode
miss counter and stops sending a mode that has produced nothing three calls
running (see [How retries and pacing interact](https://docs.cognee.ai/rust/configuration/runtime.md#how-retries-and-pacing-interact)).
`LLM_STRUCTURED_OUTPUT_MODE` skips the discovery when you already know the answer:

* `auto` (default) — cascade as above, bounded by the miss counter.
* `json_schema` — try constrained decoding first, then cascade. See below.
* `tools` — only native tool-calling.
* `functions` — only the legacy `functions`/`function_call` pair.
* `json` — only JSON mode.

Pick `json` when the server **rejects** `tools`/`functions` outright. If it
merely *ignores* them — accepting the request and echoing JSON in `content` —
leave it on `auto`: tool-calling mode already succeeds there in one request, and
it is the only mode that puts the real JSON schema on the wire (JSON mode sends a
prose example derived from the schema, which means weaker schema pressure).

Pinning a mode means the other modes are **never sent**, and exhausting the pinned
mode fails the call rather than falling through — the error names the pin. An
unrecognised value falls back to `auto`.

> [!WARNING]
>
> **Python's `LLM_INSTRUCTOR_MODE` is not an alias for this.** It covers similar
> ground but does not mean the same thing, so Rust does not read it. If a shared
> `.env` sets `LLM_INSTRUCTOR_MODE`, translate it to `LLM_STRUCTURED_OUTPUT_MODE`
> yourself.

The two differ in four ways:

* **When it applies.** Python reads `llm_instructor_mode` only on its
  `instructor` structured-output framework (`STRUCTURED_OUTPUT_FRAMEWORK=instructor`).
  The default framework, `litellm_native`, ignores it. Aliasing it would make a
  value that does nothing in Python change behaviour in Rust.
* **Values.** Python takes an `instructor.Mode` value: `json_schema_mode`,
  `tool_call`, `tools_strict`, `markdown_json_mode`, `anthropic_tools`,
  `mistral_tools` and so on, many of them specific to one provider. Only
  `json_mode` and `function_call` are also spelled the same in Rust; anything
  else falls back to `auto` here.
* **Empty value.** Empty means the provider's entry in Python's static table
  (`instructor_modes.py`, for example `json_schema_mode` for OpenAI). In Rust,
  `auto` means the cascade.
* **Scope and pinning.** Python's mode applies to every adapter, and
  `json_schema_mode` is a pin. Rust's knob applies only to the OpenAI-compatible
  and Azure providers (not Anthropic or Bedrock), and `json_schema` is not a pin (see below).

Rough translation: `json_schema_mode` → `json_schema`; `tool_call` /
`tools_strict` / `parallel_tool_call` → `tools`; `function_call` → `functions`;
`json_mode` / `markdown_json_mode` → `json`.

**`json_schema` — constrained decoding.** `json_schema` is **not a pin**. It sends

```json theme={null}
"response_format": {"type": "json_schema",
                    "json_schema": {"name": "extract_structured_data",
                                    "strict": true, "schema": {…}}}
```

ahead of the cascade, and falls back to it when the endpoint says no. On a
backend that supports it the sampler cannot emit a token that would break the
schema. It is opt-in because there is no capability table for an arbitrary
OpenAI-compatible base URL, and some backends (Baseten's `gpt-oss-120b`) answer
**HTTP 501** to a constrained request. The mode demotes rather than failing:

1. `strict: true` with the all-required / `additionalProperties: false` schema;
2. on rejection, the same envelope without `strict`, carrying the ordinary schema;
3. on rejection again, out of the mode entirely: the `tools` → `functions` →
   `json` cascade runs exactly as under `auto`.

Only an outright refusal of the request *shape* moves that ladder — HTTP 501, or
an HTTP 400 whose body names `response_format` / `json_schema` / `schema` /
`strict`. Rate limits, other 5xx, network errors, model refusals and unrelated
400s leave the (process-lifetime, monotonic) memo untouched. The probe costs one
request and is single-flighted per schema. A schema whose root is not an object
never enters the mode, and a schema using a feature outside OpenAI's strict subset
(a map field, `patternProperties`, `if`/`then`/`else`) is refused and demoted.

The Bedrock adapter does not read this knob; it picks Converse's native
`outputConfig.textFormat.jsonSchema` or a synthetic `json_tool_call` from its own
capability table.

> [!NOTE]
>
> **Ollama embeddings:** set `EMBEDDING_ENDPOINT` explicitly when using
> `EMBEDDING_PROVIDER=ollama`. The Ollama embedder needs the `/api/embed` route, and
> when `EMBEDDING_ENDPOINT` is unset the embedding endpoint falls back to
> `LLM_ENDPOINT`, which for Ollama points at the `/v1` chat base, so the embedder
> would POST to the wrong path. Set
> `EMBEDDING_ENDPOINT=http://localhost:11434/api/embed`.

## Embedding

The `embedding` section is registered by the embedding kind and shared by every
provider. Each provider's factory registers the settings only it reads (see
[Component config sections](https://docs.cognee.ai/rust/configuration.md#component-config-sections)). The Python flat name of
each key is accepted as an alias, shown in brackets.

| Env var (aliases) | Key (alias) | Default |
| - | - | - |
| `EMBEDDING_PROVIDER` | `embedding.provider` (`embedding_provider`) | `openai` (`onnx` on Android) |
| `EMBEDDING_MODEL` | `embedding.model` (`embedding_model_name`, `embedding_model`) | `text-embedding-3-small` (`BGE-Small-v1.5` on Android) |
| `EMBEDDING_DIMENSIONS` | `embedding.dimensions` (`embedding_dimensions`) | `1536` (`384` on Android) |
| `EMBEDDING_ENDPOINT` | `embedding.endpoint` (`embedding_endpoint`) | *(empty)* |
| `EMBEDDING_API_KEY` (falls back to `LLM_API_KEY`) | `embedding.api_key` (`embedding_api_key`) | *(empty)* |
| `EMBEDDING_MAX_SEQUENCE_LENGTH` | `embedding.max_sequence_length` (`embedding_max_sequence_length`) | `512` (local ONNX/fastembed only; remote providers use `EMBEDDING_MAX_COMPLETION_TOKENS`) |
| `EMBEDDING_MAX_COMPLETION_TOKENS` | `embedding.max_completion_tokens` (`embedding_max_completion_tokens`) | `8191` (input-token limit of a remote embedding API; the embedding term of the chunk-size auto-calculation for remote providers) |
| `EMBEDDING_BATCH_SIZE` | `embedding.batch_size` (`embedding_batch_size`) | `36` (texts per embedding request; raise for providers that allow larger batches. The OpenAI-compatible engine also dispatches up to 8 sub-batches concurrently) |
| `EMBEDDING_RATE_LIMIT_ENABLED` / `_REQUESTS` / `_INTERVAL` | `embedding.rate_limit_*` (`embedding_rate_limit_*`) | `false` / `60` / `60` |
| `MOCK_EMBEDDING` | `embedding.mock` (`mock_embedding`) | *(empty)*. `true`/`1`/`yes`/`deterministic` select the mock engine with deterministic SHA-256-derived vectors; `zero` selects all-zero vectors (cosine KNN then returns nothing) |

Provider sections, registered by the provider's factory and present only when it
is compiled in:

| Env var (aliases) | Key (alias) | Provider | Default |
| - | - | - | - |
| `EMBEDDING_MODEL_PATH` / `COGNEE_E2E_EMBED_MODEL_PATH` | `embedding.onnx.model_path` (`embedding_model_path`) | `onnx` / `fastembed` (`onnx` feature) | *(empty: `<cache_root_directory>/models/BGE-Small-v1.5-model_quantized.onnx`)* |
| `EMBEDDING_TOKENIZER_PATH` / `COGNEE_E2E_TOKENIZER_PATH` | `embedding.onnx.tokenizer_path` (`embedding_tokenizer_path`) | `onnx` / `fastembed` | *(empty: `<cache_root_directory>/models/bge-small-tokenizer.json`)* |
| `EMBEDDING_ONNX_BATCH_SIZE` | `embedding.onnx.batch_size` (`embedding_onnx_batch_size`) | `onnx` / `fastembed` | `32` (inference batch size; independent of `embedding.batch_size`. Lower it under memory pressure on edge devices) |
| `EMBEDDING_API_VERSION` | `embedding.openai_compatible.api_version` (`embedding_api_version`) | `openai` / `openai_compatible` (Azure `api-version`) | *(unset)* |
| `HUGGINGFACE_TOKENIZER` | `embedding.ollama.huggingface_tokenizer` (`huggingface_tokenizer`) | `ollama` | *(unset)* |
| `AWS_*` | `aws.*` | `bedrock` (`bedrock` feature) | see [Bedrock](#bedrock) |

Provider values: `onnx`, `fastembed`, `openai`, `openai_compatible`, `ollama`,
`bedrock`, `mock`, plus any provider a custom registry adds with
`ComponentRegistry::register_embedding`. A provider whose factory is not
registered (a misspelling, or a backend compiled out) is an error, never a silent
ONNX fallback.

**`MOCK_EMBEDDING` diverges from the Python SDK, deliberately.** Python's
embedding engines return `[0.0] * dimensions` under a truthy `MOCK_EMBEDDING`.
Rust defaults to SHA-256-derived vectors instead, because zero vectors are
silently discarded by cosine-distance KNN (the distance is `NaN`, and LanceDB
filters `NaN` rows out), making retrieval vacuously empty. The mock engine is
test-only. Set `MOCK_EMBEDDING=zero` to reproduce Python's behaviour.

**`EMBEDDING_PROVIDER=bedrock`** uses Amazon Bedrock's **InvokeModel** API
(`POST {endpoint}/model/{modelId}/invoke`) — Converse is never used here. Two
model families are supported, selected from the normalised model id: **Titan**
(`amazon.titan-embed-text-v1`, `amazon.titan-embed-text-v2:0`,
`amazon.titan-embed-image-v1`), which takes one text per request, and **Cohere**
(`cohere.embed-*`), which batches. Set `EMBEDDING_MODEL` to the Bedrock model id;
`EMBEDDING_DIMENSIONS` drives the engine's reported `dimension()` and is sent as
`dimensions` on Titan v2, so it must match what the model returns. AWS
configuration comes from the same `AWS_*` variables as the LLM path (see
[Bedrock](#bedrock)), and `EMBEDDING_API_KEY` (falling back to `LLM_API_KEY`) is
the Bedrock API key, which short-circuits to a bearer header with no SigV4;
leaving it unset runs the ambient credential ladder. Returned vectors are always
L2-normalised (server-side on Titan v2 via `normalize: true`, client-side for the
others).

> [!NOTE]
>
> **Bedrock embeddings and `EMBEDDING_ENDPOINT`:** through `Settings`,
> `EMBEDDING_ENDPOINT` defaults to empty, and a Bedrock embedder inherits
> `LLM_ENDPOINT` only when the LLM provider is also `bedrock`. Direct users of
> `EmbeddingConfig::default()` start from the OpenAI base URL instead; the Bedrock
> engine ignores an endpoint that points at `api.openai.com` rather than POSTing
> Bedrock bodies there. Set it only to a real Bedrock runtime host (or use
> `AWS_BEDROCK_RUNTIME_ENDPOINT`).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.