> ## Documentation Index
> Fetch the complete documentation index at: https://docs.cognee.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Rust SDK Improve

> improve in cognee-rust: the self-improvement loop's nine stages, background runs, locking and the global context index.

## What is the improve operation

`improve` runs the self-improvement loop over a dataset. It turns session activity (ratings, Q\&A, agent traces, preferences) into lasting graph memory and keeps dataset-wide structures such as the global context index up to date. `remember` runs it automatically unless you turn that off.

## Stages

Runs the self-improvement loop over a dataset as nine gated stages, in
this order: `feedback_weights`, `persist_session_qa`,
`persist_agent_traces`, `extract_agent_context`, `distill_sessions`,
`update_user_preferences`, `build_truth_subspace`, `review_conflicts`,
`global_context_index`. The first seven read the given sessions and are
skipped with `no_session_ids` without them; the last two work on the graph
alone and run by default. Each stage reports a `StageResult` (`status`, `reason`,
`error`, `counts`, `duration_ms`) on the returned `ImproveResult` (`dataset_id`,
`dataset_name`, `session_ids`, `stages`, `background`, `finished`, `error`,
`memify_run`), whose `status` summarises them (`completed`, `errored`,
`skipped`, `running`). The earlier `memify` (triplet enrichment) and
`sync_graph_to_session` stages are removed, and the old counters
(`stages_run`, `feedback_entries_processed`, `sessions_persisted`, …) are now
stage `counts`. Stages named in `improve.stages_disabled`
(`IMPROVE_STAGES_DISABLED`, a comma-separated list) are skipped first, with
`disabled_by_config`; an unknown name, or the fatal `persist_session_qa`, makes
the configuration invalid and the run fails before any stage. `feedback_alpha`
defaults to `improve.feedback_alpha` (0.1). The options that only scoped or
fed the removed memify stage are gone too
(`node_name` and the CLI `--node-name` flag, `extraction_tasks`,
`enrichment_tasks`, `data`, and the bindings' `nodeName`). `ImproveResult` still serializes the deprecated `memify_run` key,
as `{}`. The bindings' improve options accept `datasetId` (a UUID string): the
improve targets that dataset, possibly
one shared with the caller, instead of looking `datasetName` up among the
caller's own datasets.

## Background runs

**Background runs.** `cognee::api::improve_in_background` runs improve
in the background: it validates the configuration and
`feedback_alpha` and resolves the dataset first — a name resolves to the
caller's dataset or is created for it, a dataset id that names no dataset
(`InvalidArgument`; `PermissionDenied` when an ACL store is wired, which checks
the grant first) or one the caller may not write (`PermissionDenied`; the
ACL's `write` grant, or ownership without an ACL) is refused — then spawns the
run as a process-wide background task and returns the `running` result with
the resolved dataset id and name. Wait for it with
`cognee::wait_for_background_tasks`; every binding exposes the same wait
(`waitForBackgroundTasks` in TS and Java, `wait_for_background_tasks` in
Python, `cg_sdk_wait_for_background_tasks` in C). The wait takes an optional
`timeoutSeconds` (`timeout_seconds` in Python), which cancels nothing when it
elapses, and reports `{finished, pending}`: `finished` is `false` only when
the timeout elapsed with work still running. An `improve` op with
`runInBackground: true` uses this entry point. `ImproveParams::run_in_background`
on `improve()` itself is only a label, for a host that dispatches the run on
its own. The bindings' improve options also accept `buildGlobalContextIndex`
and `reviewConflicts`, which follow their `improve.*` setting when omitted (an
explicit value wins either way), and `buildTruthSubspace`, which opts in to
that stage.

## Feedback weights

Besides an explicit
`feedback_score`, a turn's answer is rated implicitly by the latest
`kind = "feedback"` session-context row that references it, and an implicit
rating moves the graph elements at `alpha * 0.5`; an explicit score still wins.
Each turn records which node and edge ids it already moved, so a retry moves
only the rest; ids missing from the graph keep the turn pending, and a turn is
marked done after 3 attempts. A graph or session-store error on one turn counts
it as skipped instead of aborting the stage.

**Breaking:** `SessionQAEntry::memify_metadata` (and
`SessionQAUpdate::memify_metadata`) is now
`Option<HashMap<String, serde_json::Value>>` instead of
`Option<HashMap<String, bool>>` (TypeScript: `Record<string, unknown> | null`),
so it holds the lists and numbers Python writes there; read a flag with
`Value::as_bool`. The old bool-only decode dropped the whole map when it met one.
For callers of `cognee_cognify::memify`, `FeedbackApplyResult` gained public
fields (`done`, `errored`, `elements_moved`, `elements_pruned`,
`elements_failed`), so a struct literal no longer compiles, and `FeedbackError`
gained the `MetadataNotUpdated` and `InvalidSessionIds` variants.

## Failures and skipped stages

The run fails open — a failed stage is recorded and the next one runs —
except that a failure in `persist_session_qa` (losing session Q\&A would be data
loss) stops the run with `ApiError::ImproveAborted` carrying the partial
result, and an exhausted LLM budget skips every later stage with
`budget_exhausted`. In the bindings that abort is `SdkError::ImproveAborted`
(code `IMPROVE_ABORTED`): Python's `CogneeImproveAbortedError.improve_result`,
JavaScript's `ImproveAbortedError.improveResult`; C (`CG_ERR_RUNTIME`) and Java
append `improve_result: <json>` to the message. The stages that draft text with
an LLM are skipped with `no_llm_configured` when none is configured.
`review_conflicts` and `global_context_index` run by default
(`improve.review_conflicts`, `improve.global_context_index`; a per-call
`review_conflicts` / `build_global_context_index` wins either way) and report
`turned_off` when off. `review_conflicts` is not ported yet and always reports
a skip reason (`turned_off`, then `no_llm_configured`, then
`edge_evidence_disabled` while `cognify.edge_evidence_enabled` is off, then
`not_ported`).

**Breaking:** both stages used to be opt-in, and a switched-off one used to
report `opt_in_disabled` (`build_truth_subspace` still does).
`ImproveParams::build_global_context_index` is now `Option<bool>`, and
`ImproveParams` / `OwnedImproveParams` gained `review_conflicts: Option<bool>`,
so a struct literal needs both (`None` follows the setting). Set
`IMPROVE_GLOBAL_CONTEXT_INDEX=false` and `IMPROVE_REVIEW_CONFLICTS=false` to
keep the old behavior.

## Agent context lessons

`extract_agent_context` runs while automatic session context is on
(`auto_feedback` with session caching; otherwise `auto_feedback_disabled`)
and an LLM is configured. Per session it reads the trace steps newer than the
session's extraction watermark (plus 3 already-processed ones, at most 40),
shows the model the session's existing agent lessons,
and stores at most 5 new lessons as `agent`-profile session-context entries
(sections `tool_rules`, `workflow_state`, `success_patterns`,
`failure_lessons`, `environment_facts`). A lesson is stored only with
confidence ≥ 0.75, cut to 280 characters; one whose normalized text an
entry of the same section already has is not stored again. The watermark
advances only after the session's extraction succeeded. Counts: `lessons`,
`sessions_failed`; the stage is `errored` when any session failed, and the
other sessions still run. Error texts shown to the model are redacted
(tokens, secrets, UUIDs, long numbers).

## User preferences

`update_user_preferences` runs while `personalization_enabled` is on. It folds
the sessions' turns into one `UserPreference` node per (user, dataset), an
internal node (never embedded, never returned by a search) in the
`user_preferences` node set, and weighted `prefers` edges from it to the graph
nodes the turns' answers used (`used_graph_element_ids.node_ids`). A turn's
rating is its `feedback_score`, else the rating a feedback row inferred for
it; a rating `r` pulls each node's weight toward `(r - 1) / 4` by
`preference_alpha`, from its decayed value
`0.5 + (w - 0.5) * (1 - preference_beta)^(idle turns)`. Every turn advances
the node's turn counter once; an edge that decayed to within 0.01 of neutral
is retired. The node's text collects the usable `preferences`-section
session-context entries newer than its watermark, newest first, capped at
2000 characters on line boundaries. Turns are marked in `memify_metadata`
(`preference_turn_counted`, `preference_weights_applied`), so a re-run
counts and applies each once. Counts: `turns_applied`, `edges_written`,
`edges_pruned`, `text_lines_added`; nothing to fold is `completed` with the
reason `no_changes`. In Rust, `preferences`-section entries come from a session
search's turn analysis (while `auto_feedback` is on with session caching), which
also records the rating a feedback row infers for the previous answer.

With personalization on, searches read these back for the requesting user in
the search's single dataset: the decayed `prefers` weights scale the matched
nodes' distances in graph triplet ranking (every graph completion type) and
re-rank RAG completion's chunks (fetching `wide_search_top_k` candidates
first, then trimming to `top_k`), by at most `personalization_influence`;
graph and RAG completion also append the preference text (under
`## What this user prefers`) to the question and context. The hybrid search
uses them too: the weights re-rank its chunk lane, and its completion and
`only_context` preview carry the same preference guidance. On a session search
the preference lines instead join the `Preferences` section of the one
`## Active session guidance` block (budgets of 400 characters per section and
1200 in total), served after the question and context together with the
session's stored guidance entries; there is no second block. An answered turn
records the ids of the guidance entries it served (`used_session_context_ids`),
so the next turn's ratings of that guidance apply. Session history now goes
into the user prompt, ahead of the rendered template, rather than into the
system prompt. A RAG completion turn now records its chunks' ids, so
ratings can weight chunks.

## Locking

A run holds its dataset's lock for its whole length, then a claim on every
session it targets (`session:{user_id}:{session_id}`), and releases both once
it ends. Both wait, with no timeout: improves that share a dataset or a session
run one after the other, as pipeline runs on one dataset do, instead of the
later one skipping; runs on different datasets and sessions do not wait. The
dataset lock is one shared registry: `cognee_core::DatasetLocks::global()`
is the default of `add` (`AddPipeline::new_with_algorithm`; `AddPipeline::new`
takes no lock unless one is set), dataset-scoped deletes
(`DeleteService`) and `improve` (`ImproveParams::dataset_locks`, else the add
pipeline's registry, else the global one), and `cognify` always
holds it for its run, so in one process those writers queue per dataset. A
host with its own registry wires it into the add pipeline, the delete service
and `improve`. The stages' own adds and
cognifies re-enter the improve's hold. A dataset that does not exist yet is
held by the id an `add` of it would create.

## Global context index

`global_context_index` (on by default: the
`improve.global_context_index` setting, `IMPROVE_GLOBAL_CONTEXT_INDEX`, also
for the automatic improve after `remember()`; per call,
`build_global_context_index` in the Rust `improve()` API,
`--build-global-context-index[=BOOL]` in the CLI or `buildGlobalContextIndex`
in the bindings overrides it either way) builds the
dataset's global context index with the default `from_entities` method,
which makes no LLM call: one `GlobalContextSummary` bucket per document
naming the entities it is distinctively about (tf·idf over the dataset's
chunks, up to 20) and up to 40 relation lines among them, plus a root of
three lines (`Topics:`, `Entities:`, `Documents:`). Ids are uuid5
ids, so the text and ids match the index Python cognee builds on a shared
store byte for byte. Each run computes the
whole index, writes only the nodes whose text changed (graph node, vector in
`GlobalContextSummary_text`, `summarized_in` edge to the root and
`summarizes` edge to the document) and deletes the index nodes it did not
compute; the nodes' ownership rows name the dataset, so deleting the
dataset removes them. Its count `nodes_written` is the nodes it wrote. The
opt-in LLM method (vector-placed buckets of `TextSummary` nodes, levels, LLM
bucket and root summaries) is
`cognee_cognify::build_global_context_index` with
`GlobalContextIndexMethod::Llm`; `improve()` does not use it. A graph or hybrid completion search with `include_global_context_index`
prepends the root and the `global_context_index_top_k` (3) nearest bucket
summaries to its context (`World summary:` / `Relevant areas:`). A
dataset-scoped search reads only its datasets' root and summaries (their
`dataset_id`), so datasets sharing one graph and vector store never see each
other's index. With no triplet found, the prelude alone is the context, and an
`only_context` search previews the prompt built from it.

`global_context_index` reports `already_completed` (`no_writes_since_last_improve`)
when no `pipeline_runs` row other than a `search`, a `recall` or a host's
`improve_pipeline` record was written since it last ran. A delete records a
`forget` row on every dataset it touched, and an improve run whose stage may
have written to the graph (anything but the index itself, unless its write
counts say it wrote nothing) records one `improve` row, so both count as
changes. A run that did not end `errored` stamps the stage's start in the `run_info` of
its `improve` operation row (`{"global_context_index": {"status", "started_at"}}`),
where Python cognee also keeps it; the newest such row is the watermark, and the row itself
is not a change. Without a relational database, a stamp, or on any error
reading them, the stage runs. (The `graph_sync_checkpoints` table that held the
stamps before is gone, and with it `CheckpointStore` and
`ImproveParams::checkpoint_store`; a migration moves existing stamps into `improve` rows, so
no stage reruns because of the upgrade.)
What is not ported yet is tracked in
[not-implemented.md](https://github.com/topoteretes/cognee-rs/blob/main/docs/roadmap/not-implemented.md).
rustdoc: `api::improve`.

## Related

> [card] [Remember](https://docs.cognee.ai/rust/core-concepts/main-operations/remember.md)
>
> Runs improve after storing

> [card] [Sessions](https://docs.cognee.ai/rust/core-concepts/sessions-and-caching.md)
>
> Where session lessons live

> [card] [Improve configuration](https://docs.cognee.ai/rust/configuration/search.md#improve-self-improvement-loop)
>
> improve.\* settings


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.