Skip to main content
The vocabulary behind cognee-rust: the stores that hold memory, the building blocks that produce it, and the terms that show up across the API, CLI, and config. This page is the conceptual map — API/type detail lives in rustdoc (cargo doc --no-deps --open). For what the system does see operations; for how it fits together see architecture.

​
Architecture: three stores

Cognee keeps memory in three complementary backends. Every cognify run writes to all three; search reads across them. Backend selection and connection settings are covered in tools/backends and configuration; the layering is in architecture.

​
Building blocks

​
DataPoints

A DataPoint is the base storage-layer unit: a structured record that carries a stable UUID, timestamps, a type discriminator, free-form metadata, and provenance fields (source_pipeline, source_task, source_node_set, source_content_hash). Typed graph nodes — Entity, EntityType, EdgeType, DocumentChunk, etc. — embed a DataPoint as their base, exposed through the HasDataPoint trait so provenance stamping can walk any node uniformly. When a DataPoint is indexed, its vector-store payload has the shape of Python’s IndexSchema (index_schema_payload()): the indexed text, a few reference scalars (document_id, chunk_index, source_chunk_id, …) and the base fields at their IndexSchema defaults, plus the Rust-only dataset_id that dataset-scoped search filters on. LanceDB stores it without the metadata key, as Python’s LanceDB adapter does; pgvector keeps it. (This replaces vector_metadata(); rows written by earlier versions stay readable.) Python’s serialization and versioning helpers are mirrored as to_json_string() / from_json(), to_dict() / from_dict() and update_version() (bumps version and refreshes updated_at). Rust: DataPoint / HasDataPoint in cognee-models.

​
Tasks

A Task is one reusable unit of work that transforms data — classify, chunk, extract, summarize, embed. Tasks come in eight execution flavours (sync/async × single/iterator/stream × single-value/batch) so a step can stream, fan out, or process whole batches. They are composed with optional per-task config (TaskInfo: name, batch size, weight, rate limiter) and the pipeline executor routes values between them. A stream task that fails after it has started yielding fails the run: the failure is not retried, because the stream is already partly consumed, and items already passed downstream are not rolled back. Rust: Task / TypedTask / TaskInfo in cognee-core.

​
Pipelines

A Pipeline is an orchestrated sequence of Tasks with shared context (database, graph, vector, cancellation, progress) and a watcher for status events. The concrete pipelines are: Pipeline orchestration primitives (PipelineWatcher, ExecStatusManager, thread pool) live in cognee-core. The end-to-end flow is described in operations.

​
Key concepts

​
Datasets

A Dataset is the organizational scope for memory operations: named, owned by a user, optionally tenant-scoped. As in Python, a data row belongs to exactly one dataset (data.dataset_id) and gets a random uuid4 id, so the same content added to two datasets is two rows. add / cognify / search / delete all operate within a dataset scope. Dataset IDs are deterministic (UUID5 of name, owner and tenant) for cross-SDK reproducibility. Rust: Dataset in cognee-models; lifecycle helpers in DatasetManager (cognee api::datasets).

​
Sessions

A Session is a temporary memory context — search/answer history and feedback for a single conversational thread — distinct from permanent, graph-backed storage. Passing a --session-id to remember / recall scopes a turn to that session and lets retrieval reuse prior context; omitting it on remember persists input as permanent graph memory (see the memory API in operations). A completion search / recall (a retriever that prepares session turns; not chunks, summaries, lexical, Cypher, …) that omits it now uses the default session when the caller is known and session memory is on (session.caching); before, such a search was stateless. As in Python’s resolve_session_id, the id is default_session_<dataset_id> when one distinct dataset is searched, so separate single-dataset searches never share a session, else default_session. A search over several datasets, or naming none (unless per-dataset storage leaves the caller exactly one readable dataset, which scopes it), is one logical turn in the unscoped default session. This diverges from Python in two cases: Python’s access-control path uses one session per dataset, and on a shared store Python scopes a search naming no dataset to the user’s only dataset. Such a search reads the session’s history and, while AUTO_FEEDBACK is on, its stored guidance entries, analyzes the turn (also only while AUTO_FEEDBACK is on) and saves its Q&A; pass your own session id to keep it out of the default session, or turn session.caching off to keep it stateless. The store backend is pluggable. Rust: SessionStore trait in cognee-session, with FsSessionStore (crate feature fs), RedisSessionStore (crate feature redis), and RelationalSessionStore backends. Setting session.store = "fs" or "redis" requires a cognee built with its session-fs / session-redis feature. Those features register the store factory and its session.fs / session.redis config section; enabling cognee-session/fs or cognee-session/redis alone does neither. Next to its Q&A entries and agent trace steps, a session keeps a list of session-context rows (Python’s session context): JSON objects with a "kind" field. "context" rows are active guidance entries (SessionContextEntry: goals, rules, preferences, lessons); other kinds are feedback records or internal state rows, such as the StateRowWatermark rows the improve stages use to remember how far they got. Only "context" rows are ever rendered (coerce_active_context_entries). SessionManager::{get,create,update}_session_context_entry read and write them on every store (session_context_entries table, {session_id}.context.json file, session_context:{user}:{session} Redis list); deleting or pruning a session removes them. The table holds one row per (user_id, session_id, entry_id) (unique index uq_session_context_entry, as Python’s uq_cache_session_context_entry), so on the relational store a create of an existing entry id replaces it in place (the file and Redis stores append); a session without a user is the "default" user’s, as on the file and Redis stores. apply_candidate_updates is the one writer of guidance entries (confidence ≥ 0.75, at most 280 characters, an exact duplicate links its new source instead of a second entry); improve’s extract_agent_context stage feeds it the agent-profile lessons it draws from the session’s trace steps. The session graph-context snapshot is removed: SessionStore / SessionManager::{get,set}_graph_context, cognee::session::{get,set}_graph_context and the bindings’ graph-context ops are gone, a migration drops the session_graph_context table, and session history no longer starts with a “Background knowledge from the knowledge graph” prefix. The recall scope "graph_context" is a deprecated alias of "graph" (it logs a warning), and "all" means graph, session and trace. With personalization on (personalization_enabled), improve’s update_user_preferences stage keeps one internal UserPreference node per user and dataset, with weighted prefers edges to the graph nodes the user’s rated answers used and the user’s stated preferences as text; graph, RAG, and hybrid searches by that user rank by those weights and carry that text (see Configuration: feedback & personalization).

​
Node Sets

A node set is a tag attached to ingested data and the DataPoints derived from it, used to categorize and later scope the knowledge base. It is partially realized in Rust today:
  • Tagging at ingest — add accepts a node_set, stored on Data.node_set and under external_metadata["node_set"], where cognify reads it and propagates it to derived DataPoints as source_node_set. Earlier versions wrote only the column, so data added with a node set was never tagged and node_name searches missed it; cognify now falls back to the column for those rows. Adding the same content again with a node set replaces its stored tags (without one, they are kept). Every name must be non-empty, and an item’s own external_metadata["node_set"] must be a list of names or null; otherwise the whole add fails with IngestionError::InvalidNodeSet before anything is stored. The pipeline executor can attach node_set provenance to task outputs (Tagged / TaggedMeta in cognee-core).
  • Scoping — a graph subset can be selected by node type and node name (the graph trait’s get_nodeset_subgraph, used by search’s node-name filter). The internal persist_sessions step tags cached session data with a fixed node set. Memify’s --node-type / --node-name scoping went away with memify.
Note: this is type/name-based subgraph filtering rather than a fully general named-node-set query surface; treat node sets as a tagging-and-scoping primitive, not a finished feature.

​
Ontologies

An ontology grounds extracted entities in external, structured knowledge. cognee-rust loads RDF/OWL ontologies (Turtle, RDF/XML, N-Triples, JSON-LD) and uses them for fuzzy entity matching and subgraph enrichment during cognify. The default is a no-op resolver (no grounding), matching Python’s ontology_file=None. Rust: OntologyResolver trait in cognee-ontology, with NoOpOntologyResolver (default) and RdfLibOntologyResolver. Enabled per run via cognify’s --ontology-file; see configuration.

​
Loaders & Chunkers

Loaders handle file-format reading at ingest: a loader registry dispatches by MIME type / extension to per-format loaders (text, PDF, CSV, HTML, image, audio, and the unstructured office formats), most behind feature flags. As in Python, they run at add, which stores the extracted text (an image’s vision-LLM description, an audio file’s transcript) and keeps the source file in original_data_location. Chunkers then segment a document into token-bounded pieces through a word → sentence → paragraph hierarchy, sizing chunks with a pluggable TokenCounter (WordCounter, or the feature-gated HuggingFace / tiktoken counters). Rust: loaders (LoaderRegistry, DocumentLoader) in cognee-ingestion; chunking (text_chunker, TokenCounter) in cognee-chunking. Token-counter selection is configured in configuration.

​
See also