> ## Documentation Index
> Fetch the complete documentation index at: https://docs.cognee.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Rust SDK DataPoints

> DataPoints in cognee-rust: the storage-layer unit every graph node embeds, with provenance and indexing.

## What are DataPoints

DataPoints are the units that make up memory. Every graph node is a DataPoint, and the vector store indexes them, so they are what cognify writes and what search reads.

* **Atomic**: each DataPoint holds one concept or unit of information: a document, a chunk, a summary, an entity, an entity type.
* **Structured**: each kind is a plain Rust struct that embeds a shared `DataPoint` base and is serialized with serde.
* **Contextual**: the base carries provenance, versioning and indexing hints, so every later step knows where a node came from and how to use it.

The same `id` names a DataPoint in all three stores: the graph node, its vector rows and its ownership rows in the relational database.

## Core Structure

`DataPoint` is a struct with a fixed set of standard fields. Typed nodes embed it as their `base` field.

> [accordion] See the struct definition
>
> ```rust theme={null}
> // crates/models/src/data_point.rs (serde attributes and doc comments trimmed)
> pub struct DataPoint {
>     pub id: Uuid,
>     pub created_at: i64,                 // ms since epoch
>     pub updated_at: i64,                 // ms since epoch
>     pub ontology_valid: bool,
>     pub version: i32,                    // default 1
>     pub topological_rank: Option<i32>,   // default Some(0)
>     pub metadata: HashMap<String, serde_json::Value>,
>     #[serde(rename = "type")]
>     pub data_type: String,
>     pub belongs_to_set: Option<Vec<serde_json::Value>>,
>     pub source_pipeline: Option<String>,
>     pub source_task: Option<String>,
>     pub source_node_set: Option<String>,
>     pub source_user: Option<String>,
>     pub source_content_hash: Option<String>,
>     pub feedback_weight: f64,            // default 0.5
>     pub importance_weight: Option<f64>,  // default Some(0.5)
> }
> ```
>
> `DataPoint::new(data_type, dataset_id)` and `DataPoint::with_metadata(data_type, dataset_id, metadata)` build one with a random UUID v4, both timestamps set to now, and the defaults above. A `Some(dataset_id)` is recorded in `belongs_to_set` as the dataset id's string.
>
> Key fields:
>
> * `id`: the identifier shared by the graph node, its vector rows and its ownership rows. The built-in types replace the random default with a deterministic id (see [Deduplication](#deduplication-stable-ids)).
> * `created_at`, `updated_at`: timestamps in milliseconds since the epoch. `touch()` refreshes `updated_at`.
> * `version`: starts at `1`. `update_version()` increments it.
> * `topological_rank`: the 1-based position of the pipeline task that produced the DataPoint. `0` means unset.
> * `metadata`: free-form map. The built-in types store their `index_fields` here.
> * `data_type`: the type discriminator, serialized as `type` (`"Entity"`, `"DocumentChunk"`, `"TextDocument"`, ...).
> * `belongs_to_set`: dataset and node-set tags. See [Node sets](https://docs.cognee.ai/rust/core-concepts/further-concepts/node-sets.md).
> * `ontology_valid`: set when the node was matched against an ontology.
>
> Some fields are filled in by cognee. Leave them at their defaults:
>
> * `source_pipeline`, `source_task`, `source_user`, `source_node_set`, `source_content_hash`, `topological_rank`: the pipeline executor stamps them on every `Document`, `DocumentChunk`, `Entity`, `EntityType` and `EdgeType` a task returns (bare or in a `Vec`), only where they are still unset. cognify stamps its `TextSummary` values itself. See [Provenance stamping](https://docs.cognee.ai/rust/core-concepts/building-blocks/pipeline-context.md#provenance-stamping).
> * `importance_weight`: copied from the source `Data` record onto its document, chunks, summaries and extracted entities. Triplet ranking reads it. `0.5` is neutral.
> * `feedback_weight`: adjusted by improve's feedback stage and read by triplet ranking. See [Feedback weights](https://docs.cognee.ai/rust/core-concepts/main-operations/improve.md#feedback-weights) and [Triplet ranking](https://docs.cognee.ai/rust/core-concepts/main-operations/legacy-operations/search.md#triplet-ranking).

A typed node is a struct whose `base` is flattened into it, so the stored node is one flat object:

```rust theme={null}
// crates/models/src/entity.rs
pub struct Entity {
    #[serde(flatten)]
    pub base: DataPoint,
    pub name: String,
    pub is_a: Option<Uuid>,   // the EntityType this entity is an instance of
    pub description: String,
}
```

```rust theme={null}
use cognee::models::{Entity, EntityType};

let person = EntityType::new("Person", "", None);
let alice = Entity::new("Alice", Some(person.base.id), "A software engineer", None);

assert_eq!(alice.base.data_type, "Entity");
assert_eq!(alice.base.id, Entity::id_for("Alice"));
```

## How DataPoints work in Rust

A **DataPoint** is the base storage-layer unit: a structured record that carries
a stable UUID, timestamps, a `type` discriminator, free-form `metadata`, and
provenance fields (`source_pipeline`, `source_task`, `source_node_set`,
`source_content_hash`). Typed graph nodes — `Entity`, `EntityType`, `EdgeType`,
`DocumentChunk`, etc. — embed a DataPoint as their `base`, exposed through the
`HasDataPoint` trait so provenance stamping can walk any node uniformly. When a
DataPoint is indexed, its vector-store payload has the `IndexSchema`
shape (`index_schema_payload()`): the indexed `text`, a few reference
scalars (`document_id`, `chunk_index`, `source_chunk_id`, …) and the base fields
at their IndexSchema defaults, plus the Rust-only `dataset_id` that
dataset-scoped search filters on. LanceDB stores it without the `metadata` key;
pgvector keeps it. (This replaces
`vector_metadata()`; rows written by earlier versions stay readable.)
Serialization and versioning helpers are `to_json_string()`
/ `from_json()`, `to_dict()` / `from_dict()` and `update_version()` (bumps
`version` and refreshes `updated_at`).

Rust: `DataPoint` / `HasDataPoint` in [`cognee-models`](https://github.com/topoteretes/cognee-rs/blob/main/crates/models/).

## Indexing & Embeddings

Each DataPoint type has one or more **index fields**. Their values are embedded and written to the vector store, which is what semantic search runs against.

* Each indexed field gets its own vector collection named `Type_field`: an entity's `name` goes to `Entity_name`, a chunk's `text` to `DocumentChunk_text`.
* Only the index field's value is embedded. An entity's `description`, for example, is not.
* Non-indexed fields stay as properties of the graph node.

For the built-in types the indexed fields are fixed by cognify's add-data-points stage, which writes each collection explicitly. The types also declare them as an `INDEX_FIELDS` constant (`Entity::INDEX_FIELDS == ["name"]`) and under `metadata["index_fields"]`. For a [custom type](#custom-datapoint-types), the `index_fields` you declare in the schema choose the collections.

The vector row is not a copy of the DataPoint. It holds the embedded `text`, the shared `id`, a few reference fields and the `dataset_id`, as described in [How DataPoints work in Rust](#how-datapoints-work-in-rust). A write replaces any existing row with the same `id`.

> [!NOTE]
>
> **Cross-store retrieval:** a vector hit carries the DataPoint's `id`, which is also its graph node id. Search reads the node's full properties from the graph store, since the vector row holds only the indexed text.

Vectors are written in windows, and are not kept in the cognify result unless you ask for them. See [Writing vectors](https://docs.cognee.ai/rust/core-concepts/main-operations/legacy-operations/cognify.md#writing-vectors).

## From DataPoints to the Graph

cognify writes DataPoints in two stages (both in [`crates/cognify/src/tasks.rs`](https://github.com/topoteretes/cognee-rs/blob/main/crates/cognify/src/tasks.rs)): graph extraction (`extract_graph_from_data`) writes the extracted entities and the edges between them, and the add-data-points stage (`add_data_points`) writes the documents, chunks, entity types and summaries, their structural edges, and every vector collection. Together they:

* Record in the relational database which data item owns each node and edge, before anything reaches the graph or vector store, so a failed write can be swept and `forget` can remove them with their document.
* Write each DataPoint as a graph **node**, with its fields as node properties.
* Turn its relationship fields into graph **edges**.
* Embed the index fields and write the vector collections.

A relationship field holds the target's `Uuid`, not the target struct. Each built-in type declares its outgoing relationships through the `GraphExtractable` trait, and `get_graph_from_model` collects them into deduplicated edges named after the field:

| Source | Field | Target |
| - | - | - |
| `DocumentChunk` | `is_part_of` | its `Document` |
| `DocumentChunk` | `contains` | each `Entity` extracted from the chunk |
| `Entity` | `is_a` | its `EntityType` |
| `TextSummary` | `made_from` | the `DocumentChunk` it summarizes |

The relationships the LLM extracts between entities become edges named by their `relationship_name`. Edges can carry an `edge_text` sentence. The `EdgeType_relationship_name` collection embeds each edge's non-blank `edge_text`, or its `relationship_name` when there is none.

This is how one unit gives both **semantic similarity** (vector) and **structural reasoning** (graph).

## Examples and details

> [accordion] Example: relationship fields
>
> ```rust theme={null}
> use cognee::cognify::{GraphExtractable, get_graph_from_model};
> use cognee::models::{Entity, EntityType};
>
> let person = EntityType::new("Person", "", None);
> let alice = Entity::new("Alice", Some(person.base.id), "A software engineer", None);
>
> let edges = get_graph_from_model(&[&alice as &dyn GraphExtractable]);
> // One edge: (alice.id, person.id, "is_a", { updated_at })
> assert_eq!(edges[0].2, "is_a");
> ```
>
> `belongs_to_set` is a tag, not a relationship: it never becomes an edge.

> [accordion] Built-in DataPoint types
>
> | Type | Struct | Indexed field → collection |
> | - | - | - |
> | Documents | `Document`, with `type` set to the concrete kind: `TextDocument`, `PdfDocument`, `CsvDocument`, `ImageDocument`, `AudioDocument`, `UnstructuredDocument`, `DltRowDocument` | `name` → `TextDocument_name`, `PdfDocument_name`, ... |
> | Chunks | `DocumentChunk` | `text` → `DocumentChunk_text`. A DLT row is a `DocumentChunk` with type `DltRow`, indexed into `DltRow_text`. |
> | Summaries | `TextSummary` (in `cognee::cognify`) | `text` → `TextSummary_text` |
> | Global context summaries | `GlobalContextSummary`, built by improve's [global context index](https://docs.cognee.ai/rust/core-concepts/main-operations/improve.md#global-context-index) stage | `text` → `GlobalContextSummary_text` |
> | Entities | `Entity` | `name` → `Entity_name` |
> | Entity types | `EntityType` | `name` → `EntityType_name` |
> | Edge types | `EdgeType` | `relationship_name` → `EdgeType_relationship_name` |
>
> `EdgeType` rows exist only in the vector store. cognify does not add them to the graph as nodes. Each one is keyed on an edge's retrieval text: its non-blank `edge_text`, else its `relationship_name`.
>
> All of these implement `HasDataPoint`. All but `TextSummary` are in `cognee::models` ([`crates/models/`](https://github.com/topoteretes/cognee-rs/blob/main/crates/models/)).

> [accordion] Updating DataPoints
>
> The Ladybug and Postgres graph adapters upsert a node by `id`: a node written with an existing `id` replaces the stored node's name, type and properties. The LanceDB and pgvector stores replace a row with the same `id` too. So when cognify writes an entity whose deterministic id already exists, it updates that node rather than adding a second one.
>
> To change a document you added, use [`update`](https://docs.cognee.ai/rust/core-concepts/main-operations/overview#additional-operations). It drops the document's memory, re-adds the new content under the same `data_id` and re-cognifies the dataset.
>
> Versioning is manual. Nothing in cognify calls `update_version()`: a DataPoint you change keeps `version` at its old value until you call it yourself.
>
> ```rust theme={null}
> let mut alice = Entity::new("Alice", None, "An engineer", None);
> alice.description = "A staff engineer".to_string();
> alice.base.update_version(); // version 2, updated_at refreshed
> ```

> [accordion] Deleting DataPoints
>
> DataPoints are deleted with the data they came from, not one by one. `forget` removes a data item, a dataset or everything, across the relational, graph and vector stores and file storage, using the ownership rows add-data-points recorded. A node that several data items produced keeps all of them as owners and is removed only with the last one.
>
> See [What you can forget](https://docs.cognee.ai/rust/core-concepts/main-operations/forget.md#what-you-can-forget) for the targets, and [delete](https://docs.cognee.ai/rust/core-concepts/main-operations/forget.md#delete) for the lower-level service with a dry-run preview.

> [accordion] Dataset routing: which dataset receives the DataPoints?
>
> The add-data-points stage reads the dataset from its input: the `dataset_id`, `user_id` and `tenant_id` fields of the `SummarizedData` that the earlier cognify stages fill in for the dataset being processed. There is no separate dataset argument.
>
> The dataset is used in two places:
>
> * **Ownership rows**: each one records the tenant, user, dataset and pipeline run alongside the node or edge it owns. When no user is given, the rows are recorded under a default user.
> * **Vector rows**: every row carries `dataset_id`, which dataset-scoped search filters on. When a row with the same `id` is indexed again for a second dataset, the vector adapter keeps the union of both datasets in the row's `dataset_ids`.
>
> See [Datasets](https://docs.cognee.ai/rust/core-concepts/further-concepts/datasets.md) for how datasets are named and resolved.

> [accordion] Deduplication: stable ids
>
> The built-in types do not use the random UUID v4 that `DataPoint::new` gives. Each one derives a deterministic id, so the same thing written twice lands on the same node:
>
> | Type | Id |
> | - | - |
> | `Entity` | `Entity::id_for(name)` |
> | `EntityType` | `EntityType::id_for(name)` |
> | `EdgeType` | `EdgeType::deterministic_id(retrieval_text)` |
> | `TextSummary` | UUID v5 of `"TextSummary"` in the namespace of its chunk id |
> | `Document` | the id of the `Data` record it was classified from |
>
> `Entity::id_for` and `EntityType::id_for` call `data_point_id_for` in [`cognee_utils`](https://github.com/topoteretes/cognee-rs/blob/main/crates/utils/src/id_generation.rs), which computes `uuid5(NAMESPACE_OID, "ClassName:value1|value2|...")` with each value lower-cased, spaces turned into `_` and apostrophes removed. The class name namespaces the id, so an `Entity` and an `EntityType` with the same name never collide.
>
> Because an extracted entity's id comes from its name, the same name in two chunks or two documents is one node. When several nodes in one chunk's extraction share a name, each gets a chunk-scoped id instead. See [Entity ids](https://docs.cognee.ai/rust/core-concepts/main-operations/legacy-operations/cognify.md#entity-ids).
>
> **Checking whether a DataPoint already exists**
>
> Compute the id and ask the graph store:
>
> ```rust theme={null}
> use cognee::graph::GraphDBTrait;
> use cognee::models::Entity;
>
> let id = Entity::id_for("Alice");
> if graph_db.has_node(&id.to_string()).await? {
>     let node = graph_db.get_node(&id.to_string()).await?;
>     println!("Already in the graph: {node:?}");
> }
> ```
>
> Here `graph_db` is any `GraphDBTrait` implementation, such as the Ladybug or Postgres adapter.
>
> For a custom type, list the fields that identify it in `identity_fields`. See the [schema contract](https://docs.cognee.ai/rust/guides/custom-graph-schema.md#schema-contract).

> [accordion] Custom DataPoint types
>
> cognify stores only the built-in structs and the types described by a custom graph schema. A struct of your own that embeds `DataPoint` and implements `HasDataPoint` is not picked up by the add-data-points stage, and the pipeline executor does not stamp it.
>
> To have cognify extract and store your own node types, pass a JSON Schema as `CognifyConfig.graph_schema` (or a spec with `--graph-spec`). Each model in it becomes a DataPoint type at runtime, and its `metadata` default sets the same policy the built-in types have:
>
> * `index_fields`: the scalar fields embedded into `Model_field` collections.
> * `identity_fields`: the fields that make up the deterministic id.
> * `transparent`: the model is a container, not a stored node.
>
> Fields that reference another model become edges named after the field. See [Custom Graph Schema](https://docs.cognee.ai/rust/guides/custom-graph-schema.md) for the full contract, validation, entry points and the Rust-only [`FromIdentity`](https://docs.cognee.ai/rust/guides/custom-graph-schema#fromidentity-references-rust-only-keyword) keyword.

> [accordion] Transparent containers: nodes that group rather than describe
>
> In a custom graph schema, a model whose `metadata` default sets `"transparent": true` exists only to hold other models, for example a root model that wraps a list of entities.
>
> Wherever a transparent node appears, it is replaced by its DataPoint children:
>
> * The container is never stored: no node, and never an edge endpoint.
> * Its children take its place, so a transparent root yields its children as top-level nodes, and a field that points at a transparent node gets edges to that node's children.
> * Nesting works: a transparent child of a transparent container resolves too.
> * A scalar field on a transparent model has nowhere to be stored. When it holds a value, cognee logs a warning and drops it.
>
> A transparent root needs at least one reachable non-transparent descendant; otherwise the custom graph stores no nodes. See the [schema contract](https://docs.cognee.ai/rust/guides/custom-graph-schema.md#schema-contract).

## Related

> [card] [Tasks](https://docs.cognee.ai/rust/core-concepts/building-blocks/tasks.md)
>
> What produces DataPoints

> [card] [Pipelines](https://docs.cognee.ai/rust/core-concepts/building-blocks/pipelines.md)
>
> How tasks are chained

> [card] [Custom graph schema](https://docs.cognee.ai/rust/guides/custom-graph-schema.md)
>
> Define your own node types

> [card] [Cognify](https://docs.cognee.ai/rust/core-concepts/main-operations/legacy-operations/cognify.md)
>
> The pipeline that writes DataPoints

> [card] [Forget](https://docs.cognee.ai/rust/core-concepts/main-operations/forget.md)
>
> Delete data and the DataPoints built from it


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.