What is PipelineContext
PipelineContext tells a running task which pipeline run it belongs to, who started it, which dataset it works on and which data item is being processed. Tasks use it to attribute what they write, for example to stamp provenance on DataPoints.
How tasks reach it
Every task runs with aTaskContext (cognee-core, task_context.rs). It carries the shared resources of the run:
TaskContext::pipeline() returns the PipelineContext. It panics when none is set, so call it only from tasks that always run inside a pipeline executor; otherwise read the pipeline_ctx option.
A Rust task always receives the TaskContext and reads what it needs from it.
Fields
user_label() gives the value used as a DataPoint’s source_user: the email when set, else the user id, else nothing.
current_data
current_data is the original value that entered the pipeline for this data item, not the previous task’s output. The executor sets it once per item, and every task in the chain sees the same value, batch tasks included.
Before cognee-rs #120 (August 2026), current_data held the previous task’s output, and batch tasks saw None. A custom task written against that behaviour must use its input argument instead.
Because the value stays reachable until the item’s whole chain finishes, a pipeline whose input holds a large buffer (such as DataInput::Binary) keeps it in memory longer. With Pipeline::with_concurrency above 1, more items overlap. Stream large payloads instead of putting them in the input value.
Provenance stamping
provenance_visited makes sure a DataPoint that passes through several tasks is stamped once, with the first task’s name.
Setting the context
The caller builds thePipelineContext and puts it on the TaskContext with TaskContextBuilder::pipeline_context. Its fields are public and it has no constructor, so you write all of them:
current_data and run_id unset: execute fills them in. At the start of each run it also clears provenance_visited, so a reused context does not carry stamps from an earlier run. The user and dataset ids decide the run’s attribution: execute derives the run’s pipeline_id from the user id, the pipeline name and the dataset id, and runs in the dataset’s scope when a dataset is set. See How pipelines run.
Custom pipeline state
PipelineContext has no free-form map for your own values. A task is a Send + Sync + 'static closure, so it takes its settings by capturing them:
TaskContext::publish_payload_field(key, value) with a JSON value:
TaskContext (TaskContextBuilder::pipeline_watcher), not to the one passed to execute, and the call does nothing when that watcher is unset or the task runs outside execute. DbPipelineWatcher stores each field in the pipeline_run_payload_fields table, one row per run and key, where a later value for the same key replaces the earlier one. PipelineRunRepository::get_payload(run_id) reads them back as a JSON map.
Examples and details
Accessing user and dataset in a task
Accessing user and dataset in a task
Read the identity fields when a task writes its own records or applies per-user logic:The stores on the
TaskContext already point at the run’s dataset: when dataset_id is set, execute runs every task in that dataset’s scope, so under per-dataset storage graph_db and vector_db reach the dataset’s own databases.State persists across chained tasks
State persists across chained tasks
Every task of a run sees the same
PipelineContext values: the same user, dataset, run id and provenance_visited set. Only current_data changes, once per data item. Tasks receive the context behind an Arc and cannot change it, so data flows from task to task through task outputs. For settings several tasks share, capture clones of one Arc:Making the context optional
Making the context optional
TaskContext::pipeline() panics without a PipelineContext. A task that may also run outside a pipeline executor, in a test or called directly, reads the option instead:Related
Tasks
The units of work that read the context
Pipelines
The runs the context describes
DataPoints
Where provenance is stamped