Building the Global Context Index

This guide shows you how to build a global context index over a dataset, inspect the buckets and root it produces, run improve() again to see an unchanged dataset skipped, and then add more data to see the index update.

Before You Start

Complete Quickstart to understand basic operations
Ensure you have LLM Providers configured for the ingestion step that extracts entities. The index itself makes no LLM call.
Read Global Context Index for the conceptual overview — this guide focuses on the runnable example, not the full parameter surface
Be familiar with improve()

Code in Action

import asyncio

import cognee
from cognee.infrastructure.databases.graph import get_graph_engine

DATASET = "global_context_index_demo"

INITIAL_FACTS = [
    "Alice hikes in the Alps every summer.",
    "Bob sails along the Adriatic coast every summer.",
    "Alice says hiking helps her disconnect from work.",
    "Bob says sailing is the best way to unwind after a busy winter.",
    "Last year Alice hiked a new trail near Lake Como.",
    "Last year Bob sailed to a small island he had never visited.",
]

ADDITIONAL_FACT = (
    "This year, Bob decided to join Alice's hiking trip to the Alps instead of sailing."
)


async def print_index_structure(label):
    """Print the index and return its texts, so two runs can be compared."""
    graph_engine = await get_graph_engine()
    nodes_data, _edges_data = await graph_engine.get_graph_data()

    root = None
    buckets = []

    for node_id, properties in nodes_data:
        if properties.get("type") != "GlobalContextSummary":
            continue
        if properties.get("is_root"):
            root = (node_id, properties.get("text", ""))
        else:
            buckets.append((node_id, properties.get("text", "")))

    print(f"\n{label}")
    print(f"Buckets: {len(buckets)}")
    for bucket_id, text in buckets:
        print(f"  - [{bucket_id}]\n{indent(text)}")
    if root:
        print(f"Root [{root[0]}]:\n{indent(root[1])}")

    return sorted(text for _, text in buckets), root[1] if root else None


def indent(text):
    return "\n".join(f"      {line}" for line in text.splitlines())


async def main():
    await cognee.remember(
        INITIAL_FACTS,
        dataset_name=DATASET,
        self_improvement=False,
    )
    await cognee.improve(dataset=DATASET, build_global_context_index=True)
    first = await print_index_structure("Index structure after the initial build:")

    # Nothing changed in between, so this run skips the build: the stage reports
    # already_completed and the index stays as it is.
    await cognee.improve(dataset=DATASET, build_global_context_index=True)
    second = await print_index_structure("Index structure after a second improve():")
    print(f"\nSecond run produced identical texts: {first == second}")

    await cognee.remember(
        ADDITIONAL_FACT,
        dataset_name=DATASET,
        self_improvement=False,
    )
    await cognee.improve(dataset=DATASET, build_global_context_index=True)
    await print_index_structure("Index structure after adding one more fact:")


if __name__ == "__main__":
    asyncio.run(main())

What Just Happened

Step 1: Ingest the Initial Facts

await cognee.remember(
    INITIAL_FACTS,
    dataset_name=DATASET,
    self_improvement=False,
)

Six short facts describe two people, the trips they take, and how they each feel about their favorite way to spend free time. Each string in the list becomes its own document, and cognify extracts the entities each one mentions. This is the step that uses the LLM.

Step 2: Build the Index

await cognee.improve(dataset=DATASET, build_global_context_index=True)
first = await print_index_structure("Index structure after the initial build:")

improve() builds the global context index as its last stage, without an LLM call. It writes one bucket per document that has entities, naming that document’s most distinctive entities (ranked by tf-idf over the dataset’s chunks) and the relations among them, plus one root that lists the dataset’s entity types, its most widespread names, and its documents — see Global Context Index for the exact text layout. print_index_structure() reads the graph directly through get_graph_engine().get_graph_data() — see the “Inspect extracted graph schema” accordion (its “Python SDK” subsection) on the Cognify page for exactly what it returns — a (node_id, properties) pair per node. It keeps only GlobalContextSummary nodes, splits them into the single root (is_root) and the buckets, prints each text, and returns the texts so two builds can be compared. It doesn’t filter by dataset; to read one dataset’s root, see Scope to a dataset.

Step 3: Run improve() Again Without Changes

await cognee.improve(dataset=DATASET, build_global_context_index=True)
second = await print_index_structure("Index structure after a second improve():")
print(f"\nSecond run produced identical texts: {first == second}")

Only reads have touched the dataset since the first build, so the global_context_index stage reports already_completed and rewrites nothing. The comparison prints True, but the build is deterministic, so the texts would match even if it had run again; the skip itself shows on the result improve() returns, where the global_context_index stage has status already_completed.

Step 4: Add a Fact and Rebuild

await cognee.remember(
    ADDITIONAL_FACT,
    dataset_name=DATASET,
    self_improvement=False,
)
await cognee.improve(dataset=DATASET, build_global_context_index=True)
await print_index_structure("Index structure after adding one more fact:")

The new fact is a new document, so it gets a new bucket, and the root’s Documents: line now names it. The build recomputes the whole index but writes only the buckets and root whose text changed; a bucket whose text is the same as before is left untouched. Every bucket keeps the same id across builds, because the id is derived from the dataset and the document.

Under the Hood

How the Index Is Actually Built and Updated

Bucket text: the document’s name, then one Type: name, name line per entity type (types ordered by their best entity), then relation lines between the bucket’s own entities, ranked so a relation scores high only when both ends do. A bucket names at most 20 entities and 40 relations.
Root text: Topics: (entity types by how many entities carry them), Entities: (names by how many documents mention them), and Documents: (document names by chunk count). Each line is capped at 20 types or 40 names; past 40 documents the line ends with (+N more).
Ids: a bucket’s id derives from the dataset id and its document id, and the root’s id from the dataset id alone, so neither changes between builds.
Updates and deletions: every build computes the full desired index, writes the nodes whose text changed, and deletes the dataset’s index nodes it did not compute. Deleting a document removes its bucket and the dataset’s root right away. With the default method, the root is then rebuilt at once from the remaining documents (one embedding, no LLM call), so broad queries keep routing to the dataset. If that rebuild fails, a warning is logged and the root returns on the next improve().
Skipping: the stage runs only when something other than a read (search, recall, or an improve() that ran no stage) has run on the dataset since its last build. With ENABLE_BACKEND_ACCESS_CONTROL=false, datasets share one graph, so a write to any dataset triggers a rebuild.

An advanced version of this guide is available as a smoke-check script on our GitHub. It ingests a multi-day scheduling conversation in which meetings are booked, moved, and cancelled, builds the index with improve(build_global_context_index=True), then runs the same query with include_global_context_index off and on to verify the index’s context prelude actually reaches recall() — and finishes by asking questions whose answers depend on the latest state of the whole thread.

Global Context Index

The full concept, configuration options, and when to use it

Improve

Understand improve()‘s other stages

Reading the Global Context Index

See how include_global_context_index changes retrieval, once the index above exists

Getting Started

Core Concepts

Setup Configuration

Guides

Cookbooks

Examples

CLI

OSS

Before You Start

Code in Action

What Just Happened

Step 1: Ingest the Initial Facts

Step 2: Build the Index

Step 3: Run improve() Again Without Changes

Step 4: Add a Fact and Rebuild

Under the Hood

Global Context Index

Improve

Reading the Global Context Index

​Before You Start

​Code in Action

​What Just Happened

​Step 1: Ingest the Initial Facts

​Step 2: Build the Index

​Step 3: Run improve() Again Without Changes

​Step 4: Add a Fact and Rebuild

​Under the Hood

Global Context Index

Improve

Reading the Global Context Index

Before You Start

Code in Action

What Just Happened

Step 1: Ingest the Initial Facts

Step 2: Build the Index

Step 3: Run improve() Again Without Changes

Step 4: Add a Fact and Rebuild

Under the Hood