Folder Presort

A minimal guide to pre-organizing a folder before it reaches your graph. Presort scans a directory without touching the files on disk, reports what is junk, duplicated, versioned, personal, or already in cognee, and then ingests the groups you approve — one dataset per group.

Before You Start

Complete Quickstart to understand basic operations
Ensure you have LLM Providers configured — the analyze phase is deterministic, but the apply phase runs cognify
Read Datasets — presort ingests each proposed group into its own dataset
Set PRESORT_FOLDER to the folder you want to scan, or let the script use ~/Downloads
Name that folder as a permitted scan root before running the script — the defaults are the working directory, the temp directory, and cognee’s own storage, so a folder under $HOME is always outside them: export COGNEE_ALLOWED_LOCAL_FILE_ROOTS="$HOME/Downloads". See Presort scan roots

Code in Action

import asyncio
import os
from pathlib import Path

import cognee

FOLDER = os.environ.get("PRESORT_FOLDER", str(Path.home() / "Downloads"))


async def main():
    # Phase 1: analyze. Returns a PresortReport (also auto-saved under
    # cognee's system directory as <scan-id>.presort.json).
    report = await cognee.remember(FOLDER, dry_run="presort")

    summary = report.summary()
    print(f"Scanned {summary['files']} files ({summary['junk']} junk skipped)")
    print(f"Already in cognee: {summary['cognee_status']}")
    print(
        f"Duplicate clusters: {summary['duplicate_clusters']} ({summary['wasted_bytes']} wasted bytes)"
    )
    print(f"Potential personal data: {summary['pii_findings']} findings")
    for group in report.groups:
        print(
            f"  group {group.name!r} -> dataset {group.dataset_name!r} ({len(group.file_paths)} files)"
        )

    # Review/adjust the apply decisions on the report itself.
    report.exclude_pii = True  # keep files with personal data out of the graph
    report.skip_duplicates = True  # ingest one copy per duplicate cluster
    report.apply_groups = [group.name for group in report.groups if group.kind != "code_project"]

    # Phase 2: apply. One dataset per proposed group; re-running is idempotent
    # (already-cognified content is skipped by incremental loading).
    results = await cognee.remember(report)
    for dataset_name, result in results.items():
        print(f"ingested dataset {dataset_name!r}: {result}")

    # The presorted data is now queryable per dataset.
    answers = await cognee.recall("What documents do I have?", datasets=list(results))
    for answer in answers:
        print(answer)


if __name__ == "__main__":
    asyncio.run(main())

The complete runnable script is on GitHub: examples/guides/presort_downloads.py.

What Just Happened

Step 1: Scan the Folder

FOLDER = os.environ.get("PRESORT_FOLDER", str(Path.home() / "Downloads"))

# Phase 1: analyze. Returns a PresortReport (also auto-saved under
# cognee's system directory as <scan-id>.presort.json).
report = await cognee.remember(FOLDER, dry_run="presort")

dry_run="presort" turns remember() into an analyzer: it walks the folder, reads samples of the files, and returns a PresortReport instead of ingesting anything. Nothing on disk is moved, renamed, or deleted. The report is also saved to <SYSTEM_ROOT_DIRECTORY>/presort/<scan_id>.presort.json, so the analyze result survives a failed apply — report.report_path holds the exact location, or None when SYSTEM_ROOT_DIRECTORY is unset or on S3 and the report was not persisted.

Step 2: Review the Report

summary = report.summary()
print(f"Scanned {summary['files']} files ({summary['junk']} junk skipped)")
print(f"Already in cognee: {summary['cognee_status']}")
print(
    f"Duplicate clusters: {summary['duplicate_clusters']} ({summary['wasted_bytes']} wasted bytes)"
)
print(f"Potential personal data: {summary['pii_findings']} findings")
for group in report.groups:
    print(
        f"  group {group.name!r} -> dataset {group.dataset_name!r} ({len(group.file_paths)} files)"
    )

report.summary() is the at-a-glance view: file and junk counts, exact-duplicate clusters with the bytes they waste, version candidates, potential personal data, and a cognee_status breakdown of how many files are new, staged, or already cognified. report.groups holds the proposal itself — each group carries the dataset_name it would land in and the files it would take there.

Step 3: Set the Apply Decisions

report.exclude_pii = True  # keep files with personal data out of the graph
report.skip_duplicates = True  # ingest one copy per duplicate cluster
report.apply_groups = [group.name for group in report.groups if group.kind != "code_project"]

The report is editable, and the apply phase reads these three fields off it. exclude_pii defaults to False, so setting it is what actually changes the run — it drops files with potential personal data. skip_duplicates is already on by default; it is spelled out to keep the decision visible. apply_groups narrows the run to the groups you name — here every group except code projects, which belong in a code graph rather than a document dataset.

Step 4: Apply the Report and Recall

results = await cognee.remember(report)
for dataset_name, result in results.items():
    print(f"ingested dataset {dataset_name!r}: {result}")

answers = await cognee.recall("What documents do I have?", datasets=list(results))
for answer in answers:
    print(answer)

Passing the report back to remember() runs the second phase: each approved group goes through the normal add → cognify chain into its own dataset, and you get back a {dataset_name: result} mapping. Ingestion is incremental, so re-running the script after adding a few files only processes what is new. From there the folder is ordinary cognee memory — recall() takes the dataset names as its scope.

Advanced Usage

Deterministic scan vs. LLM analysis

The analyze phase needs no LLM or embedding configuration: junk filtering, duplicate detection, version candidates, and folder-based grouping are all deterministic. Pass use_llm=True to remember(folder, dry_run="presort", use_llm=True) for LLM content classification, deeper PII detection, and semantic grouping.The apply phase runs cognify and does need a configured LLM. Without one, presort degrades rather than fails: the deterministic scan still runs, use_llm is downgraded, and apply stages files with add() only — each reported as a warning on the report.

Skipping the review step

When you trust the defaults, remember(FOLDER, dry_run="presort", auto_apply=True) collapses both phases into one call instead of the two this guide makes — see auto_apply for what comes back.

Running presort from the CLI

cognee-cli remember <folder> --presort runs the same two phases from the shell, with --allow-root in place of the environment variable. The commands and their apply-time flags are in the CLI reference.

Datasets

How the datasets presort proposes organize documents, permissions, and processing.

remember()

Every option the two presort phases accept, including the apply overrides.

Getting Started

Core Concepts

Setup Configuration

Guides

Examples

CLI

OSS

Before You Start

Code in Action

What Just Happened

Step 1: Scan the Folder

Step 2: Review the Report

Step 3: Set the Apply Decisions

Step 4: Apply the Report and Recall

Advanced Usage

Datasets

remember()

​Before You Start

​Code in Action

​What Just Happened

​Step 1: Scan the Folder

​Step 2: Review the Report

​Step 3: Set the Apply Decisions

​Step 4: Apply the Report and Recall

​Advanced Usage

Datasets

remember()

Before You Start

Code in Action

What Just Happened

Step 1: Scan the Folder

Step 2: Review the Report

Step 3: Set the Apply Decisions

Step 4: Apply the Report and Recall

Advanced Usage