Google Drive Ingestion

A minimal guide to ingesting a Google Drive folder into cognee. The Google Drive connector extracts Google Docs, Sheets and Slides, PDFs, text files (any text/* type, plus JSON and XML), and Office and document files (.docx, .xlsx, .pptx, .odt, .ods, .odp, .rtf, .html, .epub) from a folder — and, by default, its subfolders — then cognifies them like any other document. Re-running the sync only processes files that changed, and files removed from the folder are forgotten. Office files, .rtf and .epub need a document loader: install cognee[docling] (or cognee[docs], which brings the unstructured loader) alongside the connector extra. Drawings, Forms, images, audio, video, archives and legacy .doc/.xls/.ppt files are skipped, and a Google Sheet contributes its first sheet only, which is all Drive’s export returns. There is no file size limit by default — downloads are streamed to a temporary file — but you can set one with GOOGLE_DRIVE_MAX_FILE_SIZE_MB. File content is fetched on 8 threads in parallel, with rate-limit and server errors retried; GOOGLE_DRIVE_MAX_WORKERS changes the thread count. See What gets indexed for the full table — the hosted connector reads files through the same code path.

Before You Start

Complete Quickstart to understand basic operations
Ensure you have LLM Providers configured — the script calls cognee.recall(), which needs an LLM for the final completion
Read Datasets — the folder is synced into the google_drive_demo dataset
Install the connector extra: pip install "cognee[google-drive]"
Enable the Google Drive API and set up a service account or OAuth client — follow Set Up Google Access below
Set GOOGLE_DRIVE_FOLDER_ID to the folder’s Drive ID, taken from its URL

This guide drives the connector from your own script, with credentials you hold on disk. If you instead want users to connect their own Drive account to a running Cognee backend over a web OAuth consent, use the Google Drive integration — it is configured with a different set of GOOGLE_DRIVE_* variables (GOOGLE_DRIVE_CLIENT_ID, GOOGLE_DRIVE_REDIRECT_URI, and so on) and syncs over its own REST endpoints.

Code in Action

import asyncio

import cognee
from cognee.tasks.ingestion.connectors import google_drive_source


async def main():
    drive_source = google_drive_source()  # reads GOOGLE_DRIVE_* env vars

    print("=== Initial sync ===")
    result = await cognee.remember(
        drive_source,
        dataset_name="google_drive_demo",
        primary_key="id",
        # "merge" is required: it's what makes re-runs incremental and what
        # makes deletions propagate via orphan cleanup. The default
        # ("replace") would re-extract every file on every run instead.
        write_disposition="merge",
        # The DLT ingestion default caps a table at 50 rows; Drive folders
        # commonly exceed that, so lift the cap.
        max_rows_per_table=0,
    )
    print(result)

    answer = await cognee.recall("Summarize what's in the Drive folder.")
    print("Recall:", answer)

    print("\n=== Incremental re-sync (only changed/removed files are processed) ===")
    result = await cognee.remember(
        google_drive_source(),
        dataset_name="google_drive_demo",
        primary_key="id",
        write_disposition="merge",
        max_rows_per_table=0,
    )
    print(result)


if __name__ == "__main__":
    asyncio.run(main())

What Just Happened

Step 1: Create the Drive Source

drive_source = google_drive_source()  # reads GOOGLE_DRIVE_* env vars

google_drive_source() returns a dlt source for the folder. Called with no arguments, it takes the auth mode, credentials, token path, and folder ID from the GOOGLE_DRIVE_* environment variables set up in Before You Start.

Step 2: Run the Initial Sync

result = await cognee.remember(
    drive_source,
    dataset_name="google_drive_demo",
    primary_key="id",
    # "merge" is required: it's what makes re-runs incremental and what
    # makes deletions propagate via orphan cleanup. The default
    # ("replace") would re-extract every file on every run instead.
    write_disposition="merge",
    # 0 = no row cap. This is already the default; pass it to be explicit,
    # or to override a positive DLT_MAX_ROWS_PER_TABLE.
    max_rows_per_table=0,
)
print(result)

Passing the source to remember() extracts every file, chunks it, and builds the graph. primary_key="id" keys each row by its Drive file ID, and write_disposition="merge" upserts on that key — which is what makes later runs incremental and lets deletions propagate. max_rows_per_table=0 means no row cap, but it changes nothing here: Google Drive is a document source, and Cognee always reads a document source’s whole staging table, whatever max_rows_per_table or DLT_MAX_ROWS_PER_TABLE says. (The upstream script’s inline comment still describes an older 50-row default, which cognee dropped in favor of unlimited ingestion.)

Step 3: Recall From the Folder

answer = await cognee.recall("Summarize what's in the Drive folder.")
print("Recall:", answer)

Once synced, the folder’s contents are ordinary cognee memory, so recall() can answer questions about them.

Step 4: Re-Sync Incrementally

result = await cognee.remember(
    google_drive_source(),
    dataset_name="google_drive_demo",
    primary_key="id",
    write_disposition="merge",
    max_rows_per_table=0,
)
print(result)

The second call uses a fresh source with the same dataset, key, and write disposition. Only files that were added or changed since the previous run are processed, and files deleted or trashed in Drive are removed from memory. Run this on a schedule to keep the dataset in step with the folder.

Additional Information

Set Up Google Access

Do this once in the Google Cloud Console:

Create a project, or select an existing one.
Open the Google Drive API page and click Enable.

Then set up one of these auth modes, in your shell or in your .env. Use absolute paths in .env, which doesn’t expand ~.Service account (recommended, no browser step):

Open Service Accounts and click Create service account. Enter a name, skip the optional permission steps, and click Done. The account needs no roles.
Click the new account, open the Keys tab, click Add key → Create new key, choose JSON, and click Create. The key file downloads. If your organization blocks key creation, use OAuth instead.
Copy the account’s email (...@....iam.gserviceaccount.com). In Google Drive, share the folder with that address as Viewer.

Point the script at the key:

export GOOGLE_DRIVE_AUTH_MODE=service_account
export GOOGLE_DRIVE_CREDENTIALS_PATH=/path/to/service-account-key.json

OAuth user credentials (for a folder in your own My Drive):

Open Google Auth Platform and click Get started. Enter an app name and your email, choose External as the audience, and finish the wizard. Google Auth Platform isn’t in the left menu by default; you can also reach it through ☰ → APIs & Services → OAuth consent screen.
Open Audience. If it shows User type: External and Publishing status: Testing, click Add users under Test users and add the Google account whose Drive folder you want to sync. If the user type is Internal (a Google Workspace organization) or the app is In production, there is no Test users section and you can skip this step.
Under Data Access, click Add or remove scopes, filter for drive.readonly, tick .../auth/drive.readonly, click Update, then Save.
Under Clients, click Create client, choose Desktop app, and click Create. Click Download JSON in the dialog that follows. A Web application client, such as the one from the Google Drive integration, fails at sign-in with Error 400: redirect_uri_mismatch.

Point the script at the downloaded file:

export GOOGLE_DRIVE_AUTH_MODE=oauth
export GOOGLE_DRIVE_CREDENTIALS_PATH=/path/to/client_secret_XXXX.json
# caches the token; without it, every run opens the browser again
export GOOGLE_DRIVE_TOKEN_PATH=/path/to/drive-token.json

dlt (Data Load Tool)

The dlt ingestion path the connector builds on, including write dispositions.

remember()

Every option remember() accepts when ingesting a source.

Getting Started

Core Concepts

Setup Configuration

Guides

Cookbooks

Examples

CLI

OSS

Before You Start

Code in Action

What Just Happened

Step 1: Create the Drive Source

Step 2: Run the Initial Sync

Step 3: Recall From the Folder

Step 4: Re-Sync Incrementally

Additional Information

dlt (Data Load Tool)

remember()

​Before You Start

​Code in Action

​What Just Happened

​Step 1: Create the Drive Source

​Step 2: Run the Initial Sync

​Step 3: Recall From the Folder

​Step 4: Re-Sync Incrementally

​Additional Information

dlt (Data Load Tool)

remember()

Before You Start

Code in Action

What Just Happened

Step 1: Create the Drive Source

Step 2: Run the Initial Sync

Step 3: Recall From the Folder

Step 4: Re-Sync Incrementally

Additional Information