Loaders

Documentation Index

Fetch the complete documentation index at: https://docs.cognee.ai/llms.txt

Use this file to discover all available pages before exploring further.

​

Loader Selection

1. TextLoader: For plain text files (.txt, .md, .json, .xml, etc.).
2. PyPdfLoader: For PDF files (requires pypdf).
3. ImageLoader: For images (uses vision models to describe content).
4. AudioLoader: For audio files (uses transcription models).
5. CsvLoader: For CSV files (converts rows to text).
6. UnstructuredLoader: For complex formats like .docx, .pptx, .epub (requires unstructured).
7. AdvancedPdfLoader: For layout-aware PDF extraction (requires unstructured).
8. DoclingLoader: If no other loader can ingest the file type, Docling is used for conversion (if the type is supported).

​

Available Loaders

​

Core Loaders

• TextLoader: Reads text files with UTF-8 encoding.
• CsvLoader: Reads CSV files and converts each row into a structured text format (Row N: key: value).
• ImageLoader: Uses an LLM vision model to transcribe or describe the image content.
• AudioLoader: Uses an audio transcription API to transcribe audio files. This depends on your configured LLM provider supporting transcription endpoints; see LLM Providers for provider-specific caveats.

​

External Loaders

• PyPdfLoader: Extracts text from PDFs page by page using the pypdf library, preserving page boundaries with Page N: markers in the extracted text. Requires pip install cognee[docs] (or pip install pypdf).
• AdvancedPdfLoader: Layout-aware PDF extraction using the unstructured library. Extracts text, tables (as HTML), and image placeholders per page. Falls back to PyPdfLoader automatically if extraction fails. Requires pip install cognee[docs], plus poppler and tesseract installed on the system.
• UnstructuredLoader: Handles many office and document formats (.docx, .xlsx, .pptx, .odt, .rtf, .eml, .epub, .html, and more) via unstructured’s auto-partition. Requires pip install cognee[docs].
• BeautifulSoupLoader: Extracts text from HTML files using BeautifulSoup. Applies CSS selector rules to pull structured content from specific tags. Requires pip install cognee[scraping].
• DoclingLoader: Catch-all fallback that converts a wide range of document formats (PDF, DOCX, XLSX, PPTX, HTML, Markdown, and more) to plain text via Docling. Supported extensions are discovered dynamically from Docling’s FormatToExtensions map. Requires pip install cognee[docling].

​

Supported File Extensions

​

Usage

import cognee

await cognee.remember(
    data=["example_website.html"],
    preferred_loaders=[
        {
            "beautiful_soup_loader": {
                "extraction_rules": {
                    "title": "h1",
                    "body": "article.main-content"
                }
            }
        }
    ]
)

# Japanese
sudo apt-get install tesseract-ocr-jpn tesseract-ocr-jpn-vert

# Chinese (Simplified / Traditional)
sudo apt-get install tesseract-ocr-chi-sim tesseract-ocr-chi-tra

# Korean
sudo apt-get install tesseract-ocr-kor

import cognee

# Japanese PDF
await cognee.remember(
    data=["japanese_document.pdf"],
    preferred_loaders=[
        {
            "advanced_pdf_loader": {
                "strategy": "ocr_only",
                "languages": ["jpn"]
            }
        }
    ]
)

import cognee

await cognee.remember(
    data=["presentation.pptx"],
    preferred_loaders=[
        {
            "unstructured_loader": {
                "strategy": "fast"
            }
        }
    ]
)

import cognee

await cognee.remember(
    data=["report.pdf"],
    preferred_loaders=[{"docling_loader": {}}]
)

import cognee

await cognee.remember(
    data=["page.html"],
    preferred_loaders=[
        {
            "beautiful_soup_loader": {
                "extraction_rules": {
                    "title": {"selector": "h1", "all": False},
                    "body": {"selector": "article.main-content", "all": True, "join_with": "\n\n"},
                    "og_image": {"selector": "meta[property='og:image']", "attr": "content"}
                }
            }
        }
    ]
)

await cognee.remember(
    data=["page.html"],
    preferred_loaders=[
        {
            "beautiful_soup_loader": {
                "extraction_rules": {
                    "content": {"xpath": "//div[@class='content']//p"}
                }
            }
        }
    ]
)

# .env — enable vision by choosing a vision-capable model
LLM_PROVIDER="openai"
LLM_MODEL="gpt-4o-mini"
LLM_API_KEY="sk-..."

from cognee.infrastructure.loaders import use_loader
from cognee.infrastructure.loaders.LoaderInterface import LoaderInterface

class MyCustomLoader(LoaderInterface):
    loader_name = "my_custom_loader"
    supported_extensions = [".custom"]
    supported_mime_types = ["application/x-custom"]

    async def load(self, file_path, **kwargs):
        # Your custom logic to read the file and return text
        return "Extracted text content"

# Register the loader so Cognee can use it
use_loader("my_custom_loader", MyCustomLoader)

await cognee.remember("data/file.custom")