> ## Documentation Index
> Fetch the complete documentation index at: https://docs.cognee.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Rust SDK Loaders & Chunkers

> Loaders and chunkers in cognee-rust: how files are read at add and split into token-bounded chunks at cognify.

## What are loaders and chunkers

Loaders read files of each format into text when you add them. Chunkers then split that text into pieces small enough to embed and extract from.

## How loaders & chunkers work in Rust

**Loaders** handle file-format reading at ingest: a loader registry dispatches by
MIME type / extension to per-format loaders (text, PDF, CSV, HTML, image, audio,
and the `unstructured` office formats), most behind feature flags. They
run at `add`, which stores the extracted text (an image's vision-LLM
description, an audio file's transcript) and keeps the source file in
`original_data_location`. **Chunkers**
then segment a document into token-bounded pieces through a word → sentence →
paragraph hierarchy, sizing chunks with a pluggable `TokenCounter`
(`WordCounter`, or the feature-gated HuggingFace / tiktoken counters).

Rust: loaders (`LoaderRegistry`, `DocumentLoader`) in
[`cognee-ingestion`](https://github.com/topoteretes/cognee-rs/blob/main/crates/ingestion/); chunking (`text_chunker`, the
`Chunker` trait that [`chunk_strategy`](https://docs.cognee.ai/rust/configuration/cognify#chunking--tokenizer) selects from,
`TokenCounter`) in [`cognee-chunking`](https://github.com/topoteretes/cognee-rs/blob/main/crates/chunking/). Token-counter
selection is configured in [configuration](https://docs.cognee.ai/rust/configuration/cognify.md#chunking--tokenizer).

## Related

> [card] [Add](https://docs.cognee.ai/rust/core-concepts/main-operations/legacy-operations/add.md#loaders)
>
> How add picks a loader

> [card] [Cognify configuration](https://docs.cognee.ai/rust/configuration/cognify.md#chunking--tokenizer)
>
> Chunking and tokenizer settings

> [card] [Cognify](https://docs.cognee.ai/rust/core-concepts/main-operations/legacy-operations/cognify.md)
>
> Where chunking runs


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.