agrag.chunking

Chunking: how a Document becomes Chunks.

Chunker packs the sections of a document into chunks. ChunkedDocument carries the chunks with where each chunk hangs. Graph takes one.

Modules:

• chunker – The section chunker: one packer that serves every source format.

Classes:

• ChunkPlacement – Where one chunk hangs in the structure of its document.
• ChunkedDocument – The chunks of one document with where each chunk hangs.
• Chunker – Packs the sections of a Document into chunks.
• ChunkingError – A chunker could not split a document without changing its text.

Attributes:

• DEFAULT_TOKENIZER –