agrag.loaders.base.Loader

Bases: ABC

Reads one source and yields Document objects.

A Loader keeps no state between calls, so a worker process can reuse one instance across many sources.

Attributes:

• extensions (frozenset[str]) – The file extensions this loader claims, each with a leading dot.
• mime_types (frozenset[str]) – The MIME types this loader claims. Empty when the loader relies on the extension alone.
• family (DocumentFamily) – The document family this loader produces.
• extra (str | None) – The package extra this loader needs. None for core loaders. The registry raises MissingExtraError for a loader whose extra is not installed.
• extra_module (str | None) – The importable module that proves the extra is installed. None means the module has the same name as extra.

Functions:

• is_available – Return whether the package that the loader's extra installs is present.
• load – Yield documents read from one source.

extensions

extensions: frozenset[str]

extra

extra: str | None = None

extra_module

extra_module: str | None = None

family

family: DocumentFamily

is_available

is_available() -> bool

Return whether the package that the loader's extra installs is present.

A loader with no extra is always available. The check looks the module up without importing it, so an installed package that fails to import still counts.

load

load(source:SourceRef, stream:BinaryIO, opts:ReadOptions, *, start_at:int = 0) -> Iterator[Document]

Yield documents read from one source.

Parameters:

• source (SourceRef) – The source to read.
• stream (BinaryIO) – The open binary stream for the source, positioned at the start.
• opts (ReadOptions) – The read options for this call.
• start_at (int) – The record index to resume from. Prose loaders ignore this argument.

Yields:

• Document – One Document per unit the source contains, in a fixed order.

mime_types

mime_types: frozenset[str] = frozenset()