agrag.loaders.records.JsonlLoader

Bases: RecordLoader

Reads JSON Lines files as one document per line.

A line that holds a JSON object becomes a record Document. A line that holds another JSON value becomes a prose Document of its JSON text. With json_mode=DOCUMENT the loader yields one prose Document for the whole source.

Attributes:

• extensions – The .jsonl and .ndjson extensions.

Functions:

• is_available – Return whether the package that the loader's extra installs is present.
• load – Yield one Document per non-blank line, or one for the whole source.

extensions

extensions = frozenset({'.jsonl', '.ndjson'})

extra

extra: str | None = None

extra_module

extra_module: str | None = None

family

family = DocumentFamily.RECORD

is_available

is_available() -> bool

Return whether the package that the loader's extra installs is present.

A loader with no extra is always available. The check looks the module up without importing it, so an installed package that fails to import still counts.

load

load(source:SourceRef, stream:BinaryIO, opts:ReadOptions, *, start_at:int = 0) -> Iterator[Document]

Yield one Document per non-blank line, or one for the whole source.

Parameters:

• source (SourceRef) – The source to read.
• stream (BinaryIO) – The open binary stream for the source.
• opts (ReadOptions) – The read options. json_mode=DOCUMENT reads the whole source as one prose Document.
• start_at (int) – The record index to resume from.

Yields:

• Document – One Document per non-blank line, in file order. A line that is a JSON
• Document – object gives a record Document. A line with any other JSON value gives
• Document – a prose Document.

Raises:

• MalformedRecordError – A line is not valid JSON.

mime_types

mime_types: frozenset[str] = frozenset()