agrag.loaders.common

Shared helpers for the corpus readers.

Functions:

• build_prose_document – Build a prose-family Document from final text.
• build_record_document – Build a record-family Document from one row.
• paragraph_units – Split text into paragraph units that point at their place in the document.
• read_within_limit – Read a source's bytes without exceeding max_document_bytes.
• record_source_hash – Hash the whole raw source bytes for a record-family document.
• resolve_text_column – Pick the column that holds document text.
• source_title – Derive a document title from the source uri.
• text_sections – Return one section with an empty heading that holds the text as paragraphs.

Attributes:

• EXTENSION_FORMAT (dict[str, SourceFormat]) –