Skip to main content
Version: 0.0.5 (stable)

agrag.loaders.common.build_prose_document

build_prose_document(*, source:SourceRef, text:str, encoding:str, source_format:SourceFormat, loader_name:str, opts:ReadOptions, title:str, sections:list[DocumentSection] | None = None, content_hash:str | None = None, loader_version:str | None = None) -> Document

Build a prose-family Document from final text.

This function hashes text to form the content hash, so callers must pass the final text (the extracted main content for HTML, the raw decoded text otherwise).

Parameters:

  • source (SourceRef) – The source the text came from.
  • text (str) – The final document text.
  • encoding (str) – The encoding used to decode the source.
  • source_format (SourceFormat) – The format the loader used.
  • loader_name (str) – The name to record for the loader.
  • opts (ReadOptions) – The read options, used for the store_text flag.
  • title (str) – The document title.
  • sections (list[DocumentSection] | None) – The sections of the document. Dropped when the read options do not store text, because their units point into the text.
  • content_hash (str | None) – The content hash to record. Defaults to the hash of text. A loader whose parsed output is unstable across versions passes the hash of its raw source bytes instead.
  • loader_version (str | None) – The version of the loader package, when known.

Returns: