Skip to main content
Version: latest

agrag.loaders.common.build_prose_document

build_prose_document(*, source:SourceRef, text:str, encoding:str, source_format:SourceFormat, loader_name:str, opts:ReadOptions, title:str, sections:list[DocumentSection] | None = None, source_hash:str | None = None, loader_version:str | None = None) -> Document

Build a prose-family Document from final text.

The content hash is the sha256 of text, so callers must pass the final text (the extracted main content for HTML, the raw decoded text otherwise).

Parameters:

  • source (SourceRef) – The source the text came from.
  • text (str) – The final document text.
  • encoding (str) – The encoding used to decode the source.
  • source_format (SourceFormat) – The format the loader used.
  • loader_name (str) – The name to record for the loader.
  • opts (ReadOptions) – The read options, used for the store_text flag.
  • title (str) – The document title.
  • sections (list[DocumentSection] | None) – The sections of the document. Dropped when the read options do not store text, because their units point into the text.
  • source_hash (str | None) – The sha256 of the raw source bytes. File loaders pass it; inline text has no source file and leaves it None.
  • loader_version (str | None) – The version of the loader package, when known.

Returns: