Skip to main content
Version: latest

agrag.common.data_models.document.Document

Bases: DataPoint

One unit of source text, before chunking.

A prose source, such as a Markdown file, makes one Document. A record source, such as a CSV file, makes one Document per row.

content_hash is always the sha256 of text, whatever the loader. The chunk offsets index text, so the hash names the text they point into. source_hash is the sha256 of the raw source bytes, and it is kept apart so that a change of converter output is still visible as a content change.

id is always the persisted Document node id for document_key. The system computes it, and an explicit id must equal that value.

Attributes:

  • text (str) – The document text. For a docling source, this holds the docling Markdown export. The chunker does not read it for a docling source: the chunks come from sections. The loaders leave it empty when store_text is off.
  • title (str) – The document title.
  • published_at (date | None) – The date the source says it was published, or None.
  • content_date (date | None) – The date the content describes, or None.
  • uri (str) – The location of the source. It is not part of the node id, which follows document_key.
  • source_format (SourceFormat) – The format the loader used to read this document.
  • family (DocumentFamily) – The shape of the source: one document per file, or one document per record.
  • content_hash (str) – The sha256 hex digest of text as UTF-8 bytes.
  • loader_name (str) – The name of the loader that produced this document, for example "text" or "docling".
  • loader_version (str | None) – The version of the loader package. Does not affect the document id.
  • encoding (str | None) – The text encoding. Text loaders set this field. Other loaders leave it empty.
  • source_hash (str | None) – The sha256 hex digest of the raw bytes of the source file. Every document read from a file sets this field. Inline text leaves it None.
  • char_count (int) – The number of characters in text.
  • line_count (int | None) – The number of lines in text. Some loaders do not set this field.
  • record_index (int | None) – The 0-based row number in the source. Record-family documents set this field.
  • record_id (str | None) – The value from the configured id column. Record-family documents set this field only when the caller configures an id column.
  • raw_record (dict[str, JsonValue] | None) – The original record data. A loader sets this field only when the caller asks for it.
  • sections (list[DocumentSection]) – The headings of the document and the content under them, in reading order. A non-blank document with no headings has one section with an empty heading. A record row has none: its text is one unit of content. A document read without its text (store_text off) has none either.
  • document_key (str | None) – The stable identifier for this document's persisted graph node. Independent of id, which changes with every content edit. Defaults to uri when not supplied.
  • normalization (Normalization | None) – How the loader normalized text. None for a document that no text loader made, such as a docling document or one built by hand.
  • schema_name (str | None) – The name of the schema of the run that wrote the document. None for a document that no run stamped.
  • schema_version (str | None) – The version string of that schema.
  • schema_fingerprint (str | None) – The structure hash of that schema (see agrag.common.schema_fingerprint).
  • extraction_fingerprint (str | None) – The hash of the descriptions, the extraction settings and the model name of that run.

Functions:

  • node_id_for – Compute the persisted Document graph node id.
  • to_node_record – Return this document as a GraphStore write record for its graph node.

char_count​

char_count: int

content_date​

content_date: date | None = None

content_hash​

content_hash: str

created_at​

created_at: datetime = Field(default_factory=lambda: datetime.now(UTC))

document_key​

document_key: str | None = None

encoding​

encoding: str | None = None

extraction_fingerprint​

extraction_fingerprint: str | None = None

family​

family: DocumentFamily

id​

id: UUID | None = None

line_count​

line_count: int | None = None

loader_name​

loader_name: str

loader_version​

loader_version: str | None = None

metadata​

metadata: dict[str, Any] = Field(default_factory=dict)

node_id_for​

node_id_for(*, document_key:str) -> UUID

Compute the persisted Document graph node id.

The id is keyed on document_key, not the content hash, so it stays the same across content changes to the same logical document. Every content version of a document shares one graph node with a changing content hash.

Parameters:

  • document_key (str) – The document's stable key.

Returns:

  • UUID – The Document graph node id.

normalization​

normalization: Normalization | None = None

published_at​

published_at: date | None = None

raw_record​

raw_record: dict[str, JsonValue] | None = None

record_id​

record_id: str | None = None

record_index​

record_index: int | None = None

resolved_document_key​

resolved_document_key: str

Return the document key. It is never None after construction succeeds.

document_key is typed as optional because callers can omit it and let _resolve_document_key default it to uri. Every constructed Document has a non-None document key by the time callers see it. Use this property instead of document_key where a non-optional value is required, such as computing the persisted Document node id.

Raises:

  • RuntimeError – document_key is still None, which means a validator was bypassed, for example via model_construct.

schema_fingerprint​

schema_fingerprint: str | None = None

schema_name​

schema_name: str | None = None

schema_version​

schema_version: str | None = None

sections​

sections: list[DocumentSection] = Field(default_factory=list)

source_format​

source_format: SourceFormat

source_hash​

source_hash: str | None = None

text​

text: str

title​

title: str

to_node_record​

to_node_record() -> NodeRecord

Return this document as a GraphStore write record for its graph node.

The record excludes text: the persisted node exists for traversal and the update no-op check, not to duplicate the document body already held per-chunk.

uri​

uri: str