Skip to main content
Version: latest

agrag.common.data_models.chunk.Chunk

Bases: DataPoint

One retrieval-sized piece of a Document.

Attributes:

  • document_id (UUID) – The id of the persisted Document graph node this chunk belongs to (see Document.node_id_for). It stays stable across content versions of the same logical document. Per-version identity lives in Chunk.id instead.
  • content_hash (str) – The content hash of the document version this chunk was cut from, as a 64-character lowercase hexadecimal SHA-256 digest. It stays on the chunk after the document changes, so a superseded chunk still names the text it came from.
  • index (int) – The position of the chunk within its document, from 0.
  • text (str) – The chunk text.
  • provenance (TextProvenance | PageProvenance) – The location of this chunk in its source. A text source gives character offsets. A source with page layout gives pages and boxes, and a source with neither gives no page spans.
  • heading_path (list[str]) – The headings that contain this chunk, from outermost to innermost. Empty for a chunk with no heading above it.
  • content_kind (Literal['text', 'table']) – "text" for a chunk of running text and "table" for a chunk made from a table.
  • chunker (str | None) – The name of the chunker that made this chunk. None for a chunk written before chunkers were recorded.
  • chunker_hash (str | None) – The fingerprint of the settings of the chunker that made this chunk. None for a chunk written before chunkers were recorded.
  • schema_fingerprint (str | None) – The structure hash of the schema of the run that made this chunk. None for a chunk written before runs were recorded.
  • extraction_fingerprint (str | None) – The hash of the descriptions, extraction settings and model name of that run. None for the same chunks.
  • section_ids (list[UUID]) – The node ids of the sections whose text the chunk holds, in reading order. Empty for a document with no sections.

Functions:

  • from_node – Build a Chunk from the flat properties of a stored Chunk node.
  • section_label – Return the heading path as one line, or None when the path is empty.
  • to_node_record – Return this chunk as a GraphStore write record.

chunker​

chunker: str | None = None

chunker_hash​

chunker_hash: str | None = None

content_hash​

content_hash: str = Field(min_length=64, max_length=64, pattern='^[0-9a-f]{64}$')

content_kind​

content_kind: Literal['text', 'table'] = 'text'

contextual_text​

contextual_text: str

The text with its heading path above it, for embedding.

The stored text and its offsets do not change. A chunk with no heading path returns its text.

created_at​

created_at: datetime = Field(default_factory=lambda: datetime.now(UTC))

document_id​

document_id: UUID

embedding​

embedding: list[float] | None = None

extraction_fingerprint​

extraction_fingerprint: str | None = None

from_node​

from_node(properties:Mapping[str, Any]) -> Chunk

Build a Chunk from the flat properties of a stored Chunk node.

Parameters:

  • properties (Mapping[str, Any]) – The stored node properties, including id as the node id string. to_node_record().properties omits id, which lives on the record, so merge it in first. provenance may be the JSON text that to_node_record writes or the decoded mapping.

Returns:

  • Chunk – The validated Chunk.

Raises:

  • ValueError – A property is missing, is not JSON when it should be, or does not validate. Pydantic's ValidationError is a ValueError.

heading_path​

heading_path: list[str] = Field(default_factory=list)

id​

id: UUID | None = None

index​

index: int = 0

metadata​

metadata: dict[str, Any] = Field(default_factory=dict)

provenance​

provenance: TextProvenance | PageProvenance = Field(discriminator='kind')

schema_fingerprint​

schema_fingerprint: str | None = None

section_ids​

section_ids: list[UUID] = Field(default_factory=list)

section_label​

section_label() -> str | None

Return the heading path as one line, or None when the path is empty.

Whitespace runs in a heading become one space, and runs of three or more dashes become one dash, so a heading cannot end the text block of the extraction prompt.

text​

text: str

to_node_record​

to_node_record() -> NodeRecord

Return this chunk as a GraphStore write record.

Provenance is flattened to a plain JSON-safe dict via model_dump. GraphStore's own serialize.node_params only converts UUIDs and walks containers.

Raises:

  • ValueError – id is None.