Skip to main content
Version: latest

Consolidate duplicates

Run resolution over every entity in the graph. Review the matches it finds. Then apply them.

When to use this guide​

Graph.add already compares new mentions with the entities that the graph stores. Run Graph.consolidate in two cases. You added data before you configured an LLM, so some pairs stayed unmatched. Or you want to review the whole graph again.

Before you start​

  • A running Neo4j and the NEO4J_* variables. The Quickstart shows how.
  • The extras for local extraction:
uv pip install "agentic-graphrag[neo4j,extract]"
  • Optional: an LLM for the last tier. Install the llm extra, and set LLM_BASE_URL, LLM_API_KEY, and LLM_MODEL_ID. Without an LLM, that tier cannot decide any pair. Those pairs stay separate, the log shows one warning, and a later call can review them after you configure an LLM.

The resolution LLM tier runs whenever LLM settings exist. It does not depend on EXTRACTION_MODE. The mode only controls extraction, so EXTRACTION_MODE=gliner still sends ambiguous pairs to the LLM during resolution. Pass max_llm_pairs=0 to Graph(...) to send no pairs to the LLM. Ambiguous pairs that would go to the LLM then stay undecided and count as cap_truncated_pairs. Automatic exact, fuzzy, embedding, and precluster merges still run at zero.

1. Add documents that name one person in three ways​

import asyncio

from agrag.common.data_models import DEFAULT_SCHEMA
from agrag.embedding import build_embedder
from agrag.graphdb import build_graph_store
from agrag.ingestion import GlinerExtractor, Graph

DOCUMENTS = [
    "Charles Babbage designed the Analytical Engine in London.",
    "Babbage Charles taught mathematics at Cambridge.",
    "C. Babbage proposed the Difference Engine.",
]


async def main() -> None:
    graph_store = build_graph_store("neo4j")
    graph = await Graph.open(
        schema=DEFAULT_SCHEMA,
        graph_store=graph_store,
        embedder=build_embedder("ibm-granite/granite-embedding-small-english-r2"),
        extractor=GlinerExtractor(),
    )
    try:
        for text in DOCUMENTS:
            result = await graph.add(text=text)
            print(result.resolution)
    finally:
        await graph_store.close()


asyncio.run(main())

Each add prints a resolution summary. ambiguous_count counts uncertain, missing, and malformed LLM answers. failed_llm_requests, cap_truncated_pairs, and llm_unavailable_pairs count requests that raised, ambiguous pairs over the per-label cap, and pairs skipped because no LLM settings were found. These pairs stay undecided. Without an LLM, "C. Babbage" stays a separate entity.

2. Preview and apply the matches​

Graph.consolidate() is a dry run. It writes nothing. Graph.consolidate(apply=True) writes the matches. The LLM reviews the most similar pairs first for each label. Entity indices break ties, so the order is repeatable. Pairs over the cap stay undecided. This script previews, applies, and then reads the groups back.

import asyncio

from agrag.common.data_models import DEFAULT_SCHEMA
from agrag.embedding import build_embedder
from agrag.graphdb import build_graph_store
from agrag.ingestion import GlinerExtractor, Graph


async def main() -> None:
    graph_store = build_graph_store("neo4j")
    graph = await Graph.open(
        schema=DEFAULT_SCHEMA,
        graph_store=graph_store,
        embedder=build_embedder("ibm-granite/granite-embedding-small-english-r2"),
        extractor=GlinerExtractor(),
    )
    try:
        report = await graph.consolidate()
        ids = [
            str(entity_id)
            for match in report.would_match
            for entity_id in (match.entity_a_id, match.entity_b_id)
        ]
        rows = await graph_store.execute_read(
            "MATCH (e:_Node) WHERE e.id IN $ids RETURN e.id AS id, e.name AS name",
            {"ids": ids},
        )
        names = {row["id"]: row["name"] for row in rows}
        for match in report.would_match:
            first = names[str(match.entity_a_id)]
            second = names[str(match.entity_b_id)]
            print(f"{first} = {second} ({match.comparator}, score {match.score})")
        print("uncertain pairs:", report.ambiguous_count)

        applied = await graph.consolidate(apply=True)
        print("applied:", applied.applied, "failures:", len(applied.failures))
        groups = await graph_store.execute_read(
            "MATCH (r:ResolvedEntity)<-[:RESOLVED_AS]-(e) "
            "RETURN r.name AS resolved, collect(e.name) AS members"
        )
        for row in groups:
            print(row["resolved"], "<-", sorted(row["members"]))
    finally:
        await graph_store.close()


asyncio.run(main())

Without an LLM, the output shows the one match that the fuzzy tier found:

Charles Babbage = Babbage Charles (fuzzy_fast_path, score 1.0)
uncertain pairs: 0
applied: True failures: 0
Charles Babbage <- ['Babbage Charles', 'Charles Babbage']

With an LLM configured, the last tier also matches "C. Babbage", and the three names form one group:

Babbage Charles = Charles Babbage (fuzzy_fast_path, score 1.0)
Charles Babbage = C. Babbage (llm, score None)
Babbage Charles = C. Babbage (llm, score None)
uncertain pairs: 0
applied: True failures: 0
Babbage Charles <- ['Babbage Charles', 'C. Babbage', 'Charles Babbage']

The order of the names, and the name of the group, can differ in your run.

3. Read the report​

  • would_match lists every confirmed match. Each item has the two entity ids, the tier, a score when the tier has one, and the reason when an LLM decided. The tier is fuzzy_fast_path, embedding, or llm.
  • ambiguous_count is the number of LLM answers that were uncertain, missing, or malformed. Those pairs never merge.
  • failed_llm_requests is the number of LLM requests that raised an error. The pairs of those requests stay undecided.
  • cap_truncated_pairs is the number of ambiguous pairs over the per-label cap. The cap keeps the most similar pairs. Raise max_llm_pairs in Graph(...) to review more.
  • llm_unavailable_pairs is the number of ambiguous pairs that were skipped because no LLM settings were found. Set EXTRACTION_LLM_CLIENTS, or LLM_BASE_URL and LLM_MODEL_ID.
  • applied is True when the matches were written. failures lists any match that Agentic GraphRAG did not write, and any entity whose candidates it did not read. Consolidate does not compare that entity in this call. Run it again after you fix the cause.

If any of the three failure counts is not zero, the pass is partial. Run consolidate again after you fix the cause. The report lists matches that the graph already stores, so applying the same matches again is safe.