Entity Consolidation

Rewrite fragmented entity descriptions using LLM analysis of graph neighborhoods

A minimal guide to consolidating entity descriptions in an existing knowledge graph. After ingestion, entity descriptions can be fragmented or repetitive because each one is derived from a single chunk — this lower-level Memify pipeline rewrites each entity’s description using the LLM and the entity’s full neighborhood context. Use improve() for the standard self-improvement flow; use this guide when you specifically want entity-description consolidation.

This pipeline only rewrites description text — it never creates or deletes nodes. To merge near-duplicate Entity nodes into one, see Entity Deduplication.

Before You Start

• Complete Quickstart to understand basic operations
• Ensure you have LLM Providers configured
• Have an existing knowledge graph with Entity nodes — the script below builds one with remember()

Code in Action

import asyncio
import cognee

from os import path
from cognee.api.v1.visualize.visualize import visualize_graph
from cognee.memify_pipelines.consolidate_entity_descriptions import (
    consolidate_entity_descriptions_pipeline,
)

custom_prompt = """
Extract only people and cities as entities.
Connect people to cities with the relationship "lives_in".
Ignore all other entities.
"""

graph_visualization_path_before_enrichment = path.join(
    path.dirname(__file__), ".artifacts", "before_consolidate_enrichment_entity_descriptions.html"
)
graph_visualization_path_after_enrichment = path.join(
    path.dirname(__file__), ".artifacts", "after_consolidate_enrichment_entity_descriptions.html"
)


async def main():
    # Prune data and system metadata before running, only if we want "fresh" state.
    await cognee.forget(everything=True)
    await cognee.remember(
        [
            "Alice moved to Paris in 2010, while Bob has always lived in New York.",
            "Andreas was born in Venice, but later settled in Lisbon.",
            "Diana and Tom were born and raised in Helsinki. Diana currently resides in Berlin, while Tom never moved.",
        ],
        custom_prompt=custom_prompt,
        self_improvement=False,
    )

    await visualize_graph(graph_visualization_path_before_enrichment)

    await consolidate_entity_descriptions_pipeline()

    await visualize_graph(graph_visualization_path_after_enrichment)


if __name__ == "__main__":
    asyncio.run(main())

What Just Happened

Step 1: Clear Existing Data

await cognee.forget(everything=True)

Start from a clean state so the before-and-after visualizations only reflect this example run.

Step 2: Build and Visualize the Graph

custom_prompt = """
Extract only people and cities as entities.
Connect people to cities with the relationship "lives_in".
Ignore all other entities.
"""

await cognee.remember(
    [
        "Alice moved to Paris in 2010, while Bob has always lived in New York.",
        "Andreas was born in Venice, but later settled in Lisbon.",
        "Diana and Tom were born and raised in Helsinki. Diana currently resides in Berlin, while Tom never moved.",
    ],
    custom_prompt=custom_prompt,
    self_improvement=False,
)
await visualize_graph(graph_visualization_path_before_enrichment)

Create a focused graph with only people, cities, and lives_in relationships, then save a visualization of the graph before consolidation.

Step 3: Consolidate and Visualize Again

await consolidate_entity_descriptions_pipeline()
await visualize_graph(graph_visualization_path_after_enrichment)

The pipeline rewrites each existing Entity node’s description in place using LLM analysis of the entity’s neighbors and edges — no nodes are created or deleted. Descriptions become more coherent because the LLM sees each entity in the context of its graph neighborhood, not just the original chunk text, and the before and after HTML files make the change easy to inspect.

Additional Information

• Runnable guide script available on our GitHub
• Pipeline implementation: consolidate_entity_descriptions.py

Under the hood

Three tasks run in sequence:

1. get_entities_with_neighborhood — loads all Entity nodes and fetches their edges and neighbor nodes.
2. generate_consolidated_entities — sends each entity plus neighborhood to the LLM, which returns a refined description.
3. add_data_points — writes the updated Entity objects back to the graph and vector DB.

Legacy guide

import asyncio
import cognee
from cognee.memify_pipelines.consolidate_entity_descriptions import (
    consolidate_entity_descriptions_pipeline,
)

async def main():
    await cognee.prune.prune_data()
    await cognee.prune.prune_system(metadata=True)
    await cognee.add(
        [
            "Alice moved to Paris in 2010, while Bob has always lived in New York.",
            "Andreas was born in Venice, but later settled in Lisbon.",
            "Diana and Tom were born and raised in Helsinki. Diana currently resides in Berlin, while Tom never moved.",
        ]
    )
    await cognee.cognify()

    await consolidate_entity_descriptions_pipeline()

if __name__ == "__main__":
    asyncio.run(main())

Troubleshooting

• No entities found — the graph must already contain Entity nodes. Run cognee.remember() first.
• LLM errors — verify that your LLM provider is configured. See LLM Providers.
• Permission errors — the user must have write access to the target dataset. See Permissions.

Entity Deduplication

Merge near-duplicate entity nodes into one canonical node

Improve

Understand the current improvement workflow

Self-Improvement Quickstart

Bridge session memory and enrich a dataset

Search

Query the enriched graph with specialized search types