run_migrations

Apply pending relational and vector database schema migrations

cognee.run_migrations

Applies all pending database schema migrations before the rest of your application starts.

await cognee.run_migrations()

In releases before Cognee 1.5.0 this function was named run_startup_migrations. That name is gone — update calls to cognee.run_migrations() when you upgrade.

It runs two steps in sequence:

1. Relational schema — executes alembic upgrade head against your configured relational database (SQLite by default, or Postgres).
   • Revisions c5d7e9f1a3b5 and d6e8f0a2b4c6 complete the dataset-scoping of the data table — see Dataset-scoping upgrade below.
   • This step also heals the SQL session-cache table on existing deployments. Revision c3d5e7f9a1b2 deletes all but the newest row per (user_id, session_id, entry_id) in cache_session_context and creates the uq_cache_session_context_entry unique index that the cache adapter’s upsert targets. It covers both the database Alembic is connected to and the standalone SQLite cache.db that CACHE_BACKEND=sqlite (the default) keeps next to the relational database. Cache tables are created on init rather than managed by Alembic, so fresh databases already get the index from the table definition and only pre-existing tables are touched. Because the migration mutates data and builds a unique index, it can take time and hold locks on a large cache_session_context table — back up the database and run it in a maintenance window.
2. Vector schema — runs the vector adapter’s run_migrations method for every database that needs it:
   • Single-user mode (ENABLE_BACKEND_ACCESS_CONTROL=False): migrates the single default vector engine.
   • Multi-user mode (ENABLE_BACKEND_ACCESS_CONTROL=True, the default): iterates over every dataset database and migrates each one individually. A failure for one dataset is logged and skipped; the remaining datasets continue to migrate.
   • If the active vector engine has no run_migrations method, Cognee logs a warning and skips that engine.
   • If the dataset_database table does not exist yet (a fresh database), the vector migration step is skipped with a warning instead of raising. This is handled on both SQLite (OperationalError, “no such table”) and PostgreSQL/pgvector (ProgrammingError / UndefinedTableError).

Migrations that re-embed batch their embedding requests. Some data migrations re-embed rows through cognify’s indexing path; those embeddings are sent in batches, so a large migration no longer fails on provider request-size or rate limits. Batching follows the same EMBEDDING_BATCH_SIZE and EMBEDDING_MAX_CONCURRENT_DATA_POINTS settings as normal indexing; there is no migration-specific setting to tune.

Dataset-scoping upgrade

Cognee 1.5.0 makes Data rows dataset-scoped: a row belongs to exactly one dataset, and the same content added to two datasets is two independent rows (see deduplication). Databases created by an earlier release stored one row shared across datasets through a dataset_data membership table, so upgrading needs a data migration, not just a schema change.

Two Alembic revisions do the relational half:

Revision	What it does
c5d7e9f1a3b5	Adds the data.dataset_id column and the (dataset_id, owner_id, content_hash) lookup index that backs add-time deduplication. Existing rows keep dataset_id = NULL until the next revision.
d6e8f0a2b4c6	Adds data.legacy_id, backfills every legacy row from its memberships, then drops dataset_data.

The backfill splits rows per dataset:

• One membership (the overwhelming majority) — the row keeps its original id and is stamped with that dataset, so data ids you stored externally stay valid.
• Several memberships — the oldest membership keeps the original id; every other dataset gets a fresh row with a new id, all columns copied, and legacy_id recording the pre-split id. Relational ledger rows for those datasets are repointed to the new ids.
• No memberships — unreachable orphans, left untouched.

Because legacy_id is preserved, every id ever issued keeps resolving within its dataset: Cognee matches the exact id first, then falls back to legacy_id.

The graph and vector stores are then brought in line by the rekey_fork_document_ids migration in Cognee’s data-migration chain, which runs after the relational step. For each split (“fork”) document it re-keys the graph document node to the canonical relational id, updates the ledger’s node/edge references, refreshes the document_id property on the document’s chunk nodes, and re-upserts the chunks’ vector index rows — in batched embedding requests, as described above — so their document_id payload cites the canonical id. Chunk point ids are unchanged. Fork rows are rare — same user, identical content, several datasets, all before the upgrade — so on most deployments this is one indexed relational query and no per-document work.

When a fork does exist on a large graph, the graph half of that re-key is real work. Its edge-identity and node-id lookups run as chunked index seeks and its provenance restore is attached in batches grouped by pipeline run, so the re-key completes within the worker subprocess deadline instead of timing out — but on a graph of ~100k nodes and ~300k edges it can still run for tens of minutes. On the default embedded Ladybug backend (this does not apply to Neo4j or Postgres graph stores, which have no such cap), give the store headroom under its size cap — the kuzu_max_db_size setting (KUZU_MAX_DB_SIZE) — before starting, and let the run finish: killing a worker mid-checkpoint can leave .lbug.shadow / .wal.checkpoint recovery files behind that block the next open, and repeated interrupted runs can exhaust the cap even while the database is small on disk.

Much of that graph work is set-based rather than row-by-row. On Ladybug/Kuzu, the provenance move is one replace() statement over the nodes and one over the edges still carrying the pre-fork key, so the batched restore described above handles only the residue: artifacts that already carry both keys are deliberately left to that generic attach-then-remove sweep, which dedupes them, so the migration still converges on re-run. Every other graph backend — and any engine that rejects those statements — takes the unchanged generic path for the whole move, and the migrated result is identical either way. The survivor-edge repair in the sibling namespace_entity_type_node_ids migration is trimmed the same way on any backend that can list its edges in one scan: it asks which at-risk edges were actually dropped and re-asserts only those, and blind-upserts every at-risk edge only where that scan isn’t answerable. On a large fork subgraph these shortcuts make the migration substantially faster.

Both migrations are reversible, but the rollback order is the inverse of the upgrade order: run the rekey_fork_document_ids downgrade before the d6e8f0a2b4c6 downgrade. The reverse map is built from legacy_id, so dropping that column first strands fork graphs on canonical ids with no way back. Downgrading also loses fork lineage permanently — the old schema has nowhere to keep it — so a later re-upgrade re-splits shared rows with fresh ids.

The backfill is not an online migration: an old-version replica still writes shared rows and reads dataset_data, both of which the migration retires. Stop all replicas, upgrade, then start them again. A concurrent second run self-cancels — both transactions contain the dataset_data drop, and the loser rolls back whole — and an interrupted run rolls back cleanly and simply reruns.

Entrypoints

All migration functions live in the cognee.run_migrations module. run_migrations is also re-exported at the top level as cognee.run_migrations, so it is the recommended entrypoint for most applications.

Function	Import	Migrates
run_migrations	cognee.run_migrations (or from cognee.run_migrations import run_migrations)	Relational schema and the graph/vector stores (recommended)
run_relational_migrations	from cognee.run_migrations import run_relational_migrations	Relational schema only (alembic upgrade head)

When to call it

Scenario	Why
After upgrading the cognee package	New versions may add tables or columns to the relational schema.
Upgrading to Cognee 1.5.0 from an earlier release	Required: the dataset-scoping backfill rewrites existing Data rows and drops the dataset_data table. Stop all replicas first.
First run against an external database	The automatic startup run migrates Postgres and other external databases just like SQLite, but an explicit run lets you build the schema before the app rolls out.
Kubernetes / Docker init containers	Run migrations once before starting the main application pods.
Switching to a new relational DB provider	The new database starts empty and needs all migrations applied.
Running migrations on your own schedule	Turn off the automatic runs with ENABLE_AUTO_MIGRATIONS=false and migrate explicitly instead.

For the default local setup (SQLite + LanceDB), Cognee handles migrations automatically when the API server starts. You only need to call run_migrations() explicitly in server deployments or CI pipelines where you manage database lifecycle yourself.

Disabling automatic migrations

run_migrations() is called for you in a few places: the FastAPI server’s startup lifespan, the first cognify() or remember() call in a process, and — in the Docker image — the container entrypoint before the server binds its port.

Set ENABLE_AUTO_MIGRATIONS=false to disable all of those automatic runs:

ENABLE_AUTO_MIGRATIONS=false

The variable defaults to true. It is disabled by false, 0, or no (case-insensitive); any other value leaves automatic migrations on. When disabled, run_migrations() logs that it was skipped and returns an empty list without touching any database.

Disabling automatic migrations does not migrate anything on its own — Cognee will run against whatever schema already exists. Pair it with an explicit cognee-cli upgrade step before rolling out a new version: the CLI deliberately ignores this flag and runs the same locked relational + graph/vector sequence regardless. Note that the init container below calls run_migrations(), so it is gated by this flag too — use cognee-cli upgrade as its command, or leave the flag unset in the init container’s own environment.

For the container-startup side of this flag, see Docker → Database Migrations on Startup.

Example

import asyncio
import cognee

async def main():
    # Apply all pending schema migrations before starting
    await cognee.run_migrations()

    # Normal usage
    await cognee.add("Hello, world!", dataset_name="demo")
    await cognee.cognify()

asyncio.run(main())

Kubernetes init container

Run migrations as a one-shot init container so the main pod only starts after the schema is ready:

initContainers:
  - name: migrate
    image: your-cognee-image:latest
    command: ["python", "-c", "import asyncio, cognee; asyncio.run(cognee.run_migrations())"]
    envFrom:
      - secretRef:
          name: cognee-env

Concurrency

Every migration flow runs under a single cross-process lock, so a host performs at most one migration of any kind at a time. If several processes start at once — multiple workers of the same server, parallel SDK runs, or several init containers — only one acquires the lock and migrates; the others block until it finishes, then re-read the stored revision and skip work that is already done. Nothing runs migrations in parallel.

Because of this, startup can block (and time-to-ready can increase) while another process holds the lock and migrates. This is expected under the Kubernetes init container and multi-worker scenarios above — the wait is the coordination working as intended, not a hang.

The lock backend depends on your relational database:

• Postgres — a session-scoped advisory lock, which also serializes migrations across hosts. Use Postgres metadata when multiple hosts may start and migrate at the same time.
• SQLite — an OS advisory file lock placed next to the database file. It serializes multiple processes on a single host (multi-worker servers, parallel SDK runs) but not across hosts or over NFS.

Errors

Error	Cause	Fix
FileNotFoundError	alembic.ini or the migrations directory is missing from the installed package.	Reinstall cognee — the package may be corrupted or partially installed.
MigrationError	Alembic exited with a non-zero return code.	Check the error message logged at ERROR level; usually a DB connection problem or a SQL conflict.
RuntimeError: The session cache lives in a separate database this migration cannot reach	You use a SQL session-cache backend and CACHE_DB_URL points at a non-SQLite database other than the one Alembic connects to, so revision c3d5e7f9a1b2 cannot dedupe and index cache_session_context there.	Run the DELETE and CREATE UNIQUE INDEX statements printed in the error message against that cache database, then re-run migrations. There is deliberately no fallback: the migration blocks the deploy rather than leaving a cache database the upsert would fail on.

Set LOG_LEVEL=DEBUG to see the full Alembic output when diagnosing migration failures.

Related

• Relational Databases — configure SQLite or Postgres
• Deployment Overview — how to structure Cognee in production
• Kubernetes (Helm) — full Kubernetes deployment guide