Session Distillation in Cognee Cloud

Endpoint and automatic triggers for bridging agent sessions into permanent memory

Session distillation bridges a conversation from the short-lived session cache into your permanent knowledge graph: it weights existing memories by the session’s feedback, persists the Q&A and agent traces, distills durable guidance into reusable lessons (session_learnings), and enriches the graph so future sessions recall from it. Surfaced in the UI as Self-improvement.

This page covers the Cloud endpoint and how it fires automatically for connected agents. For the concept and SDK walkthroughs, see Session Distillation (guide), the Self-Improving Memory Quickstart, and the improve operation.

Improve

POST /api/v1/improve — Enrich the graph and, when session_ids is passed, bridge those sessions into it.

curl -X POST https://your-tenant.aws.cognee.ai/api/v1/improve \
  -H "X-Api-Key: your-key" \
  -H "Content-Type: application/json" \
  -d '{"dataset_name": "project_memory", "session_ids": ["conversation_1"], "run_in_background": true}'

Without session_ids, the call runs default graph enrichment only. With session_ids, it runs the full session-to-graph pipeline described above for each session.

Request body

Parameter	Type	Description
dataset_name	string	Target dataset name. Provide this or dataset_id.
dataset_id	UUID	Target dataset UUID, alternative to dataset_name.
session_ids	string[]	Sessions to bridge into the permanent graph. Omit to run enrichment only.
run_in_background	boolean	Run every stage as one background task and return immediately with status: "running". Default false.
build_global_context_index	boolean	Build the global context index after enrichment. Default false.
node_name	string[]	Restrict enrichment to specific named entities.
extraction_tasks	string[]	Replace the extraction tasks of the triplet_enrichment stage with tasks named from the memify task registry. Default: none, so the stage runs its built-in triplet indexing (skipped when triplet embeddings are off).
enrichment_tasks	string[]	Replace the enrichment tasks of the triplet_enrichment stage, using names from the same registry. Default: none.
data	string	Custom input for the custom tasks, used in place of the dataset’s graph. Setting it skips the graph projection, so node_name is not applied. Default: none, so the tasks read the projected graph.

Either dataset_name or dataset_id is required. extraction_tasks, enrichment_tasks, and data are power-user overrides most callers never set. They are passed to memify() as they are. Over HTTP, tasks can only be given by name. To pass Task objects with custom arguments, use the Python SDK. If you set only one of the two task lists, the other keeps its default task (get_triplet_datapoints for extraction when triplet embeddings are on, index_data_points for enrichment).

For example, to merge near-duplicate entities with default settings instead of running the default enrichment (for thresholds or dry_run, see Entity Deduplication):

curl -X POST https://your-tenant.aws.cognee.ai/api/v1/improve \
  -H "X-Api-Key: your-key" \
  -H "Content-Type: application/json" \
  -d '{"dataset_name": "project_memory", "extraction_tasks": ["detect_entity_duplicates"], "enrichment_tasks": ["merge_entity_duplicates"]}'

Error responses

Status	Meaning
400	Neither dataset_name nor dataset_id was provided.
402	Insufficient credits to run improve — the tenant’s prepaid budget cannot cover the estimated cost. Raised pre-flight, before any work starts, or mid-run when persist_session_qa stops on an exhausted LLM budget — the body then carries error and the partial improve_result. Add credits and retry.
403	{"detail": "<message> [PermissionDeniedError]"} — you lack write access to the target dataset.
404	{"detail": "<message> [DatasetNotFoundError]"} — the requested dataset does not exist or resolved to nothing.
409	{"error": "An error occurred during graph improvement."} — an unexpected, non-Cognee server error interrupted the improve run. Cognee’s own errors no longer collapse into this status; they keep their own code and message.
420	The pipeline run itself completed with an error; the response body carries the failed run’s details.

Personalized ranking

Cognee Cloud runs with per-user preference personalization switched on for every workspace. Self-hosted Cognee ships with it off (PERSONALIZATION_ENABLED, default false); on Cloud it is set for the whole fleet — a workspace that is asleep picks it up when it next wakes — and there is no per-user switch in the app.

Personalization changes your own results only. Nothing a teammate asks is re-ranked by what you rated, and nothing you rate is visible to them.

Where the ratings come from. Cloud runs with automatic feedback on, so the per-turn analysis of an agent session can read a 1–5 rating of the previous answer out of what you say next — “that was exactly right”, “no, wrong again”. An explicit rating sent through the API wins over an inferred one, and a 3 is neutral and does nothing.

When it is written. The ratings are folded into preferences by the same improve run that bridges a session into the graph — the one connected agents fire for you. A call without session_ids never touches preferences, and a failure in that stage is logged without failing the rest of the run.

What is stored. One internal preference record per user and dataset: weighted prefers links to the graph nodes that built the answers you rated, plus the preferences you stated in words, newest first and capped at 2,000 characters. Weights sit at neutral until a rating moves them and decay back toward neutral as the conversation goes on, so a preference you stop reinforcing fades instead of hardening. This record is internal: it is never embedded, never returned as a search result, and never appears in the knowledge graph you browse.

What changes at retrieval. On your recalls, GRAPH_COMPLETION (and its variants), HYBRID_COMPLETION, and RAG_COMPLETION move ranking by at most 30% — toward what you rated up and away from what you rated down — and pass your stated preferences to the model as guidance. Other search types are unaffected, and so is a search that spans several datasets: personalization only applies when a search targets one dataset. A missing or unreadable preference record never fails a search — the query runs as if the feature were off.

For the mechanism in full — how a rating becomes a weight, how decay and pruning work, and the knobs a self-hosted deployment can tune — see Personalized Memory per User and the Feedback System guide.

Automatic distillation for agent sessions

You rarely call this endpoint directly. When you connect a coding agent through the Agent Integrations, the plugin fires it for you.

The Cognee coding-agent plugins (Claude Code, Codex) run session distillation for you — you never call improve() by hand. A distillation pass fires on two automatic triggers:

Trigger	When it fires	How
Session end	You quit the agent	The SessionEnd hook drains buffered turns into the server session cache, then fires POST /api/v1/improve for the session id in the background. A launch-exit watcher covers exits where the hook never fires — a hard exit, or Codex CLI shutdowns that skip SessionEnd.
Every N tool calls	Long sessions, incrementally	A per-session counter fires an improve every COGNEE_AUTO_IMPROVE_EVERY stored tool calls/stops (default 150), so a long session bridges into the graph without waiting for the end.

The every-N trigger has a per-session cooldown: after an improve, the next automatic one waits at least COGNEE_IMPROVE_COOLDOWN seconds (default 86400, 24 hours), and a failed attempt starts the same wait as a backoff. Session end, the sync skill, and a dataset switch improve regardless of it. Going idle does not trigger an improve.

Overlapping triggers are safe. A per-session improve lock on the server serializes concurrent runs, and unchanged session content dedups server-side by content hash — so a repeat improve over content that hasn’t changed is a cheap no-op, not duplicated work.

Configuration

All triggers are tuned through environment variables read by the plugin. The defaults are chosen so distillation stays out of your way; you rarely need to change them.

Variable	Default	What it controls
COGNEE_AUTO_IMPROVE_EVERY	150	Stored tool calls/stops between automatic mid-session improves; 0 turns the trigger off
COGNEE_IMPROVE_COOLDOWN	86400	Minimum seconds between automatic improves of one session
COGNEE_IMPROVE_SUBMIT_TIMEOUT	420	Read timeout for the improve POST (distillation runs inside the request)
COGNEE_IDLE_POLL	10	How often, in seconds, the background watcher wakes to refresh shared agent memory and to notice a stop
COGNEE_IDLE_DISABLED	unset	1 / true turns off the background watcher (update and LLM-key checks, shared agent memory refresh). In Claude Code cloud mode it also stops the worker that uploads captured turns to your tenant, so don’t set it there to tune distillation

The plugin READMEs document additional advanced knobs — timing (poll deadlines, busy-retry intervals for a held session lock), session-sync retries, and the update-notification variables (COGNEE_UPDATE_CHECK, COGNEE_UPDATE_CHECK_INTERVAL). You almost never need them — reach for the table above first.

Turning it down or off

• Reduce mid-session improves: raise COGNEE_IMPROVE_COOLDOWN, or set COGNEE_AUTO_IMPROVE_EVERY=0 to turn the every-N trigger off.
• Session-end distillation always runs when the plugin is active — it’s how a finished session reaches permanent memory.

Confirming it happened

• Cloud UI: the Self-improvement card at the top of a session on the Sessions page shows the status of the last graph enrichment and the dataset it wrote to.
• Plugin hook log: each automatic run emits an improve_fired event you can grep for when debugging (in local SDK mode, where the plugin calls the library directly instead of the HTTP endpoint, look for auto_improve_fired instead).
• improve-unsupported.json marker: if this file appears in the plugin’s shared state directory (24h TTL), the server rejected the improve endpoint and the plugin fell back to the legacy remember bridge for that window — a signal the server predates session-aware improve.