Skip to main content
cognee-rust is a Rust port of the Python cognee library — an AI memory pipeline that turns raw data into persistent, queryable knowledge graphs. It targets both edge devices (Android, embedded) with local models and drop-in parity with the Python cognee SDK (90%+ correctness). Core pipeline: add (ingest) → cognify (knowledge-graph extraction) → search (context retrieval). See Operations for what each operation does and Configuration for how to configure them. This page is the single source of truth for the workspace layout, the crate breakdown, and the cross-cutting design patterns. .claude/CLAUDE.md and the root README.md link here rather than duplicating it.

​
Workspace structure

​
Crate breakdown

cognee-models — Core data types shared across crates: Data, Dataset, DataInput, Document, DocumentChunk, Entity, KnowledgeGraph, etc. Pure data structures, no traits. cognee-storage — Abstract file storage layer. Trait: StorageTrait (+ StorageExt, StorageWriter). Impls: LocalStorage, MockStorage. cognee-database — Relational database abstraction. RelationalDb is the one object components hold (Arc<dyn RelationalDb>): a supertrait of the domain traits IngestDb, DeleteDb, SearchHistoryDb, NotebookDb, DatasetConfigDb, SessionLifecycleDb, SessionQaDb (session Q&A tables) and ProvenanceDb (cognify’s provenance ledger and run rows), plus the pipeline-run / checkpoint / sync repositories and lifecycle (ping, close). Callers upcast to the narrow trait they need. Impl: DatabaseConnection (SQLite/Postgres via SeaORM); no code outside the crate depends on the concrete type. cognee-ingestion — Pipeline for ingesting data: streams content, computes hashes, deduplicates, and stores. Main type: AddPipeline. No trait abstraction — uses StorageTrait and IngestDb from sibling crates. cognee-chunking — Text chunking strategies ported from the Python chunking hierarchy (word → sentence → paragraph). Main entry point: ExtractTextChunksPipeline. Trait: TokenCounter. Impls: WordCounter (whitespace fallback), HuggingFaceTokenCounter (BPE/WordPiece, behind hf-tokenizer), TikTokenCounter (cl100k_base BPE, behind tiktoken). TokenCounterKind::resolve() auto-selects the counter from already-resolved settings: cognify.token_counter, the HuggingFace tokenizer id, the embedding provider and the ONNX tokenizer path. cognee-cognify — Knowledge-graph extraction pipeline: classify documents, chunk text, extract entities/relationships via LLM, summarize, store to graph and vector DBs. Entry points: cognify() / cognify_datasets(); main types: CognifyConfig, CognifyInput, CognifyResult, FactExtractor, SummaryExtractor. Run orchestration’s failure policy — which scope a finished run sweeps, which items it marks complete, and what its run record says — lives in the rollback module and calls into cognee-delete’s RunSweeper. Also houses the memify sub-module (MemifyConfig, MemifyResult, memify()): reads the existing graph, creates triplet embeddings, indexes them for SearchType::TripletCompletion. cognee-search — Unified search orchestration across multiple retrieval strategies. Main types: SearchBuilder, SearchOrchestrator. SearchType enum defines 16 search modes with corresponding retriever implementations; the newest, HybridCompletion, blends vector search over chunks/summaries/entities/edge-facts, and 1-hop graph-neighborhood expansion, then answers via LLM completion. cognee-session — Session management and QA-history storage. Trait: SessionStore. Impls: FsSessionStore, RedisSessionStore, RelationalSessionStore (over any SessionQaDb; formerly SeaOrmSessionStore). cognee-embedding — Text vectorization engine. Trait: EmbeddingEngine. Impls: OnnxEmbeddingEngine (local ONNX Runtime, BGE-Small-v1.5), OpenAICompatibleEmbeddingEngine (OpenAI/Azure/vLLM/llama.cpp/TEI via HTTP), OllamaEmbeddingEngine, BedrockEmbeddingEngine (AWS Bedrock InvokeModel — Titan + Cohere families, bedrock feature), MockEmbeddingEngine. EmbeddingConfig::from_env() + create_engine() factory select the provider. See Configuration. cognee-llm — Async LLM abstraction with structured JSON output. Trait: Llm (+ auto-implemented LlmExt). Impls: OpenAIAdapter (OpenAI-compatible APIs, works with Ollama/vLLM/Azure), AnthropicAdapter (native Messages API), BedrockAdapter (AWS Bedrock Converse, bedrock feature), MockLlm (cassette-backed, testing feature). An on-device LiteRT adapter is not part of OSS. cognee-graph — Graph database abstraction for knowledge-graph storage and traversal. Trait: GraphDBTrait (+ GraphDBTraitExt), including the batched get_neighborhood(ids, depth) -> (nodes, edges) method that expands a set of seed nodes and returns the true stored edge direction (unlike get_connections, whose Ladybug impl reports the queried node as the source regardless of stored direction — see crates/graph/src/traits.rs and the get_neighborhood_preserves_edge_direction test in crates/graph/src/mock.rs). Impls: LadybugAdapter (embedded Ladybug), PgGraphAdapter (feature postgres), MockGraphDB. Concurrency: Rust matches Python’s default single-owning-process model for file-backed Ladybug; cross-process locking is intentionally out of scope (see roadmap/). get_neighborhood_scoped(ids, depth, scope) is the optional narrowing form — property projection and an opt-in per-seed neighbour cap — whose trait default is get_neighborhood narrowed client-side by apply_neighborhood_scope, so a backend only overrides it to push the narrowing into the query (PgGraphAdapter does; see COGNEE_TRIPLET_NEIGHBOR_CAP in Configuration). cognee-vector — Vector database abstraction for similarity search. Trait: VectorDB. Impls: LanceDbAdapter (embedded Apache-Arrow / Lance, on-disk; default on non-Android targets), BruteForceVectorDB (pure-Rust in-memory; default on Android and via vector_db_url = ":memory:"), PgVectorAdapter (Postgres + pgvector extension, feature pgvector), MockVectorDB. An embedded Qdrant adapter is not part of OSS. begin_bulk_load / end_bulk_load is a nestable, no-op-by-default hint — cognify wraps one pipeline run in a scope — that lets a backend defer per-row ANN index maintenance to one build at the end; a backend that defers must leave the store correct (exact scans) if the scope never closes, and repairable by create_missing_vector_indexes (see Pluggable Backends). cognee-ontology — RDF/OWL ontology integration for entity validation. Trait: OntologyResolver. Impls: RdfLibOntologyResolver, NoOpOntologyResolver (pass-through). cognee-delete — Cascading deletion of data/datasets across all backends (relational → graph → vector → file storage), plus the run-scoped rollback sweep cognee-cognify calls when a run fails. Main types: DeleteService, AuthorizedDeleteService, RunSweeper. cognee-core — Async runtime, task scheduling, and pipeline-execution primitives. Traits: PipelineWatcher, ExecStatusManager. Impls: NoopWatcher, RayonThreadPool, NoopExecStatusManager. cognee-components — Shared backend construction. Owns ComponentError, the BackendBuildContext (the resolved, env-free input both callers lower their config into), the adapter factory traits (VectorDbFactory, GraphDbFactory, LlmFactory, EmbeddingFactory), and the ComponentRegistry (provider-id → factory) with with_builtins(). Sits below cognee (and below any downstream consumer such as an HTTP server); they delegate their backend construction here, so the two paths can’t drift. The registry is the explicit-DI extension seam — external adapter crates implement a factory trait and register_* it at their binary entry point. See Operations. cognee-visualization — Single-file HTML knowledge-graph visualization (d3.js v7) with four tabs — Graph, Schema, Memory, Semantic — plus a node inspector. preprocessor derives the whole payload (node/link normalization, stage + topological_rank layout, colour maps, memory map, type-schema graph, operation layer); html substitutes it into the frontend assets under assets/, which are vendored verbatim from Python and must never be hand-edited (see assets/README.md). Entry points: visualize/render/render_multi_user. Surfaces via the CLI visualize subcommand. Payload gaps in roadmap/not-implemented.md. cognee-migration — COGX export: writes the graph store as a portable archive that Python cognee can re-import, either locally (cognee.remember(COGXArchiveSource(path))) or over HTTP (POST /api/v1/remember, content_type=cogx-archive, packed .cogx.tar.gz). COGX is the only one of Python’s five export formats with a reader on the other side. Entry points: write_cogx (pure, nodes+edges → archive), export_graph (reads a GraphDBTrait), pack_archive (archive feature). Surfaces via the CLI export subcommand, whose --dataset is a label only: cognee-rs has no per-dataset graph partition, so an export always covers the whole store. The manifest must declare source_system = "cognee" — Python derives id preservation from that exact string. Contract tests in tests/python_import_contract.rs; the cross-SDK roundtrip lives in e2e-cross-sdk. cognee-observability — OpenTelemetry tracing pipeline. Bridges #[tracing::instrument] sites into an OTLP exporter. Entry point: init_telemetry (tracing layer + RAII TelemetryGuard). Activated by COGNEE_TRACING_ENABLED=true or a non-empty OTEL_EXPORTER_OTLP_ENDPOINT; real exporter behind the telemetry feature. See observability/opentelemetry.md. cognee-telemetry — Product-analytics client (send_telemetry). Fire-and-forget POST to https://test.prometh.ai per public API call; on by default, with a process-wide switch (TelemetryConfig, set_enabled / is_enabled, exposed by every binding and the CLI’s --no-telemetry); env opt-outs TELEMETRY_DISABLED, ENV=test|dev; compiled out with --no-default-features. See observability/send_telemetry.md. cognee-logging — Shared file-based logging: rotation, the Python-compatible plain-text formatter, and a noise-suppressing EnvFilter. Entry point: init_logging, called by the CLI and by other embedders. Env-var surface documented in Configuration: logging. cognee-bindings-common — Shared SDK facade for the Neon JS, C-API, Python and Java (JNI) bindings: SdkError (+ code()), HandleState, CogneeServices, and neon-free JSON wire helpers. Not a new user-facing Rust API — that remains cognee::api. cognee — Unified public API facade. Re-exports all crates and adds an api/ module mirroring the Python SDK: forget, update, prune, recall, remember, improve, plus DatasetManager. Houses the shared Settings/ConfigManager, the setting-key registry (config/keys.rs: one row per setting binds its env vars, the generic set, and cognee-cli config) and runtime setters. ComponentManager (lazy, version-cached) delegates backend construction to a cognee-components ComponentRegistry; use ComponentManager::with_registry (or HandleState::from_settings_with_registry in the bindings) to inject external adapter factories. The registry API is re-exported here so downstream entry points use cognee:: paths. (The crate lives in crates/lib/; it shipped as cognee-lib through v0.1.3 and was renamed to cognee.) cognee-lib — Deprecated re-export shim (crates/cognee-lib/) that keeps the old cognee-lib crate name compiling after the rename. It is pub use cognee::* with every Cargo feature forwarded 1:1 to cognee. New code should depend on cognee directly. cognee-cli — Command-line binary (cognee-cli). See CLI Reference. cognee-utils — Shared utilities: retry logic, deterministic ID generation (generate_node_id, NAMESPACE_OID, …), secret redaction (redact), and tracing attribute keys. cognee-config-core — The configuration boundary, and the machinery for component-owned config sections (config_section!, ConfigSection, ConfigSchema, unknown_key). EnvSource (a read-only view of an environment), ProcessEnv (the one sanctioned read of the process env; constructing it is clippy-banned outside a declared boundary), MapEnv (an in-memory source, for tests or for resolving Settings without touching the process env), and load_dotenv (dotenv feature). A leaf crate with no cognee-* dependencies. See Configuration: configuration model. cognee-truth-subspace — Pure alignment math (cosine projection, query-relevance-weighted truth scoring, stable id signatures) for the hybrid-retriever truth-subspace re-ranking feature, plus VectorDB-backed centroid persistence (load_centroids/upsert_centroids). Used by cognee-search’s HybridRetriever (use_truth_weight, off by default) and by cognee-cognify. cognee-test-utils — Test helpers and mock implementations for integration tests.

​
Architecture patterns

  • Feature strategy — Most leaf crates define optional features with no defaults (default = []); cognee-components and cognee-bindings-common carry backend defaults. The umbrella library (cognee) and the CLI (cognee-cli) enable all non-platform-specific features by default, so a plain cargo build gives a fully-featured binary. Platform- and deployment-specific extras (e.g. on-device LiteRT inference, embedded Qdrant) are not part of this repository; the testing feature stays opt-in. New feature-gated capabilities should be propagated up through cognee/cognee-cli defaults unless platform- or test-only.
  • Trait-based abstractions — StorageTrait, IngestDb, GraphDBTrait, VectorDB, EmbeddingEngine, Llm, SessionStore, etc. enable backend swapping and mock testing.
  • Open to external providers — adding a vector / graph / LLM backend must never require a change to cognee-rs. An outside crate implements a factory trait, register_*s it on a ComponentRegistry, and selects it by provider id in configuration. Two rules follow: (1) nothing validates a provider id against a fixed list — the registry’s own lookup is the only authority; and (2) every resolved input a provider might need (notably the assembled Postgres URLs in BackendBuildContext) is populated for all provider ids, never only for the built-in ones, and is never gated on a built-in’s cargo feature. A built-in id may be replaced by registering over it. See guides/external-providers.md.
  • Prefer dyn Trait — object-safe traits via &dyn Trait / Arc<dyn Trait> at call sites; monomorphized generics only when performance-critical.
  • Zero-copy where possible — WordChunk<'a>, SentenceChunk<'a>, ParagraphChunk<'a> borrow &str slices via byte-offset tracking.
  • Arc for shared ownership — Arc<dyn Trait> in pipelines; Arc<Mutex<T>> in mocks.
  • Async-first — all I/O via tokio; trait methods are async since components may be local or remote.
  • Streaming-first — DataInput::process_by_chunks(), StorageTrait::store_stream(), ContentHasher::hash_content_stream() avoid loading full files into memory.
  • Deterministic IDs — same content + owner ⇒ same UUID via uuid5(NAMESPACE_OID, …) (content-addressed dedup). Chunk IDs: uuid5(NAMESPACE_OID, "{document_id}-{chunk_index}").
  • Configuration at the boundary — configuration is resolved once, at a config boundary, into a typed struct (Settings per SDK instance; LoggingConfig and the observability/telemetry settings per process); components receive resolved values through constructors and BackendBuildContext, and never read the process environment. Env vars are defaults; everything they set is also settable from code. Enforced by clippy::disallowed_methods in the root clippy.toml plus the scripts/check_env_boundary.sh per-file ratchet on direct-access call sites. See Configuration: configuration model. Component-specific settings are declared by the component (config_section!, returned from its factory’s config_sections()), so registering a factory registers its settings and a compiled-out feature contributes none.
  • Error types per crate — each crate defines its own thiserror enum (StorageError, ChunkingError, IngestionError, …).

​
Key dependencies

​
Browsing the API docs (rustdoc)

API and type detail are documented inline in the code and rendered by rustdoc — the rest of these docs link to it rather than restating signatures.
CI already runs cargo doc --no-deps on every push (no hosted docs.rs site — build locally). Each crate’s lib.rs carries a top-level //! summary; start from cognee (the facade) and follow the re-exports.