> ## Documentation Index
> Fetch the complete documentation index at: https://docs.lancedb.com/llms.txt
> Use this file to discover all available pages before exploring further.

# LanceDB

> Multimodal lakehouse for AI.

**LanceDB** is a [multimodal lakehouse](https://lancedb.com/blog/multimodal-lakehouse/) for AI teams that need
one data layer for curation, feature engineering, search and retrieval, and model training.
It is built on top of [Lance](https://docs.lancedb.com/oss/v0-41-0/basics/lance.md), an open-source lakehouse format designed for multimodal AI data.

Move from data exploration to model training on one unified platform without needing to manage a
fragmented stack of storage, feature, retrieval, and training systems.

## Train on the right data faster

Search, curate, and train at petabyte scale, with all your multimodal data in one table.

Preparing training data and running experiments slow down when raw data, metadata, embeddings, features, and governance
artifacts live in separate systems. LanceDB keeps them together in one versioned multimodal table, so AI teams spend less
time stitching infrastructure together and more time improving datasets, testing features, and keeping GPUs fed.

## Accelerate the data flywheel for model research

Keep your AI data in one table.
Find the failure, fix the data, and retrain.

**1. Find a curated training subset for every failure case**

Start from one failed grasp or evaluation example. [Search](https://docs.lancedb.com/oss/v0-41-0/search.md) multimodal data, metadata, and embeddings
together, retrieve similar examples, and [curate](https://docs.lancedb.com/oss/v0-41-0/curation.md) them into a targeted training subset.

**2. A single dataset for every modality**

Keep episodes, frames, metadata, features, and embeddings together in one schema-enforced table, without
pipelines to keep them in sync. Table [versions](https://docs.lancedb.com/oss/v0-41-0/management/table.md#versioning) let you trace each training
run back to the exact data it used.

**3. More hillclimbing, less babysitting preprocessing jobs**

Define the [computed columns](https://docs.lancedb.com/oss/v0-41-0/feature-engineering/add-computed-column.md) and
[materialized views](https://docs.lancedb.com/oss/v0-41-0/feature-engineering/materialized-views.md) your experiment needs, such as extracted features
or embeddings, and refresh them as your data changes. On LanceDB Enterprise, column refreshes run as background
jobs on the cluster instead of in your code.

**4. Higher GPU utilization, faster convergence**

Keep GPUs fed. [`StreamingDataset`](https://docs.lancedb.com/oss/v0-41-0/training/data-loading) streams multimodal training data into PyTorch with
globally shuffled, deterministic batches, combining fast random access with high-throughput scans.

Use the same table to curate training data, add derived features, retrieve examples, and feed training jobs that rely on expensive GPUs.
Training workloads can sample, shuffle, and scan projected columns from local storage or object storage, then assemble
GPU-ready batches from a tagged dataset version. Training pipelines need curation, derived features, reproducible
splits, fast random access, and a clean path into frameworks such as PyTorch -- LanceDB keeps all of it connected
through the same table model, whether a workflow uses one table or several related ones.

## Architecture: Embedded and Enterprise

LanceDB ships as two offerings on the same [Lance](https://docs.lancedb.com/oss/v0-41-0/basics/lance.md) format and table model, which stores table data,
versions, and indexes in open, columnar files. What changes between them is how storage and compute are organized
around that format: start locally with Embedded, then move to Enterprise when your team needs distributed scale,
managed infrastructure, private deployment, or higher-throughput curation, feature engineering, search and
retrieval, and training workflows.

> [card] [LanceDB Embedded](https://docs.lancedb.com/oss/v0-41-0/basics/lancedb-embedded.md)
>
> An open-source, embedded retrieval library that runs inside your application process, with no separate
> database server to deploy.

> [card] [LanceDB Enterprise](https://docs.lancedb.com/oss/v0-41-0/enterprise/index.md)
>
> A distributed multimodal lakehouse service for production workloads that need managed infrastructure and
> higher-throughput workflows.

See [Embedded vs Enterprise](https://docs.lancedb.com/oss/v0-41-0/basics/embedded-vs-enterprise.md) for a feature-by-feature comparison, including
architecture, latency, throughput, and day-to-day usage differences.

## Projects using LanceDB

### Training

* [stable-worldmodel](https://github.com/galilai-group/stable-worldmodel) -- a platform for reproducible world-model
  research built on a LanceDB data layer, reporting 3-4x faster data loading on Push-T versus HDF5 / MP4 at a
  fraction of the disk footprint.
* [le-wm](https://github.com/lucas-maes/le-wm) -- a joint-embedding predictive world model from pixels, trained on
  the stable-worldmodel platform and its LanceDB data layer.
* [lerobot-lancedb](https://github.com/lancedb/lerobot-lancedb) -- a drop-in LanceDB backend for HuggingFace
  LeRobot datasets with faster loading across robotics datasets.

### Agentic applications

* [OpenClaw LanceDB Plugin](https://docs.openclaw.ai/plugins/memory-lancedb) -- stores an agent's long-term memory
  in LanceDB, automatically recalling relevant memories before a model turn and capturing new facts afterward.
* [Cognee](https://github.com/topoteretes/cognee) -- an open-source AI memory platform that gives agents persistent
  memory across sessions, using LanceDB as one of its vector store backends.
* [CrewAI](https://github.com/crewAIInc/crewAI) -- a multi-agent framework that
  [rebuilt its agent memory on LanceDB](https://www.lancedb.com/blog/crewai-rebuilt-agent-memory-on-lancedb).

Across these projects, LanceDB and Lance provide the multimodal data layer that keeps raw observations, annotations,
features, and training or memory access patterns in one format instead of scattering them across task-specific
stores.

## Start with your workload

> [card] [Ingest your data](https://docs.lancedb.com/oss/v0-41-0/ingestion.md)
>
> Create tables, append records, and update or delete rows as your data changes.

> [card] [Build search and retrieval](https://docs.lancedb.com/oss/v0-41-0/search.md)
>
> Use vector search, full-text search, hybrid search, reranking, filtering, and SQL.

> [card] [Curate training data](https://docs.lancedb.com/oss/v0-41-0/curation.md)
>
> Profile, sample, chunk, join, and deduplicate records before training on them.

> [card] [Engineer features in place](https://docs.lancedb.com/oss/v0-41-0/feature-engineering.md)
>
> Derive computed columns and materialized views without moving data elsewhere.

> [card] [Load data into PyTorch](https://docs.lancedb.com/oss/v0-41-0/training/data-loading.md)
>
> Use LanceDB tables and permutations for projected, shuffled, random-access training reads.

> [card] [Manage catalog objects](https://docs.lancedb.com/oss/v0-41-0/management.md)
>
> Create, list, and drop tables, namespaces, and views from Python, TypeScript, or SQL.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.