Quickstart

This quickstart uses LanceDB Embedded with a local database and works without an account or API key. LanceDB Embedded runs in your application’s process; your data can live on local disk or in object storage.

1. Install LanceDB

Choose your language. Python uses Pandas to display the search results. TypeScript needs Node.js 22.18 or later to run the .mts example directly.

pip install "lancedb==0.41.0" pandas

uv venv
uv pip install "lancedb==0.41.0" pandas

npm install @lancedb/lancedb@0.41.0

cargo init --name lancedb_quickstart
cargo add lancedb@0.31.0 --features polars,remote
cargo add tokio --features macros,rt-multi-thread

Run the Rust commands in a new, empty directory. The polars feature provides DataFrame output; remote also lets this example connect to Enterprise.

Python pre-release builds and older CPUs

Python pre-release builds

To pick up the latest features and bug fixes before the next stable release, install a pre-release from LanceDB’s Fury index.

pip install --pre --extra-index-url https://pypi.fury.io/lancedb/ lancedb

uv venv
uv pip install --prerelease allow --index https://pypi.fury.io/lancedb/ lancedb

# To add to pyproject.toml, use:
uv add --prerelease allow --index https://pypi.fury.io/lancedb/ lancedb

Most applications only need lancedb. Install the optional pylance extra and include the Lance preview index only when building query engines or connectors, calling format-level lance APIs directly, converting plain-scan lazy blob handles with to_pandas(), or reading legacy v1 blob bytes into pandas with Python 0.41.0 or 0.42.0b8. Those clients use the native bytes path for supported v2 blob queries; they still need pylance for legacy pandas bytes conversion:

pip install --pre \
  --extra-index-url https://pypi.fury.io/lancedb/ \
  --extra-index-url https://pypi.fury.io/lance-format/ \
  "lancedb[pylance]"

uv pip install --prerelease allow \
  --index https://pypi.fury.io/lancedb/ \
  --index https://pypi.fury.io/lance-format/ \
  --index-strategy unsafe-best-match \
  "lancedb[pylance]"

The LanceDB preview index contains old pylance releases; current Lance previews are published to the lance-format index. By default, uv only considers versions from the first index containing a package, which can select an old pylance or backtrack to an older LanceDB release. unsafe-best-match searches all configured indexes, like pip; use it only with indexes you trust. Install the extra rather than an unconstrained pylance so LanceDB’s minimum supported version is enforced.

Pre-release builds receive the same level of testing as stable releases, but their availability is not guaranteed for more than 6 months after release. For real-world workloads, we recommend you use the latest stable release as far as possible.

Older x86_64 CPUs

The default lancedb wheel targets x86-64-haswell (AVX2 + FMA + F16C). On older x86_64 CPUs without AVX2 (Intel Sandy Bridge, Ivy Bridge, Westmere; AMD Bulldozer, Piledriver, Steamroller), import lancedb crashes with Illegal instruction.Install lancedb-compat instead. It exposes the same API (import lancedb still works), is built at the x86-64-v2 baseline, and uses runtime SIMD dispatch to still use AVX2, FMA, or AVX-512 when available:

pip install lancedb-compat

The two packages share the same lancedb/ namespace and conflict at install time, so pick one. To switch, uninstall the other first (pip uninstall lancedb && pip install lancedb-compat).

2. Connect to a database

Local directory

LanceDB creates an ex_lancedb directory to store your tables. No server is needed.

import lancedb

uri = "ex_lancedb"
db = lancedb.connect(uri)

import * as lancedb from "@lancedb/lancedb";

const db = await lancedb.connect("ex_lancedb");

use lancedb::connect;

let db = connect("ex_lancedb").execute().await?;

For Python’s async API, see Connecting to a database.

Object storage

Use an s3://, gs://, or az:// URI in place of the local path. See Object storage for credentials and provider-specific options.

import lancedb

uri = "s3://your-bucket/path"
# You can also use "gs://your-bucket/path" or "az://your-container/path".
db = lancedb.connect(uri)

import * as lancedb from "@lancedb/lancedb";

const uri = "s3://your-bucket/path";
// You can also use "gs://your-bucket/path" or "az://your-container/path".
const db = await lancedb.connect(uri);

use lancedb::connect;

let uri = "s3://your-bucket/path";
// You can also use "gs://your-bucket/path" or "az://your-container/path".
let db = connect(uri).execute().await?;

Enterprise

Use the database URI, API key, and endpoint provided by your administrator or the LanceDB team. Replace the placeholders below with those details. Use the deployment’s region if provided; otherwise keep us-east-1.

import lancedb

db = lancedb.connect(
    uri="db://your-database",
    api_key="your-api-key",
    region="us-east-1",
    host_override="https://your-endpoint",
    sql_host_override="grpc+tls://your-sql-endpoint:10026",
)

import * as lancedb from "@lancedb/lancedb";

const db = await lancedb.connect("db://your-database", {
  apiKey: "your-api-key",
  region: "us-east-1",
  hostOverride: "https://your-endpoint",
});

use lancedb::connect;

let db = connect("db://your-database")
    .api_key("your-api-key")
    .region("us-east-1")
    .host_override("https://your-endpoint")
    .execute()
    .await?;

In application code, load credentials from environment variables or a secrets manager. See connection settings for an example.

Need a deployment? Contact the LanceDB team. For differences between local and server execution, see LanceDB Enterprise.

3. Create a table and add data

Each row describes a frame from a robot video clip and stores an embedding alongside its video, caption, and quality stats.

These embeddings are made up for this example. In your application, use an embedding model to generate them from your data.

Frame records

data = [
    {
        "frame_id": "1",
        "video_id": "clip-1",
        "caption": "a robot arm reaching toward a cup",
        "stats": {"motion": 3, "sharpness": 4, "exposure": 4, "framing": 3},
        "embedding": [0.7, 0.1, 0.9, 0.7],
    },
    {
        "frame_id": "2",
        "video_id": "clip-1",
        "caption": "a two-finger gripper closing on a cup",
        "stats": {"motion": 5, "sharpness": 4, "exposure": 5, "framing": 2},
        "embedding": [0.2, 0.9, 0.4, 0.9],
    },
    {
        "frame_id": "3",
        "video_id": "clip-2",
        "caption": "a robot arm holding a cup steady",
        "stats": {"motion": 2, "sharpness": 3, "exposure": 3, "framing": 4},
        "embedding": [0.9, 0.1, 0.5, 0.4],
    },
]

const data = [
  {
    frame_id: "1",
    video_id: "clip-1",
    caption: "a robot arm reaching toward a cup",
    stats: { motion: 3, sharpness: 4, exposure: 4, framing: 3 },
    embedding: [0.7, 0.1, 0.9, 0.7],
  },
  {
    frame_id: "2",
    video_id: "clip-1",
    caption: "a two-finger gripper closing on a cup",
    stats: { motion: 5, sharpness: 4, exposure: 5, framing: 2 },
    embedding: [0.2, 0.9, 0.4, 0.9],
  },
  {
    frame_id: "3",
    video_id: "clip-2",
    caption: "a robot arm holding a cup steady",
    stats: { motion: 2, sharpness: 3, exposure: 3, framing: 4 },
    embedding: [0.9, 0.1, 0.5, 0.4],
  },
];

use std::sync::Arc;

use lancedb::arrow::arrow_array::types::Float32Type;
use lancedb::arrow::arrow_array::{
    FixedSizeListArray, Int8Array, LargeStringArray, RecordBatch, RecordBatchIterator,
    RecordBatchReader, StructArray,
};
use lancedb::arrow::arrow_schema::{DataType, Field, FieldRef, Fields, Schema};

// Define structs representing the data schema
#[derive(Debug, Clone)]
struct Stats {
    motion: i8,
    sharpness: i8,
    exposure: i8,
    framing: i8,
}

#[derive(Debug, Clone)]
struct Frame {
    frame_id: String,
    video_id: String,
    caption: String,
    stats: Stats,
    embedding: [f32; 4],
}

fn frames_schema() -> Arc<Schema> {
    Arc::new(Schema::new(vec![
        Field::new("frame_id", DataType::LargeUtf8, false),
        Field::new("video_id", DataType::LargeUtf8, false),
        Field::new("caption", DataType::LargeUtf8, false),
        Field::new(
            "stats",
            DataType::Struct(Fields::from(vec![
                Arc::new(Field::new("motion", DataType::Int8, false)),
                Arc::new(Field::new("sharpness", DataType::Int8, false)),
                Arc::new(Field::new("exposure", DataType::Int8, false)),
                Arc::new(Field::new("framing", DataType::Int8, false)),
            ])),
            false,
        ),
        Field::new(
            "embedding",
            DataType::FixedSizeList(Arc::new(Field::new("item", DataType::Float32, true)), 4),
            false,
        ),
    ]))
}

fn frames_to_reader(schema: Arc<Schema>, rows: &[Frame]) -> Box<dyn RecordBatchReader + Send> {
    let frame_ids =
        LargeStringArray::from_iter_values(rows.iter().map(|row| row.frame_id.as_str()));
    let video_ids =
        LargeStringArray::from_iter_values(rows.iter().map(|row| row.video_id.as_str()));
    let captions = LargeStringArray::from_iter_values(rows.iter().map(|row| row.caption.as_str()));

    let motion = Int8Array::from_iter_values(rows.iter().map(|row| row.stats.motion));
    let sharpness = Int8Array::from_iter_values(rows.iter().map(|row| row.stats.sharpness));
    let exposure = Int8Array::from_iter_values(rows.iter().map(|row| row.stats.exposure));
    let framing = Int8Array::from_iter_values(rows.iter().map(|row| row.stats.framing));
    let stats_fields: Vec<FieldRef> = vec![
        Arc::new(Field::new("motion", DataType::Int8, false)),
        Arc::new(Field::new("sharpness", DataType::Int8, false)),
        Arc::new(Field::new("exposure", DataType::Int8, false)),
        Arc::new(Field::new("framing", DataType::Int8, false)),
    ];
    let stats = StructArray::new(
        stats_fields.into(),
        vec![
            Arc::new(motion),
            Arc::new(sharpness),
            Arc::new(exposure),
            Arc::new(framing),
        ],
        None,
    );

    let embeddings = FixedSizeListArray::from_iter_primitive::<Float32Type, _, _>(
        rows.iter()
            .map(|row| Some(row.embedding.iter().copied().map(Some).collect::<Vec<_>>())),
        4,
    );

    let batch = RecordBatch::try_new(
        schema.clone(),
        vec![
            Arc::new(frame_ids),
            Arc::new(video_ids),
            Arc::new(captions),
            Arc::new(stats),
            Arc::new(embeddings),
        ],
    )
    .unwrap();

    Box::new(RecordBatchIterator::new(
        vec![Ok(batch)].into_iter(),
        schema,
    ))
}
let data = vec![
    Frame {
        frame_id: "1".to_string(),
        video_id: "clip-1".to_string(),
        caption: "a robot arm reaching toward a cup".to_string(),
        stats: Stats {
            motion: 3,
            sharpness: 4,
            exposure: 4,
            framing: 3,
        },
        embedding: [0.7, 0.1, 0.9, 0.7],
    },
    Frame {
        frame_id: "2".to_string(),
        video_id: "clip-1".to_string(),
        caption: "a two-finger gripper closing on a cup".to_string(),
        stats: Stats {
            motion: 5,
            sharpness: 4,
            exposure: 5,
            framing: 2,
        },
        embedding: [0.2, 0.9, 0.4, 0.9],
    },
    Frame {
        frame_id: "3".to_string(),
        video_id: "clip-2".to_string(),
        caption: "a robot arm holding a cup steady".to_string(),
        stats: Stats {
            motion: 2,
            sharpness: 3,
            exposure: 3,
            framing: 4,
        },
        embedding: [0.9, 0.1, 0.5, 0.4],
    },
];

Create a frames table from those records:

table = db.create_table("frames", data=data, mode="overwrite")

const table = await db.createTable("frames", data, { mode: "overwrite" });

use lancedb::database::CreateTableMode;

let schema = frames_schema();
let table = db
    .create_table("frames", frames_to_reader(schema.clone(), &data))
    .mode(CreateTableMode::Overwrite)
    .execute()
    .await
    .unwrap();

This example uses overwrite mode so you can run it again. It replaces any existing frames table in the database you connect to. Add one more frame:

table.add(
    [
        {
            "frame_id": "4",
            "video_id": "clip-2",
            "caption": "a gripper lowering a cup onto the table",
            "stats": {"motion": 5, "sharpness": 4, "exposure": 4, "framing": 2},
            "embedding": [0.3, 0.9, 0.6, 0.8],
        }
    ]
)

await table.add([
  {
    frame_id: "4",
    video_id: "clip-2",
    caption: "a gripper lowering a cup onto the table",
    stats: { motion: 5, sharpness: 4, exposure: 4, framing: 2 },
    embedding: [0.3, 0.9, 0.6, 0.8],
  },
]);

let more_data = vec![Frame {
    frame_id: "4".to_string(),
    video_id: "clip-2".to_string(),
    caption: "a gripper lowering a cup onto the table".to_string(),
    stats: Stats {
        motion: 5,
        sharpness: 4,
        exposure: 4,
        framing: 2,
    },
    embedding: [0.3, 0.9, 0.6, 0.8],
}];
table
    .add(frames_to_reader(schema.clone(), &more_data))
    .execute()
    .await
    .unwrap();

4. Search the table

Find the two embeddings closest to the query embedding. This example uses a fixed embedding; it does not call an embedding model.

# Search for frames similar to "a gripper closing on an object"
query_vector = [0.2, 0.8, 0.4, 0.9]

# Ensure you run `pip install pandas` beforehand
result = (
    table.search(query_vector)
    .select(["frame_id", "video_id", "caption", "_distance"])
    .limit(2)
    .to_pandas()
)
print(result)

// Search for frames similar to "a gripper closing on an object"
const queryVector = [0.2, 0.8, 0.4, 0.9];

const result = await table
  .search(queryVector)
  .select(["frame_id", "video_id", "caption", "_distance"])
  .limit(2)
  .toArray();
console.table(result);

use lancedb::arrow::IntoPolars;
use lancedb::query::{ExecutableQuery, QueryBase, Select};

// Search for frames similar to "a gripper closing on an object"
let query_vector = [0.2, 0.8, 0.4, 0.9];

let result = table
    .query()
    .nearest_to(&query_vector)
    .unwrap()
    .select(Select::Columns(vec![
        "frame_id".to_string(),
        "video_id".to_string(),
        "caption".to_string(),
        "_distance".to_string(),
    ]))
    .limit(2)
    .execute()
    .await
    .unwrap()
    .into_polars()
    .await
    .unwrap();
println!("{result:?}");

Frames 2 and 4 come back first — the gripper closing on the cup, then lowering it. Python displays a Pandas DataFrame, Rust displays a Polars DataFrame, and TypeScript displays an array of results. The _distance column measures how far each result is from your query embedding, with smaller values indicating a closer match.

5. Curate data with SQL

Step 5 needs LanceDB Enterprise: SQL is an Enterprise feature. If you’re using LanceDB Embedded, skip ahead to Flatten nested fields below.

LanceDB Enterprise accepts SQL over the same connection, for sampling, joins, and curation tasks that go beyond a single table scan. Draw a quick sample to look at the data:

sample = db.execute_query(
    "SELECT frame_id FROM frames TABLESAMPLE (2 ROWS)"
).read_all()
print(sample.to_pylist())

See SQL for the Flight SQL client and CLI alternatives to execute_query(). Now add a near-duplicate frame — frame 5 is one video frame after frame 2, with an almost identical embedding — and build the vector index that dedupe reads:

# Frame 5 is a near-duplicate of frame 2: the gripper is still closing,
# one video frame later, with an almost identical embedding.
table.add(
    [
        {
            "frame_id": "5",
            "video_id": "clip-1",
            "caption": "the two-finger gripper still closing on the cup",
            "stats": {"motion": 5, "sharpness": 4, "exposure": 5, "framing": 2},
            "embedding": [0.2, 0.9, 0.4, 0.89],
        }
    ]
)
from lancedb.index import IvfFlat

table.create_index(
    "embedding", config=IvfFlat(num_partitions=1, distance_type="l2")
)
table.wait_for_index(["embedding_idx"])

Collapse near-duplicates into a materialized view that stays current as new frames arrive, then read it back:

db.execute_query("""
    CREATE MATERIALIZED VIEW frames_deduped AS
    SELECT * FROM dedupe(
        'frames',
        'embedding',
        distance_threshold => 0.03
    )
""")

deduped = db.execute_query(
    "SELECT frame_id FROM frames_deduped ORDER BY frame_id"
).read_all()
print(deduped.to_pylist())

Frames 2 and 5 collapse into one representative — five rows become four. Which of the two ids survives is not guaranteed to stay the same across appends, so don’t treat it as a stable identifier for the cluster. See Deduplicate near-duplicate records for duplicate_pairs, resolution strategies, and tuning distance_threshold, and Statistical Sampling for percentage samples, repeatable draws, and sampling across a join.

6. Add a derived feature

Flatten nested fields

A UDF binds to top-level columns, so flatten the fields inside stats first. This is a plain eager add_columns call, so it works the same on LanceDB Embedded and LanceDB Enterprise — step 8 needs these columns either way:

table.add_columns(
    {
        "motion": "cast(stats.motion as float)",
        "sharpness": "cast(stats.sharpness as float)",
        "exposure": "cast(stats.exposure as float)",
        "framing": "cast(stats.framing as float)",
    }
)

The rest of this step needs LanceDB Enterprise — Functions are an Enterprise feature. If you’re using LanceDB Embedded, skip ahead to step 7.

Register a UDF as a Function, then call it to fill a column computed from those flattened fields. This fills every existing row once, immediately:

from lancedb import col, udf

@udf
def quality_score(
    motion: float, sharpness: float, exposure: float, framing: float
) -> float:
    return (motion + sharpness + exposure + framing) / 4.0

version = db.create_function_async(quality_score).wait()
table.add_columns(
    {
        "quality_score": version(
            motion=col("motion"),
            sharpness=col("sharpness"),
            exposure=col("exposure"),
            framing=col("framing"),
        )
    }
)
table.refresh_column("quality_score")

Read it back like any other column:

features = table.search().select(["frame_id", "video_id", "quality_score"]).limit(10).to_pandas()
print(features)

refresh_column fills only rows added since the last refresh and recomputes rows whose inputs changed, leaving unchanged rows alone. Run the same refresh as a background job instead, so it doesn’t block:

job = table.refresh_column_async("quality_score")
job.wait()

See Create a Function for UDFs that call external models or services, and Computed columns for the SQL-expression form that doesn’t need a registered Function.

7. Store multimodal data

Store images, audio, or other binary data alongside vectors and metadata, in a column of raw bytes. This creates a new table from one image file — replace the path with one of your own:

from pathlib import Path

image_bytes = Path("data/sample-frame.jpg").read_bytes()

multimodal_table = db.create_table(
    "frame_images",
    data=[
        {
            "frame_id": "thumb-1",
            "caption": "Sample frame thumbnail",
            "image": image_bytes,
            "embedding": [0.9, 0.1, 0.5, 0.4],
        }
    ],
    mode="overwrite",
)

const arrow = await import("apache-arrow");
const { readFile } = await import("node:fs/promises");

const imageBytes = await readFile("data/sample-frame.jpg");
const imageSchema = new arrow.Schema([
  new arrow.Field("frame_id", new arrow.Utf8()),
  new arrow.Field("caption", new arrow.Utf8()),
  new arrow.Field("image", new arrow.Binary()),
  new arrow.Field(
    "embedding",
    new arrow.FixedSizeList(
      4,
      new arrow.Field("item", new arrow.Float32(), true),
    ),
  ),
]);
const imageData = lancedb.makeArrowTable(
  [
    {
      frame_id: "thumb-1",
      caption: "Sample frame thumbnail",
      image: imageBytes,
      embedding: [0.9, 0.1, 0.5, 0.4],
    },
  ],
  { schema: imageSchema },
);
const multimodalTable = await db.createTable(
  "frame_images",
  imageData,
  { mode: "overwrite" },
);

use lancedb::arrow::arrow_array::BinaryArray;

let image_bytes = std::fs::read("data/sample-frame.jpg").unwrap();

let image_schema = Arc::new(Schema::new(vec![
    Field::new("frame_id", DataType::LargeUtf8, false),
    Field::new("caption", DataType::LargeUtf8, false),
    Field::new("image", DataType::Binary, false),
    Field::new(
        "embedding",
        DataType::FixedSizeList(Arc::new(Field::new("item", DataType::Float32, true)), 4),
        false,
    ),
]));
let image_embeddings = [[0.9_f32, 0.1, 0.5, 0.4]];
let image_batch = RecordBatch::try_new(
    image_schema.clone(),
    vec![
        Arc::new(LargeStringArray::from_iter_values(["thumb-1"])),
        Arc::new(LargeStringArray::from_iter_values([
            "Sample frame thumbnail",
        ])),
        Arc::new(BinaryArray::from_iter_values([image_bytes.as_slice()])),
        Arc::new(
            FixedSizeListArray::from_iter_primitive::<Float32Type, _, _>(
                image_embeddings
                    .iter()
                    .map(|vector| Some(vector.iter().copied().map(Some).collect::<Vec<_>>())),
                4,
            ),
        ),
    ],
)
.unwrap();
let image_reader: Box<dyn RecordBatchReader + Send> = Box::new(RecordBatchIterator::new(
    vec![Ok(image_batch)].into_iter(),
    image_schema,
));
let multimodal_table = db
    .create_table("frame_images", image_reader)
    .mode(CreateTableMode::Overwrite)
    .execute()
    .await
    .unwrap();

See Multimodal data types for larger payloads and the Blob v2 column type, which avoids loading binary data on every scan.

8. Train on your data

Read table data into PyTorch, projecting and shuffling only the columns a training step needs:

import torch
from lancedb.streaming import StreamingDataset

dataset = StreamingDataset(table, columns=["motion", "sharpness"], shuffle_seed=42)
dataloader = torch.utils.data.DataLoader(dataset, batch_size=1024, num_workers=0)

StreamingDataset wraps a table for PyTorch training reads; columns projects before reading, so a training step touches only the columns it selects, not unused ones such as caption or embedding. It works the same way against LanceDB Embedded and LanceDB Enterprise. See PyTorch Data Loader for shuffling, transforms, resuming, and multi-worker loading, and Permutation for the lower-level row-ordering primitive it’s built on.

Where to go next

Basic table operations

Create, update, and manage your tables.

Build a RAG app

Use retrieved context in a language model application.

Indexing

Speed up queries on larger datasets.

Embedding models

Generate vectors from your own data.

Introduction

Getting Started

Concepts

Data Ingestion

Search

Data Curation

Feature Engineering

Training

Management

Maintenance

Security

Deployment

SQL Reference

1. Install LanceDB

Python pre-release builds

Older x86_64 CPUs

2. Connect to a database

Local directory

Object storage

Enterprise

3. Create a table and add data

4. Search the table

5. Curate data with SQL

6. Add a derived feature

Flatten nested fields

7. Store multimodal data

8. Train on your data

Where to go next

Basic table operations

Build a RAG app

Indexing

Embedding models

​1. Install LanceDB

Python pre-release builds

Older x86_64 CPUs

​2. Connect to a database

​Local directory

​Object storage

​Enterprise

​3. Create a table and add data

​4. Search the table

​5. Curate data with SQL

​6. Add a derived feature

​Flatten nested fields

​7. Store multimodal data

​8. Train on your data

​Where to go next

Basic table operations

Build a RAG app

Indexing

Embedding models

1. Install LanceDB

2. Connect to a database

Local directory

Object storage

Enterprise

3. Create a table and add data

4. Search the table

5. Curate data with SQL

6. Add a derived feature

Flatten nested fields

7. Store multimodal data

8. Train on your data

Where to go next