Polars - LanceDB

LanceDB supports Polars, a blazingly fast DataFrame library for Python written in Rust. Under the hood, both Lance and Polars speak Arrow, so passing data back and forth stays zero-copy and ergonomic.

Create & Query a Table

Import the required libraries, including the optional Pydantic helpers if you plan to define schemas.

Python

import tempfile
from pathlib import Path

import lancedb
import polars as pl
from lancedb.pydantic import LanceModel, Vector

Build a Polars DataFrame, convert it to Arrow, and use it directly when creating a LanceDB table.

Python

birds = pl.DataFrame(
{
"text": ["phoenix", "sparrow"],
"vector": [
[0.1, 0.2, 0.3],
[0.8, 0.6, 0.5],
],
}
)
polars_db = lancedb.connect(str(Path(tempfile.mkdtemp()) / "polars-demo"))
polars_table = polars_db.create_table(
"birds", data=birds.to_arrow(), mode="overwrite"
)

Run vector search and keep the results as a Polars DataFrame for further processing or visualization.

Python

polars_results = (
polars_table.search([0.1, 0.2, 0.3])
.select(["text", "_distance"])
.limit(1)
.to_polars()
)
print(polars_results)

Work with LazyFrames

When you want to operate on the entire table (potentially larger than RAM), convert to a Polars LazyFrame so you can chain transformations without loading everything at once.

Python

lazy_frame = polars_table.to_polars().lazy()
print(lazy_frame.select(["text"]).collect())

Define Schemas with Pydantic

You can also describe your table via LanceModel and continue ingesting data from Polars. This is useful when multiple teams share a schema or when you want validation.

Python

class BirdModel(LanceModel):
text: str
vector: Vector(3)

schema_table = polars_db.create_table(
"birds_schema", schema=BirdModel, mode="overwrite"
)
schema_table.add(birds.to_dicts())

​Create & Query a Table

​Work with LazyFrames

​Define Schemas with Pydantic

Create & Query a Table

Work with LazyFrames

Define Schemas with Pydantic