Skip to main content
Applies to: Embedded ❌, Enterprise ✅ Refresh a computed column to fill new rows and recompute values whose inputs changed. The example below continues with the doubled column from the computed-column guide.
Computed columns require LanceDB Enterprise.
table.refresh_column(...) (sync) is also a local, blocking form, but it raises on an Enterprise connection — use refresh_column_async, shown above, or the SQL form instead. See Run a refresh on Enterprise below for what the returned job supports.
The refresh commits ordinary table versions (the values, then a metadata-only record of the inputs they were computed from), so versioning applies to it like any other write.
The SQL form is REFRESH COLUMN, with the same resource options.

​
What a refresh recomputes

Without a WHERE predicate, a refresh fills unassigned rows and recomputes values whose inputs changed since they were computed. Every other row is left alone, so a second refresh with nothing changed does nothing. This supports two common workflows. Appending is cheap. Rows added since the last refresh are filled by the next one; filled rows whose inputs are unchanged are skipped. Backfilling a column and then ingesting more data does not mean recomputing the backfill. A filled value catches up with its input at the next refresh. If doubled derives from x and a row’s x is later updated, doubled keeps its old value until the next refresh recomputes it. That is what makes the call idempotent and safe to retry. However, unlike a spreadsheet formula, nothing is recomputed until you refresh, so a derived column is stale between an update and the next refresh.

​
Automatic refresh

A computed column can refresh itself on a schedule instead of waiting for an explicit call here. See Auto Refresh for enabling it, the SQL and SDK forms for changing the setting, and how catch-up works when it’s re-enabled after being off.

​
Refreshing part of a table

Enterprise In SQL a refresh can be scoped to a predicate, which is useful for filling a slice first and checking it before committing to the whole table:
A filtered refresh recomputes every matching live row, including rows whose outputs are already current. Repeating the same predicate repeats those Function calls and their cost. To select only null outputs within a slice:
This skips non-null outputs even when their inputs changed, and can recompute outputs that were successfully computed as NULL. Use an unfiltered refresh to let LanceDB track which outputs need updating.
Enterprise job type: computed_refresh for a SQL-expression column, function_refresh for one filled by a registered function.

​
Run a refresh on Enterprise

Enterprise With an Enterprise cluster that enables computed columns, the refresh runs on the cluster. refresh_column raises there; use refresh_column_async, which returns a job you can wait on or poll instead of holding a connection open:
Poll with status(), resolve with wait(). status() is a point snapshot and does not raise, which is what makes it safe in a loop; wait() raises JobFailedError, JobCancelledError or TimeoutError. A long backfill can therefore be started, left, and checked on from a different process.

​
Configure workers and resources

SQL exposes the worker budget directly, which is the lever when a backfill is too slow or too expensive:
The resource values cap the Function workers together: this example allows up to 8 CPUs, 16 GiB of memory, and 2 GPUs across the fleet. They are not per-worker allocations or a guaranteed worker count. num_workers is only accepted for SQL-expression columns; registered Function columns reject it. When setting CPU or memory, supply all three budget fields, using gpu_count = 0 for a CPU-only Function. batch_size caps rows in one Function call; LanceDB adapts actual call and work sizes. For Function columns, the refresh defaults to error_policy = 'fail'. Set error_policy = 'skip' to write null outputs for failing rows and record them in the error store, subject to failure limits. See the SQL refresh options. For a column filled by a registered Function, the Python and Rust job APIs can override the Function’s GPU placement for this refresh:
gpu_count is the aggregate GPU budget for the refresh’s Function workers, not a request for that many GPUs per worker. It must be positive. gpu_class must name a class returned by list_gpu_classes(); class names are lowercase DNS labels. Omit either option to use the Function’s registered class or the server’s configured budget. list_gpu_classes() requires System Usage permission, and raises NotImplementedError when the server predates GPU class discovery.
The Rust GPU discovery and refresh controls are not in the latest Rust release. TypeScript does not expose GPU class discovery or placement; a tracking issue is needed before adding equivalent controls.

​
When a refresh fails

What a failure keeps depends on where the refresh ran. The local, blocking refresh_column writes the column in one commit, so a failure leaves the column as it was. A refresh job in LanceDB Enterprise — always refresh_column_async or REFRESH COLUMN there, never the blocking form — commits finished batches as it goes, so some values can remain after a failure. Either way, recover by fixing the cause and refreshing again. The new refresh fills rows that are still empty or whose inputs changed, and normally skips values that were kept. That is not guaranteed: a refresh can recompute values that were already written (for example when the record of the inputs they were computed from was not saved), so do not count on a retry costing only the rows that did not complete. For a column filled by a function, the usual cause is the worker environment rather than the data. See environment mismatches for the symptoms, which rarely name their cause.

​
Related guides

Job Framework

Waiting on, inspecting, and cancelling server-side work.

Refresh a materialized view

The same idea, for a whole derived table.

Auto Refresh

Refresh on a schedule instead of calling this yourself.