Job Framework

Applies to: Embedded ❌, Enterprise ✅

Job handles

A job handle lets you track an operation, such as an index build or a computed column refresh. Use it to check progress, wait for completion and inspect failures. Methods such as Python’s create_index_async and TypeScript’s createIndexAsync return job handles. In LanceDB Embedded, the work runs in your application process; in Enterprise, it runs on server workers or through the SQL query backend. A returned handle can represent an operation that has already finished, so check its status before waiting or requesting cancellation. See the operation’s API reference for when the method returns.

job = tbl.create_index_async("vector", replace=True)

print(job.id)
print(job.job_type)
print(job.status())

Submit and wait for a job

Submit the operation. Keep the returned job handle. For an Enterprise registry job, also keep its ID if another process needs to look it up later.
Check progress. Read the current status while the operation runs.
Wait for completion. Check that the job succeeded before using its result. If it failed or was cancelled, inspect the failure details and any results it committed before stopping.

In Python, job.status() reports the current state and job.wait() waits for completion, raising if the job fails or is cancelled. Both Job.wait() and AsyncJob.wait() retry transient connection and read errors with capped exponential backoff within the caller’s timeout. Without a timeout, retries continue indefinitely. If the timeout expires while disconnected, the wait raises ConnectionError with the job ID and instructions to resume waiting. A timeout while connected raises TimeoutError. The job may still be running; timing out does not cancel it.

from datetime import timedelta
from lancedb.exceptions import JobFailedError, JobCancelledError

job = tbl.create_index_async("vector", replace=True)
job_id = job.id  # Save the ID to reopen an Enterprise registry job later.

try:
    result = job.wait(timeout=timedelta(minutes=10))
except JobFailedError:
    print(f"job failed: {job.failure}")
except JobCancelledError:
    print("job was cancelled")
except ConnectionError as exc:
    print(exc)  # The job may still be running on the server.
except TimeoutError:
    print(f"still running: {job.status()}")

After connectivity returns, reopen an Enterprise registry job by its saved ID and continue waiting on the existing operation:

job = db.open_job(job_id)
result = job.wait(timeout=timedelta(minutes=10))

For SQL-backed jobs, call wait() again on the original handle. These jobs cannot be reopened with db.open_job(); see SQL-backed job handles.

Inspect and cancel jobs

Task	What to do
Inspect progress	Check status and refresh the handle to retrieve details. Enterprise job events can help explain ongoing work or failures.
Reattach from another process	Open an existing server job with its ID, such as Python’s `db.open_job(job_id)`. The server must still retain the job record.
Diagnose a failure	Inspect failure details and relevant events. Enterprise SQL also provides `SHOW ERRORS`.
Request cancellation	Call the job’s cancellation method. Cancellation may take time, and work already committed is not rolled back.

Reopening a job retrieves the existing operation; it does not restart failed work. Treat the ID as an opaque identifier and pass it unchanged to the jobs API.

job = db.open_job(job_id)  # raises JobNotFoundError if unknown
job.refresh()              # pull the latest status from the server
print(job.status())

events = job.events(limit=100)
print(events.to_pylist())

job.cancel()

List jobs on a connection, paging through the results:

for info in db.list_jobs(page_limit=50):
    print(info.job_id, info.job_type, info.state)

Event responses have row limits. An explicit limit can request more than the default, but it does not guarantee a complete event history.

Embedded and Enterprise execution

The following table describes LanceDB Embedded operations and registry jobs on LanceDB Enterprise. SQL-backed job handles follow the query lifecycle described below.

	Embedded	Enterprise registry job
Execution	In your application process	On server workers
Job identity	No server ID	A server ID for looking up the job
Event history	No server event history	Available through the jobs API
Reattaching after a client restart	No server job to reopen	Reopen by ID while the record is retained

SQL-backed job handles

On Enterprise, several catalog operations return a job handle backed by the SQL query engine rather than the job registry:

Operation	SDK methods	Behavior
Materialized views	`create_materialized_view_async()`, `refresh_async()`, `drop_materialized_view_async()`	Synchronous variants wait on the same SQL execution. Drop removes the catalog binding immediately; physical cleanup continues in the background.
Functions	Registration, `drop_function_async()`	Registration waits for catalog publication before returning the immutable FunctionVersion. Drop confirms catalog removal while cleanup continues separately.
Secrets	Create, alter, list, describe, drop	Credential values are bound as SQL parameters, never embedded in SQL text or returned in results. `DESCRIBE SECRET` additionally exposes rotation history.
Views	Create, describe, list, drop	Create returns the stored definition and schema. Drop confirms catalog removal while cleanup continues separately.

All of these report job.job_type as "query", with job.id holding the connection-scoped SQL query UUID — use describe_query(job.id) to inspect one. Unlike registry handles, they don’t expose event history, can’t be reopened through open_job() or reattached after a client restart, and cancellation depends on the SQL server’s support for it.

Object teardown

The “physical cleanup continues in the background” behavior above, for a dropped table, view, materialized view, Secret or Function, is a job type of its own: drop_object deletes the dropped object’s content, then clears it from the database’s cleanup backlog. You trigger it indirectly, by dropping one of those objects or a namespace — it runs as this job rather than inline with the call. Whether a dropped table runs as drop_object or as drop_table, which fences the table’s directory with its own marker instead, depends on how the deployment stores its catalog.

Learn more

Incremental Indexing: submit index work.
Refresh a computed column: populate derived values.
SQL job commands: list jobs and inspect errors.
SDKs: find the job API for your client.

Introduction

Getting Started

Concepts

Data Ingestion

Search

Data Curation

Feature Engineering

Training

Management

Maintenance

Security

Deployment

SQL Reference

Job handles

Submit and wait for a job

Inspect and cancel jobs

Embedded and Enterprise execution

SQL-backed job handles

Object teardown

Learn more

​Job handles

​Submit and wait for a job

​Inspect and cancel jobs

​Embedded and Enterprise execution

​SQL-backed job handles

​Object teardown

​Learn more

Job handles

Submit and wait for a job

Inspect and cancel jobs

Embedded and Enterprise execution

SQL-backed job handles

Object teardown

Learn more