Skip to main content
Enterprise only

Job handles

A job handle lets you track an operation, such as an index build or a computed column refresh. Use it to check progress, wait for completion and inspect failures. Methods such as Python’s create_index_async and TypeScript’s createIndexAsync return job handles. In LanceDB Embedded, the work runs in your application process; in Enterprise, it runs on server workers or through the SQL query backend. A returned handle can represent an operation that has already finished, so check its status before waiting or requesting cancellation. See the operation’s API reference for when the method returns.

Cached properties and live calls

Python job properties describe the last observation; reading them does not contact the server. Submission need not fetch the full registry record, so some properties can initially be None. For example, poll a registry handle and then retrieve its final details:
The four lifecycle strings below are the usual SDK contract; newer server states may pass through an older client. Do not infer execution progress from state alone. Structured progress methods and queue fields require matching SDK/server support; see refresh admission.

Submit and wait for a job

  1. Submit the operation. Keep the returned job handle. For an Enterprise registry job, also keep its ID if another process needs to look it up later.
  2. Check progress. Read the current status while the operation runs.
  3. Wait for completion. Check that the job succeeded before using its result. If it failed or was cancelled, inspect the failure details and any results it committed before stopping.
In Python, job.status() returns the current state: job.status() checks the current state without waiting for completion. It returns failed or cancelled for unsuccessful jobs. Use job.wait() to wait for completion and raise if the job fails or is cancelled. Both Job.wait() and AsyncJob.wait() retry transient connection and read errors with capped exponential backoff within the caller’s timeout. Without a timeout, retries continue indefinitely. If the timeout expires while disconnected, the wait raises ConnectionError with the job ID and instructions to resume waiting. A timeout while connected raises TimeoutError. The job may still be running; timing out does not cancel it.
After connectivity returns, reopen an Enterprise registry job by its saved ID and continue waiting on the existing operation:
For SQL-backed jobs, call wait() again on the original handle. These jobs cannot be reopened with db.open_job(); see SQL-backed job handles.

Inspect and cancel jobs

Reopening a job retrieves the existing operation; it does not restart failed work. Treat the ID as an opaque identifier and pass it unchanged to the jobs API.
List jobs on a connection, paging through the results:
list_jobs yields JobInfo summaries, newest creation time first, rather than live Job handles. Open an ID to inspect its full record. page_limit bounds each server request, not the total yielded items, and page_token resumes a listing. It is a lazy iterator: iteration performs requests and can raise request errors. Use list(db.list_jobs(...)) to collect all remaining results; there is no .to_list() method.
Job filters and synchronous Listing indexing, slicing, and len() require a Python SDK release that includes these APIs. Older SDKs expose only page_token and page_limit; collect results with list(...) before indexing or counting them. Filtering also requires a matching Enterprise server that supports the requested filters.
Filter jobs on the server before pagination with table, state, job_type or job_types, and creation-time bounds. table names a table in namespace_path (the root by default) and includes all its branches. Requests stay scoped to the connected database. Creation times are Unix milliseconds: created_at_gte_millis is inclusive and created_at_lt_millis is exclusive. Choose either job_type or job_types. The state filter uses registry states: in_progress, succeeded, failed, canceled, and timed_out. These differ from the SDK lifecycle labels such as running and finished returned in JobInfo.state.
Synchronous indexing and slicing refer to the remaining results without consuming them. A bounded forward slice such as jobs[:5] fetches only enough pages to reach its stop. len(jobs) returns the remaining count by fetching and caching every remaining result. Negative indexes and open-ended or reverse slices also fetch and cache all remaining results. These operations can raise request errors; later iteration drains their cache. AsyncListing remains an async iterator: use async for to read it. Listing.num_page_results() reports cached items available without another request, including items retained by indexing, slicing, or counting. Listing.page_token() reports the next request’s token. Drain all cached items before saving that token, or resuming can skip them. Resume with the same filters, for example db.list_jobs(table="events", state="in_progress", page_token=saved_token, page_limit=5). A None token after the final page can coexist with unread cached items. A resumed listing is not a transactionally frozen view of jobs submitted in the meantime. events.schema describes the Arrow response. Use events.to_pylist() for Python row dictionaries. Common event columns are event_seq (int64), state (string), updated_at_millis (int64), updated_by (string), and nullable emitted_from, emitted_by and claim_entity strings. Optional uint64 work counters such as rows_processed and diagnostic details depend on job type and server version. If a payload column is returned, its value is a nullable JSON string, not an already decoded Python dictionary. Empty results retain their schema. Events are newest first and belong to the requested job; child events are separate. An event count is not a completed-row count, and a running job can have no terminal result while producing events. For a Function refresh, wait() returns a RefreshColumnResult such as rows_assigned, rows_failed, rows_remaining, source_version and optional published_version. Other job types have different results. Do not parse the printed representation to recover a spec, result or progress measurement. For object-shaped registry data, job.spec and job.result are decoded Python dictionaries: access fields with keys, such as job.result.get("summary") after refreshing a successful Function job. The separate typed object from wait() uses attributes, such as result.rows_assigned. Event responses have row limits. An explicit limit can request more than the default, but it does not guarantee a complete event history.

Embedded and Enterprise execution

The following table describes LanceDB Embedded operations and registry jobs on LanceDB Enterprise. SQL-backed job handles follow the query lifecycle described below.

SQL-backed job handles

On Enterprise, several catalog operations return a job handle backed by the SQL query engine rather than the job registry: All of these report job.job_type as "query", with job.id holding the connection-scoped SQL query UUID — use describe_query(job.id) to inspect one. Unlike registry handles, they don’t expose event history, can’t be reopened through open_job() or reattached after a client restart, and cancellation depends on the SQL server’s support for it.

Object teardown

The “physical cleanup continues in the background” behavior above, for a dropped table, view, materialized view, Secret or Function, is a job type of its own: drop_object deletes the dropped object’s content, then clears it from the database’s cleanup backlog. You trigger it indirectly, by dropping one of those objects or a namespace — it runs as this job rather than inline with the call. Whether a dropped table runs as drop_object or as drop_table, which fences the table’s directory with its own marker instead, depends on how the deployment stores its catalog.

Learn more