> ## Documentation Index
> Fetch the complete documentation index at: https://docs.lancedb.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Job Framework

> Track background work from submission to completion.

Enterprise only

## Job handles

A job handle lets you track an operation, such as an index build or a computed
column refresh. Use it to check progress, wait for completion and inspect
failures.

Methods such as Python's `create_index_async` and TypeScript's
`createIndexAsync` return job handles. In LanceDB Embedded, the work runs in your application
process; in Enterprise, it runs on server workers or through the SQL query
backend. A returned handle can represent an operation that has already finished,
so check its status before
waiting or requesting cancellation. See the operation's API reference for when
the method returns.

```python Python icon="python" theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
job = tbl.create_index_async("vector", replace=True)

print(job.id)
print(job.job_type)
print(job.status())
```

## Cached properties and live calls

Python job properties describe the last observation; reading them does not
contact the server. Submission need not fetch the full registry record, so
some properties can initially be `None`.

| Surface | Contract |
| - | - |
| `state` | Last observed lifecycle state. It does not update on its own. |
| `job_type`, `creation_ms` | Cached job kind and creation time in epoch milliseconds. |
| `spec` | Complete, job-specific submitted specification. Read this separately from the bounded `repr(job)` display. |
| `result` | Cached, job-specific reported data; it is not necessarily the typed object returned by `wait()`. |
| `failure` | Cached `JobFailureInfo`, when the server reports a terminal failure. |
| `status()` | Live lifecycle request. For registry jobs it also caches the full record. A failed state is returned without raising a job-failure exception; request errors can still raise. |
| `refresh()` | Fetch and cache the current record without waiting for completion. |
| `events(...)` | Separate live request returning a `pyarrow.Table` of recorded events; it does not refresh the properties. |
| `wait(...)` | Poll until completion and return the operation's typed result, or raise on failure/cancellation. Refresh afterward when you need the complete final registry record. |

For example, poll a registry handle and then retrieve its final details:

```python theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
import time

state = job.status()
while state == "running":
    print(job.state)
    time.sleep(2)
    state = job.status()

result = job.wait()
job.refresh()
print(result)
print(job.spec)
```

The four lifecycle strings below are the usual SDK contract; newer server
states may pass through an older client. Do not infer execution progress from
`state` alone. Structured progress methods and queue fields require matching
SDK/server support; see [refresh admission](https://docs.lancedb.com/enterprise/v1-14-0/feature-engineering/refresh-column.md#admission).

## Submit and wait for a job

1. **Submit the operation.** Keep the returned job handle. For an Enterprise
   registry job, also keep its ID if another process needs to look it up later.
2. **Check progress.** Read the current status while the operation runs.
3. **Wait for completion.** Check that the job succeeded before using its
   result. If it failed or was cancelled, inspect the failure details and any
   results it committed before stopping.

In Python, `job.status()` returns the current state:

| State | Meaning |
| - | - |
| `running` | The operation is in progress. |
| `finished` | The operation completed successfully. |
| `failed` | The operation failed. |
| `cancelled` | The operation was cancelled. |

`job.status()` checks the current state without waiting for completion.
It returns `failed` or `cancelled` for unsuccessful jobs.

Use `job.wait()` to wait for completion and raise if the job fails or is
cancelled. Both `Job.wait()` and `AsyncJob.wait()` retry transient connection
and read errors with capped
exponential backoff within the caller's timeout. Without a timeout, retries
continue indefinitely.

If the timeout expires while disconnected, the wait raises `ConnectionError`
with the job ID and instructions to resume waiting. A timeout while connected
raises `TimeoutError`. The job may still be running; timing out does not cancel it.

```python Python icon="python" theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
from datetime import timedelta
from lancedb.exceptions import JobFailedError, JobCancelledError

job = tbl.create_index_async("vector", replace=True)
job_id = job.id  # Save the ID to reopen an Enterprise registry job later.

try:
    result = job.wait(timeout=timedelta(minutes=10))
except JobFailedError:
    print(f"job failed: {job.failure}")
except JobCancelledError:
    print("job was cancelled")
except ConnectionError as exc:
    print(exc)  # The job may still be running on the server.
except TimeoutError:
    print(f"still running: {job.status()}")
```

After connectivity returns, reopen an Enterprise registry job by its saved ID
and continue waiting on the existing operation:

```python theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
job = db.open_job(job_id)
result = job.wait(timeout=timedelta(minutes=10))
```

For SQL-backed jobs, call `wait()` again on the original handle. These jobs
cannot be reopened with `db.open_job()`; see [SQL-backed job handles](#sql-backed-job-handles).

## Inspect and cancel jobs

| Task | What to do |
| - | - |
| Inspect progress | Check status and refresh the handle to retrieve details. Enterprise job events can help explain ongoing work or failures. |
| Reattach from another process | Open an existing server job with its ID, such as Python's `db.open_job(job_id)`. The server must still retain the job record. |
| Diagnose a failure | Inspect failure details and relevant events. Enterprise SQL also provides `SHOW ERRORS`. |
| Request cancellation | Call the job's cancellation method. Cancellation may take time, and work already committed is not rolled back. |

Reopening a job retrieves the existing operation; it does not restart failed
work. Treat the ID as an opaque identifier and pass it unchanged to the jobs API.

```python Python icon="python" theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
job = db.open_job(job_id)  # raises JobNotFoundError if unknown
job.refresh()              # pull the latest status from the server
print(job.status())

events = job.events(limit=100)
print(events.to_pylist())

job.cancel()
```

List jobs on a connection, paging through the results:

```python Python icon="python" theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
for info in db.list_jobs(page_limit=50):
    print(info.job_id, info.job_type, info.state)
```

`list_jobs` yields `JobInfo` summaries, newest creation time first, rather than
live `Job` handles. Open an ID to inspect its full record. `page_limit` bounds
each server request, not the total yielded items, and `page_token` resumes a
listing. It is a lazy iterator: iteration performs requests and can raise
request errors. Use `list(db.list_jobs(...))` to collect all remaining results;
there is no `.to_list()` method.

> [!NOTE]
>
> Job filters and synchronous `Listing` indexing, slicing, and `len()` require
> a Python SDK release that includes these APIs. Older SDKs expose only
> `page_token` and `page_limit`; collect results with `list(...)` before indexing
> or counting them. Filtering also requires a matching Enterprise server that
> supports the requested filters.

Filter jobs on the server before pagination with `table`, `state`, `job_type`
or `job_types`, and creation-time bounds. `table` names a table in
`namespace_path` (the root by default) and includes all its branches. Requests
stay scoped to the connected database. Creation times are Unix milliseconds:
`created_at_gte_millis` is inclusive and `created_at_lt_millis` is exclusive.
Choose either `job_type` or `job_types`.

The `state` filter uses registry states: `in_progress`, `succeeded`, `failed`,
`canceled`, and `timed_out`. These differ from the SDK lifecycle labels such
as `running` and `finished` returned in `JobInfo.state`.

```python theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
jobs = db.list_jobs(table="events", state="in_progress", page_limit=5)
first_five = jobs[:5]
for info in first_five:
    print(info.job_id, info.state)
```

Synchronous indexing and slicing refer to the remaining results without
consuming them. A bounded forward slice such as `jobs[:5]` fetches only enough
pages to reach its stop. `len(jobs)` returns the remaining count by fetching
and caching every remaining result. Negative indexes and open-ended or reverse
slices also fetch and cache all remaining results. These operations can raise
request errors; later iteration drains their cache. `AsyncListing` remains an
async iterator: use `async for` to read it.

`Listing.num_page_results()` reports cached items available without another
request, including items retained by indexing, slicing, or counting.
`Listing.page_token()` reports the next request's token. Drain all cached
items before saving that token, or resuming can skip them. Resume with the same
filters, for example `db.list_jobs(table="events", state="in_progress",
page_token=saved_token, page_limit=5)`. A `None` token after the final page can
coexist with unread cached items. A resumed listing is not a transactionally
frozen view of jobs submitted in the meantime.

`events.schema` describes the Arrow response. Use `events.to_pylist()` for
Python row dictionaries. Common event columns are `event_seq` (int64), `state`
(string), `updated_at_millis` (int64), `updated_by` (string), and nullable
`emitted_from`, `emitted_by` and `claim_entity` strings. Optional uint64 work
counters such as `rows_processed` and diagnostic details depend on job type
and server version. If a `payload` column is returned, its value is a nullable
JSON string, not an already decoded Python dictionary.
Empty results retain their schema. Events are newest first and belong to the
requested job; child events are separate. An event count is not a completed-row
count, and a running job can have no terminal `result` while producing events.

For a Function refresh, `wait()` returns a `RefreshColumnResult` such as
`rows_assigned`, `rows_failed`, `rows_remaining`, `source_version` and optional
`published_version`. Other job types have different results. Do not parse the
printed representation to recover a spec, result or progress measurement.
For object-shaped registry data, `job.spec` and `job.result` are decoded Python
dictionaries: access fields with keys, such as `job.result.get("summary")`
after refreshing a successful Function job. The separate typed object from
`wait()` uses attributes, such as `result.rows_assigned`.

Event responses have [row limits](https://docs.lancedb.com/enterprise/v1-14-0/basics/limits.md#jobs). An explicit limit can
request more than the default, but it does not guarantee a complete event
history.

## Embedded and Enterprise execution

The following table describes LanceDB Embedded operations and registry jobs on
LanceDB Enterprise. SQL-backed job handles follow the query lifecycle
described below.

| | Embedded | Enterprise registry job |
| - | - | - |
| Execution | In your application process | On server workers |
| Job identity | No server ID | A server ID for looking up the job |
| Event history | No server event history | Available through the jobs API |
| Reattaching after a client restart | No server job to reopen | Reopen by ID while the record is retained |

### SQL-backed job handles

On Enterprise, several catalog operations return a job handle backed by the
SQL query engine rather than the job registry:

| Operation | SDK methods | Behavior |
| - | - | - |
| Materialized views | `create_materialized_view_async()`, `refresh_async()`, `drop_materialized_view_async()` | Synchronous variants wait on the same SQL execution. Drop removes the catalog binding immediately; physical cleanup continues in the background. |
| Functions | Registration, `drop_function_async()` | Registration waits for catalog publication before returning the immutable FunctionVersion. Drop confirms catalog removal while cleanup continues separately. |
| Secrets | Create, alter, list, describe, drop | Credential values are bound as SQL parameters, never embedded in SQL text or returned in results. `DESCRIBE SECRET` additionally exposes rotation history. |
| Views | Create, describe, list, drop | Create returns the stored definition and schema. Drop confirms catalog removal while cleanup continues separately. |

All of these report `job.job_type` as `"query"`, with `job.id` holding the
connection-scoped SQL query UUID -- use `describe_query(job.id)` to inspect
one. Unlike registry handles, they don't expose event history, can't be
reopened through `open_job()` or reattached after a client restart, and
cancellation depends on the SQL server's support for it.

### Object teardown

The "physical cleanup continues in the background" behavior above, for a
dropped table, view, materialized view, Secret or Function, is a job type of
its own: `drop_object` deletes the dropped object's content, then clears it
from the database's cleanup backlog. You trigger it indirectly, by dropping
one of those objects or a [namespace](https://docs.lancedb.com/enterprise/v1-14-0/management/namespace.md) -- it runs as
this job rather than inline with the call. Whether a dropped table runs as
`drop_object` or as [`drop_table`](https://docs.lancedb.com/enterprise/v1-14-0/management/table#drop), which fences the
table's directory with its own marker instead, depends on how the deployment
stores its catalog.

## Learn more

* [Incremental Indexing](https://docs.lancedb.com/enterprise/v1-14-0/indexing/incremental-indexing.md): submit index work.
* [Refresh a computed column](https://docs.lancedb.com/enterprise/v1-14-0/feature-engineering/refresh-column.md): populate derived values.
* [SQL job commands](https://docs.lancedb.com/enterprise/v1-14-0/sql/ddl/job.md): list jobs and inspect errors.
* [SDKs](https://docs.lancedb.com/enterprise/v1-14-0/basics/sdk.md): find the job API for your client.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.