Enterprise only
Function registration, query execution and column refresh have different
lifecycles. Match your SDK and Enterprise server release before relying on
optional GPU discovery, progress fields or warm-worker reuse. The examples
use the public @function decorator; see
authoring guidance and
Function signatures.
Query-time cold starts
A query that invokes a Python Function can pay for admission, worker placement, image preparation, Python startup, dependency import, model download and model initialization before producing its first row. These are separate kinds of warmth:
Use a class Function to initialize expensive state once per instance and reuse
it across that instance’s rows/batches. Each instance owns its own model copy.
Supporting refresh deployments can reuse initialized instances across jobs;
do not assume a new refresh resets mutable instance state. Query-time SQL
execution has its own lifetime, and refresh-worker reuse does not promise a
resident query-serving pool. A cached image is not a warm-latency guarantee.
Measure time to first row separately from steady-state batch throughput, and
record server/SDK versions, GPU class, model/dependencies and whether image,
model files and initialized instances were retained. Do not extrapolate a
single warm measurement to an independently submitted query or cold worker.
Latency-sensitive query embeddings
Keep document embeddings in a stored Function column and refresh them during ingestion or backfill. Compute each interactive query vector in an already initialized application model or a separately operated embedding service, then send the numeric vector to LanceDB. For example, with a long-lived SentenceTransformer instance in the application:encoder once at application startup. Use the same model revision,
dimension, pooling and normalization as the stored document embeddings. A
separate service needs its own capacity, authentication and timeout management;
a thin HTTP Function does not make that service’s GPU part of LanceDB’s budget.
SQL Function calls remain useful for batch transformations, exploratory
queries and workloads that can tolerate startup. Precomputing document
embeddings does not remove a query-time model cold start if each search still
calls a remote GPU Function to embed the query string.
GPU placement and budgets
Discover the configured classes with Pythondb.list_gpu_classes(), SQL
SHOW GPU CLASSES, or REST GET /v1/gpu/classes. These require supporting
SDK/server versions and System Usage permission. The response describes the
operator’s configured GPU resource name, optional model and memory per device,
per-worker device ceiling and worker resources. It is configuration, not live
inventory or a guarantee of currently available capacity. Missing metadata is
unknown; a missing per-worker limit leaves the aggregate budget as the
adaptive-growth ceiling.
The deployment operator configures available classes and their per-worker
limits. Choose a class supported by your deployment and a model that fits its
worker shape.
Choose a returned class when registering a Function with
@function(gpu=True, gpu_class="h100"), or override the class for a refresh:
gpu_count caps devices across the Function worker fleet. It does not
request two GPUs in one Python process, guarantee two workers, or configure
your model library’s tensor parallelism. gpu=True requests GPU execution;
it does not describe model memory requirements. A deployment’s adaptive worker
shapes and maximum devices per worker are distinct from a supported explicit
per-Function multi-device request. Do not assume a proposed registration option
such as gpus_per_worker is available. Use a model that fits the supported
worker shape, or a separately managed model service, until your release
provides the required multi-device contract.
Inspect the Function environment report
for observed devices and libraries. Configured memory is per device; it does
not establish that one model can allocate the sum of every device’s memory.
Refresh admission and concurrency
Submission creates a job, then source preparation, dispatcher admission and worker startup happen separately. Poor table layout can require compaction before a refresh;WHERE does not promise to limit that prerequisite to only
the selected rows. An earlier column refresh can also block a later one.
Dispatcher concurrency, cluster worker capacity and automatic-refresh admission
are deployment-level limits. They are separate from a refresh’s aggregate
cpu_millis, memory_bytes, gpu_count budget and a Function’s call-level
max_concurrency. Ask your deployment operator about effective limits; defaults
do not guarantee available capacity or a universal supervisor cap. GPU quotas,
placement constraints and model startup can add waits after admission.
Distinguish waiting from execution
Use the Console’s job detail andSHOW JOBS. On matching
SDK/server versions, job.progress() exposes optional phase, queue_reason
and row/worker counters; see the detailed
refresh progress reference. Older deployments
can report only lifecycle running while work has not begun.
Missing counters and timestamps are unknown.
creation_ms is registry
creation time, while supervisor_started_at_millis is execution start. In a
finished job’s summary, their difference includes admission/queue time, not
only Python startup. DESCRIBE JOB is a completed-run report, not a live queue
ETA. Sampled worker readiness does not mean a model has initialized.
Request cancellation with job.cancel() or
CANCEL JOB, then observe the terminal state. Cancellation
also applies while waiting; it does not roll back outputs already committed.
A timeout in the client does not cancel the server job. Avoid cancelling and
resubmitting merely to get a new ID; the replacement encounters the same
admission constraints.