# LanceDB: Embedded 0.41.0 Documentation ## Documentation ### Introduction - [LanceDB](https://docs.lancedb.com/oss/v0-41-0/index.md): Multimodal lakehouse for AI. - [LanceDB Embedded](https://docs.lancedb.com/oss/v0-41-0/basics/lancedb-embedded.md): The open-source, embedded retrieval library that runs inside your application process. - [LanceDB Enterprise](https://docs.lancedb.com/oss/v0-41-0/enterprise/index.md): Features and benefits of LanceDB Enterprise. - [Embedded vs Enterprise](https://docs.lancedb.com/oss/v0-41-0/basics/embedded-vs-enterprise.md): A feature-by-feature comparison between LanceDB Embedded and LanceDB Enterprise. - [About this version](https://docs.lancedb.com/oss/v0-41-0/about-this-version.md): The release and sources of the LanceDB Embedded 0.41.0 documentation. ### Getting Started - [Quickstart](https://docs.lancedb.com/oss/v0-41-0/quickstart.md): Install LanceDB, create a table, and run your first vector search. - [SDKs](https://docs.lancedb.com/oss/v0-41-0/basics/sdk.md): Choose a LanceDB SDK and find the API reference for your language. - [SQL](https://docs.lancedb.com/oss/v0-41-0/basics/sql.md): Query LanceDB Enterprise with SQL from Python or an Arrow Flight SQL client. - [Console](https://docs.lancedb.com/oss/v0-41-0/enterprise/console.md): A web UI for running SQL, browsing tables, and monitoring jobs on LanceDB Enterprise. - [Agentic skills](https://docs.lancedb.com/oss/v0-41-0/basics/agentic-skills.md): Give a coding agent a current, maintained reference for the LanceDB APIs. - [Support](https://docs.lancedb.com/oss/v0-41-0/deployment/support.md): Where to get help: community Discord and email for LanceDB Embedded, dedicated support for LanceDB Enterprise. ### Concepts - [Lance format](https://docs.lancedb.com/oss/v0-41-0/basics/lance.md): The open format for multimodal data behind LanceDB. - [LanceDB Lakehouse](https://docs.lancedb.com/oss/v0-41-0/basics/catalog.md): LanceDB Lakehouse objects are organized into a catalog of databases, namespaces and tables -- connect to a database to work with them. - [Job Framework](https://docs.lancedb.com/oss/v0-41-0/basics/jobs.md): Track background work from submission to completion. - [Real Time engine (RTE)](https://docs.lancedb.com/oss/v0-41-0/tables/realtime-engine.md): Absorb small, frequent writes cheaply and make them queryable within milliseconds. - [Consistency](https://docs.lancedb.com/oss/v0-41-0/tables/consistency.md): Understand when readers see changes and choose the freshness your application needs. - [Guarantees](https://docs.lancedb.com/oss/v0-41-0/basics/guarantees.md): Understand table commits, concurrent writes and the boundaries of recovery. - [Limits](https://docs.lancedb.com/oss/v0-41-0/basics/limits.md): Find fixed constraints, configurable bounds and what happens when you reach them. - [Naming rules](https://docs.lancedb.com/oss/v0-41-0/basics/naming.md): Choose portable names for your data objects and namespace paths. ### Data Ingestion - [Ingesting data](https://docs.lancedb.com/oss/v0-41-0/ingestion/index.md): Choose how to create a table, append records or change existing data. - [Creating a table](https://docs.lancedb.com/oss/v0-41-0/ingestion/create-table.md): Create a table from records or a schema and choose input formats. - [Insert data](https://docs.lancedb.com/oss/v0-41-0/ingestion/insert.md): Append records, load file-backed data or batches, and verify ingestion. - [Update rows](https://docs.lancedb.com/oss/v0-41-0/ingestion/update.md): Select existing rows, change values or expressions, and verify the changes. - [Delete rows](https://docs.lancedb.com/oss/v0-41-0/ingestion/delete.md): Select and delete records, verify the remaining rows, and understand storage retention. - [RTE Write (Upsert)](https://docs.lancedb.com/oss/v0-41-0/ingestion/realtime-write.md): Enable the real time engine on a table and absorb small, frequent writes cheaply, each one committed in milliseconds. - [Define Storage Integration](https://docs.lancedb.com/oss/v0-41-0/ingestion/define-storage-integration.md): Set up a reusable credential for AWS, GCP, Azure or HuggingFace before importing data from it. - [Attach Database](https://docs.lancedb.com/oss/v0-41-0/ingestion/migrate-from-embedded.md): Point Enterprise at an existing database in object storage with ATTACH DATABASE, without copying data. - [Import Datasets](https://docs.lancedb.com/oss/v0-41-0/ingestion/import.md): Create a table directly from a HuggingFace dataset or an existing Lance dataset in object storage, with one CREATE TABLE AS SELECT. - [Distributed Write](https://docs.lancedb.com/oss/v0-41-0/enterprise/distributed-write.md): How LanceDB Enterprise distributes a CTAS import's writes across plan executors. - [Distributed RTE Write](https://docs.lancedb.com/oss/v0-41-0/enterprise/distributed-real-time-write.md): How Enterprise scales RTE writes horizontally across a dedicated serving tier, and the throughput limits that come with it. ### Search - [Search](https://docs.lancedb.com/oss/v0-41-0/search/index.md): Choose vector, full-text or hybrid search and return the records your application needs. - [Vector Search](https://docs.lancedb.com/oss/v0-41-0/search/vector-search.md): Search by vector similarity, choose a distance metric and tune your queries. - [Full-Text Search (FTS)](https://docs.lancedb.com/oss/v0-41-0/search/full-text-search.md): Find records by words, phrases or partial matches using a full-text index. - [Hybrid Search](https://docs.lancedb.com/oss/v0-41-0/search/hybrid-search.md): Combine vector similarity and keyword matches in a single ranked result set. - [Metadata Filtering in LanceDB](https://docs.lancedb.com/oss/v0-41-0/search/filtering.md): Use predicates to select rows and control when filters apply to search. - [Reranking Search Results](https://docs.lancedb.com/oss/v0-41-0/reranking/index.md): Choose a reranker, apply it to search candidates and evaluate relevance and latency. - [RTE reads](https://docs.lancedb.com/oss/v0-41-0/search/realtime-read.md): Query a table with the real time engine enabled and get writes back within milliseconds of being written, merged transparently with the base table. - [Distributed RTE Read](https://docs.lancedb.com/oss/v0-41-0/enterprise/distributed-real-time-read.md): How a query combines the real time engine's fresh tier with the ordinary distributed query-serving path, machine to machine. - [Distributed Search](https://docs.lancedb.com/oss/v0-41-0/enterprise/distributed-search.md): How LanceDB Enterprise distributes index execution, data reads, and caching across query nodes and executors. - [Data Page Cache](https://docs.lancedb.com/oss/v0-41-0/enterprise/data-page-cache.md): How LanceDB Enterprise caches table data pages to avoid re-reading object storage. - [Tiered Index Cache](https://docs.lancedb.com/oss/v0-41-0/enterprise/tiered-index-cache.md): How LanceDB Enterprise caches deserialized index structures across memory and local disk. - [Cache Prewarm](https://docs.lancedb.com/oss/v0-41-0/enterprise/cache-prewarm.md): Populate the tiered index cache ahead of a request, instead of filling it lazily on first read. ### Data Curation - [Data curation](https://docs.lancedb.com/oss/v0-41-0/curation/index.md): Prepare datasets for search, evaluation and training by profiling, selecting and combining records. - [Inspect Datasets](https://docs.lancedb.com/oss/v0-41-0/curation/inspect-datasets.md): Query a table, a HuggingFace dataset, or a dataset in object storage the same way, with plain SELECT, WHERE, projection and LIMIT. - [Create a Function](https://docs.lancedb.com/oss/v0-41-0/curation/create-function.md): Register Python as a column: CPU, GPU, and remote-service Functions, called once per row or once per batch. - [Chunk records into multiple rows](https://docs.lancedb.com/oss/v0-41-0/curation/chunking.md): Expand one source row into many derived rows -- documents into passages, videos into clips, images into tiles. - [Statistical Sampling](https://docs.lancedb.com/oss/v0-41-0/curation/statistical-sampling.md): Draw a random subset of a table with SQL TABLESAMPLE, by rows or by fragments. - [Selective Sampling](https://docs.lancedb.com/oss/v0-41-0/curation/selective-sampling.md): Fetch specific rows by row ID, row address or offset instead of drawing a random fraction. - [Profiling a dataset](https://docs.lancedb.com/oss/v0-41-0/curation/aggregate.md): Measure coverage, missing values and class balance before selecting a dataset. - [Joining datasets](https://docs.lancedb.com/oss/v0-41-0/curation/join.md): Combine records with labels or metadata, and check for missing or duplicate matches. - [Blob Late Materialization](https://docs.lancedb.com/oss/v0-41-0/curation/blob-late-materialization.md): Deferring a wide BlobV2 or vector column's read until after a join has already narrowed down which rows survive. - [Distributed Shuffle](https://docs.lancedb.com/oss/v0-41-0/curation/distributed-shuffle.md): How a SQL query's intermediate results move between plan executors for a join, aggregate or sort. - [Ranking](https://docs.lancedb.com/oss/v0-41-0/curation/windowing.md): Choose records with explicit ranking rules and deterministic tie handling. - [Cluster records by index partition](https://docs.lancedb.com/oss/v0-41-0/curation/clustering.md): Group records by the partition an existing IVF vector index already assigns them to, without training a new clustering model. - [Deduplicate near-duplicate records](https://docs.lancedb.com/oss/v0-41-0/curation/dedupe.md): Collapse records that are only approximately alike, judged by embedding distance, without a pairwise scan. - [Scope curation to what changed](https://docs.lancedb.com/oss/v0-41-0/curation/changes.md): Process only the rows added since your last curation run, instead of rescanning the whole table. - [Create a view](https://docs.lancedb.com/oss/v0-41-0/feature-engineering/view.md): Define a derived dataset as a saved query, with no storage or refresh of its own. - [Vend a Secret to a Function](https://docs.lancedb.com/oss/v0-41-0/curation/vend-secret.md): Bind a Secret to a Function by reference, and how its value reaches running code during SQL evaluation and refresh. ### Feature Engineering - [Feature engineering](https://docs.lancedb.com/oss/v0-41-0/feature-engineering/index.md): Derive columns and tables from data already in the lakehouse, without moving it somewhere else to compute. - [Create a computed column](https://docs.lancedb.com/oss/v0-41-0/feature-engineering/add-computed-column.md): Declare a column from a SQL expression. Nothing is evaluated until you refresh it. - [Refresh a computed column](https://docs.lancedb.com/oss/v0-41-0/feature-engineering/refresh-column.md): Populate a computed column and update it when its inputs change. - [Create a materialized view](https://docs.lancedb.com/oss/v0-41-0/feature-engineering/materialized-views.md): Store the result of a query as a table, ready to query, index and search. - [Refresh a materialized view](https://docs.lancedb.com/oss/v0-41-0/feature-engineering/refresh-materialized-view.md): Bring a materialized view up to date after its source data changes. - [Incremental Refresh](https://docs.lancedb.com/oss/v0-41-0/enterprise/incremental-refresh.md): Why refreshing a computed column only recomputes fragments that hold a row whose inputs actually changed. - [Auto Refresh](https://docs.lancedb.com/oss/v0-41-0/enterprise/auto-refresh.md): Let a computed column or materialized view refresh itself on a schedule, instead of calling refresh explicitly. - [Function Refresh Checkpointing](https://docs.lancedb.com/oss/v0-41-0/enterprise/checkpointing.md): How a Function refresh's progress survives a worker failure or restart, part by part. - [Adaptive Execution](https://docs.lancedb.com/oss/v0-41-0/enterprise/adaptive-execution.md): How a Function refresh's part size, resource class, worker count and checkpoint frequency adjust themselves to observed behavior. #### Geneva - [Multimodal Feature Engineering with Geneva](https://docs.lancedb.com/oss/v0-41-0/geneva/index.md): Learn how to do multimodal feature engineering in LanceDB Enterprise to transform raw data into meaningful features for AI models. - [What is Feature Engineering?](https://docs.lancedb.com/oss/v0-41-0/geneva/overview/index.md): Learn how to transform raw data into meaningful features for AI models using LanceDB's feature engineering capabilities. Scale your feature engineering workflows with distributed processing and UDFs. - [Getting Started](https://docs.lancedb.com/oss/v0-41-0/geneva/getting-started.md): Connect to LanceDB Enterprise, define a UDF, and run a distributed backfill — from a notebook or a script. - [End-to-End Example](https://docs.lancedb.com/oss/v0-41-0/geneva/end-to-end.md): A complete Geneva example on LanceDB Enterprise — create a table, backfill computed columns, and build a materialized view with embeddings. - [Geneva Python SDK](https://docs.lancedb.com/oss/v0-41-0/geneva/reference.md): LanceDB Feature Engineering Python SDK Reference ##### Transforms - [Understanding Transforms](https://docs.lancedb.com/oss/v0-41-0/geneva/udfs/index.md): Understand the three types of user-defined functions in Geneva — UDFs, chunkers (scalar UDTFs), and batch UDTFs — and when to use each. - [User-Defined Functions (UDFs)](https://docs.lancedb.com/oss/v0-41-0/geneva/udfs/udfs.md): Define 1:1 transforms that add computed columns to your tables — embeddings, enrichment, scoring, and more. - [Chunkers (Scalar UDTFs)](https://docs.lancedb.com/oss/v0-41-0/geneva/udfs/scalar-udtfs.md): Use chunkers (scalar UDTFs) for 1:N row expansion — split videos into clips, chunk documents, or tile images with automatic parent column inheritance and incremental refresh. - [Batch User-Defined Table Functions (UDTFs)](https://docs.lancedb.com/oss/v0-41-0/geneva/udfs/batch-udtfs.md): Use batch UDTFs for N:M transformations like deduplication, clustering, and aggregation across entire tables or partitions. - [Error Handling in Geneva UDFs](https://docs.lancedb.com/oss/v0-41-0/geneva/udfs/error_handling.md): Learn how configure retry, skip, and fail behaviors for UDFs. - [Profiling Stateful UDF Memory](https://docs.lancedb.com/oss/v0-41-0/geneva/udfs/profiling-memory.md): Find memory leaks and runaway peak usage in stateful UDFs with memray, before they cause worker OOMs in production. - [Blob Types in Geneva UDFs](https://docs.lancedb.com/oss/v0-41-0/geneva/udfs/blobs.md): Learn how to work with Lance Blobs in Geneva UDFs for handling large binary objects efficiently with lazy reading capabilities. ##### Built-in Transforms - [Built-in LLM and Embedding UDFs](https://docs.lancedb.com/oss/v0-41-0/geneva/udfs/providers/index.md) - [OpenAI](https://docs.lancedb.com/oss/v0-41-0/geneva/udfs/providers/openai.md) - [Gemini](https://docs.lancedb.com/oss/v0-41-0/geneva/udfs/providers/gemini.md) - [Sentence Transformers](https://docs.lancedb.com/oss/v0-41-0/geneva/udfs/providers/sentence-transformers.md) ##### Running Jobs - [Geneva Jobs](https://docs.lancedb.com/oss/v0-41-0/geneva/jobs/index.md): Learn about Geneva's job execution framework for distributed feature engineering workflows. - [Backfilling](https://docs.lancedb.com/oss/v0-41-0/geneva/jobs/backfilling.md): Learn how to trigger backfill operations to populate column values in your LanceDB table using Geneva's distributed framework. - [Bulk Loading & Updating Columns](https://docs.lancedb.com/oss/v0-41-0/geneva/jobs/bulk-load-columns.md): Load or update column data from external sources (Parquet, Lance, IPC) into your LanceDB table using a primary-key join. - [Materialized Views with UDFs](https://docs.lancedb.com/oss/v0-41-0/geneva/jobs/materialized-views.md): Learn how to use Geneva's materialized view feature to declaratively manage batch updates of expensive operations using UDFs. - [Advanced Job Configuration](https://docs.lancedb.com/oss/v0-41-0/geneva/jobs/advanced-job-configuration.md): Pin the dependency manifest a transform's distributed job runs with using @udf, @chunker, and @udtf. - [Job Lifecycle](https://docs.lancedb.com/oss/v0-41-0/geneva/jobs/lifecycle.md): Understanding how Geneva jobs work, their lifecycle states, and how to monitor and manage them. - [Backfill Conflicts](https://docs.lancedb.com/oss/v0-41-0/geneva/jobs/conflicts.md): Learn how Geneva handles conflicts during backfill operations and what to do when they occur. - [Distributed Job Performance and Cluster Sizing](https://docs.lancedb.com/oss/v0-41-0/geneva/jobs/performance.md): Learn how to tune Geneva distributed job performance by scaling compute resources and balancing write bandwidth. - [Job Metrics (Diagnostics)](https://docs.lancedb.com/oss/v0-41-0/geneva/jobs/job_metrics.md): Use Geneva Job Metrics to monitor and troubleshoot jobs in real time. - [Geneva Console](https://docs.lancedb.com/oss/v0-41-0/geneva/jobs/console.md) - [Troubleshooting Geneva Jobs](https://docs.lancedb.com/oss/v0-41-0/geneva/jobs/troubleshooting.md): Diagnose and fix common issues when running Geneva backfills, materialized views, and other distributed jobs. - [Advanced Execution Contexts](https://docs.lancedb.com/oss/v0-41-0/geneva/jobs/contexts.md): Configure the distributed execution backend — clusters and dependency manifests — for Geneva jobs on object-storage and local-filesystem connections. ##### Deployment - [Deploy Geneva using Helm](https://docs.lancedb.com/oss/v0-41-0/geneva/deployment/helm.md): Learn how to deploy Geneva on Kubernetes using the Geneva Helm Chart - [Job and Session Startup Optimizations](https://docs.lancedb.com/oss/v0-41-0/geneva/jobs/startup.md): Learn how to optimize Geneva job and session startup times for faster interactive development and production workflows. - [Dependency Verification](https://docs.lancedb.com/oss/v0-41-0/geneva/deployment/dependency-verification.md): Diagnose and resolve package version mismatches between local and distributed worker environments. - [Advanced Configuration](https://docs.lancedb.com/oss/v0-41-0/geneva/udfs/advanced-configuration.md): Learn about environment variables for configuring Geneva behavior. - [Manual Deployment on Kubernetes](https://docs.lancedb.com/oss/v0-41-0/geneva/deployment/index.md): Learn how to deploy Geneva on Kubernetes using KubeRay for distributed feature engineering workflows on GKE and EKS. - [Troubleshooting Geneva Deployments](https://docs.lancedb.com/oss/v0-41-0/geneva/deployment/troubleshooting.md): Learn how to diagnose and resolve common issues with Geneva deployments, including version compatibility, permissions, and serialization errors. ### Training - [PyTorch Data Loader](https://docs.lancedb.com/oss/v0-41-0/training/data-loading.md): Stream, shuffle, transform, and resume model training data with StreamingDataset. - [Training Cache](https://docs.lancedb.com/oss/v0-41-0/training/training-cache.md): Connect the PyTorch data loader to LanceDB Enterprise and training reads automatically benefit from the plan executor's data page cache. - [Permutation](https://docs.lancedb.com/oss/v0-41-0/training/permutation.md): The low-level, customizable row ordering that StreamingDataset is built on. - [Observability](https://docs.lancedb.com/oss/v0-41-0/training/observability.md): Pipeline counters for diagnosing StreamingDataset bottlenecks. - [Distributed Training](https://docs.lancedb.com/oss/v0-41-0/training/distributed-training.md): Shard StreamingDataset across ranks and workers, with elastic determinism across topologies. - [Checkpointing](https://docs.lancedb.com/oss/v0-41-0/training/checkpointing.md): Save StreamingDataset's read position alongside your model, so a failed run can pick back up. - [Resuming](https://docs.lancedb.com/oss/v0-41-0/training/resuming.md): Load a saved StreamingDataset checkpoint back, on the same topology or a different one. ### Management - [Management](https://docs.lancedb.com/oss/v0-41-0/management/index.md): Create, list, and drop the objects in a catalog, from Python, TypeScript, or SQL. - [Catalog](https://docs.lancedb.com/oss/v0-41-0/management/catalog.md): The unnamed root of an Enterprise deployment: initial access, the superuser bootstrap, and where privileges apply. - [Database](https://docs.lancedb.com/oss/v0-41-0/management/database.md): Connect to LanceDB and configure storage, credentials and read consistency. - [Namespaces](https://docs.lancedb.com/oss/v0-41-0/management/namespace.md): Grouping tables into a hierarchy, and addressing them through it. - [Tables](https://docs.lancedb.com/oss/v0-41-0/management/table.md): The object most operations act on: creating, opening, renaming, cloning and dropping one. - [Compress Function output columns](https://docs.lancedb.com/oss/v0-41-0/enterprise/function-column-compression.md): Choose Lance storage compression before refreshing a Function output column. - [Views](https://docs.lancedb.com/oss/v0-41-0/management/view.md): A saved query, evaluated when read. - [Materialized view](https://docs.lancedb.com/oss/v0-41-0/management/materialized-view.md): Create, open, list and drop a table whose contents are defined by a query. - [Secret](https://docs.lancedb.com/oss/v0-41-0/management/secret.md): Create, rotate, list, describe and drop a named credential that Functions bind by reference. - [Functions](https://docs.lancedb.com/oss/v0-41-0/management/function.md): Register a Python function once and apply it as a column, for work SQL cannot express. - [Storage Integration](https://docs.lancedb.com/oss/v0-41-0/management/storage-integration.md): A namespace-scoped, reusable credential for reading or importing from storage outside the deployment's own tables. ### Maintenance - [Indexing Data](https://docs.lancedb.com/oss/v0-41-0/indexing/index.md): Optimize search performance with LanceDB using vector indexes, full-text search, scalar indexes, and more. - [Intelligent Auto-Maintenance](https://docs.lancedb.com/oss/v0-41-0/enterprise/intelligent-auto-maintenance.md): How Enterprise continuously tracks table activity in the background and decides what maintenance -- compaction, cleanup, indexing, remap catch-up -- a table needs next. - [Vector Indexing](https://docs.lancedb.com/oss/v0-41-0/indexing/vector-index.md): Build and manage LanceDB vector indexes. - [Full-Text Search (FTS) Indexing](https://docs.lancedb.com/oss/v0-41-0/indexing/fts-index.md): Create and tune BM25-based full-text search indexes in LanceDB. - [Scalar Indexing](https://docs.lancedb.com/oss/v0-41-0/indexing/scalar-index.md): Learn how to use scalar indexes in LanceDB for efficient metadata filtering and query optimization. - [Incremental Indexing](https://docs.lancedb.com/oss/v0-41-0/indexing/incremental-indexing.md): Keep an index up-to-date as new data arrives, without a full rebuild. - [Distributed Indexing](https://docs.lancedb.com/oss/v0-41-0/enterprise/distributed-indexing.md): How LanceDB Enterprise distributes a vector index build across a fleet of workers, stage by stage. - [GPU Indexing](https://docs.lancedb.com/oss/v0-41-0/indexing/gpu-indexing.md): Build a vector index faster with GPU acceleration, manually in LanceDB Embedded or automatically in LanceDB Enterprise. - [Compaction and cleanup](https://docs.lancedb.com/oss/v0-41-0/maintenance/compaction.md): Merge small files, prune old versions, and fold new rows into existing indexes. - [Real Time Engine compaction](https://docs.lancedb.com/oss/v0-41-0/enterprise/realtime-engine-compaction.md): How Enterprise automatically folds a Real Time Engine shard's SSTables into the base table and catches its indexes up. - [Distributed Compaction](https://docs.lancedb.com/oss/v0-41-0/enterprise/distributed-compaction.md): How LanceDB Enterprise distributes compaction across a fleet of job workers. - [Advanced Conflict Resolution](https://docs.lancedb.com/oss/v0-41-0/enterprise/advanced-conflict-resolution.md): How compaction and indexing stay safe to run concurrently, using Lance's fragment reuse index. - [Garbage Collection](https://docs.lancedb.com/oss/v0-41-0/maintenance/cleanup.md): Pruning old versions and the files only they referenced. ### Security - [Authentication](https://docs.lancedb.com/oss/v0-41-0/enterprise/authentication.md): Authentication modes for LanceDB Enterprise: API keys and OAuth credentials. - [Authorization](https://docs.lancedb.com/oss/v0-41-0/governance/authorization.md): Who may do what: an overview of LanceDB Enterprise's access control model. - [Principals](https://docs.lancedb.com/oss/v0-41-0/governance/principals.md): Users, groups, and roles: where each comes from, and how a GRANT or REVOKE statement names one. - [Privileges](https://docs.lancedb.com/oss/v0-41-0/governance/privileges.md): Every privilege, which object it's granted on, and the GRANT/REVOKE syntax that covers each one. - [Audit Log](https://docs.lancedb.com/oss/v0-41-0/governance/audit-log.md): The durable record of access attempts in the cluster: its schema, how to query it, and how to grant access. - [Compliance](https://docs.lancedb.com/oss/v0-41-0/enterprise/security.md): Learn about LanceDB Enterprise security features and best practices. ### Deployment - [Horizontal Scaling](https://docs.lancedb.com/oss/v0-41-0/enterprise/horizontal-scaling.md): How LanceDB Enterprise spreads serving capacity for a table across replica groups, and replicates it within each one. - [Monitor LanceDB with OpenTelemetry](https://docs.lancedb.com/oss/v0-41-0/deployment/opentelemetry.md): Export LanceDB object store request counts, bytes, latency, errors, and throttles to any OpenTelemetry backend. - [Kubernetes deployment guide](https://docs.lancedb.com/oss/v0-41-0/enterprise/deployment/kubernetes.md): Learn how LanceDB Enterprise deploys on Kubernetes with private networking, object storage, and workload identity. - [Multicloud](https://docs.lancedb.com/oss/v0-41-0/deployment/multicloud.md): Keep deployment orchestration in your cloud while dispatching compute-heavy work, like GPU UDF execution, to run in a different cloud. - [Slurm](https://docs.lancedb.com/oss/v0-41-0/deployment/slurm.md): Run LanceDB Enterprise's own pods as Slurm-scheduled jobs, and reach a Neocloud's GPU capacity through Slurm in the multicloud case. ### SQL Reference - [SQL reference](https://docs.lancedb.com/oss/v0-41-0/sql/index.md): SQL statements, expressions and data types supported by LanceDB Enterprise. - [SQL data types](https://docs.lancedb.com/oss/v0-41-0/sql/data-types.md): How Lance columns appear to SQL, including the types SQL has no name for. #### DDL - [Database](https://docs.lancedb.com/oss/v0-41-0/sql/ddl/database.md): Create, describe, list, attach, and drop databases with SQL. - [Namespace](https://docs.lancedb.com/oss/v0-41-0/sql/ddl/namespace.md): Create, alter, describe, list, and drop namespace paths with SQL. - [Table](https://docs.lancedb.com/oss/v0-41-0/sql/ddl/table.md): CREATE, DESCRIBE, ALTER, CLONE and DROP TABLE. - [View](https://docs.lancedb.com/oss/v0-41-0/sql/ddl/view.md): Create, describe, list, and drop saved queries with SQL. - [Materialized View](https://docs.lancedb.com/oss/v0-41-0/sql/ddl/materialized-view.md): CREATE MATERIALIZED VIEW and SHOW MATERIALIZED VIEWS; refreshing is a REFRESH statement. - [Secret](https://docs.lancedb.com/oss/v0-41-0/sql/ddl/secret.md): CREATE, ALTER, DROP, SHOW and DESCRIBE SECRET, and granting privileges on Secrets. - [Function](https://docs.lancedb.com/oss/v0-41-0/sql/ddl/function.md): Create, describe, alter, list, and drop registered functions with SQL. - [Storage Integration](https://docs.lancedb.com/oss/v0-41-0/sql/ddl/storage-integration.md): CREATE, DESCRIBE, SHOW and DROP STORAGE INTEGRATION: a namespace-scoped, reusable credential for reading or importing from external storage. - [Index](https://docs.lancedb.com/oss/v0-41-0/sql/ddl/index.md): CREATE INDEX and DROP INDEX: one column, with the algorithm chosen by USING. - [Job](https://docs.lancedb.com/oss/v0-41-0/sql/ddl/job.md): SHOW JOBS, SHOW JOB EVENTS, DESCRIBE JOB, SHOW ERRORS, and controlling a running job with PAUSE, RESUME and CANCEL. - [Grant](https://docs.lancedb.com/oss/v0-41-0/sql/ddl/grant.md): GRANT and REVOKE: privileges on every catalog object, from a database down to a Function. #### DML - [Insert](https://docs.lancedb.com/oss/v0-41-0/sql/dml/insert.md): INSERT INTO, including INSERT ... SELECT to move rows server-side. - [Update](https://docs.lancedb.com/oss/v0-41-0/sql/dml/update.md): UPDATE ... SET ... WHERE, and why it does not recompute derived columns. - [Delete](https://docs.lancedb.com/oss/v0-41-0/sql/dml/delete.md): DELETE FROM ... WHERE: rows are marked, not erased. - [Copy](https://docs.lancedb.com/oss/v0-41-0/sql/dml/copy.md): COPY ... TO: writing a query result to object storage as files. - [REFRESH COLUMN](https://docs.lancedb.com/oss/v0-41-0/sql/dml/refresh-column.md): Fill computed outputs and select rows to recompute. - [REFRESH MATERIALIZED VIEW](https://docs.lancedb.com/oss/v0-41-0/sql/dml/refresh-materialized-view.md): Bringing a materialized view's stored result up to date. - [REFRESH](https://docs.lancedb.com/oss/v0-41-0/sql/dml/refresh.md): Filling computed columns and bringing materialized views up to date. #### DQL - [SELECT](https://docs.lancedb.com/oss/v0-41-0/sql/dql/select.md): Projection, filtering, ordering, and composing queries out of other queries. - [Subqueries](https://docs.lancedb.com/oss/v0-41-0/sql/dql/subqueries.md): Queries inside queries: scalar, IN, EXISTS, correlated, and derived tables. - [Joins](https://docs.lancedb.com/oss/v0-41-0/sql/dql/join.md): Combining rows from more than one table, including across namespaces. - [Operators and literals](https://docs.lancedb.com/oss/v0-41-0/sql/dql/operators.md): Comparison, logical and arithmetic operators, literals, and precedence. - [Functions](https://docs.lancedb.com/oss/v0-41-0/sql/dql/functions.md): Where to find each kind of function, and the dialect version they come from. - [Scalar functions](https://docs.lancedb.com/oss/v0-41-0/sql/dql/scalar-functions.md): The per-row function set: math, string, regex, date/time, array, struct, and what Lance adds. - [Aggregation](https://docs.lancedb.com/oss/v0-41-0/sql/dql/aggregation.md): GROUP BY, HAVING, and the aggregate function set. - [Window functions](https://docs.lancedb.com/oss/v0-41-0/sql/dql/window.md): Compute over a set of rows related to the current one, without collapsing them. - [Random sampling](https://docs.lancedb.com/oss/v0-41-0/sql/dql/sampling.md): TABLESAMPLE: draw a random subset of a table, by rows or by fragments. - [Changes since a version](https://docs.lancedb.com/oss/v0-41-0/sql/dql/changes.md): CHANGES: read the rows appended to a table since an earlier version. - [Full-Text Search](https://docs.lancedb.com/oss/v0-41-0/sql/dql/fts.md): The fts() table function: match, fuzzy, phrase, boolean, boost and multi-match queries from SQL. - [CONTAINS_TOKENS](https://docs.lancedb.com/oss/v0-41-0/sql/dql/contains-tokens.md): The contains_tokens() scalar function: an index-accelerated keyword predicate usable directly in WHERE. - [IVF_PARTITION](https://docs.lancedb.com/oss/v0-41-0/sql/dql/ivf-partition.md): The ivf_partition() scalar function: assign each row to its trained IVF index partition. - [DEDUPE](https://docs.lancedb.com/oss/v0-41-0/sql/dql/dedupe.md): The dedupe() table function: one representative row per near-duplicate group, scanned over the index. - [DUPLICATE_PAIRS](https://docs.lancedb.com/oss/v0-41-0/sql/dql/duplicate-pairs.md): The duplicate_pairs() table function: every close pair in a vector column, scanned over the index. - [MinHash SQL functions](https://docs.lancedb.com/oss/v0-41-0/enterprise/sql-minhash.md): Create lexical text signatures and probabilistic duplicate candidates. - [Vector distance functions](https://docs.lancedb.com/oss/v0-41-0/sql/dql/vector-distance.md): cosine_distance, distance, exact_distance and inner_product: per-row vector distance, and the ORDER BY rewrite into vector search. - [BLOB_DESCRIPTOR](https://docs.lancedb.com/oss/v0-41-0/sql/dql/blob-descriptor.md): The blob_descriptor() scalar function: a BlobV2 column's payload location, without reading it. - [GENERATE_SERIES](https://docs.lancedb.com/oss/v0-41-0/sql/dql/generate-series.md): The generate_series() table function: rows from a range. - [Explain](https://docs.lancedb.com/oss/v0-41-0/sql/dql/explain.md): EXPLAIN, EXPLAIN ANALYZE and EXPLAIN VERBOSE: reading and measuring a plan from SQL.