> ## Documentation Index
> Fetch the complete documentation index at: https://docs.lancedb.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Storage Integration

> CREATE, DESCRIBE, SHOW and DROP STORAGE INTEGRATION: a namespace-scoped, reusable credential for reading or importing from external storage.

**Applies to:** Embedded ❌, Enterprise ✅

A storage integration is a namespace-scoped catalog object that lets a
query read from, or import from, a storage location outside the
deployment's own tables -- an S3, GCS or Azure prefix, or a HuggingFace
dataset repository -- without embedding a credential in the query itself.
It is addressed and scoped the same way a [Secret](https://docs.lancedb.com/sql/ddl/secret.md) is: a
1-3 part identifier, `name`, `namespace.name` or `database.namespace.name`,
with a nested namespace path `$`-joined into that middle part.

## CREATE STORAGE INTEGRATION

```sql SQL icon="database" theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
CREATE STORAGE INTEGRATION <name>
    STORAGE_PROVIDER = 'S3' | 'AZURE' | 'GCP' | 'HUGGINGFACE'
    STORAGE_PREFIX = '<uri-prefix>'
    [ <provider option> = <value> [...] ]
```

`STORAGE_PROVIDER` and `STORAGE_PREFIX` are required for every provider.
`STORAGE_PREFIX` is the URI prefix the integration covers -- a query only
matches an integration whose prefix is a prefix of the URI it reads.
Options ending in `_SECRET` take a [Secret](https://docs.lancedb.com/sql/ddl/secret.md) identifier, not
a quoted string: `name`, `namespace.name` or `database.namespace.name`, with a
nested namespace path `$`-joined in the middle part (``db.`team$prod`.key``).
Omitted parts default to the session's current database and namespace.
Creating the integration requires `USAGE` on each referenced Secret and on its
database and namespace. Passing an option a provider does not accept, such as
an Azure option on an `S3` integration, is refused.

### S3

```sql SQL icon="database" theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
CREATE STORAGE INTEGRATION raw
    STORAGE_PROVIDER = 'S3'
    STORAGE_PREFIX = 's3://customer-data/raw/'
    STORAGE_AWS_ROLE_ARN = 'arn:aws:iam::123456789012:role/lancedb-access'

CREATE STORAGE INTEGRATION raw
    STORAGE_PROVIDER = 'S3'
    STORAGE_PREFIX = 's3://customer-data/raw/'
    STORAGE_AWS_ACCESS_KEY_ID_SECRET = bootstrap.credentials.aws_key_id
    STORAGE_AWS_SECRET_ACCESS_KEY_SECRET = bootstrap.credentials.aws_key_secret
```

| Option | Type | Meaning |
| - | - | - |
| `STORAGE_AWS_ROLE_ARN` | string | an IAM role to assume, by cross-account trust (see [below](#trust)) |
| `STORAGE_AWS_ACCESS_KEY_ID_SECRET` | Secret ref | a static access key ID, paired with the secret key below |
| `STORAGE_AWS_SECRET_ACCESS_KEY_SECRET` | Secret ref | the matching static secret access key |
| `STORAGE_AWS_REGION` | string, optional | the bucket's region |
| `STORAGE_AWS_ENDPOINT` | string, optional | an S3-compatible endpoint URL (MinIO, Tigris, ...) instead of AWS itself |

Give either `STORAGE_AWS_ROLE_ARN` alone, or both key-pair options together
\-- mixing the two, or supplying only one half of the key pair, is refused.

### Azure

```sql SQL icon="database" theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
CREATE STORAGE INTEGRATION blobs
    STORAGE_PROVIDER = 'AZURE'
    STORAGE_PREFIX = 'az://customer-data/raw/'
    STORAGE_AZURE_STORAGE_ACCOUNT = 'customeraccount'
    STORAGE_AZURE_TENANT_ID_SECRET = bootstrap.credentials.tenant
    STORAGE_AZURE_CLIENT_ID_SECRET = bootstrap.credentials.client
    STORAGE_AZURE_CLIENT_SECRET_SECRET = bootstrap.credentials.credential
```

| Option | Type | Meaning |
| - | - | - |
| `STORAGE_AZURE_STORAGE_ACCOUNT` | string | the storage account name |
| `STORAGE_AZURE_TENANT_ID_SECRET` | Secret ref | Entra ID tenant |
| `STORAGE_AZURE_CLIENT_ID_SECRET` | Secret ref | service principal client ID |
| `STORAGE_AZURE_CLIENT_SECRET_SECRET` | Secret ref | service principal client secret |

All four are required together; there is no role-based alternative for
Azure.

### GCP

```sql SQL icon="database" theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
CREATE STORAGE INTEGRATION gcs
    STORAGE_PROVIDER = 'GCP'
    STORAGE_PREFIX = 'gs://customer-data/raw/'
    GCP_CREDENTIALS_JSON_SECRET = bootstrap.credentials.gcp
```

`GCP_CREDENTIALS_JSON_SECRET` -- note the option name has no `STORAGE_`
prefix, unlike every other provider's options -- names a Secret holding
the full service-account JSON key. It is required; GCP has no role-based
alternative here.

### HuggingFace

```sql SQL icon="database" theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
CREATE STORAGE INTEGRATION hub
    STORAGE_PROVIDER = 'HUGGINGFACE'
    STORAGE_PREFIX = 'hf://datasets/owner/private/data/'
    HUGGINGFACE_TOKEN_SECRET = bootstrap.credentials.hf_token
```

`HUGGINGFACE_TOKEN_SECRET` is the only HuggingFace option, and it is
required: creating an integration without it is refused. For anonymous
access, at HuggingFace's default anonymous rate limits, register no
integration for that prefix at all. See
[Inspect Datasets](https://docs.lancedb.com/curation/inspect-datasets.md) for the full
read and import workflow this integration unlocks.

### The S3 trust policy

Creating an `STORAGE_AWS_ROLE_ARN` integration does not finish the AWS
side of the setup. The result row's `trust_policy`, `permissions_policy`
and `configuration_instructions` columns -- also visible from
[`DESCRIBE`](#describe) -- are the IAM policy documents your role needs:
attach `trust_policy` as the role's trust relationship, scoped to
`aws_account_id`, `aws_iam_principal_arn` and `aws_external_id` from the
same row, and attach `permissions_policy` to grant that role read access
to the bucket. `access_verified` reports whether the deployment already
confirmed it can assume the role and read the prefix; `false` typically
means the trust policy has not been attached yet.

## Reading and importing through the integration

A query or `CREATE TABLE ... AS SELECT` can reference a matching URI
directly, once an integration covers it -- the integration is never named
in the statement. What the URI reads depends on the provider:

* **HuggingFace** (`hf://datasets/...`) reads Parquet files from a dataset
  repository. See [Inspect Datasets](https://docs.lancedb.com/curation/inspect-datasets.md)
  for the full workflow, including how a CTAS import distributes across
  plan executors.
* **S3, GCS and Azure** (`s3://`, `gs://`, `az://`) read an existing Lance
  dataset at that prefix, the same way as a managed table:

```sql SQL icon="database" theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
SELECT * FROM 's3://customer-data/raw/events.lance' LIMIT 10

CREATE TABLE events_copy AS
SELECT * FROM 's3://customer-data/raw/events.lance'
```

A CTAS import distributes the same way regardless of provider: disjoint
file and row-group ranges go to plan-executor workers, each writes
uncommitted Lance files, and a coordinator commits the table once every
worker finishes. See [`CREATE TABLE`](https://docs.lancedb.com/sql/ddl/table#create) for
`CREATE TABLE AS SELECT` syntax in general.

## DESCRIBE STORAGE INTEGRATION

```sql SQL icon="database" theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
DESCRIBE STORAGE INTEGRATION raw
DESC STORAGE INTEGRATION platform.hub
```

Returns one row with every column `CREATE` returns: the provider, prefix,
the Secret references and plain-string options that apply to its
provider (the rest are null), `created_at_millis`, `access_verified`, and
\-- for an `STORAGE_AWS_ROLE_ARN` integration -- the generated AWS policy
documents from [above](#trust). No call ever returns a Secret's value,
only the reference naming it.

## SHOW STORAGE INTEGRATIONS

```sql SQL icon="database" theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
SHOW STORAGE INTEGRATIONS
SHOW STORAGE INTEGRATIONS IN platform
LIST STORAGE INTEGRATIONS
```

`LIST` is an alias for `SHOW` here. Returns one `name` column; `IN`
narrows the listing to one namespace.

## DROP STORAGE INTEGRATION

```sql SQL icon="database" theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
DROP STORAGE INTEGRATION raw
DROP STORAGE INTEGRATION IF EXISTS raw
```

Returns `dropped`, `true` or `false`. There is no `ALTER STORAGE
INTEGRATION`: to rotate a credential, rotate the Secret it references
(see [Secret](https://docs.lancedb.com/sql/ddl/secret.md)) rather than dropping and recreating the
integration. A storage integration an attached database still depends on
blocks the drop; detach the database first.

## GRANT and REVOKE

```sql SQL icon="database" theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
GRANT CREATE STORAGE INTEGRATION ON SCHEMA platform TO ROLE platform_admins
GRANT USAGE ON STORAGE INTEGRATION platform.hub TO ROLE research
GRANT OWNERSHIP ON STORAGE INTEGRATION platform.hub TO ROLE platform_admins
REVOKE USAGE ON STORAGE INTEGRATION platform.hub FROM ROLE research
```

`CREATE_STORAGE_INTEGRATION` on the namespace creates one; `USAGE` on the
integration itself lets a query match and read through it; `OWNERSHIP`
grants full control, including dropping it. See
[Privileges](https://docs.lancedb.com/governance/privileges.md#sql) for the full statement syntax,
every grantee form, and how a query picks a matching integration without
naming one.

## What to read next

> [card] [Inspect Datasets](https://docs.lancedb.com/curation/inspect-datasets.md)
>
> Query and import HuggingFace or object-storage datasets, the most common use of a storage integration today.

> [card] [Secret](https://docs.lancedb.com/sql/ddl/secret.md)
>
> The credential storage integration options reference.

> [card] [Privileges](https://docs.lancedb.com/governance/privileges.md)
>
> Every privilege, and the grant and revoke syntax that covers each one.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.