> ## Documentation Index
> Fetch the complete documentation index at: https://docs.lancedb.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Define Storage Integration

> Set up a reusable credential for AWS, GCP, Azure or HuggingFace before importing data from it.

**Applies to:** Embedded ❌, Enterprise ✅

Before you can [import data](https://docs.lancedb.com/ingestion/import.md) from external storage or
[attach an existing database](https://docs.lancedb.com/ingestion/migrate-from-embedded.md), define a
storage integration once per location. It is a namespace-scoped catalog
object that stores the credential -- neither operation names the
integration directly, LanceDB matches the URI you read against whichever
integration's prefix covers it.

## What each provider needs

| Provider | Credential | Notes |
| - | - | - |
| AWS S3 | An IAM role to assume, or a static access key pair | A role needs one extra step outside LanceDB: [trust policy setup](#trust). |
| Google Cloud Storage | A service account's JSON key | No role-based alternative. |
| Azure Blob Storage | A service principal's tenant, client ID and client secret | All three required together. |
| HuggingFace | An access token | Always required to create an integration. For anonymous access at HuggingFace's default rate limits, register no integration for that prefix instead. |

Every credential option takes a [Secret](https://docs.lancedb.com/management/secret.md) reference,
never the value directly -- create the Secret first.

## Create one

### AWS S3

Give either an IAM role to assume, or a static access key pair -- mixing
the two, or supplying only one half of the key pair, is refused.

```sql SQL icon="database" theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
CREATE STORAGE INTEGRATION raw
    STORAGE_PROVIDER = 'S3'
    STORAGE_PREFIX = 's3://customer-data/raw/'
    STORAGE_AWS_ROLE_ARN = 'arn:aws:iam::123456789012:role/lancedb-access'

CREATE STORAGE INTEGRATION raw
    STORAGE_PROVIDER = 'S3'
    STORAGE_PREFIX = 's3://customer-data/raw/'
    STORAGE_AWS_ACCESS_KEY_ID_SECRET = aws_key_id
    STORAGE_AWS_SECRET_ACCESS_KEY_SECRET = aws_key_secret
```

```python Python icon="python" theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
db.create_storage_integration(
    "raw", "s3://customer-data/raw/",
    storage_provider="S3",
    aws_role_arn="arn:aws:iam::123456789012:role/lancedb-access",
)

db.create_storage_integration(
    "raw", "s3://customer-data/raw/",
    storage_provider="S3",
    aws_access_key_id_secret="aws_key_id",
    aws_secret_access_key_secret="aws_key_secret",
)
```

```rust theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
use lancedb::storage_integration::CreateStorageIntegrationRequest;

let info = db.create_storage_integration(
    "raw",
    CreateStorageIntegrationRequest::s3(
        "s3://customer-data/raw/",
        "arn:aws:iam::123456789012:role/lancedb-access",
    ),
    &[],
).await?;
```

### Google Cloud Storage

```sql SQL icon="database" theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
CREATE STORAGE INTEGRATION gcs
    STORAGE_PROVIDER = 'GCP'
    STORAGE_PREFIX = 'gs://customer-data/raw/'
    GCP_CREDENTIALS_JSON_SECRET = gcp_service_account
```

```python Python icon="python" theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
db.create_storage_integration(
    "gcs", "gs://customer-data/raw/",
    storage_provider="GCP",
    gcp_credentials_json_secret="gcp_service_account",
)
```

The Secret holds the full service-account JSON key. GCP has no
role-based alternative.

### Azure Blob Storage

```sql SQL icon="database" theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
CREATE STORAGE INTEGRATION blobs
    STORAGE_PROVIDER = 'AZURE'
    STORAGE_PREFIX = 'az://customer-data/raw/'
    STORAGE_AZURE_STORAGE_ACCOUNT = 'customeraccount'
    STORAGE_AZURE_TENANT_ID_SECRET = azure_tenant
    STORAGE_AZURE_CLIENT_ID_SECRET = azure_client
    STORAGE_AZURE_CLIENT_SECRET_SECRET = azure_credential
```

```python Python icon="python" theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
db.create_storage_integration(
    "blobs", "az://customer-data/raw/",
    storage_provider="AZURE",
    azure_storage_account="customeraccount",
    azure_tenant_id_secret="azure_tenant",
    azure_client_id_secret="azure_client",
    azure_client_secret_secret="azure_credential",
)
```

All four options are required together; there is no role-based
alternative for Azure.

### HuggingFace

```sql SQL icon="database" theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
CREATE STORAGE INTEGRATION hub
    STORAGE_PROVIDER = 'HUGGINGFACE'
    STORAGE_PREFIX = 'hf://datasets/owner/private/data/'
    HUGGINGFACE_TOKEN_SECRET = hf_token
```

```python Python icon="python" theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
db.create_storage_integration(
    "hub", "hf://datasets/owner/private/data/",
    storage_provider="HUGGINGFACE",
    huggingface_token_secret="hf_token",
)
```

`HUGGINGFACE_TOKEN_SECRET` is required to create this integration; omitting
it is rejected, not treated as anonymous. For anonymous access at
HuggingFace's default rate limits, register no integration for that prefix
at all.

### Finish the AWS trust policy

An IAM role integration needs one step outside LanceDB: describing it
returns the `trust_policy` and `permissions_policy` documents your role
needs. Attach them in AWS, then describe the integration again and check
`access_verified` to confirm LanceDB can assume the role. A static access
key pair skips this step entirely.

See [Storage Integration](https://docs.lancedb.com/sql/ddl/storage-integration.md) for every
provider's complete option reference, S3's two credential modes in full,
and the output schema, or [Storage Integration](https://docs.lancedb.com/management/storage-integration.md)
for the Python and SQL CRUD operations at a glance.

## What to read next

> [card] [Import Datasets](https://docs.lancedb.com/ingestion/import.md)
>
> Read or import through the integration once it exists.

> [card] [Attach Database](https://docs.lancedb.com/ingestion/migrate-from-embedded.md)
>
> Attach an existing database at a location the integration covers.

> [card] [Storage Integration](https://docs.lancedb.com/sql/ddl/storage-integration.md)
>
> The complete SQL reference: every provider's options and output columns.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.