> ## Documentation Index
> Fetch the complete documentation index at: https://docs.lancedb.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Scope curation to what changed

> Process only the rows added since your last curation run, instead of rescanning the whole table.

**Applies to:** Embedded ❌, Enterprise ✅

A curation task you run repeatedly -- chunking, deduplication, profiling --
does not need to rescan rows it already processed. `CHANGES` reads only the
rows appended to a table since an earlier
[version](https://docs.lancedb.com/management/table.md#versioning), so a recurring job can scope itself
to what is new.

## Example data

`curation_batches` gets a batch of new documents appended periodically, with
no updates or deletes between appends:

```sql SQL icon="database" theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
CREATE TABLE curation_batches (doc_id BIGINT, source VARCHAR, text VARCHAR);
INSERT INTO curation_batches VALUES
    (1, 'web', 'A cat on a chair'),
    (2, 'web', 'A dog in a field');
-- note the version after this insert, for example 5
INSERT INTO curation_batches VALUES
    (3, 'books', 'A bird in a tree'),
    (4, 'books', 'A cat by a window');
-- the table is now at a later version, for example 7
```

## Read only what is new

`CHANGES (INFORMATION => APPEND_ONLY) AT (VERSION => 5)` reads the rows
appended after version 5 -- documents 3 and 4 -- without rescanning 1 and 2:

```sql SQL icon="database" theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
SELECT * FROM curation_batches CHANGES (INFORMATION => APPEND_ONLY) AT (VERSION => 5)
```

Record the table's version after each curation run, then start the next
run's `CHANGES` from it. See [Versioning](https://docs.lancedb.com/management/table.md#versioning) for
the APIs that read a table's current version.

## Chain it into a curation task

`CHANGES` is a relation, like any other table reference, so it composes with
the rest of this page: chunk, dedupe, or join only the new rows, instead of
the whole table. Materializing the result keeps a running, append-only table
of work already done:

```sql SQL icon="database" theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
INSERT INTO curated_batches
SELECT doc_id, source, text
FROM curation_batches CHANGES (INFORMATION => APPEND_ONLY) AT (VERSION => 5)
WHERE source = 'books'
```

A [chunker Function](https://docs.lancedb.com/curation/chunking.md) or a [`dedupe`](https://docs.lancedb.com/sql/dql/dedupe)
call works the same way: put the `CHANGES` relation where the source table
would otherwise go, and only the new rows flow through it.

## Strictly append-only

The clause is named for what it requires, not just what it does. If anything
other than a pure append happened to the table between the starting version
and the current one -- an update, a delete, a merge -- the query is refused
rather than silently returning an incomplete or wrong result:

```sql SQL icon="database" theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}}
-- refused if any row in curation_batches was updated or deleted after version 5
SELECT * FROM curation_batches CHANGES (INFORMATION => APPEND_ONLY) AT (VERSION => 5)
```

When that happens, either re-run from a later starting version that is
itself after the non-append write, or read the table in full instead of
incrementally for that run. See
[Changes since a version](https://docs.lancedb.com/sql/dql/changes.md#strict) for the complete set of
refused forms.

## What to read next

> [card] [CHANGES](https://docs.lancedb.com/sql/dql/changes.md)
>
> The full SQL syntax, placement rules, and unsupported forms.

> [card] [Table](https://docs.lancedb.com/management/table.md#versioning)
>
> How table versions work, and how to find one to start from.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.