> ## Documentation Index > Fetch the complete documentation index at: https://docs.lancedb.com/llms.txt > Use this file to discover all available pages before exploring further. # Intelligent Auto-Maintenance > How Enterprise continuously tracks table activity in the background and decides what maintenance -- compaction, cleanup, indexing, remap catch-up -- a table needs next. **Applies to:** Embedded ❌, Enterprise ✅ Keeping a table in good condition takes several different kinds of work -- compacting files, catching indexes up, cleaning old versions, folding a RTE write tier into the base table -- each with its own trigger. In LanceDB Enterprise, a fleet of agent processes decides when each of these needs to run, continuously, per table, in the background across your cluster -- nothing to schedule, trigger, or monitor by hand unless you want to. ```mermaid theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}} %%{init: {"theme": "base", "themeVariables": { "background": "#ffffff", "primaryColor": "#ffffff", "primaryTextColor": "#221d19", "primaryBorderColor": "#ad9e95", "lineColor": "#6b625c", "textColor": "#221d19", "edgeLabelBackground": "#ffffff", "fontFamily": "Inter, sans-serif", "fontSize": "18px" }}}%% flowchart LR E["Table event notifications"] --> P S["Periodic sweep
every table, on a timer"] --> P P["Planner
evaluates one table"] P --> W["Possible next step:
WAL index catch-up"] P --> I["Possible next step:
Index operations"] P --> C["Possible next step:
Compaction"] P --> CL["Possible next step:
Cleanup"] P --> R["Possible next step:
Remap catch-up"] classDef default fill:#ffffff,stroke:#ad9e95,stroke-width:1px,color:#221d19 classDef accent fill:#fff1ec,stroke:#e8582d,stroke-width:1.5px,color:#221d19 class P accent ``` ## Tracking activity An agent's long-running process watches two independent sources for a table: **table events** (Job Registry notifications, so it learns about a write, a schema change, or a new index almost immediately) and a **periodic sweep** that revisits every table it owns on a timer -- 600 seconds by default -- so a table that stops producing events still gets re-evaluated rather than going stale. A hot table producing a flood of events does not get to monopolize evaluation: notifications and the periodic sweep share a bounded per-tick budget, and reads for a given table run at most once at a time. Each evaluation reopens the table and reads its manifest, indexes and in-flight jobs fresh, rather than deciding from a cached guess. Those are separate reads rather than one atomic snapshot, so a concurrent write between them is possible; the next evaluation -- triggered by that same write's own event notification -- is what catches anything the current one missed. ## Deciding what to run A table is not reduced to one "next best action" -- each evaluation walks a fixed, priority-ordered sequence of independent decisions, launching whichever ones are due: 1. **WAL index catch-up.** Checked first and gated by nothing else, because it is correctness-driven rather than discretionary: while an index lags behind the compacted generation, the RTE write tier cannot trim its log. See [RTE Compaction](https://docs.lancedb.com/enterprise/realtime-engine-compaction.md). 2. **Index operations.** Reindexing a column, unless a compaction that cannot defer its remap is about to run this same tick -- deferred-remap compaction lets index work proceed alongside it instead of waiting. See [Incremental Indexing](https://docs.lancedb.com/indexing/incremental-indexing.md). 3. **Compaction.** Whether enough small fragments or deleted rows have accumulated to merge. See [Compaction](https://docs.lancedb.com/maintenance/compaction.md) and [Distributed Compaction](https://docs.lancedb.com/enterprise/distributed-compaction.md). 4. **Cleanup.** Independent of the other four -- it can always run in parallel with them. [See below](#cleanup) for exactly when it's due. 5. **Deferred-remap catch-up.** The batched, amortized follow-up for index entries a deferred compaction didn't remap synchronously. See [Advanced Conflict Resolution](https://docs.lancedb.com/enterprise/advanced-conflict-resolution.md). More than one of these can fire for the same table in the same evaluation -- a compaction and a cleanup are unrelated and can both launch together -- but some gate each other where running them at the same time would be unsafe, like index operations waiting on a compaction that can't defer its remap. ### When cleanup is due [Cleanup](https://docs.lancedb.com/maintenance/cleanup.md) prunes old table versions and the files only they referenced. In LanceDB Embedded, that always happens because you asked for it -- a call to `optimize()`. LanceDB Enterprise does not support a manual `optimize()` call at all; cleanup there is entirely this planner's own decision, off by default unless the deployment sets a minimum version age -- with no minimum age configured, nothing is ever old enough to clean up automatically, and old versions simply accumulate until an operator enables it. A table can also disable it individually even when the deployment allows it, overriding the deployment-wide setting for just that table. Once enabled, a table becomes due for cleanup when any one of these is true: | Trigger | Condition | | - | - | | Enough new versions | The current version has advanced far enough past the version the last successful cleanup started from | | Enough time has passed | The last successful cleanup finished long enough ago -- the default is 24 hours -- so an idle table whose versions simply age past the retention window still gets revisited | | Version went backward | The current version is lower than the version the last successful cleanup recorded, which re-triggers a run immediately | A table that has never had a successful cleanup run is held to its own initial version threshold first, so a brand-new table is not claimed for cleanup before it has accumulated enough versions to be worth the pass. A failed attempt is retried, but not immediately -- a minimum delay (60 seconds by default) must pass after the failure before the table becomes eligible again, even if one of the triggers above would otherwise fire. ## Confirming it's actually running Rather than asking you to trust that evaluation is happening, the agent exposes coverage metrics scoped to *successful* evaluations specifically -- a failed or timed-out attempt does not count: | Metric (`lance_agent_` prefix) | What it tells you | | - | - | | `coverage_never_evaluated` | Tables this agent owns that have never completed an evaluation | | `coverage_oldest_success_age_seconds` | How long it's been since the longest-overdue table was last successfully evaluated | | `coverage_cycle_target` / `coverage_cycle_pending` | How much work the current full sweep has left | | `coverage_cycles_completed_total` | How many full sweeps have finished | A cycle freezes its target set of tables up front; new tables join the *next* cycle rather than resetting progress on the current one, and a table that disappears or changes ownership is dropped from the target rather than counted as covered. ## What to read next > [card] [Cleanup](https://docs.lancedb.com/maintenance/cleanup.md) > > What a cleanup pass actually removes, and the retention window it respects. > [card] [Advanced Conflict Resolution](https://docs.lancedb.com/enterprise/advanced-conflict-resolution.md) > > Why compaction and index maintenance need the deferred-remap decision at all. > [card] [RTE Compaction](https://docs.lancedb.com/enterprise/realtime-engine-compaction.md) > > The WAL-tier compaction this page's Decision 1 depends on. > [card] [Incremental Indexing](https://docs.lancedb.com/indexing/incremental-indexing.md) > > What an index operation actually does once scheduled. > [card] [OpenTelemetry](https://docs.lancedb.com/deployment/opentelemetry.md) > > Viewing the `lance_agent_*` coverage metrics above. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.