Skip to main content
Applies to: Embedded ❌, Enterprise ✅ Distributed compaction splits a table’s fragments into independent, fragment-disjoint tasks and fans them out across job workers in LanceDB Enterprise. Unlike an index build, a compaction plan needs no bin-packing: Lance’s own planner already produces disjoint tasks, so workers claim and execute them directly, with no coordination needed to decide who does what. See Architecture.
No task depends on another’s result, so a worker that finishes early simply claims another task rather than waiting on its peers. Committing is where the fleet does coordinate: finished tasks go through a shared eligibility gate, and only the one worker that claims the commit actually writes it, rather than each worker committing its own task independently. Whether a failed task can still let the rest commit depends on the remap mode. Inline remap is all-or-nothing: one failed task fails the whole job, since the commit updates every affected index inline and can’t skip a task’s share of that. Deferred remap can commit a subset — a failed task’s fragments are abandoned and replanned in a later round, while the rest of the job’s tasks commit without waiting on it. See Advanced Conflict Resolution for when compaction defers a remap at all.

​
Compaction interfaces

  • Compaction for triggering and configuring a compaction.
  • Garbage Collection for removing files compaction and other operations leave behind.
  • Job Framework for the job types a deployment runs on its own schedule.