Skip to main content
Applies to: Embedded ❌, Enterprise ✅

Projection

Name the columns you need instead of using SELECT *, particularly for tables containing images, video or large vectors. Projection lets the engine avoid loading unnecessary columns, although columns used by filters, joins or other expressions may still need to be read. Row metadata is available as pseudo-columns (_rowid, _rowaddr, _rowoffset) but only when named, so they stay out of SELECT *. DISTINCT deduplicates the projected rows.

Filtering

The SDK where takes the same expression language, so a predicate moves between them unchanged. A supported predicate can use a scalar index; confirm the chosen plan with explain_plan. See Filtering for the operators and their pushdown behaviour.

Ordering and limiting

ORDER BY on a large table is a sort over the whole result, so pair it with a LIMIT unless you genuinely need every row ordered.

Set operations

UNION, UNION ALL, INTERSECT and EXCEPT combine whole queries:
UNION deduplicates and UNION ALL does not, which makes UNION ALL the cheaper one when you already know the inputs are disjoint.

Common table expressions

A CTE is scoped to the statement. One you want to keep is a view; one whose rows you want stored is a materialized view.

Subqueries

Subqueries are allowed anywhere an expression or a relation is (in FROM, in WHERE with IN or EXISTS, and as a scalar in the select list):

Namespaced tables

A table inside a namespace is written with its schema — the $-joined namespace path, not a dotted one — before the table name; see Schema for why:

Reading the plan

EXPLAIN and EXPLAIN ANALYZE work as DataFusion defines them, and produce the same operators the SDK methods do:
See Explain and Explain Analyze.