Full Text Search
The full-text search allows you to incorporate keyword-based search (based on BM25) in your retrieval solutions. LanceDB can deliver 26ms query latency for full-text search. Our benchmark tests have more details.
Newly added or modified records become searchable immediately. The full-text search (FTS) index updates automatically in the background, ensuring continuous search availability without blocking queries.
Advanced Search Features
Fuzzy Search
Fuzzy search allows you to find matches even when the search terms contain typos or slight variations. LanceDB uses the classic Levenshtein distance to find similar terms within a specified edit distance.
Parameter | Type | Default | Description |
---|---|---|---|
fuzziness | int | 0 | Maximum edit distance allowed for each term. If not specified, automatically set based on term length: 0 for length ≤ 2, 1 for length ≤ 5, 2 for length > 5 |
max_expansions | int | 50 | Maximum number of terms to consider for fuzzy matching. Higher values may improve recall but increase search time |
Let’s create a sample table and build full-text search indices to demonstrate fuzzy search capabilities and relevance boosting features.
To demonstrate fuzzy search’s ability to handle typos and misspellings, let’s perform a search with a deliberately misspelled word. The search engine will attempt to match similar terms within the specified edit distance.
Phrase Match
Phrase matching enables you to search for exact sequences of words. Unlike regular text search which matches individual terms independently, phrase matching requires words to appear in the specified order with no intervening terms. This is particularly useful when:
- Searching for specific multi-word expressions
- Matching exact titles or quotes
- Finding precise word combinations in a specific order
Search with Boosting
Boosting allows you to control the relative importance of different search terms or fields in your queries. This feature is particularly useful when you need to:
- Prioritize matches in certain columns
- Promote specific terms while demoting others
- Fine-tune relevance scoring for better search results
Parameter | Type | Default | Description |
---|---|---|---|
positive | Query | required | The primary query terms to match and promote in results |
negative | Query | required | Terms to demote in the search results |
negative_boost | float | 0.5 | Multiplier for negative matches (lower values = stronger demotion) |
- Use fuzzy search when handling user input that may contain typos or variations
- Apply field boosting to prioritize matches in more important columns
- Combine fuzzy search with boosting for robust and precise search results