Skip to main content
Phidata is a framework for building AI Assistants with long-term memory, contextual knowledge, and the ability to take actions using function calling. It helps turn general-purpose LLMs into specialized assistants tailored to your use case by extending its capabilities using memory, knowledge, and tools.
  • Memory: Stores chat history in a database and enables LLMs to have long-term conversations.
  • Knowledge: Stores information in a vector database and provides LLMs with business context. (Here we will use LanceDB)
  • Tools: Enable LLMs to take actions like pulling data from an API, sending emails or querying a database, etc.
example Memory & knowledge make LLMs smarter while tools make them autonomous. LanceDB is a vector database and its integration into Phidata makes it easy for us to provide a knowledge base to LLMs. It enables us to store information as embeddings and search for the results similar to ours using query.
What is a Knowledge Base?Knowledge Base is a database of information that the Assistant can search to improve its responses. This information is stored in a vector database and provides LLMs with business context, which makes them respond in a context-aware manner.While any type of storage can act as a knowledge base, vector databases offer the best solution for retrieving relevant results from dense information quickly.
Let’s see how using LanceDB inside Phidata helps in making LLM more useful:

​
Prerequisites: install and import necessary dependencies

Create a virtual environment
  1. install virtualenv package
bash
pip install virtualenv
  1. Create a directory for your project and go to the directory and create a virtual environment inside it.
bash
mkdir phi
bash
cd phi
bash
python -m venv phidata_
Activating virtual environment
  1. from inside the project directory, run the following command to activate the virtual environment.
bash
phidata_/Scripts/activate
Install the following packages in the virtual environment
bash
pip install lancedb phidata youtube_transcript_api openai ollama numpy pandas
Create python files and import necessary libraries You need to create two files — transcript.py and ollama_assistant.py or openai_assistant.py
If creating Ollama assistant, download and install Ollama from here and then run the Ollama instance in the background. Also, download the required models using ollama pull <model-name>. Check out the models here
Run the following command to deactivate the virtual environment if needed
bash
deactivate

​
Step 1 - Create a Knowledge Base for AI Assistant using LanceDB

Check out the list of embedders supported by Phidata and their usage here. Here we have used TextKnowledgeBase, which loads text/docx files to the knowledge base. Let’s see all the parameters that TextKnowledgeBase takes -
NameTypePurposeDefault
pathUnion[str, Path]Path to text file(s). It can point to a single text file or a directory of text files.provided by user
formatsList[str]File formats accepted by this knowledge base.[".txt"]
vector_dbVectorDbVector Database for the Knowledge Base. Phidata provides a wrapper around many vector DBs, you can import it like this - from phi.vectordb.lancedb import LanceDbprovided by user
num_documentsintNumber of results (documents/vectors) that vector search should return.5
readerTextReaderPhidata provides many types of reader objects which read data, clean it and create chunks of data, encapsulate each chunk inside an object of the Document class, and return List[Document].TextReader()
optimize_onintIt is used to specify the number of documents on which to optimize the vector database. Supposed to create an index.1000
??? Tip “Wonder! What is Document class?” We know that, before storing the data in vectorDB, we need to split the data into smaller chunks upon which embeddings will be created and these embeddings along with the chunks will be stored in vectorDB. When the user queries over the vectorDB, some of these embeddings will be returned as the result based on the semantic similarity with the query. When the user queries over vectorDB, the queries are converted into embeddings, and a nearest neighbor search is performed over these query embeddings which returns the embeddings that correspond to most semantically similar chunks(parts of our data) present in vectorDB. Here, a “Document” is a class in Phidata. Since there is an option to let Phidata create and manage embeddings, it splits our data into smaller chunks(as expected). It does not directly create embeddings on it. Instead, it takes each chunk and encapsulates it inside the object of the Document class along with various other metadata related to the chunk. Then embeddings are created on these Document objects and stored in vectorDB. However, using Phidata you can load many other types of data in the knowledge base(other than text). Check out Phidata Knowledge Base for more information. Let’s dig deeper into the vector_db parameter and see what parameters LanceDb takes -
NameTypePurposeDefault
embedderEmbedderPhidata provides many Embedders that abstract the interaction with embedding APIs and utilize it to generate embeddings. Check out other embedders hereOpenAIEmbedder
distanceList[str]The choice of distance metric used to calculate the similarity between vectors, which directly impacts search results and performance in vector databases.Distance.cosine
connectionlancedb.db.LanceTableLanceTable can be accessed through .connection. You can connect to an existing table of LanceDB, created outside of Phidata, and utilize it. If not provided, it creates a new table using table_name parameter and adds it to connection.None
uristrIt specifies the directory location of LanceDB database and establishes a connection that can be used to interact with the database."/tmp/lancedb"
table_namestrIf connection is not provided, it initializes and connects to a new LanceDB table with a specified(or default) name in the database present at uri."phi"
nprobesintIt refers to the number of partitions that the search algorithm examines to find the nearest neighbors of a given query vector. Higher values will yield better recall (more likely to find vectors if they exist) at the expense of latency.20
Since we just initialized the KnowledgeBase. The VectorDB table that corresponds to this Knowledge Base is not yet populated with our data. It will be populated in Step 3, once we perform the load operation.You can check the state of the LanceDB table using - knowledge_base.vector_db.connection.to_pandas()
Now that the Knowledge Base is initialized, , we can go to step 2.

​
Step 2 - Create an assistant with our choice of LLM and reference to the knowledge base.

Assistants add memory, knowledge, and tools to LLMs. Here we will add only knowledge in this example. Whenever we will give a query to LLM, the assistant will retrieve relevant information from our Knowledge Base(table in LanceDB) and pass it to LLM along with the user query in a structured way.
  • The add_references_to_prompt=True always adds information from the knowledge base to the prompt, regardless of whether it is relevant to the question.
To know more about an creating assistant in Phidata, check out Phidata docs here.

​
Step 3 - Load data to Knowledge Base.

The above code loads the data to the Knowledge Base(LanceDB Table) and now it is ready to be used by the assistant.
NameTypePurposeDefault
recreateboolIf True, it drops the existing table and recreates the table in the vectorDB.False
upsertboolIf True and the vectorDB supports upsert, it will upsert documents to the vector db.False
skip_existingboolIf True, skips documents that already exist in the vectorDB when inserting.True
Tip · What is upsert?
Upsert is a database operation that combines “update” and “insert”. It updates existing records if a document with the same identifier exists, or inserts new records if no matching record exists. This keeps the knowledge base current without manual checks.
During the Load operation, Phidata directly interacts with the LanceDB library and performs the loading of the table with our data in the following steps -
  1. Creates and initializes the table if it does not exist.
  2. Then it splits our data into smaller chunks.
    Question · How do they create chunks?
    Phidata provides multiple knowledge-base types depending on the source data. Most of them (except the LlamaIndexKnowledgeBase and LangChainKnowledgeBase) expose a document_lists iterator. During the load operation, this iterator reads the input (for example, text files), splits it into chunks, wraps each chunk in a Document, and yields lists of those Document objects.
  3. Then embeddings are created on these chunks are inserted into the LanceDB Table
    Question · How do they insert the chunks into LanceDB?
    Each list of Document objects from the previous step is processed as follows:
    • Generate embeddings for every Document.
    • Clean the content field so only the text you care about is persisted.
    • Prepare a payload with the id, the embedding (vector), and any metadata needed for retrieval.
    • Add the prepared rows to the LanceDB table.
  4. Now the internal state of knowledge_base is changed (embeddings are created and loaded in the table ) and it ready to be used by assistant.

​
Step 4 - Start a cli chatbot with access to the Knowledge base

For more information and amazing cookbooks of Phidata, read the Phidata documentation and also visit LanceDB x Phidata docmentation.