The integration of advanced technologies in Artificial Intelligence (AI) and Machine Learning (ML) is transforming how we interact with machines, reshaping our digital experiences. The evolution of chatbots from simple, script-based responders to digital assistants capable of engaging in meaningful conversations represents a significant technological leap. In this introductory article, we will discuss how integrating Langchain, OpenAI, and pgvector is shaping the future of AI chatbots.
1. A simple use case
Let us assume a simple use case to improve the user experience for all the visitors of our website (https://hexacluster.ai/). We want to help users not spend an inordinate amount of time trying to decipher our company’s specialties.
While looking for a solution, we might consider leveraging advanced digital solutions, but the first option that often comes to mind is the traditional chatbot. Unfortunately, these chatbots tend to be robotic, limited, and honestly, more frustrating than helpful. It’s like asking for directions and getting a response in Morse code!
What if we could transform this experience? What if a chatbot could not just respond, but actually understand the context, assist, and guide our users with the finesse of a real person ?
1.1. Solution
Yes, It’s possible in the world of AI and machine learning, when combined with the power of Langchain, OpenAI, and pgvector. Our goal is to create a chatbot that doesn’t just answer questions but understands the essence of a user inquiry, making web interactions smoother, more personal, and frankly, a lot more enjoyable.
2. Dive into AI components with Code
So, what goes into creating a chatbot that’s more like a savvy companion than a mechanical responder? Let’s break down the key components: Vector Databases, Langchain, and OpenAI’s Large Language Models (LLMs). Here, the focus is to have a sample code to work on and get a basic understanding of the important components.
2.1 Set Up Your Environment
The first basic step will be to create an environment to create a smart bot. In this tutorial, we used the conda environment. Alternatively, you can opt for Python’s built-in virtual environment tool, venv, which is another effective option. Install the required libraries using the following pip command.
pip install langchain openai tiktoken langchainhub gradio beautifulsoup4 pgvector psycopg2-binary
We would be using the following libraries for our solution.
Langchain: Langchain is a toolkit designed for building language model applications.
OpenAI, tiktoken: OpenAI is known for its advanced AI models and tools, including language models like GPT-3. Tiktoken, on the other hand, is a library used for token management specifically for API’s with token-based authentication.
BeautifulSoup4 (bs4): BeautifulSoup4 is commonly used for web scraping, allowing easy extraction of data from websites.
pgvector, psycopg2-binary: pgvector is a library for efficient similarity search in databases, especially useful for applications involving vector data. psycopg2-binary is a PostgreSQL database adapter for Python.
2.2 Import the required libraries
In this section, we will import the important libraries for data retrieval, data embedding, accessing the Open AI LLM model, and other important components from the langchain.
import os
os.environ["OPENAI_API_KEY"] = "Set-Your-Key"
# For retriving the data from the website
import bs4
from bs4 import BeautifulSoup as Soup
from langchain.document_loaders.recursive_url_loader import RecursiveUrlLoader
# For accessing the database
from langchain.vectorstores.pgvector import PGVector
# For Embedding and accessing LLM Model
from langchain.embeddings import OpenAIEmbeddings
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.chat_models import ChatOpenAI
# For Rag with LangChain
from langchain.prompts import ChatPromptTemplate
from langchain.memory import ConversationSummaryMemory
from langchain.chains import ConversationalRetrievalChain
# For Chat Interface
import gradio as gr2.3 Data Ingestion
To make the scope of this tutorial precise, we have considered using the data from our own website(https://hexacluster.ai/). To get the textual part of the website, we will be using two main libraries: RecursiveUrlLoader from LangChain and BeautifulSoup. RecursiveUrlLoader will go into the depth of the pages to fetch the data and BeautifulSoup parses the text from those html’s.
url = "https://www.hexacluster.ai/"
loader = RecursiveUrlLoader(
url=url, max_depth=3, extractor=lambda x: Soup(x, "html.parser").text
)
docs = loader.load()
2.4 Data Transformation
We will now delve into the data transformation for optimizing chatbot performance which is a crucial stage of chatbot development. Additionally, performing preliminary data cleaning and basic transformations would yield better results. But, now we focus on a key transformation step essential for enhancing a chatbot’s functionality.
At this stage, we utilize the ‘docs’ dataset, previously extracted from our website. Our objective is to divide this dataset into manageable segments, a process vital for efficient data processing and analysis in chatbot development.
2.5 Let us look at our Chunking Strategy
Chunk Size: We’ve chosen a chunk size of 1000 characters. This means each segment of the text will contain up to 1000 characters.
Chunk Overlap: To maintain context continuity, we’re implementing a 200-character overlap between consecutive chunks. This overlap ensures that the chatbot has sufficient context to understand and respond to user queries accurately.
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
doc_chunks = text_splitter.split_documents(docs)
Let us consider a sample text as follows : “Welcome to our chatbot blog. Here, we explore advanced techniques in chatbot dev”
Using the RecursiveCharacterTextSplitter with our specified chunk size and overlap, the text would be split as follows.
- First Chunk: “Welcome to our chatbot blog. Here, we”
- Second Chunk: “og. Here, we explore advanced techniq”
- Third Chunk: “e advanced techniques in chatbot dev”
2.6 Heart of the Smart Bot
So far, we were able to scrape the website and split the crawled pages into chunks. Before we embark on the journey of creating a vector database with embeddings, it’s imperative to grasp the underlying concepts that drive this technology.
2.6.1 Embeddings: The Foundation of Understanding Language
What exactly are embeddings? In the domain of AI and natural language processing, embeddings transform words, phrases, or entire documents into numerical vectors.
2.6.1.1 What are Embeddings?
In the realm of AI and natural language processing, embeddings are a way to represent words, phrases, or even entire documents as vectors of numbers. Imagine translating the complexities of language into a form that a computer can not only understand but also analyze in depth. That’s what embeddings do. They capture the essence, meaning, and context of language elements in a multi-dimensional space.
2.6.1.2 Why are Embeddings Important?
The beauty of embeddings lies in their ability to capture the nuances of language. Words with similar meanings are positioned closely in this multi-dimensional space, allowing the AI to understand synonyms, context, and even the tone of the conversation. This forms the backbone of any system that aims to comprehend and respond to human language in a nuanced and meaningful way.
With this understanding of embeddings, we can better appreciate the role of vector databases, which leverage these embeddings to revolutionize how we search and interact with information.
2.6.2 Vector Databases: Beyond Traditional Text Matching
Traditional databases work well for exact queries but fall short when it comes to understanding the nuances of human language. Enter Vector Databases, a game-changer in handling semantic search. Unlike traditional text matching, which relies on exact words or phrases, vector databases like Postgres with pgvector process information semantically.
This article/blog aims to use pgvector as a database, highlighting its major advantage: seamless integration with PostgreSQL. As an extension to one of the most trusted and widely-used open-source relational database systems, pgvector inherits PostgreSQL’s robustness, security, and reliability. It reduced the need for additional infrastructure or complex integration processes.
Think of it like this: instead of searching for a needle in a haystack with a magnet (traditional text matching), pgvector uses a map that understands the layout of the haystack, making it easier to find the needle based on its position and relation to other objects (semantic search). This means our chatbot can grasp the intent behind your words, not just the words themselves.
After having a basic understanding of the vector database and embedding, you can use the below code to create the pgvector in the database. The last section of the code will create a vector store with the embeddings retrieved from Open AI for the doc chunks of our website (https://hexacluster.ai/).
CONNECTION_STRING = "postgresql+psycopg2://postgres@localhost:5432/ml"
COLLECTION_NAME = "hexacluster_website"
embeddings = OpenAIEmbeddings()
vecdb = PGVector.from_documents(
embedding=embeddings,
documents=doc_chunks, # from data transformation section
collection_name=COLLECTION_NAME,
connection_string=CONNECTION_STRING,
pre_delete_collection=True, # To override the vectorstore
)
retriever = vecdb.as_retriever(search_kwargs={"k": 2}) #Fetch top 2 results from vector store
2.7 Final Task: Bot Creation
So, we have a vector store brimming with data from our website (https://hexacluster.ai/), all neatly organized into embeddings. This brings a question: how do we transform this rich data into the brainpower for a smart, intuitive chatbot? This is where the magic of combining advanced AI components comes into play, each playing a pivotal role in elevating the chatbot from a simple responder to an intelligent conversationalist.
2.7.1 Langchain: Harnessing Conversational Context
Now, understanding the query is one thing, but maintaining a natural, context-aware conversation is another. That’s where Langchain with LLM comes in. It’s like the maestro orchestrating the flow of our chatbot’s conversation, ensuring it understands the context and history of the interaction. Integrating an LLM into Langchain enables our chatbot to remember previous parts of the conversation, making its responses not just accurate, but also relevant and coherent.
When augmented with the concept of Retrieval-Augmented Generation (RAG), the bot will not only understand the context of a conversation but also enhances it with a depth of knowledge and relevance.
2.7.1.1 What is RAG?
Retrieval-Augmented Generation is a technique where the system retrieves information from a large database (like documents, websites, or books) and uses this information to generate responses. Imagine a librarian who not only remembers your past questions but also constantly refers to a vast library to give you the best possible answers. That’s RAG in action.
2.7.1.2 Is Memory Relevant to Conversation?
Imagine interacting with a smart bot that, while intelligent, still behaves in a very robotic manner, lacking the ability to remember previous parts of the conversation. For instance, if you ask, “What does Hexacluster do?” and follow up with, “Could you tell me more about it?”, a bot without memory might respond with a generic, “What do you want to know more about?”, causing frustration due to its lack of contextual understanding. To address this, we can integrate ‘ConversationSummaryMemory’ from LangChain, which is designed to store summaries of previous conversations.
2.7.1.3 Build the Chain and Deploy
It’s essential to establish a pipeline, often referred to as a ‘chain’. This is crucial for facilitating a dialogue that draws upon previously gathered documents. The chain processes the chat history, which includes a series of messages, along with new inquiries, and subsequently provides a response to these questions. To create this, we utilize the ‘ConversationalRetrievalChain’ from langchain.
Additionally, we employ gradio’s ChatInterface for the creation of the ChatBot User Interface.
2.7.2 Large Language Models: The Brain of the Bot
Finally, the core of our chatbot’s intelligence – OpenAI’s Large Language Models. These LLMs, like GPT-4, are the brains of the operation. They process and generate human-like text, enabling our chatbot to not only understand and respond to queries but to do so in a way that feels natural and human-like. This is crucial because the essence of our chatbot’s success lies in its ability to interact like a human, making the user’s experience as comfortable and intuitive as possible.
As mentioned earlier, chatbot’s intelligence in this demo is powered by OpenAI’s model, serving as a Language Model (LLM). However, for those seeking local and private implementations, alternatives such as LLama 2, GPT4All, llama.cpp etc are available. In our upcoming blog posts, we will delve into these models, providing a thorough examination of their distinctive features and exploring their potential applications to elevate chatbot intelligence.
In essence, by integrating pgvector’s efficient semantic search capabilities, Langchain’s conversational context awareness, and the advanced language processing capabilities of OpenAI’s LLMs, we are creating a chatbot that’s not just a tool, but a conversational partner, ready to make your web experience smoother and more enjoyable.
3. Final Showcase
You now possess a fundamental grasp of how to use OpenAI’s LLM and Lang Chain with RAG. Below is the code you can use to construct a RAG Chatbot. Moreover, the complete source code is accessible here.
template = """You are representing HexaCluster. Consider yourself as an intelligent Sales person from HexaCluster. Answer to be the below question in a pleasing tone from the information available in the below context.
{context}
Question: {question}
"""
prompt = ChatPromptTemplate.from_template(template)
llm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0)
memory = ConversationSummaryMemory(
llm=llm, memory_key="chat_history", return_messages=True
)
qa = ConversationalRetrievalChain.from_llm(llm, retriever=retriever, memory=memory, combine_docs_chain_kwargs={"prompt": prompt})
def qa_answer(message, history):
return qa(message)["answer"]
demo = gr.ChatInterface(qa_answer)
if __name__ == "__main__":
demo.launch()
4. Conclusion
Following the dedicated efforts, it’s time to unveil the results of our work through a practical demonstration. We’ve utilized Gradio, a powerful tool, to craft a user-friendly interface that allows seamless interaction with the data on our website (Hexacluster.ai). Below, you can observe the impressive performance of the system. While it works as expected, there’s always room for improvement.
Stay tuned for our upcoming articles where we’ll explore further enhancements and developments, continuously striving to refine and advance our application. This is just the beginning of a journey towards more sophisticated and user-centric solutions.
Are you looking to build an AI Chatbot to simplify you user experience and optimize IT costs ? HexaCluster is your trusted partner. Contact us today to learn more about how we can help you. In addition to Machine Learning, we also offer PostgreSQL Consulting services that include Remote DBA, Database Migrations and Health Audits. Submit the following form and one of our experts will contact you soon.
Subscribe to our Newsletters and Stay tuned for more interesting topics.
Contact Us Today!
