Retrieval-Augmented Generation for Newbies

What is a Prompt-based Model?

Compared to Retrieval-Augmented Generation models, Prompt-based models are currently the most common form of generative AI model and power some of the most popular AI tools. When you begin interacting with a model like ChatGPT or Bing Chat powered by GPT-4, you start by issuing a prompt to elicit a response. To demonstrate, we’ll start with a prompt that a client of ours, Let’s Play Soccer (LPS), a chain of indoor soccer facilities, receives daily.  Typically, one of their customers will want to rent a field. 

“We are wondering how much it would cost to rent a field for an hour weekly/bi-weekly during a weekday early in the morning (around 6am) Facility: Timpanogos Indoor Soccer”

Even if the grammar isn’t perfect, it still offers enough metadata, including a facility name, to generate a response. If we feed this response into a bot like GPT-4, it spits out the following:

“The cost of reserving a field at Let’s Play Soccer depends on the facility, the time, and the frequency of your reservation. According to the Let’s Play Soccer websitehttps://www.letsplaysoccer.com, the prices vary from $50 to $150 per hour for different facilities. You can also get discounts for booking multiple hours or sessions in advance https://www.letsplaysoccer.com, etc…”

The completion is decent, considering the LLM model only knows what is fed into it from the LPS website. However, the response we just displayed is admittedly merely a paraphrase, because GPT-4 starts it starts talking in circles afterward. Alternatively, the model could provide a response that involves topics beyond the scope of the original prompt.  The LLM isn’t suited to succinctly answer the question because there’s nothing to prevent the logic from breaking like this.  This type of error is commonly referred to as a “hallucination.”

Let’s Play still wants to implement AI to answer queries like field rental costs, but we then run into the dilemma of designing the model to answer succinctly without divulging into a bizarre and clearly inhuman conversation. The solution is either try to build a multi-billion parameter model from scratch, which requires millions of GPU compute hours, or you can leverage Retrieval-Augmented Generation (RAG).

As an advanced AWS partner, we bring unparalleled expertise to architect, deploy, and optimize cloud solutions tailored to your unique needs. 

What is Retrieval-Augmented Generation?

The Retrieval-Augmented Generation model draws responses from a dedicated data source. These external data sources can contain content related to expected queries the model will receive and can consist of various data types, whether imagery, text documents, or spreadsheets.  By limiting the scope of what it can respond with to prevent hallucinations, RAG improves upon the LLM model without having to incur the expense of building a new model.

Data Preparation

If your proprietary data is housed in a database, PDF, or Word document, you must extract it from these sources. Next, you’ll store this data in a vector database.

A vector database will store the records of your database table or document as rows.  Instead of having a scalar value as the index, the records will be indexed by vector embeddings that attempt to capture the meaning of the data in the record. This process involves using linear algebra to convert the prompt vector embedding into a dot product.  The dot product is then compared to the record’s vector embedding or an index with a similar enough dot product.

Returning to the Let’s Play Soccer example, they have a database table that contains rates for fields on specific days of the week by facility. These records are put into the vector database with a vector embedding that would be the equivalent of “renting a field at Timpanogos.”

Application Integration

The Challenge of Implementing Retrieval-Augmented Generation