Here is the rewritten article in HTML format:
Introduction
As organizations generate vast amounts of data, it is crucial to extract valuable insights to drive better business outcomes. Generative AI and foundation models play a significant role in creating applications that enhance customer experiences and employee productivity using an organization’s proprietary data.
Organizations and Data Insights
Organizations generate vast amounts of data that is proprietary to them, and it’s critical to get insights out of the data for better business outcomes. Generative AI and foundation models (FMs) play an important role in creating applications using an organization’s data that improve customer experiences and employee productivity.
The FMs are typically pretrained on a large corpus of data that’s openly available on the internet. They perform well at natural language understanding tasks such as summarization, text generation, and question answering on a broad variety of topics. However, they can sometimes hallucinate or produce inaccurate responses when answering questions that they haven’t been trained on. To prevent incorrect responses and improve response accuracy, a technique called Retrieval Augmented Generation (RAG) is used to provide models with contextual data.
SageMaker JumpStart
SageMaker JumpStart is a powerful feature within the Amazon SageMaker ML platform that provides ML practitioners a comprehensive hub of publicly available and proprietary foundation models.
Llama 3 Overview
Llama 3 (developed by Meta) comes in two parameter sizes—8B and 70B with 8K context length—that can support a broad range of use cases with improvements in reasoning, code generation, and instruction following. Llama 3 uses a decoder-only transformer architecture and new tokenizer that provides improved model performance with 128K size. In addition, Meta improved post-training procedures that substantially reduced false refusal rates, improved alignment, and increased diversity in model responses.
BGE Large Overview
The embedding model BGE Large stands for BAAI general embedding large. It’s developed by BAAI and is designed to enhance retrieval capabilities within large language models (LLMs). The model supports three retrieval methods:
- Dense retrieval (BGE-M3)
- Lexical retrieval (LLM Embedder)
- Multi-vector retrieval (BGE Embedding Reranker).
You can use the BGE embedding model to retrieve relevant documents and then use the BGE reranker to obtain final results.
RAG Overview
Retrieval-Augmented Generation (RAG) is a technique that enables the integration of external knowledge sources with FM. RAG involves three main steps: retrieval, augmentation, and generation.
Solution Overview
You will construct a RAG QnA system on a SageMaker notebook using the Llama3-8B model and BGE Large embedding model. The following diagram illustrates the step-by-step architecture of this solution, which is described in the following sections.
Prerequisites
To implement this solution, you need the following:
- An AWS account with privileges to create AWS Identity and Access Management (IAM) resources.
Clean up
To avoid incurring unnecessary costs, when you’re done, delete the SageMaker endpoints and OpenSearch Service domain, either using the following code snippets or the SageMaker JumpStart UI.
Conclusion
In this post, we showed you a powerful RAG solution using SageMaker JumpStart to deploy the Llama 3 8B Instruct model and the BGE Large En v1.5 embedding model. We demonstrated the ability to prepare custom prompts tailored for the Llama 3 model, ensuring context-aware responses, and presented these context-specific answers in a human-friendly manner.
Frequently Asked Questions
Q1: What is RAG?
Retrieval-Augmented Generation (RAG) is a technique that enables the integration of external knowledge sources with foundation models (FMs). RAG involves three main steps: retrieval, augmentation, and generation.
Q2: What is Llama 3?
Llama 3 (developed by Meta) is a foundation model that comes in two parameter sizes—8B and 70B with 8K context length—that can support a broad range of use cases with improvements in reasoning, code generation, and instruction following.
Q3: What is BGE Large?
BGE Large is an embedding model developed by BAAI that enhances retrieval capabilities within large language models (LLMs). The model supports three retrieval methods: dense retrieval, lexical retrieval, and multi-vector retrieval.
Q4: What is SageMaker JumpStart?
SageMaker JumpStart is a powerful feature within the Amazon SageMaker ML platform that provides ML practitioners a comprehensive hub of publicly available and proprietary foundation models.
Q5: How do I implement this solution?
To implement this solution, you need an AWS account with privileges to create AWS Identity and Access Management (IAM) resources. You will also need to deploy the Llama 3 8B Instruct model and the BGE Large En v1.5 embedding model using SageMaker JumpStart.
