Introduction
Amazon Bedrock is a fully managed service that enables developers to create and manage high-performing foundation models (FMs) from leading artificial intelligence (AI) companies. It provides a unified API for accessing and generating text and data from a wide range of sources. In this post, we will explore how to use metadata filtering with Knowledge Bases for Amazon Bedrock to retrieve accurate results from a large tabular dataset.
Solution Overview
The solution consists of the following high-level steps:
1. Prepare data for metadata filtering.
2. Create and ingest data and metadata into the knowledge base.
3. Retrieve data from the knowledge base using metadata filtering.
Prepare Data for Metadata Filtering
To enable metadata filtering, we need to split a large tabular dataset into rows and set up a knowledge base with metadata for each of those records. We can achieve this by creating a data frame using the pandas library and converting the data to a suitable format.
For this example, we will use the Food.com – Recipes and Reviews dataset. The dataset contains a table with columns such as Recipe Instructions, Total Time, Cholesterol Content, Sugar Content, and others. We can create a data frame from this table and split it into rows.
Here is an example code snippet:
“`
import pandas as pd
# Load the dataset
df = pd.read_csv(‘food.csv’)
# Convert the data to a suitable format
df[‘TotalTime’] = df[‘TotalTime’].apply(convert_to_minutes)
# Split the data into rows
rows = df.shape[0]
# Create a list to store the data rows
data_rows = []
for i in range(rows):
row = df.iloc[i]
data_rows.append({
‘id’: i,
‘data’: row[‘RecipeInstructions’],
‘metadata’: {
‘TotalTime’: row[‘TotalTime’],
‘CholesterolContent’: row[‘CholesterolContent’],
‘SugarContent’: row[‘SugarContent’]
}
})
# Save the data rows to a file
with open(‘data.json’, ‘w’) as f:
json.dump(data_rows, f)
“`
This code snippet loads the dataset, converts the data to a suitable format, splits the data into rows, and saves the data rows to a file.
Create and Ingest Data and Metadata into the Knowledge Base
Once we have the data rows, we can create and ingest them into the knowledge base using the Amazon Bedrock SDK.
Here is an example code snippet:
“`
import boto3
# Create a new knowledge base
kb = boto3.client(‘bedrock-runtime’).create_knowledge_base(Name=’food-kb’, Description=’Food Recipes and Reviews’)
# Create and ingest data and metadata into the knowledge base
for row in data_rows:
boto3.client(‘bedrock-runtime’).ingest_data(
KnowledgeBaseId=kb[‘knowledgeBaseId’],
DataSourceId=row[‘id’],
Data=row[‘data’],
Metadata=row[‘metadata’]
)
“`
This code snippet creates a new knowledge base, and then creates and ingests the data rows into the knowledge base.
Retrieve Data from the Knowledge Base using Metadata Filtering
Once the data and metadata are ingested into the knowledge base, we can retrieve data from the knowledge base using metadata filtering.
Here is an example code snippet:
“`
import boto3
# Retrieve data from the knowledge base using metadata filtering
query = “Tell me a recipe that I can make under 30 minutes and has cholesterol less than 10”
response = boto3.client(‘bedrock-runtime’).retrieve(
KnowledgeBaseId=kb[‘knowledgeBaseId’],
Query=query,
Filter={
‘andAll’: [
{
‘lessThan’: {
‘key’: ‘TotalTime’,
‘value’: 30
}
},
{
‘lessThan’: {
‘key’: ‘CholesterolContent’,
‘value’: 10
}
}
]
}
)
# Print the retrieved data
print(response[‘output’][‘text’])
“`
This code snippet retrieves data from the knowledge base using metadata filtering. The query specifies the recipe instructions, and the filter specifies the metadata filtering conditions.
Conclusion
In this post, we explored how to use metadata filtering with Knowledge Bases for Amazon Bedrock to retrieve accurate results from a large tabular dataset. We also demonstrated how to create and ingest data and metadata into the knowledge base, and how to retrieve data from the knowledge base using metadata filtering.
Frequently Asked Questions
Q: What is Amazon Bedrock?
Amazon Bedrock is a fully managed service that enables developers to create and manage high-performing foundation models (FMs) from leading artificial intelligence (AI) companies.
Q: What is metadata filtering?
Metadata filtering is a technique used to filter data based on specific conditions. In the context of Amazon Bedrock, metadata filtering is used to filter the results of a query based on the metadata associated with the data.
Q: How do I create a knowledge base in Amazon Bedrock?
To create a knowledge base in Amazon Bedrock, you can use the Amazon Bedrock SDK or the Amazon Bedrock console. You can also create a knowledge base by importing data into Amazon Bedrock using a data ingestion tool.
Q: How do I ingest data into Amazon Bedrock?
You can ingest data into Amazon Bedrock using the Amazon Bedrock SDK or the Amazon Bedrock console. You can also ingest data into Amazon Bedrock using a data ingestion tool.
Q: How do I retrieve data from Amazon Bedrock?
You can retrieve data from Amazon Bedrock using the Amazon Bedrock SDK or the Amazon Bedrock console. You can also retrieve data from Amazon Bedrock using a query tool.
