Published @ AWS Builder Center

In this post, we will implement an Agentic Retrieval-Augmented Generation (Agentic RAG) system on AWS using Terraform, Amazon S3, Amazon Bedrock Knowledge Bases (KB), Amazon OpenSearch Serverless, AWS Strands Agents, and Amazon Bedrock AgentCore Runtime.

If you are new to RAG or want to understand how to build the underlying infrastructure, please check out my previous post:

Implementing RAG with Terraform using AWS S3, Bedrock Knowledge Bases, OpenSearch Serverless, and IAM.

I won't cover the RAG implementation details again in this post.

Instead, we will focus on how to use AI agents to retrieve information from Amazon Bedrock Knowledge Bases and how to improve RAG quality by combining Reranker models and Agentic RAG.

We will explore how a reranker improves retrieval precision by selecting the most relevant document chunks, while an AI agent dynamically generates retrieval queries, evaluates the retrieved context with agent-as-tool, and performs additional retrievals when necessary to improve answer completeness.

TLDR: We will build on top of the existing Simple RAG architecture and extend it with a Reranker model and Agentic RAG using Strands Agents and Amazon Bedrock AgentCore Runtime.

What is Agentic RAG?

Agentic Retrieval-Augmented Generation (Agentic RAG) is an AI architecture that combines Retrieval-Augmented Generation (RAG) with autonomous AI agents. Unlike traditional RAG, where the retrieval process follows a predefined sequence of steps, Agentic RAG allows an AI agent to decide when to retrieve information, which tools to use, and how to use the retrieved information to answer a user's question.

An AI agent can interact with external knowledge sources, execute retrieval tools, evaluate the retrieved information, and perform additional searches when the initial results are insufficient. This enables more flexible, iterative, and context-aware information retrieval.

Document Ingestion

- Upload documents → Amazon S3 - Split documents → Chunks - Generate embeddings → Embedding Model - Store embeddings + text + metadata → Vector Database

Agentic-RAG Query & Retrieval with Agent/s

- User submits a question - AI Agent receives and analyzes the question - Agent decides whether external knowledge is required - Agent selects and invokes the appropriate retrieval tool - Retrieve relevant document chunks → Amazon Bedrock Knowledge Bases - Agent evaluates the retrieved context by calling another agent as tool - Agent decides whether additional retrieval is needed - If necessary, performs additional searches or tool calls - Agent combines the retrieved information and reasoning - Generate the final answer with relevant citations
  • Traditional RAG: Follows a predefined retrieval pipeline (retrieve → augment → generate).
  • Agentic RAG: The agent dynamically controls the retrieval process (decide → retrieve → evaluate → iterate → generate).

In this project, Amazon S3 acts as the document source, while Amazon Bedrock Knowledge Bases (KB) manages the document ingestion pipeline.

agentAcrh

When an ingestion job is triggered, the following steps take place:

  • User uploads a document through the Streamlit UI.
  • The application writes the document directly to the S3 docs bucket.
  • User clicks Sync Knowledge Base in the UI.
  • The application triggers Bedrock's StartIngestionJob API.
  • Bedrock KB reads the documents from S3 and splits their content into smaller pieces called chunks.
  • Bedrock KB generates embeddings for each chunk using the configured embedding model (Titan Text Embeddings v2).
  • Bedrock KB stores the resulting vectors, together with the original text and document metadata, in Amazon OpenSearch Serverless for vector search.

Ingestion

At query/retrieval time, instead of relying on a single retrieval operation, we use an AI agent built with Strands Agents and deployed on Amazon Bedrock AgentCore Runtime.

  • User submits a question through the Streamlit chat input.
  • The application invokes the AgentCore Runtime agent.
  • The agent analyzes the question and generates a focused retrieval query.
  • The agent calls its retrieve_kb(query) tool, which invokes the Bedrock Knowledge Bases Retrieve API.
  • Bedrock KB performs a vector search against OpenSearch Serverless, retrieves 20 candidate chunks, and applies the configured Reranker model (Amazon Rerank 1.0) to select the 5 most relevant chunks.
  • The agent calls its critique_context(question) tool to evaluate whether the retrieved information is sufficient to answer the original question.
  • If the context is insufficient, the agent identifies missing aspects, reformulates the retrieval query, and performs another retrieval.
  • This retrieval and evaluation loop can run up to 3 retrieval calls per question.
  • Once sufficient context is collected, the agent generates the final answer using the retrieved document chunks and includes source citations.

query

If you have read this far, we can go deeper 😊

Why Combine Reranking and Agentic RAG?

The Reranker model and Agentic RAG improve different aspects of the retrieval process:

  • Reranker, Improves Precision: Vector search retrieves candidate chunks based on embedding similarity. The reranker evaluates the question and candidate chunks together, re-scores their relevance, and selects the most useful chunks. In our implementation, it reduces 20 candidates to the best 5.
  • Agentic RAG, Improves Recall: Instead of relying on a single retrieval, the agent can generate multiple focused queries, evaluate the retrieved context, identify missing information, and perform additional retrievals when necessary.

By combining both approaches, we aim to retrieve more complete information while ensuring that the most relevant document chunks are passed to the generation model.

The key difference is that Simple RAG performs a single retrieval and reranking operation, while Agentic RAG dynamically controls an iterative retrieval and evaluation process, applying reranking to every retrieval.

Code GitHub Link: Project on GitHub

Whether you're exploring to build your own system, this will give you a clear, practical starting point 😉

Table of Contents

  • Terraform: Bedrock AgentCore, ECR, CloudWatch, Bedrock KnowledgeBase, OpenSearch Serverless, S3, IAM
  • Agent Dockerfile, Agent.py
  • Streamlit App: Digesting & Querying
  • All Code & Demo
  • Conclusion
  • References

Terraform: Bedrock AgentCore, ECR, CloudWatch, Bedrock KnowledgeBase, OpenSearch Serverless, S3, IAM

Bedrock AgentCore, ECR-Repository, Agent-Core Runtime, Runtime IAM Policy:

# AWS ECR to store Agent Docker Image resource "aws_ecr_repository" "agent" { name = "${var.project_name}-agent" image_tag_mutability = "MUTABLE" force_delete = true image_scanning_configuration { scan_on_push = true } } # IAM Policy for AgentCore Runtime data "aws_iam_policy_document" "agent_runtime_trust" { statement { effect = "Allow" actions = ["sts:AssumeRole"] principals { type = "Service" identifiers = ["bedrock-agentcore.amazonaws.com"] } condition { test = "StringEquals" variable = "aws:SourceAccount" values = [data.aws_caller_identity.current.account_id] } condition { test = "ArnLike" variable = "aws:SourceArn" values = ["arn:aws:bedrock-agentcore:${var.aws_region}:${data.aws_caller_identity.current.account_id}:*"] } } } resource "aws_iam_role" "agent_runtime_execution" { name = "${var.project_name}-agent-runtime-role" assume_role_policy = data.aws_iam_policy_document.agent_runtime_trust.json } data "aws_iam_policy_document" "agent_runtime_permissions" { statement { sid = "BedrockModelInvoke" effect = "Allow" actions = ["bedrock:InvokeModel", "bedrock:InvokeModelWithResponseStream"] resources = [ "arn:aws:bedrock:*::foundation-model/*", "arn:aws:bedrock:*:${data.aws_caller_identity.current.account_id}:inference-profile/*", ] } statement { sid = "KnowledgeBaseRetrieve" effect = "Allow" actions = ["bedrock:Retrieve"] resources = [aws_bedrockagent_knowledge_base.this.arn] } # Retrieve calls with a rerankingConfiguration also need Rerank on the caller. statement { sid = "Rerank" effect = "Allow" actions = ["bedrock:Rerank"] resources = ["*"] } statement { sid = "EcrPull" effect = "Allow" actions = ["ecr:GetDownloadUrlForLayer", "ecr:BatchGetImage", "ecr:BatchCheckLayerAvailability"] resources = [aws_ecr_repository.agent.arn] } statement { sid = "EcrAuth" effect = "Allow" actions = ["ecr:GetAuthorizationToken"] resources = ["*"] } statement { sid = "Logs" effect = "Allow" # PutResourcePolicy lets AgentCore deliver the agent's spans into its own log group. actions = ["logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents", "logs:DescribeLogStreams", "logs:PutResourcePolicy"] resources = ["arn:aws:logs:${var.aws_region}:${data.aws_caller_identity.current.account_id}:log-group:/aws/bedrock-agentcore/*"] } statement { sid = "LogsDescribe" effect = "Allow" actions = ["logs:DescribeLogGroups"] resources = ["arn:aws:logs:${var.aws_region}:${data.aws_caller_identity.current.account_id}:log-group:*"] } statement { sid = "Xray" effect = "Allow" actions = ["xray:PutTraceSegments", "xray:PutTelemetryRecords", "xray:GetSamplingRules", "xray:GetSamplingTargets"] resources = ["*"] } # AgentCore Observability metrics (sessions, latency, tokens) from ADOT. statement { sid = "Metrics" effect = "Allow" actions = ["cloudwatch:PutMetricData"] resources = ["*"] condition { test = "StringEquals" variable = "cloudwatch:namespace" values = ["bedrock-agentcore"] } } } resource "aws_iam_role_policy" "agent_runtime_permissions" { name = "${var.project_name}-agent-runtime-permissions" role = aws_iam_role.agent_runtime_execution.id policy = data.aws_iam_policy_document.agent_runtime_permissions.json } # AgentCore Runtime resource "aws_bedrockagentcore_agent_runtime" "agent" { count = var.agent_container_image_uri == "" ? 0 : 1 agent_runtime_name = replace("${var.project_name}_agent", "-", "_") role_arn = aws_iam_role.agent_runtime_execution.arn agent_runtime_artifact { container_configuration { container_uri = var.agent_container_image_uri } } network_configuration { network_mode = "PUBLIC" } environment_variables = { AWS_REGION = var.aws_region KNOWLEDGE_BASE_ID = aws_bedrockagent_knowledge_base.this.id GENERATION_MODEL_ID = var.generation_model_id RERANK_MODEL_ARN = var.enable_reranking ? local.rerank_model_arn : "" } } # AgentCore Observability, CloudWatch resource "aws_cloudwatch_log_delivery_source" "agent_traces" { count = length(aws_bedrockagentcore_agent_runtime.agent) name = "${var.project_name}-agent-traces" log_type = "TRACES" resource_arn = aws_bedrockagentcore_agent_runtime.agent[0].agent_runtime_arn } resource "aws_cloudwatch_log_delivery_destination" "agent_traces" { count = length(aws_bedrockagentcore_agent_runtime.agent) name = "${var.project_name}-agent-traces" delivery_destination_type = "XRAY" } resource "aws_cloudwatch_log_delivery" "agent_traces" { count = length(aws_bedrockagentcore_agent_runtime.agent) delivery_source_name = aws_cloudwatch_log_delivery_source.agent_traces[0].name delivery_destination_arn = aws_cloudwatch_log_delivery_destination.agent_traces[0].arn }

IAM Policies for Bedrock KnowledgeBase, Bedrock Invoke Models:

data "aws_iam_policy_document" "bedrock_kb_trust" { statement { actions = ["sts:AssumeRole"] principals { type = "Service" identifiers = ["bedrock.amazonaws.com"] } condition { test = "StringEquals" variable = "aws:SourceAccount" values = [data.aws_caller_identity.current.account_id] } } } resource "aws_iam_role" "bedrock_kb" { name = "${var.project_name}-kb-role" assume_role_policy = data.aws_iam_policy_document.bedrock_kb_trust.json } data "aws_iam_policy_document" "bedrock_kb_permissions" { statement { sid = "S3Read" actions = ["s3:GetObject", "s3:ListBucket"] resources = [aws_s3_bucket.docs.arn, "${aws_s3_bucket.docs.arn}/*"] } statement { sid = "InvokeEmbeddingModel" actions = ["bedrock:InvokeModel"] resources = ["arn:aws:bedrock:${var.aws_region}::foundation-model/${var.embedding_model_id}"] } # Reranking inside a Retrieve call runs as this service role (not the caller). statement { sid = "Rerank" actions = ["bedrock:Rerank"] resources = ["*"] } statement { sid = "InvokeRerankModel" actions = ["bedrock:InvokeModel"] resources = [local.rerank_model_arn] } statement { sid = "OpenSearchServerlessAccess" actions = ["aoss:APIAccessAll"] resources = [aws_opensearchserverless_collection.kb.arn] } } resource "aws_iam_role_policy" "bedrock_kb" { name = "${var.project_name}-kb-policy" role = aws_iam_role.bedrock_kb.id policy = data.aws_iam_policy_document.bedrock_kb_permissions.json }

S3 Bucket, Bucket Policy:

resource "aws_s3_bucket" "docs" { bucket = var.docs_bucket_name != null ? var.docs_bucket_name : "${var.project_name}-docs-${data.aws_caller_identity.current.account_id}" } resource "aws_s3_bucket_versioning" "docs" { bucket = aws_s3_bucket.docs.id versioning_configuration { status = "Enabled" } } resource "aws_s3_bucket_public_access_block" "docs" { bucket = aws_s3_bucket.docs.id block_public_acls = true block_public_policy = true ignore_public_acls = true restrict_public_buckets = true }

Bedrock KnowledgeBase for Vector Database Config (embedding model, embedding dimensions, opensearch serverless collection, chunking), Data Source (S3 Bucket, Vector Ingestion, Chunking Strategy):

locals { generation_model_arn = "arn:aws:bedrock:${var.aws_region}:${data.aws_caller_identity.current.account_id}:inference-profile/${var.generation_model_id}" embedding_model_arn = "arn:aws:bedrock:${var.aws_region}::foundation-model/${var.embedding_model_id}" rerank_model_arn = "arn:aws:bedrock:${var.aws_region}::foundation-model/${var.rerank_model_id}" } resource "aws_bedrockagent_knowledge_base" "this" { name = "${var.project_name}-kb" role_arn = aws_iam_role.bedrock_kb.arn knowledge_base_configuration { type = "VECTOR" vector_knowledge_base_configuration { embedding_model_arn = local.embedding_model_arn embedding_model_configuration { bedrock_embedding_model_configuration { dimensions = var.embedding_dimension } } } } storage_configuration { type = "OPENSEARCH_SERVERLESS" opensearch_serverless_configuration { collection_arn = aws_opensearchserverless_collection.kb.arn vector_index_name = opensearch_index.kb_vector_index.name field_mapping { vector_field = "bedrock-knowledge-base-default-vector" text_field = "AMAZON_BEDROCK_TEXT_CHUNK" metadata_field = "AMAZON_BEDROCK_METADATA" } } } depends_on = [aws_iam_role_policy.bedrock_kb, time_sleep.wait_for_index] } resource "aws_bedrockagent_data_source" "docs" { knowledge_base_id = aws_bedrockagent_knowledge_base.this.id name = "${var.project_name}-docs-source" data_source_configuration { type = "S3" s3_configuration { bucket_arn = aws_s3_bucket.docs.arn } } vector_ingestion_configuration { chunking_configuration { chunking_strategy = "FIXED_SIZE" fixed_size_chunking_configuration { max_tokens = var.chunk_max_tokens overlap_percentage = var.chunk_overlap_percentage } } } }

OpenSearch Serverless Config for security (encryption, network), VectorDB config (embedding dimension, hnsw, faiss, knn_vector):

resource "aws_opensearchserverless_security_policy" "encryption" { name = "${var.project_name}-encrypt" type = "encryption" policy = jsonencode({ Rules = [{ ResourceType = "collection" Resource = ["collection/${var.project_name}-kb"] }] AWSOwnedKey = true }) } resource "aws_opensearchserverless_security_policy" "network" { name = "${var.project_name}-network" type = "network" policy = jsonencode([{ Rules = [{ ResourceType = "collection" Resource = ["collection/${var.project_name}-kb"] }] AllowFromPublic = true }]) } resource "aws_opensearchserverless_collection" "kb" { name = "${var.project_name}-kb" type = "VECTORSEARCH" depends_on = [ aws_opensearchserverless_security_policy.encryption, aws_opensearchserverless_security_policy.network, ] } resource "aws_opensearchserverless_access_policy" "kb" { name = "${var.project_name}-access" type = "data" policy = jsonencode([{ Rules = [ { ResourceType = "collection" Resource = ["collection/${var.project_name}-kb"] Permission = ["aoss:*"] }, { ResourceType = "index" Resource = ["index/${var.project_name}-kb/*"] Permission = ["aoss:*"] } ...
Source：https://dev.to/omerberatsezer/agentic-rag-with-terraform-using-aws-s3-bedrock-knowledgebase-opensearch-serverless-34pd
My favorites