How to Master AWS for AI?

Ciao Amici 👋

Every hiking trail in the Alps has a map at the trailhead. A good one shows you the huts, the peaks and the river crossings. What it never shows you is the stretch where the path is a loose scree slope, or the hut that closes in October, or the signpost that points you towards the wrong valley. For that, you need someone who has walked it.

A few days ago a roadmap called "How to Master AWS for AI?" landed in my feed, drawn by Umang Chaudhary (@mlwithumang). It is a lovely trailhead map: one centre (Amazon Bedrock), six branches (Foundations, Data and Compute, Amazon Bedrock, AI Agents, Knowledge and AI Services, Ops and Governance) and a suggested five-step learning path along the bottom. I have built on several of these services myself, so I did the obvious thing. I walked the map against my own scars.

This is a long article on purpose. It follows the five steps in order, and for each one it gives you three things: what is verifiably true today (as of 30 October 2026), code you can run, and the places where I would move the signposts. Every code sample in this article was executed before I wrote it down, and none of it needs an AWS account or costs a cent. I will tell you exactly how I tested it, and exactly what I did not test, at the end.

What is in here

1. The map, and my three changes to it
2. Step 0: money and blast radius (budgets, token ceilings, least privilege, logging)
3. Step 1: foundations (reading an AccessDenied properly)
4. Step 2: models and prompts (the Converse API, token accounting, stop reasons)
5. Step 3: grounding and RAG (S3 Vectors, DynamoDB vectors, and a retrieval test you can run offline)
6. Step 4: agents (Strands, AgentCore, and the two rules of distributed systems that break agents)
7. Step 5: evaluate and govern (guardrails at the tool boundary, evaluation, audit)
8. A failure cheat sheet, a four-week plan, and what I did not verify

The map in one table

Here is the learning path from the roadmap, in my words:

It is a sound order for a first pass. My notes are about what a map cannot show, because it is a map and not a diary. Here is the same path with my three changes highlighted. I will explain each one as we go:

Step 0: money and blast radius

The roadmap lists Budgets and Cost Explorer under Core Setup, which I love, but I would pull it out of the pile and put it before everything else. AI services bill by usage, and usage is exactly what an eager experiment or a runaway loop produces. There are four layers of protection, and they do different jobs.

Layer 1: an alert (slow, but it reaches you). AWS Budgets can email you when actual spend passes a percentage of a monthly limit. Here is a budget with two alerts, at 50 and 80 percent of 10 US dollars. I did not guess the structure: I validated it against the SDK's own model for create_budget (more on that in the testing section).

{ "Budget": { "BudgetName": "ai-experiments-monthly", "BudgetType": "COST", "TimeUnit": "MONTHLY", "BudgetLimit": { "Amount": "10", "Unit": "USD" } }, "NotificationsWithSubscribers": [ { "Notification": { "NotificationType": "ACTUAL", "ComparisonOperator": "GREATER_THAN", "Threshold": 50, "ThresholdType": "PERCENTAGE" }, "Subscribers": [{ "SubscriptionType": "EMAIL", "Address": "you@example.com" }] }, { "Notification": { "NotificationType": "ACTUAL", "ComparisonOperator": "GREATER_THAN", "Threshold": 80, "ThresholdType": "PERCENTAGE" }, "Subscribers": [{ "SubscriptionType": "EMAIL", "Address": "you@example.com" }] } ] } 

Create it with client.create_budget(AccountId=..., **json.load(open("budget.json"))) using boto3 and the budgets client. Be honest about what this layer is: an alert, not a brake. It reacts to billing data, which arrives with a delay, so it tells you that you have a problem rather than stopping the problem.

Layer 2: a brake inside your code. The cheapest protection is a ceiling you enforce yourself. This tiny wrapper counts the tokens from every Converse call, refuses to call again once you pass a limit, and writes a ledger line per call so you can see where the tokens went:

# code/token_budget.py """A hard token ceiling around Bedrock Converse, with a local usage ledger.""" import json import time class BudgetExceeded(Exception): pass class TokenBudget: def __init__(self, max_tokens: int, ledger_path: str = "usage.jsonl"): self.max_tokens = max_tokens self.used = 0 self.ledger_path = ledger_path def converse(self, client, **kwargs): if self.used >= self.max_tokens: raise BudgetExceeded(f"{self.used} of {self.max_tokens} tokens used") response = client.converse(**kwargs) usage = response["usage"] self.used += usage["totalTokens"] with open(self.ledger_path, "a", encoding="utf-8") as f: f.write(json.dumps({ "ts": time.time(), "model": kwargs.get("modelId"), "input": usage["inputTokens"], "output": usage["outputTokens"], "total_used": self.used, }) + "\n") return response 

Note that the check happens before the call, so one call can overshoot the ceiling by up to one response. That is fine for a safety net. The test that proves it stubs Bedrock locally, makes two calls of 100 tokens against a ceiling of 150, and confirms that the third call is refused and the ledger shows two lines.

Layer 3: least privilege. Calling Converse requires the bedrock:InvokeModel permission, and ConverseStream requires bedrock:InvokeModelWithResponseStream. So a role that may only call one cheap model looks like this:

{ "Version": "2012-10-17", "Statement": [ { "Sid": "InvokeOneModelOnly", "Effect": "Allow", "Action": ["bedrock:InvokeModel", "bedrock:InvokeModelWithResponseStream"], "Resource": "arn:aws:bedrock:us-east-1::foundation-model/amazon.nova-micro-v1:0" } ] } 

An agent, a notebook or a Lambda that runs under this role cannot suddenly start calling an expensive model, because the policy does not let it.

Layer 4: know what you are logging. Bedrock model invocation logging is off by default. When you turn it on it records the full request and response bodies, token counts and the caller identity for supported calls, and it can write to S3 and to CloudWatch Logs. That is excellent for debugging and for audit, and it also means your log bucket now contains your users' prompts. Encrypt it, restrict access, and set your own retention, because the logs are kept until you delete the configuration. You can tag calls with requestMetadata in the Converse request so you can filter the logs later.

One more money fact that surprises people: cross-Region inference profiles let a request be served by compute in several Regions. AWS states there is no extra routing cost and that the price is based on the Region from which you call the profile. Geographic profiles use standard pricing and global profiles are described as approximately 10 percent cheaper. They also have a governance side: if your organisation uses Service Control Policies to restrict Regions, the destination Regions of the profile must be allowed, and global profiles need an allowance for aws:RequestedRegion set to unspecified. Do not discover this on the day you deploy.

Rule I follow: if I cannot explain where the money stops, I do not press deploy.

Step 1: foundations, and how to read an AccessDenied

IAM, KMS, Secrets Manager, VPC endpoints, the CLI and CDK. Nobody gets excited about this branch, and everybody who skips it pays for it later.

I once wrote a whole article about the fact that the same AWS error can have two different answers, and that guessing which one you are facing is how an afternoon disappears. With AI workloads you meet that early. A model call fails, an agent cannot reach a tool, a Lambda cannot read a bucket, and the message is the same polite "access denied". The skill is to stop guessing and walk a short list, in order:

The commands behind that list are boring and worth memorising:

aws sts get-caller-identity # who am I, really? (wrong role is the most common cause) aws iam simulate-principal-policy \ --policy-source-arn <role-arn> \ --action-names bedrock:InvokeModel \ --resource-arns arn:aws:bedrock:us-east-1::foundation-model/amazon.nova-micro-v1:0 

The simulator answers the identity-policy question quickly. If it says allowed and the call still fails, the refusal is coming from somewhere else in the chain: a Service Control Policy or permissions boundary, a resource policy, a KMS key policy, or the Region and model you are actually calling. CloudTrail records the call with its error code and message, which is the receipt you want when two causes look identical.

On KMS, one design rule from the vector services is worth knowing now. When you create an S3 Vectors bucket you choose its encryption type (SSE-S3 or a KMS key) at creation time, and you cannot change it afterwards. Choosing encryption is a day-one decision, not a later hardening step.

Step 2: models and prompts

Amazon Bedrock gives you one API over many model families: Amazon Nova, Anthropic Claude, Meta, Mistral and others. For new code, the Converse API is the one to learn, because it has a consistent request and response shape across models that support messages. Here is the minimum:

import boto3 client = boto3.client("bedrock-runtime", region_name="us-east-1") response = client.converse( modelId="amazon.nova-micro-v1:0", # or an inference profile ID or ARN  system=[{"text": "You are a concise AWS tutor."}], messages=[{"role": "user", "content": [{"text": "What is idempotency, in two sentences?"}]}], inferenceConfig={"maxTokens": 300, "temperature": 0.2}, ) text = response["output"]["message"]["content"][0]["text"] usage = response["usage"] # inputTokens, outputTokens, totalTokens stop = response["stopReason"] # do not ignore this 

Three habits are hiding in those lines.

Read the stop reason. The SDK model lists these values: end_turn, tool_use, max_tokens, stop_sequence, guardrail_intervened, content_filtered, malformed_model_output, malformed_tool_use and model_context_window_exceeded. Only the first one means "the model finished its thought". A truncated answer (max_tokens) that your code treats as complete is a bug that looks like a bad model.

Count tokens on every call. The usage block is your cost meter. Input tokens include everything you send: the system prompt, the conversation history and, later in Step 3, the retrieved passages. Output tokens are what the model writes. That is why a chatty prompt and a large retrieval depth both show up on the bill, and why the ledger from Step 0 pays for itself.

Use the knobs that exist. Beyond inferenceConfig, Converse accepts additionalModelRequestFields for model-specific parameters, requestMetadata for tagging calls in your logs, guardrailConfig for attaching a guardrail, and serviceTier, whose documented values are reserved, priority, default and flex. Prompt caching works by adding cachePoint blocks, and which blocks a model accepts depends on the model. A detail from the docs that bites people: if you call a prompt from Prompt management, you cannot also pass system, inferenceConfig, toolConfig or additionalModelRequestFields, because the managed prompt owns them.

Tool use, which Step 4 builds on, is part of the same API. You describe tools in toolConfig, the model answers with stopReason set to tool_use and a toolUse block (an id, a name and an input), you run the tool, and you send the outcome back as a toolResult for that id with a status of success or error. Every agent framework is, underneath, a loop around that exchange.

The thing I would add here is a habit, not a service. A model that sounds confident is not a model that is right. I built a small game, gaslight-bench, to find out whether a model can be talked into agreeing with false statements about AWS. The game is playful, but the question behind it is serious: if a model can be pushed off the facts by a confident voice, its own tone can never be your evidence. And the first piece of evidence you can build is tiny: five questions you know the answers to, run every time you change a prompt or a model. That is where evaluation starts, and it starts here, not in Step 5. We will turn it into a real harness in Step 3.

Step 3: grounding and RAG

If you want the model to answer from your documents, the system has two halves, and it helps to keep them separate in your head:

The ingest half runs when your documents change: split them into chunks, turn each chunk into an embedding (a list of numbers), and store the embedding with the chunk's text and some metadata. The query half runs on every question: embed the question, fetch the nearest chunks, put them in the prompt, and ask the model to answer from them. Most RAG problems are in the middle: the right chunk never comes back, so the model never sees the answer, and then it improvises.

The roadmap lists four places to keep embeddings: S3 and S3 Vectors, OpenSearch Serverless, DynamoDB vector search and Aurora pgvector. Two of those have changed a lot recently, so here is what I could verify.

Amazon S3 Vectors became generally available on 2 December 2025. AWS states up to 2 billion vectors per index and up to 10,000 indexes per vector bucket, with queries around 100 ms for frequent use and under a second for infrequent ones, and claims up to 90 percent lower total cost for uploading, storing and querying vectors. It plugs into Bedrock Knowledge Bases, and into OpenSearch for hybrid search. The SDK model tells you the practical constraints: the only data type is float32, the distance metric is cosine or euclidean, and the index name, dimension, distance metric and non-filterable metadata keys are fixed when you create the index. That last point is the one to plan around: if you change your embedding model later, the dimension may change, and you will re-embed and re-index everything into a new index.

Amazon DynamoDB vector search became generally available on 5 August 2026. You store embeddings next to your normal item attributes, define a vector index (at table creation through VectorIndexes, or later through VectorIndexUpdates), and search with the SearchVectors operation, which takes the index name, the query vector, a TopK and an optional condition expression to filter on item attributes. The distance functions are COSINE, DOT_PRODUCT and EUCLIDEAN, and AWS states a target of single-digit millisecond latency at 99 percent plus recall. One detail trips people up: DynamoDB does not create the embeddings for you. You generate them with a model of your choice (a Bedrock model works) and store them. Write-ups of the launch also say that vector indexes require on-demand capacity and cap the number of results returned per search, so read the current limits page before you design around either.

Here is the same embedding stored both ways. The calls are shaped exactly to the SDK's current models (I validated every parameter offline, see the testing section), and the embed function is the only part that spends money:

# code/vector_stores.py """Two ways to store the same embedding on AWS. Calls are shaped to the current SDK models. Not run against a live account in my tests: they are validated offline (see test_vectors.py). """ import boto3 EMBED_MODEL = "amazon.titan-embed-text-v2:0" # check the model card for the request body you use  def embed(text: str, dims: int = 1024) -> list: """Real embedding call (costs a little per token). Swap in any embedding model.""" import json bedrock = boto3.client("bedrock-runtime") resp = bedrock.invoke_model(modelId=EMBED_MODEL, body=json.dumps({"inputText": text, "dimensions": dims, "normalize": True})) return json.loads(resp["body"].read())["embedding"] # ---- S3 Vectors ---------------------------------------------------------------- def s3_vectors_setup(s3v, bucket: str, index: str, dims: int = 1024): s3v.create_vector_bucket(vectorBucketName=bucket) s3v.create_index( vectorBucketName=bucket, indexName=index, dataType="float32", dimension=dims, distanceMetric="cosine", # Big text you only want back, never filter on: mark it non-filterable (immutable later).  metadataConfiguration={"nonFilterableMetadataKeys": ["source_text"]}, ) def s3_vectors_put(s3v, bucket, index, items): """items: [(key, embedding, {"source_text": ..., "topic": ...}), ...]""" s3v.put_vectors( vectorBucketName=bucket, indexName=index, vectors=[{"key": k, "data": {"float32": e}, "metadata": md} for k, e, md in items], ) def s3_vectors_query(s3v, bucket, index, query_embedding, top_k=3, topic=None): kw = {"filter": {"topic": topic}} if topic else {} resp = s3v.query_vectors( vectorBucketName=bucket, indexName=index, topK=top_k, queryVector={"float32": query_embedding}, returnMetadata=True, returnDistance=True, **kw, ) return resp["vectors"] # each has key, distance, metadata  # ---- DynamoDB vector search ---------------------------------------------------- def ddb_create_table_with_vector_index(ddb, table: str, dims: int = 1024): ddb.create_table( TableName=table, BillingMode="PAY_PER_REQUEST", # vector indexes need on-demand capacity (per AWS write-ups; check docs)  AttributeDefinitions=[{"AttributeName": "pk", "AttributeType": "S"}], KeySchema=[{"AttributeName": "pk", "KeyType": "HASH"}], VectorIndexes=[{ "IndexName": "by-embedding", "VectorAttribute": {"AttributeName": "embedding"}, "Dimensions": dims, "DistanceFunction": "COSINE", "Projection": {"ProjectionType": "ALL"}, }], ) def ddb_search(ddb, table: str, query_embedding, top_k=3): resp = ddb.search_vectors( TableName=table, IndexName="by-embedding", TopK=top_k, SearchVector=[{"N": repr(float(x))} for x in query_embedding], ) return resp["SearchResults"] 

How do I choose? Check the current documentation for limits and pricing, then pick the boring option that matches the shape of your data:

The test that matters more than the database

Choosing a store is the easy decision. The hard question is "does the right chunk come back?", and you can answer it with a page of Python, offline, before you spend anything. A retrieval test needs a golden set (questions with the chunk that should answer each one) and two numbers: recall@k (how often the right chunk is in the top k) and MRR (how high it ranks, on average).

Here is the whole harness. The embedding in it is a deliberately simple hashed bag of words, a stand-in so the harness runs anywhere; swap embed() for a real model and keep everything else:

# code/retrieval_eval.py """A retrieval test you can run with no AWS account: recall@k and MRR over a golden set. The embedding here is a hashed bag of words, a stand-in so the harness runs anywhere. Swap embed() for a real model call (see the Bedrock version in the article) and keep the rest. """ import re import zlib import numpy as np CHUNKS = { "c1": "S3 event notifications are delivered at least once so handlers must tolerate duplicates", "c2": "Lambda retries failed asynchronous invocations up to two more times by default", "c3": "The sequencer field in an S3 event orders events for the same object key"...