When we want to integrate vision models into a backend pipeline, the main problem is output reliability. Even if you explicitly instruct the model to return JSON only in the prompt, json.loads() will eventually fail because the output format is not enforced at the API level. The model can still wrap the payload in Markdown code blocks, add conversational text before or after the object, or omit keys when it's uncertain about the data, or even return numbers as strings instead of actual integers
Let's see how to send a receipt image to Claude on Amazon Bedrock using the converse API and enforce a strict JSON Schema via outputConfig. The output is guaranteed to match the schema and parse directly with json.loads().
The model can easily read the image. The part that matters in a real application is getting the answer back in a form your code can use without cleanup.
Alright, you know Python and have an AWS account where aws login works. We will work in us-east-1. Claude has to be enabled in the account. If it's your first time with Anthropic models, Bedrock will ask you for a short use case form. The script you can find in my repo https://github.com/Foxfix/agenticai_mini_course
How an image gets to the model
In the Converse API a message has a content field, and that field is a list of blocks. In previous cases I've only put text blocks in it, but a block can also be an image. An image block has two parts: the raw bytes of the file and its format, png in our case. I put the image block first, then a text block with the instruction, and the model treats them as one request.
response = bedrock.converse( modelId=CLAUDE, messages=[{"role": "user", "content": [ {"image": {"format": "png", "source": {"bytes": image}}}, {"text": "Read this receipt and extract the data."}, ]}], We don't have to base64-encode anything or upload the file to S3. I read the file in binary mode and boto3 sends the bytes as they are. Nova models accept the same structure, so switching models later means changing modelId and nothing else.
Why a schema instead of a prompt
I thought the easy way to get JSON is to write "return JSON only" in the prompt. It works for most calls. Then one response comes back wrapped in a markdown code fence, another starts with "Here is the data:", and a third leaves out the date because the model wasn't sure about it. All of these end in a JSONDecodeError in my code.
Structured output solves this by moving the format out of the prompt and into the request. I pass a JSON schema in outputConfig, and the text that comes back follows that schema. The schema for our receipt has: the store name, a date, a list of items (each with a name, a quantity and a price) and a total.
-
requiredlists the fields that must be present. Without it the model can skip a field it isn't sure about. additionalProperties: Falseforbids any field that isn't in the schema. Without it the model can add its own.The
datefield also has adescription. The model reads descriptions as instructions, so "YYYY-MM-DD" there tells it which date format you want.
The schema goes into the request as a string, which is why the script callsjson.dumps(schema). It looks odd, but that's the format the API expects.
schema = { "type": "object", "properties": { "store": {"type": "string"}, "date": {"type": "string", "description": "YYYY-MM-DD"}, "items": {"type": "array", "items": {"type": "object", "properties": { "name": {"type": "string"}, "quantity": {"type": "integer"}, "price": {"type": "number"}}, "required": ["name", "quantity", "price"], "additionalProperties": False}}, "total": {"type": "number"}, }, "required": ["store", "date", "items", "total"], "additionalProperties": False, } response = bedrock.converse( modelId=CLAUDE, messages=[{"role": "user", "content": [ {"image": {"format": "png", "source": {"bytes": image}}}, {"text": "Read this receipt and extract the data."}, ]}], outputConfig={"textFormat": {"type": "json_schema", "structure": {"jsonSchema": { "name": "receipt", "schema": json.dumps(schema)}}}}, ) Complete script
The script creates a bedrock-runtime client and reads receipt.png as bytes. Then it defines the schema. The call to converse sends one user message with two blocks, the image and a short instruction. The instruction can be short because the schema already describes the output. The answer lives in response["output"]["message"]["content"][0]["text"], which is a JSON string, so json.loads turns it into a normal dictionary. The last two lines print the result and a quick summary.
import sys, json sys.path.insert(0, __file__.rsplit("/", 2)[0]) import boto3 from config import REGION, CLAUDE, DATA bedrock = boto3.client("bedrock-runtime", region_name=REGION) image = open(f"{DATA}/receipt.png", "rb").read() schema = { "type": "object", "properties": { "store": {"type": "string"}, "date": {"type": "string", "description": "YYYY-MM-DD"}, "items": {"type": "array", "items": {"type": "object", "properties": { "name": {"type": "string"}, "quantity": {"type": "integer"}, "price": {"type": "number"}}, "required": ["name", "quantity", "price"], "additionalProperties": False}}, "total": {"type": "number"}, }, "required": ["store", "date", "items", "total"], "additionalProperties": False, } response = bedrock.converse( modelId=CLAUDE, messages=[{"role": "user", "content": [ {"image": {"format": "png", "source": {"bytes": image}}}, {"text": "Read this receipt and extract the data."}, ]}], outputConfig={"textFormat": {"type": "json_schema", "structure": {"jsonSchema": { "name": "receipt", "schema": json.dumps(schema)}}}}, ) data = json.loads(response["output"]["message"]["content"][0]["text"]) print(json.dumps(data, indent=2)) print("Items:", len(data["items"]), "· total:", data["total"]) Let's run it
python genai/a04_receipt.py I used the BrightCart receipt from the repo with three items, tax and a total of 207.83 dollars.
This is what came back:
{ "store": "BRIGHTCART", "date": "2026-10-08", "items": [ {"name": "Wireless headphones", "quantity": 1, "price": 129.0}, {"name": "USB-C charger x2", "quantity": 2, "price": 50.0}, {"name": "Screen protector", "quantity": 1, "price": 12.99} ], "total": 207.83 } The structure is guaranteed by the schema, but the values still come from the model's reading of the photo.
129 + 50 + 12.99 = 191.99. The difference to the total is 15.84, which is the tax on the receipt, so the numbers are consistent. In a real application you'd turn this into a check in code. A minimal version:
subtotal = sum(i["price"] for i in data["items"]) print("Subtotal:", round(subtotal, 2), "| total:", data["total"]) If the subtotal is larger than the total, the model misread something and the receipt should go to a person.
If the subtotal is larger than the total, the model misread something and the receipt should go to a person.
The price field is ambiguous
Look at the chargers: quantity 2, price 50.0. That is the price for the whole line, 25 per charger. The model didn't make a mistake. The schema says "price" and doesn't say which price you mean, so the model picked the one printed on the receipt line. If you need the price of one unit, say it in the description:
"price": {"type": "number", "description": "price of ONE unit"} The same applies to every field: the model follows what the schema says, not what you had in mind, so spend a minute on the descriptions.
You can check how much this will cost you in the video I recorded after completing the entire course. Even with all the model requests, it came out to less than $1. Since our receipt experiment is just one single lesson, it should literally cost pennies.
Build & Deploy an Enterprise AI
If you get "This model is not available for this account", you can fix it on the model's page in the Bedrock console. If you can't resolve this issue, it's easy to switch to another model available on your account, for example, Amazon Nova.
I haven't tried crumpled paper or receipts in other languages.
All code: github.com/Foxfix/agenticai_mini_course
I make hands-on videos about AWS, AI and deployment on my YouTube channel, @CodeAiFlow. Every article in this series has a matching episode there, if you'd rather watch it built step by step. I'm also an AWS Community Builder: my profile on AWS Builder Center.
