How CTBC Bank modernizes mainframe reverse engineering with a multi-agent pipeline on Amazon Bedrock AgentCore

For financial institutions, the hardest part of modernizing a decades-old core system is often not migrating the code but understanding what it actually does. In a regulated bank the stakes are high: the COBOL programs behind regulatory filings and business reporting encode the rules that supervisors and auditors depend on. A single misread rule can become an incorrect regulatory filing. During a core-system transformation, old and new cores must reconcile record by record and stay auditable. Automated code conversion alone doesn’t clear that bar, because reconciliation depends on knowing the original business intent. That starts with reverse engineering: recovering the original business requirements from COBOL that accumulated over two decades, written by developers who have long since moved on.

In this post, you learn how CTBC Bank moved COBOL reverse engineering from a single AI agent to a multi-agent pipeline on Amazon Bedrock AgentCore, an agentic platform for building, deploying, and operating AI agents securely at scale with any framework and foundation model (FM). A Supervisor agent coordinates seven single-responsibility sub-agents, and a quality gate checks each stage’s output before the next stage starts. You also learn how the team built in the governance and audit trail that a regulated bank needs, what the proof of concept (PoC) delivered, and five lessons for building trustworthy large language model (LLM) agents.

About CTBC Bank

CTBC Bank is the banking subsidiary of CTBC Financial Holding in Taiwan. It serves retail and corporate customers on a decades-old core system. Modernizing that core through Project NAVI is a stated group priority, alongside the bank’s broader digital and AI transformation.

Project NAVI spans CTBC Bank’s full core-system transformation, but the bottleneck is the data and reporting layer: the thousands of COBOL programs that produce the regulatory filings, internal reports, and inter-system transfers the bank depends on. Each report artifact, meaning the set of COBOL programs and jobs that produce one filing, report, or transfer, is driven by a Job Control Language (JCL) job. JCL is the scripting language that tells the mainframe which programs to run and in what order. A typical job chains six or more steps across IBM Db2 tables and Virtual Storage Access Method (VSAM) files, the mainframe’s indexed file format. It often passes data through four or five intermediate staging files that obscure the original logic. Tracing one report can span three to five COBOL programs plus copybooks, the shared source fragments that COBOL programs include.

The challenge

CTBC Bank’s first attempt paired staff with a single monolithic analysis agent driven by a prompt of nearly a thousand lines. The agent helped the team produce more than a thousand analysis documents in four months, but it hit a ceiling inherent to the single-agent pattern:

• Output drifted as the same prompt was reused across more than 100 different report types, producing inconsistent or incorrect analysis for edge cases it was never tuned for.
• Quality depended on the operator: COBOL-literate engineers cross-checked everything, while data engineers accepted about 70 percent of AI output unverified, itself a risk in a regulated bank.
• No automatic dependency tracing, so analysts had to manually chase program-to-program links to find every data source behind a report.
• Broken handoffs from Markdown to Microsoft Word to Microsoft Excel to SQL, with formatting and logic lost at each conversion step.
• A bloated prompt where each run used only 30–40 percent of the instructions, wasting tokens and increasing the chance of the model attending to irrelevant rules.

The backlog widened faster than the team could close it, slowing the transformation and delaying regulatory-reporting modernization. A different architecture was needed.

“Reverse engineering has long been one of the key challenges in core-system transformation, heavily dependent on a shrinking pool of mainframe expertise. By introducing a Multi-Agent Pipeline, we can recover the business logic embedded in legacy programs in a more efficient, consistent, and traceable way, gradually overcoming the barriers of knowledge transfer and technical handover. This allows our teams to focus more on business innovation, architecture optimization, and transformation-driven decision-making, rather than spending valuable time on repetitive analysis, ultimately accelerating the modernization of our core systems.”

— Ping Tsai, Senior Vice President, CTBC Bank

Solution overview

The platform follows an AI-Driven Development Lifecycle (AI-DLC), an AI-native approach to software delivery. In an AI-assisted process, the workflow stays the same and AI speeds up individual tasks. In AI-DLC, the workflow itself is redesigned around AI agents: agents carry out the work at each stage, from analysis and design to code and tests, while people define the intent, review the output at defined checkpoints, and remain accountable for the decisions.

The solution re-architects reverse engineering as a multi-agent pipeline on the AI-DLC Platform, which runs on AgentCore. It decomposes the near-thousand-line monolith into lean, single-responsibility sub-agents under one Supervisor, with an automated quality gate at each handoff between stages. Figure 2 in the next section shows the sub-agents and gates. The platform rests on three pillars: governance and observability (version control and an audit trail that record who changed which rule, and when), multiple agents with shared memory (agents that accumulate experience across runs), and dynamic workflow orchestration (teams compose new pipelines in the UI without writing code).

That first single-agent approach ran in an agentic integrated development environment (IDE). It helped individuals, but its output quality hinged on the operator, it couldn’t be governed or audited, and it left no reusable knowledge behind. The AI-DLC Platform turns COBOL reverse engineering from a personal skill into a governed, team-level capability:

• Quality is enforced by automated gates that measure coverage, mapping completeness, and SQL validity at each stage, rather than relying on operator expertise.
• Auditability comes from the platform’s control plane, which tracks every step.
• Knowledge compounds across runs through Amazon Bedrock AgentCore memory, a fully managed service that gives agents short-term and long-term memory. For example, when the SQL gate fails, the platform feeds the exact Oracle error back to the SQL agent and stores the correction in long-term memory, so later artifacts retrieve that lesson through semantic search instead of repeating the mistake.
• Model choice is set per agent on the console. In the PoC, most agents ran on Anthropic Claude Opus. Routing simpler classification and formatting tasks to the lower-cost Claude Sonnet is a planned next step for cost control.

The AI-DLC Platform is an internal AWS prototype, built and validated with CTBC Bank. It isn’t an AWS service or a publicly available sample, and customers who want a similar implementation can work with AWS through their account team. The reverse-engineering pipeline described here is its first proven use case. The underlying pattern, a Supervisor with single-responsibility sub-agents and quality gates checked in code, can be built on AgentCore and the Strands Agents SDK, an open source SDK that takes a model-driven approach to building AI agents in a few lines of code.

Figure 1 shows how a request flows through the platform:

1. Amazon CloudFront serves the single-page web UI from Amazon Simple Storage Service (Amazon S3), and the user signs in through Amazon Cognito.
2. UI calls go to Amazon API Gateway, which validates the user’s JSON Web Token (JWT) and forwards the request to an AWS Lambda function that acts as the governance control plane. The control plane records every call in an audit trail in Amazon DynamoDB, alongside the agent registry, prompt versions, and run state, and sends metrics, logs, and token usage to Amazon CloudWatch.
3. To start a run, the control plane asynchronously invokes a run-worker Lambda function, which calls the AgentCore InvokeAgentRuntime API. Workflows that users define in the UI run on AWS Step Functions and use the same entry point.
4. In AgentCore runtime, a capability of Amazon Bedrock AgentCore that hosts agents in a secure, serverless environment, the Supervisor runs the seven sub-agents and applies a quality gate after each stage, as Figure 2 shows.
5. The agents call Anthropic Claude models in Amazon Bedrock for inference.
6. Agents reach their tools through AgentCore Gateway, a capability of Amazon Bedrock AgentCore, which exposes Lambda functions as Model Context Protocol (MCP) tools. MCP is an open protocol for connecting AI agents to tools and data. The tool functions run in isolated private subnets of an Amazon VPC. They parse COBOL and JCL, read the COBOL index in DynamoDB and the source files in Amazon S3 through VPC endpoints, and run SQL and unit tests on Amazon Relational Database Service (Amazon RDS) for Oracle.
7. AgentCore memory carries context between agents and lessons across runs. AgentCore Identity, a capability of Amazon Bedrock AgentCore, manages agent identities and credentials, and AgentCore Observability, a capability of Amazon Bedrock AgentCore, records a trace for every run.

Key design decisions

The user starts with a single field: the artifact’s physical file name. From there, the seven sub-agents run the flow shown in Figure 2. The following design decisions shaped each stage of the pipeline.

A COBOL index replaces manual search. An offline Indexer parses the COBOL and JCL structure into a dependency graph keyed by program, file, and Db2 table. At run time, the Scanner agent gets precise dependencies, resolved from the abstract syntax tree (AST), in under a second, where analysts previously needed two to three rounds of manual search. Every result is traceable to the source program it came from.

The Lineage and Logic agents run in parallel and merge at Bridge, shortening wall-clock time while keeping each prompt focused. The Bridge agent closes what was the biggest manual gap (reverse-engineering output to target-model mapping) by cross-checking the program chain against the business-rule signals Logic captured.

Functional specification before SQL, validated for real. The FSD Writer agent consolidates everything into a business-readable functional specification document (FSD), and gate G3 measures its mapping completeness. The SQL agent then derives Oracle SQL and extract, transform, and load (ETL) scripts from the FSD and the Bridge mappings (G4). Because the new-core target tables don’t exist yet, the platform parses each statement with Oracle DBMS_SQL.PARSE without executing it and treats “object does not exist” errors, such as ORA-00942, as a pass. This catches real syntax errors before the target schema exists. The Unit Test agent then checks execution correctness (G5): it creates fixture tables with a reserved test prefix, loads test data, and runs each case inside a savepoint that is rolled back afterward. Destructive SQL is rejected unless every table it touches carries that prefix.

Five quality gates, measured in code, not by LLM self-assessment. Each gate is an automated checkpoint with a fixed, quantitative threshold. In one test run, an agent gave its own deficient output full marks, so the platform computes every gate metric in code instead of trusting the model’s score. G1 is a hard stop: if the Scanner agent misses a critical component, the run ends, because every later stage depends on it. G2 through G5 retry up to twice, injecting correction guidance into the next prompt, and then continue with the best result. Thresholds, strictness, and retry counts are configurable per report type. People stay in the loop: in step mode, the pipeline pauses after each stage so a reviewer can approve the output, reject it, or rerun the stage with revision notes. In auto mode, it runs straight through.

Built-in governance and auditability for AI agents. Regulated financial institutions need to be able to trace agent invocations. The platform’s governance control plane fronts every request and records each one in an audit trail, alongside the agent registry, prompt versions, and health monitoring. Each prompt version keeps its author, timestamp, and revision note, and agents read the active version at run time, so a prompt change takes effect on the next run without redeployment and the version history shows who made it. By default, Amazon Bedrock doesn’t store or review prompts and completions, doesn’t share them with third-party model providers, and doesn’t use them to train models.

Results

This is a PoC, so read the numbers at three levels of confidence: customer-accepted efficiency, engineering facts, and key performance indicator (KPI) targets.

Customer-accepted efficiency

Measured in person-days on the same inputs, the earlier single-agent assistant had already cut the work for one report artifact from about 8 person-days to about 6, roughly 1.3×. The multi-agent pipeline takes it to about 2, or about 4× overall. This suggests that much of the additional gain comes from decomposition and quality gates rather than from introducing an LLM. The PoC didn’t isolate that factor from other changes such as revised prompts and tooling. Figure 3 compares the three approaches.

Accepted gains by stage were about 5× for document analysis, about 4.7× for report-program generation, and about 10× for Oracle SQL and ETL generation. In end-to-end wall-clock time, the pipeline completed one artifact in about a day, and the simplest of the three artifacts in under 30 minutes. CTBC Bank estimates this is roughly 10× faster than an equivalent manual pass. The team had no instrumented baseline for manual elapsed time, so read the 4× person-day figure as the measured result and the 10× figure as the customer’s estimate.

Engineering facts

The monolithic agent was decoupled into eight agents (one Supervisor and seven sub-agents) with five automated gates and bounded retries. The pipeline was validated end to end in eight weeks on three report artifacts, compared against the customer’s ground-truth FSD. Index lookups return in under a second, and cross-run knowledge accumulates through AgentCore memory.

KPI targets (targets, not results)

Turnaround under one day, business-logic accuracy in the FSD of at least 90 percent, 100 percent SQL validity, test coverage of at least 80 percent, no more than two manual interventions per artifact, and cross-operator variance below 5 percent.

Lessons learned: Trustworthy LLM agents in a regulated bank

Each of the following five lessons came from a failure in testing, and none was fixed by a better prompt alone. In each case, the team moved a guarantee out of the model and into the code around it: the tools an agent might call, the output contract it must follow, the checks that run on its output, and the corrections fed back when a check fails.

• Encode classification rules, not templates – An early agent inflated a specification’s filters to about 30, a 3.3× bloat. After the team aligned the agent’s classification rules to the customer’s ground-truth FSD, it produced exactly nine.
• Write rules as mandatory actions, not conditional triggers – An LLM will evaluate a condition and then bypass it. The team therefore made the Scanner agent deterministic: it expands every COPY’d copybook, resolves every CALLed program, matches the call chain to the index entry by entry, and trusts only tool-reported sources. One missed dependency can drop a source that affects a filing amount.
• Truncation is silent – When an agent reaches its max_tokens limit, no exception is raised: the response ends with a max_tokens stop reason, and the output stops, possibly mid-JSON. Each agent checks for an expected end marker ([METRICS]) and, if the marker is missing, automatically requests a continuation. Without that check, truncated output can look complete and professional while being wrong.
• Semantics degrade across agents – Translating old-core COBOL into new-core specifications drifts three ways: flattening fallback chains into ANDs, dropping conditional writes, and leaking old-core names into the spec. This drift amplifies stage by stage. The fix is a cross-agent semantic contract (branch_chain, filter_role, conditional_writes, existence_check) plus an anti-pattern section with real line numbers in every prompt.
• Don’t assume UTF-8 – Mainframe source is stored in Extended Binary Coded Decimal Interchange Code (EBCDIC), and when it’s exported, comments come out in a regional encoding, which was Big5 in CTBC Bank’s exports. The platform tries an ordered list of encodings (UTF-8, then Big5, then Shift_JIS, then latin-1) to recover the comments. The risk isn’t visible garbling but silent quality loss: in one run, garbled comments led the Scanner agent to rate a program as hard to read, and no error was raised.

Conclusion

The lasting outcome is not the speed gain alone but the platform underneath it: reverse engineering that is governed, auditable, and improves with use, with quality enforced by automated gates rather than operator expertise. For CTBC Bank, that turns regulatory-reporting modernization from a bottleneck into a repeatable, auditable process. Because pipelines are composed from configured agents rather than hard-coded, the same pattern can extend to other business domains. That is the AI-DLC path: from an AI assistant, to agent automation with people at the checkpoints, to people who train and govern the agents.

If you are modernizing a core system, here’s how to get started:

1. Review the Amazon Bedrock AgentCore documentation to see how Runtime, Gateway, Memory, Identity, and Observability fit together.
2. Build a first agent with the Strands Agents quickstart.
3. Contact your AWS account team to explore a similar implementation for your own mainframe reverse engineering and regulatory reporting.

 

---

About the authors