Free AWS Security Assessment Toolkit: 42 Checks, a Local GUI, and Client-Ready HTML Reports

An engineering walkthrough of a lightweight read-only AWS security assessment toolkit with 42 controls, CLI, local dashboard, baseline diff, suppressions, and self-contained HTML reporting

There is a classic trap in cloud security: looking at a green dashboard and deciding that "AWS is secure."

I do not work that way. When I start an AWS security assessment, I first want fast, boring answers to boring questions:

**- Is MFA enabled for the root user?

• Do root access keys exist?
• Are there IAM users without MFA?
• Have access keys been living forever?
• Is SSH or RDP open to the internet?
• Are S3 buckets public?
• Is encryption enabled where I expect it?
• Are CloudTrail and AWS Config configured?
• Are GuardDuty, Security Hub, and Inspector actually enabled?
• Do EC2 instances require IMDSv2?
• Do RDS and DynamoDB have reasonable recovery controls?
• Is an EKS API endpoint accepting 0.0.0.0/0?
• Has ordinary cloud hygiene debt started piling up?**

This is not sexy zero-day research. It is plumbing. But a surprising amount of real-world security posture is built from exactly this plumbing.

That is why I built an AWS Security Assessment Toolkit, a small read-only tool for the first technical pass over an AWS account.

It grew out of consulting work I was doing through Bulwark Advisory, but this article is not a services pitch. It is an engineering walkthrough: what I wanted to automate, why I did not try to build another giant CSPM, where automation helps, and where a human still has to think.

At the time of writing, this article describes toolkit v2.0.0.

TL;DR

If you are short on time, here is the whole idea.

The toolkit performs a point-in-time AWS posture assessment through read-only APIs.
The current release includes:

• 42 controls;
• one assessment engine shared by CLI and local GUI;
• cross-account scanning through STS AssumeRole;
• baseline/diff support;
• suppressions and waivers with reasons and expiration;
• Console, JSON, Markdown, HTML, CSV, SARIF, and JUnit outputs;
• a self-contained client-facing HTML report;
• scheduled-run support for drift detection;
• no automatic remediation of target resources.

And the most important caveat:

42 PASS results do not mean "AWS is secure." They mean that 42 specific checks, in a specific scope, returned those results at a specific point in time.

That principle shaped most of the design.

New to cloud security? Five terms in sixty seconds

A little vocabulary before we go deeper.

Security posture

The current state of security-relevant configuration. Is a Security Group open? Is encryption enabled? Is an audit trail present?
Control

One concrete rule we evaluate. For example: "The root user should have MFA."
Finding

The result of evaluating a control. A finding contains status, severity, resource context, evidence, and remediation guidance.
Baseline

A saved result from an earlier assessment. It gives us something stable to compare against.
Drift

A configuration change relative to that baseline. Drift can be good, bad, or neutral.
CSPM

Cloud Security Posture Management, the category of products and services that continuously or periodically evaluate cloud configuration, standards, and misconfigurations.

In plain English, this toolkit is closer to a building inspection than a break-in attempt.

A pentest asks:

"Can I get through the window?"

A posture assessment first asks:

"Is the window even closed?"

Where the project started

I liked the core idea: understandable read-only boto3 checks, very little magic, and a fast point-in-time result.

For my own workflow, I wanted more:

• a modular architecture;
• a control registry;
• one normalized finding model;
• broader AWS coverage;
• cross-account assessment;
• baseline and diff support;
• documented suppressions;
• CI-friendly outputs;
• a local GUI;
• client-ready HTML;
• and a hard read-only boundary.

Over time, the seed became a separate toolkit.

The problem I actually wanted to solve

The AWS Shared Responsibility Model separates security of the cloud from security in the cloud.

The "in the cloud" part gets interesting fast. AWS can secure the physical infrastructure perfectly while the customer still has:

• overly permissive IAM;
• public buckets;
• Security Groups open to 0.0.0.0/0;
• weak logging coverage;
• missing recovery controls;
• long-lived credentials;
• unused keys;
• poorly scoped public endpoints.

The AWS Well-Architected Security Pillar emphasizes a strong identity foundation, traceability, defense in depth, automation of security best practices, and protection of data.

This is exactly where configuration assessment is useful as a first filter. AWS Security Hub CSPM already evaluates environments against standards such as AWS Foundational Security Best Practices and CIS AWS Foundations Benchmark. I treat those as important reference points for control design, not as competitors that my tool needs to "beat."

My requirement was narrower:

I wanted a portable first-pass scanner that I understand end to end, can adapt to an assessment workflow, can run under a read-only role, and can turn into a useful deliverable.

What the toolkit checks today

Version 2.0.0 registers 42 controls across ten categories:

These are not 42 vulnerability signatures and they are not 42 proofs of exploitability.

They are 42 deterministic posture checks.

For example, EC2-002 answers a narrow question:

Does this instance require IMDSv2?

It does not answer:

Could an attacker actually steal workload credentials through SSRF?

That requires application context and separate testing.

Why status semantics matter more than a red circle

I use five primary statuses:

For both engineers and auditors, ERROR matters a lot.

If the scanner gets AccessDenied, that is not PASS.

If a service API is unavailable, that is not PASS.

If a Region was not assessed, that is not PASS.

Unknown should not magically become green.

Otherwise we get security theater with a really nice color palette.

Severity is not business risk

Another design rule I care about:

CRITICAL, HIGH, MEDIUM, and LOW are technical triage priorities in the toolkit.

They are not CVSS scores.

They are not final business-risk decisions.

Consider the same finding twice:

Security Group exposes TCP/22 to 0.0.0.0/0 

Resource A:

Disposable training instance No sensitive data Automatically destroyed tonight 

Resource B:

Production jump host Privileged network path Customer data environment 

Same technical control.

Very different business risk.

So the workflow becomes:

Scanner severity | v Technical validation | v Asset / data / exposure context | v Compensating controls | v Likelihood + impact | v Business priority 

The scanner helps sort the queue. It does not replace the risk owner.

Scope: a green result without scope is almost useless

Before an assessment, I want to know at least:

• which AWS account is in scope;
• which principal performs the scan;
• which Regions are covered;
• which services are actually used;
• which controls are enabled;
• which controls are excluded;
• which suppressions are applied;
• when the scan runs.

Regions are especially important.

Some AWS resources are global, others are Regional. Some Regions are opt-in. Service availability also differs.

So the sentence:

"We scanned AWS."

does not tell me very much.

This is better:

"We ran toolkit v2.0.0 against account X, across all enabled/opted-in Regions, using this control set and this read-only role."

Now we have a reproducible technical statement.

Architecture: one engine, two interfaces

I did not want to build two scanners, one for a terminal and one for a pretty UI. The architecture looks roughly like this:

 AWS credential chain / optional AssumeRole | v Assessment Context account / regions / policy | v Control Registry | v Assessment Engine | v Normalized Findings | Suppression Policy | v AssessmentReport | +---------+-----------+-----------+ | | | | v v v v CLI GUI HTML JSON/CSV/ SARIF/JUnit 

The CLI and GUI call the same Assessment Engine.

That gives me one source of truth for:

• collection;
• control logic;
• statuses;
• severity;
• suppressions;
• evidence;
• reports.

Consistency in security tooling is boring. That is a compliment.

Read-only is a design constraint, not a marketing phrase

The toolkit is for assessment, not remediation. In the target account I expect inspection operations such as:

Get* List* Describe* 

not:

Create* Delete* Modify* Put* Enable* Disable* 

There is one obvious exception: sts:AssumeRole. For cross-account consulting, temporary credentials are much cleaner than asking a customer for long-lived access keys.

The AWS managed SecurityAudit policy is explicitly described as useful for software that audits security configuration metadata, so it is a reasonable starting point for an assessment role.

I still do not treat a managed policy as magic. Optional services and a particular scope can require additional read permissions.

I test the read-only boundary at release time

The repository includes a release check that:

• statically collects boto3 operations;
• validates operations and paginators against the installed botocore model;
• looks for obvious mutating API prefixes;
• explicitly allows only the expected sts:AssumeRole exception.

This is not a mathematical proof. It is a practical regression guard. If somebody accidentally adds modify_* to the assessment path tomorrow, I would rather learn about it in CI than in a customer account.

Cross-account access and why External ID matters

A typical setup looks like this:

Assessor account / workstation | | sts:AssumeRole v Customer SecurityAudit role | v Customer AWS account 

For example:

aws-sec-audit \ --profile assessor \ --role-arn arn:aws:iam::123456789012:role/SecurityAudit \ --external-id customer-123 \ --all-regions 

AWS recommends an External ID for third-party role access as a way to mitigate the confused deputy problem.

One detail that is easy to miss: the External ID is not a secret. Its purpose is different. It helps the target account constrain when the third party is allowed to assume that role on a particular customer's behalf. It is a small detail with a big impact on a clean consulting access model.

The CLI stayed

I like GUIs when I need to configure scope quickly or show results to somebody else. But automation without a CLI gets painful fast. A simple run still looks like:

aws-sec-audit --profile security-audit --all-regions 

Focused controls:

aws-sec-audit \ --include IAM-001,IAM-002,S3-001 \ --regions eu-west-1,eu-central-1 

Tag-based selection:

aws-sec-audit --tag encryption aws-sec-audit --tag cis --tag public-access aws-sec-audit --exclude-tag cost 

That matters for CI, repeatability, and boring shell automation. Boring shell automation is underrated.

Then I built a GUI anyway

At some point I noticed that every customized run ended with me reopening the README:

Right, --all-regions, then --role-arn, then --tag, then baseline... what did I call that flag again?

That usually means the interface can be better.

So I built a Local Dashboard Edition.

Install and run:

python -m pip install ".[ui]" aws-sec-audit-ui 

By default it listens on:

http://127.0.0.1:8765 

The browser does not ask for an AWS access key or secret key.

boto3 uses the normal credential provider chain:

• AWS profiles;
• AWS SSO-backed profiles;
• environment/workload credentials;
• optional AssumeRole.

A small security warning

127.0.0.1 is not a cosmetic default.

The dashboard can display:

• account metadata;
• resource names and ARNs;
• evidence;
• configuration details.

I do not treat the local UI as a ready-made multi-user web service. If it is ever exposed on a shared network, that becomes a different product requirement: authentication, TLS, authorization, tenant isolation, session/CSRF protection, and encrypted persistence.

Binding a local admin tool to 0.0.0.0 does not magically turn it into an enterprise portal. Unfortunately.

The dashboard is for triage, not dashboard-driven security

After a scan I want to understand quickly:

• how many FAIL findings exist;
• how many ERROR results indicate coverage gaps;
• whether any CRITICAL/HIGH items are open;
• which categories dominate;
• which findings are new relative to the baseline;
• which items deserve a human review first.

But the dashboard should never become a source of false confidence.
Green is a signal, not a magic amulet.

If the scope is incomplete, credentials are insufficient, or a relevant control does not exist, the color of the card will not save you.

A finding has to answer "what exactly?"

A weak scanner says:

FAIL: S3 is insecure 

Okay. Now what?

A normalized finding in the toolkit contains:

control ID title status severity account region affected resources evidence remediation references 

At minimum, a finding should answer four questions:

1. What is wrong?
2. Where exactly is it?
3. Why did the scanner reach that conclusion?
4. What is a reasonable next action?

Evidence matters to me a lot. Without evidence, a finding turns into an opinion very quickly.

Suppressions: accepted risk should not disappear

Real environments always have exceptions. For example:

• a public endpoint is public by design;
• a resource is temporarily excluded;
• a legacy workload is already being retired;
• a compensating control changes the risk;
• the risk was formally accepted;
• remediation is already scheduled for the next sprint.

The bad implementation is to delete the finding. The slightly less bad implementation is to hide it so well that nobody remembers why.

I prefer a waiver that records:

• control;
• Region;
• resource pattern;
• reason;
• optional expiration date.

A suppressed finding remains visible as SKIP, and the original evidence is retained. That helps different people in different ways.

A team lead asks:

"Who made this decision, and why?"

An auditor asks:

"Is there a traceable exception?"

Future you asks:

"Why on earth did we allow this three months ago?"

All three deserve an answer.

Baseline and diff: the second scan is more useful than the first

The first scan tells you:

What violates our controls right now?

The second tells you:

What changed?

For example:

Previous assessment 18 open findings Current assessment 6 open findings Resolved: 14 New: 2 Unchanged: 4 

Now remediation conversations get much more concrete.

You can separate:

• fixed;
• new regression;
• accepted;
• still open.

Scheduled runs through cron, systemd, or CI can provide lightweight drift detection. But I do not call that real-time detection.

Polling every few hours answers:

"Has the posture changed since the last run?"

It does not answer:

"What happened twelve seconds ago, and who did it?"

For that, I want CloudTrail, EventBridge, GuardDuty, AWS Config, Security Hub, and an event-driven detection workflow. Do not make a posture scanner cosplay as a SIEM.

Point-in-time evidence is not operating effectiveness

This is one of the most important points from an auditor's perspective.

Suppose the toolkit reports:

CloudTrail enabled: PASS 

What does that prove?

It proves that the observed configuration satisfied the control when the scan ran.

What does it not prove?

That CloudTrail was correctly enabled:

• yesterday;
• for the last 90 days;
• through the entire audit period;
• in every previously used Region;
• without interruption.

That is the difference between point-in-time configuration evidence and evidence of operating effectiveness over time.

For historical evidence I would add sources such as:

• AWS Config configuration history;
• Security Hub finding history;
• CloudTrail;
• change tickets;
• IaC history;
• operational procedures.

AWS Config is specifically designed to retain configuration history after recording begins. Security Hub updates control findings through periodic or change-triggered checks.
So in a serious assessment I would never write:

"This scan proves that the control operated effectively all quarter."

It does not.

The HTML report is client-ready, not consultant-free

Raw JSON is wonderful. Especially if your client is also jq. Most clients are still human. That is why the toolkit can generate a self-contained HTML report:

aws-sec-audit \ --all-regions \ --format html \ --output reports/aws-security-assessment.html 

It includes:

• executive summary;
• scan metadata;
• color-coded severity;
• status counters;
• category breakdown;
• filters;
• search;
• affected resources;
• evidence;
• remediation;
• references;
• print-friendly styling.

The exported HTML is standalone. It does not need the dashboard afterward.

I still do not consider the generated report a final consulting opinion.
The scanner knows configuration.
It does not know:

• why the workload exists;
• what data it processes;
• what the blast radius is;
• which compensating controls are effective;
• how much downtime is acceptable;
• which "easy fix" will break production.

That is where the human part of an assessment begins.

Turning a finding into an action plan

A team lead does not need thirty red lines. They need a queue of work.
For example:

The flow looks more like:

Automated finding | v Engineer validates | v Business / architecture context | v Remediation option | v Owner + priority + target date | v Re-scan / evidence of closure 

Automation removes repetitive work. Ownership closes risk.

Those are different jobs.

Multiple reports from one scan

I deliberately separated collection from presentation.

One scan can produce multiple representations:

aws-sec-audit --all-regions \ --export json=reports/report.json \ --export html=reports/report.html \ --export csv=reports/report.csv \ --export junit=reports/junit.xml \ --export sarif=reports/report.sarif 

The toolkit currently supports:

1. Console
2. JSON
3. Markdown
4. HTML
5. CSV
6. SARIF
7. JUnit XML

In practice, it means fewer repeated AWS API calls and less chance that two reports accidentally describe slightly different points in time.

What does "fast assessment" actually mean?

I can sometimes get a useful first picture within a couple of hours.

That does not mean:

"We fully proved AWS security in two hours."

It means something closer to:

1. agree scope and credentials;
2. run the automated posture pass;
3. investigate ERROR results and coverage gaps;
4. triage high-priority findings;
5. manually validate the most important items;
6. define the first remediation priorities.

It is a first pass.

It can tell you quickly whether the environment looks disciplined, has a handful of isolated gaps, or smells strongly of configuration debt.

The actual depth depends on:

• number of accounts;
• Regions;
• resource volume;
• architecture;
• service mix;
• permissions;
• compliance requirements;
• manual-validation needs.

Garbage in, garbage out applies to cloud security too. Give the scanner a half-blind role and you should expect a half-blind report.

Why not just Security Hub?

Security Hub CSPM is already an AWS-native service for ongoing posture management, standards, and control findings.

I am not trying to replace it. For continuous AWS governance, Security Hub is often a natural part of the architecture.

My toolkit solves a different problem:

• portable assessment;
• independent first pass;
• local execution;
• my normalized finding model;
• custom scope;
• client artifact;
• fast re-scan after remediation.

Sometimes I compare both views.

If Security Hub says one thing and a direct API check says another, that is not a reason to pick your favorite dashboard. It is a reason to investigate scope, evaluation schedule, AWS Config recording coverage, and control semantics.

Why not just Prowler?

Prowler is a mature open-source cloud security platform with hundreds of AWS checks, multiple cloud providers, compliance frameworks, UI/API/CLI, and a large community.

If your requirement is a broad open-source CSPM, it absolutely belongs on the evaluation list. I built something different.

I wanted a tool that is:

• small enough that I can understand the code path;
• easy to modify for a specific assessment workflow;
• local;
• read-only;
• infrastructure-light;
• intentionally limited to an explainable control set;
• aligned with my reporting flow.

This is not a "who has more checks" competition. Sometimes you need hundreds of controls. Sometimes you need the right forty, an architecture diagram, and an engineer willing to ask uncomfortable questions.

What a green dashboard does NOT mean

I would happily print this section next to every CSPM dashboard.
A PASS result does not prove that:

• the application has no SQL injection;
• no IAM attack path exists;
• secrets have never leaked;
• CloudTrail was not disabled yesterday;
• Kubernetes workloads are secure;
• the source code is vulnerability-free;
• incident response is ready;
• backups can actually be restored;
• every relevant resource was in scope;
• compensating controls operate exactly as described.

A posture scanner answers its controls. It does not answer every question in the security universe.

Where the toolkit sits in my assessment workflow

My AWS assessment looks more like this:

Scope / architecture | +--> AWS posture scan | +--> IAM / privilege review | +--> Network exposure | +--> Logging / detection | +--> Data protection | +--> Backup / recovery | +--> Application / API layer | +--> Manual validation | v Risk prioritization | v Action plan | v Re-test 

The toolkit covers part of posture and discovery.

It does not replace:

• threat modeling;
• penetration testing;
• source-code review;
• IAM privilege and attack-path analysis;
• architecture review;
• API/AppSec testing;
• incident-readiness assessment;
• backup restore testing;
• interviews with engineering and security teams.

That is fine. A good tool does not need to pretend to be an entire security program.

What I intentionally did NOT add

Every security side project eventually starts whispering:

"Add an LLM. Add a database. Add auto-remediation. Add multi-tenancy. Add a Slack bot. And maybe blockchain?"

I stopped earlier.

The current design does not include:

• automatic remediation;
• hosted credential storage;
• a cloud database;
• multi-user authentication;
• a real-time detection engine;
• a "magic" security score;
• an LLM deciding whether a configuration is risky.

I like this model:

Deterministic check | v Finding + evidence | v Human judgment 

If I add AI later, I would put it after the evidence:

Finding | +--> explanation +--> correlation +--> remediation draft +--> executive summary 

Root MFA is still better evaluated by a boolean. It does not hallucinate.

What comes next

Natural directions include:

• deeper IAM controls;
• AWS Organizations and multi-account orchestration;
• more framework mappings;
• historical trends;
• WAF/API efficacy testing as a separate module;
• ticketing/SIEM integrations;
• signed reports;
• additional services.

But I want to keep the main principle:

Tooling should remove repetitive work, not hide security judgment behind charts.

Final thoughts

The AWS Security Assessment Toolkit came from a simple need: get a fast, reproducible first technical view of AWS security posture.

TODAY IT INCLUDES: 42 controls + CLI + local GUI + AssumeRole + suppressions + baseline/diff + 7 report formats + a self-contained HTML report.

For me, that is a useful level of lightweight assessment tooling.
Not a full CSPM.
Not a pentest.
Not an audit-opinion generator.
Not a "Make AWS Secure" button.

Just an engineering lever that saves time where automation makes sense and leaves humans responsible for the parts that require context. Those are usually the security tools I enjoy using the most.

References

AWS

• AWS Shared Responsibility Model
• AWS Well-Architected Framework: Security Pillar
• AWS IAM security best practices
• AWS Security Hub CSPM introduction
• AWS Foundational Security Best Practices
• CIS AWS Foundations Benchmark in Security Hub CSPM
• AWS Security Hub control reference
• AWS managed policy: SecurityAudit
• Third-party access and External IDs
• How AWS Config records configuration history
• Security Hub security-check schedules

Open source

• Prowler
• AWS Security Assessment Toolkit repository by Bulwark Advisory