When managing email delivery for your domains, configuring DMARC (Domain-based Message Authentication, Reporting, and Conformance) is essential for monitoring authentication health. Once you publish a DMARC DNS record with a reporting address, receiving mail servers worldwide generate aggregate reports. These reports contain critical telemetry about which servers are sending email on behalf of your domain and whether SPF (Sender Policy Framework) and DKIM (DomainKeys Identified Mail) checks passed or failed.

However, consuming aggregate feedback is notoriously cumbersome. Receiving mail servers send reports as XML files packed inside compressed archives (.zip or .gz), often delivered as attachments to an email inbox or dumped into a directory. Manually decompressing and reading raw XML documents to troubleshoot authentication issues or discover spoofing attempts is impractical. Many teams rely on cloud services, but routing security telemetry through external providers can introduce privacy concerns, ongoing subscriptions, and unnecessary complexity.

dmarc_lens_report_viewer.py solves this problem locally. It is a single-script tool that ingests aggregate reports directly from local folders or an IMAP mailbox, uncompresses and parses the XML payloads, resolves sender reverse DNS hostnames, deduplicates reports, and stores the structured results in an embedded SQLite database.

Installation and Usage

The tool requires no external packages and relies entirely on modules available within the Python standard library.

1. Setup

Save the source file to your working directory as dmarc_lens_report_viewer.py. Ensure you have Python installed on your system. Because the script uses only standard library components (sqlite3, xml.etree.ElementTree, email, gzip, zipfile, socket, imaplib, and http.server), no package installations are needed.

Verify your environment by running:

python dmarc_lens_report_viewer.py --help

2. Basic Ingestion Workflow

The script provides programmatic functions and command-line utilities to ingest reports through two main pathways:

  • Local Directory Parsing: The process_local_directory function traverses a specified local folder. It searches for .zip, .gz, .xml, and .eml files. If it encounters .eml messages, it extracts attachments directly from the MIME structure and imports the underlying data.
  • IMAP Mailbox Retrieval: The fetch_from_imap function connects to an IMAP server over SSL on port 993. It logs in, reads unread messages from a designated folder, extracts XML or compressed attachments, parses the data into the local database, and can optionally move processed messages to an archive folder.

How It Works

The script operates as a self-contained data pipeline: ingestion, decompression, XML parsing, DNS enrichment, and relational storage.

Archive Decompression and Format Detection

Reports arrive in varying formats depending on the reporting organization. Rather than relying solely on file extensions, the script detects archive types by inspecting the magic bytes of the payload:

def unpack_and_import( raw_bytes: bytes, source_name: str, conn: sqlite3.Connection, resolve_dns: bool = True, ) -> int: """Unpack .zip, .gz, or raw .xml bytes, parse and store. Returns count of new reports added.""" count = 0 # Try ZIP  if raw_bytes[:4] == b"PK\x03\x04": try: with zipfile.ZipFile(io.BytesIO(raw_bytes)) as zf: for member_name in zf.namelist(): if member_name.endswith( ".xml" ) or not member_name.endswith("/"): with zf.open(member_name) as f: xml_content = f.read() try: data = parse_dmarc_xml(xml_content) if store_report( conn, data, resolve_dns=resolve_dns ): count += 1 except Exception as ex: sys.stderr.write( f"Warning: Failed to parse XML '{member_name}' inside zip: {ex}\n" ) return count except zipfile.BadZipFile: pass

If the payload starts with PK\x03\x04, the tool uses zipfile to read XML records from the archive. If it begins with \x1f\x8b, it decompresses the stream with gzip. If neither magic header matches, the parser treats the input as raw XML.

XML Parsing and Namespace Stripping

DMARC aggregate XML formats often include XML namespaces that complicate tag lookups. The parser removes namespace prefixes from elements during traversal to ensure reliable node extraction across report generators:

def parse_dmarc_xml(xml_bytes: bytes) -> dict: """Parse DMARC aggregate XML payload into a structured Python dictionary.""" root = ET.fromstring(xml_bytes) # Some reports use namespaces; remove them for straightforward element lookups  for elem in root.iter(): if "}" in elem.tag: elem.tag = elem.tag.split("}", 1)[1] meta_el = root.find("report_metadata") if meta_el is None: raise ValueError("Invalid DMARC XML: missing <report_metadata>") org_name = ( meta_el.findtext("org_name") or meta_el.findtext("email") or "Unknown" ).strip() contact_email = (meta_el.findtext("email") or "").strip() report_id = (meta_el.findtext("report_id") or "").strip()

The function reads the reporting organization metadata, the published policy (p, sp, pct), and each individual <record> row. For every row, it extracts the sending IP address (source_ip), message count, policy evaluation disposition (none, quarantine, reject), and authentication outcomes for SPF and DKIM.

Reverse DNS Resolution and Deduplication

When inserting parsed reports into SQLite, store_report checks the report_id column in the reports table to avoid storing duplicate submissions from repeated IMAP runs or redundant directory scans.

For each sender IP address in the report, the tool performs reverse DNS lookups via socket.gethostbyaddr(ip) to identify the hostname associated with the sending server. To prevent network latency from stalling batch imports, the script sets socket.setdefaulttimeout(2.0) and caches PTR lookups in an in-memory dictionary during the import run.

Relational Schema

The underlying database consists of three primary tables:

  • reports: Stores high-level report metadata, reporting organization names, target domains, published policies, and date ranges.
  • records: Contains individual row entries linked by report_id, storing source_ip, resolved ptr_hostname, message volume count, disposition, and authentication results (dkim_result, spf_result).
  • known_senders: Tracks identified source IP addresses alongside custom labels to help distinguish legitimate outbound services from unauthorized infrastructure.

Limitations

Before relying on this script in production environments, consider the following technical constraints present in the code:

  1. Synchronous Network Lookups: Reverse DNS resolution is handled synchronously using Python's standard socket module. While cached in memory per batch, resolving many unique IP addresses over a slow network connection can slow down ingestion.
  2. Standard Library Mail Handling: The IMAP ingestion function downloads the complete message payload (RFC822) before extracting attachments. When processing thousands of messages simultaneously, memory usage scales with the size of the messages retrieved.
  3. Evaluation Scaffolding Only: The tool does not perform cryptographic DKIM verification or query live SPF records. It processes the evaluations reported by the receiving mail servers contained within the aggregate XML files.
  4. SQLite Concurrency: SQLite is well-suited for single-user desktop inspection and local scripts. High-concurrency environments with simultaneous writes from multiple background collectors require coordinating database access locks.

Frequently Asked Questions

Does this tool require third-party libraries from PyPI?

No. The script is designed to run with Python's standard library. It does not require installing any third-party packages.

How does the tool handle duplicate reports?

The store_report function executes a query checking if report_id already exists in the reports table before writing. If the report identifier is already present in the database, the function ignores the incoming report and moves to the next file.

Can it read .eml files stored on disk?

Yes. The process_local_directory function checks file extensions. If it encounters a .eml file, it parses the MIME message using Python's standard email library, loops over every attachment, and imports any embedded XML, ZIP, or GZIP reports it finds.

Full version and the rest of the toolkit: DMARC Lens report viewer on PyForge. Product page: https://py-forge.com/products/dmarc-lens-report-viewer