AI & ML
IFRS Compliance Verification: An Open Multi-LLM Benchmark and Content-Conditioned Ensemble Classifier
arXiv cs.IR0 views
arXiv:2611.13595v1 Announce Type: cross Abstract: Verifying corporate financial disclosures against the International Financial Reporting Standards (IFRS) is a core auditing task in which experts cross-reference disclosures with hundreds of fine-grained regulatory paragraphs: a laborious, costly, and error-prone process that motivates growing interest in large language models (LLMs) as compliance assistants. Yet single-model accuracy remains insufficient for high-stakes use, and naive ensembling is vulnerable to systematic per-model biases that can drag aggregate accuracy below that of the strongest individual annotator. This gap is addressed with a multi-stage framework, supported by a new benchmark of 19,998 IFRS regulation-disclosure pairs comprising 11,161 unanimous multi-LLM labels and 8,837 human annotations on items where the LLMs disagreed. The framework couples a three-stage domain-adapted sentence embedding pipeline (BERT masked language modeling on SEC EDGAR filings, unsupervised TSDAE and SimCSE adaptation, and supervised contrastive alignment) with a learned ensemble classifier that fuses frozen embeddings and LLM probability features from GPT-4.1, Claude Sonnet 4, and Gemini 2.5 Pro via Tree-structured Parzen Estimator (TPE) optimized gradient-boosted trees. On the disagreement set, the ensemble achieves 81.7% weighted F1, exceeding the strongest single LLM by 10.4 points and majority voting by 14.6 points, and outperforms eleven unsupervised aggregation baselines; the retrieval pipeline attains Recall@15 of 45.3% on unseen regulations. The dataset, model checkpoints, annotation platform, and classifier implementation will be publicly released.
Source:https://arxiv.org/abs/2611.13595PreviousA Technique to Model Product Information and User Reviews (MPIUR)NextFrom Counterfactual Edits to Human Narratives: LLM-Driven Interpretations for Recommender System Explanations
Related posts
A useful vocal isolate is not a karaoke track
0dev.to AI
How to Pick the Right AI Development Partner for Your Business
0dev.to AI
DeepSeek R1 vs OpenAI o1-preview: Reasoning Accuracy, Token Economics, and Self-Hosting TCO
0dev.to AI

After the AI Writes, Part 1: Don't Carry Someone Else's Work Into Your Commit
0dev.to AI
My deploy script got my site removed from Bing and ChatGPT search. Seven words got a human to fix it.
0dev.to WebDev
Diluting Poisoned Data Made the Backdoor Stronger: Anthropic's 'Homeopathic Learning' Explained
0dev.to Security