Supervisors judge an XBRL validation engine on two things, in this order: does it reach the right answer, and how quickly. We benchmarked ours against Arelle, the widely used open-source reference processor, on around 290 sample filings spanning every recent EBA, EIOPA and ESEF release. Here are the results, the disagreements and the caveats.
Why validation speed matters to a supervisor
Validation is where every return starts its life. When it is slow, filers queue on deadline day, resubmission loops stretch into days, and a rule change cannot be tested against real filings before it goes live. When it takes seconds, a firm sees and fixes a failure at the point of submission, and a supervisor can re-run a new rule across years of history overnight.
What we tested
Around 290 sample filings across every release we support: EBA reporting frameworks 4.0, 4.1, 4.2 and 4.3; EIOPA Solvency II, pension funds, insurance recovery and resolution, and financial conglomerates; and ESEF. For each filing we compared every formula assertion result with Arelle, and for the EBA releases also every instance, DTS and validation-code outcome.
The disagreements
About 190 assertions disagree, but they come from a handful of causes repeated across filings, not 190 separate problems. One EIOPA Solvency II assertion disagrees in all three releases. A few other EIOPA assertions produce different counts. Around a dozen agree on the counts but bound different facts, which can be a real difference or simply a difference in how facts are keyed. ESEF shows a few classification and context-count differences. Each disagreement is resolved one of two ways: it is our bug, fixed with a test, or it is Arelle's, written up.
Speed, with the caveats
On one large FINREP filing, measured under controlled conditions, our engine took 16 seconds against Arelle's 4.2 hours. Across the full set, a first run suggests around 160 times faster: Arelle's recorded time adds up to 35.8 hours of CPU, against 13.5 minutes for our engine. Treat that wider number as rough. Both ran on a heavily loaded machine and Arelle is single-threaded, so a controlled head-to-head is still to come. We will publish it.
What it means for supervisors
Validation that runs in seconds changes what is practical. Firms get plain-language feedback while the return is still open. Supervisors can test a new taxonomy against last year's real filings before release, and re-validate history when a rule changes. And because the rules are owned by your team and versioned, a faster engine means faster policy, not just faster software. More in our guide to XBRL validation rules.
- 1How closely does it agree with a reference implementation, release by release, and where does it differ?
- 2Who investigates each disagreement, and how quickly is it fixed or explained?
- 3How long does it take to validate your largest return, measured under controlled conditions?
- 4How long would it take to re-validate a year of filings after a rule change?
- 5Can your team change and version the rules, or does every change wait for a release?
Figures are from Line4's internal benchmark, November 2026. Arelle is an open-source XBRL processor; we used it as the reference because it is widely used and freely available for anyone to check our numbers.
For supervisors
New guides and Insights, by email
One short email when we publish something worth your time, from Jess Luper. No spam, unsubscribe any time.