Nine sections, each with a concrete example of the kind of entry that would satisfy an auditor, next to the kind that wouldn't, and why.
Most writing on Annex IV either paraphrases the regulation or sells you a platform without showing what actually goes in the file. This one skips both and just shows the entries. Below, each of the nine sections gets a short note on what it's asking for, then a side-by-side: a thin entry that technically has words in it but wouldn't survive an audit, and a passing one that would.
The example system throughout is a fictional AI-assisted resume screening tool, chosen because it's a real high-risk use case under the Act (Annex III, employment) and concrete enough to make the difference between thin and passing obvious. Swap in your own system's specifics; the shape of a passing entry is what matters here, not the resume-screening details.
Purpose, provider vs. deployer, intended users, how it's delivered, hardware/software it interacts with, versioning, and what the user is shown or told.
"The system screens job applicants using AI to help recruiters find good candidates faster."
"ScreenAssist v3.2 is a SaaS tool, deployed via API integration into the customer's ATS, that scores incoming resumes against a role's stated requirements and returns a 0-100 fit score plus the top three matching and missing criteria. Provider: Ventrova (this document's author). Deployer: the hiring organization, which configures role criteria and reviews scores before any human hiring decision. The system does not reject candidates automatically; a recruiter sees the score and rationale for every applicant and makes the shortlisting decision. Interacts with the customer's ATS via REST API (read resume text, role posting; write score, rationale). Current version 3.2, released 2026-06-01; version history in Section 6."
Architecture, third-party components and how they're used, design choices and the reasoning, what the system optimizes for, and what's changed since first build.
"Built using a large language model and standard NLP techniques."
"Pipeline: resume text extraction (Apache Tika) → structured field parsing (custom regex + fine-tuned classifier, trained on 12,000 labeled resumes) → criteria match scoring (embedding similarity via a third-party model, provider and version pinned in the vendor register) → rationale generation (template-based, not generative, to keep output auditable). We chose embedding similarity over a fine-tuned classifier for the final score specifically because it lets us show which phrases drove the match, which Section 3's explainability requirement needed. Rejected approach: a single end-to-end LLM scoring call, dropped in v2.1 because it couldn't produce a traceable rationale for a given score. Optimized for recall on the top-20% band (missing a strong candidate is worse for our customers than including a weak one for human review)."
What it can and can't do, accuracy across different groups, foreseeable harms, and what a deployer needs to understand to use it correctly. This is usually the section with the most evidence attached, because "foreseeable risk" needs to be tied to something you actually tested, not a general statement.
"The system may have some bias. We recommend human review of all decisions."
"Fairness testing (2026-05-14, methodology and full results in evidence pack EV-0091): score distributions compared across gender and age-bracket proxies inferred from resume graduation years, using a 200-resume synthetic test set with controlled variation. Finding: mean score gap of 3.1 points (favoring resumes with no graduation-date gaps), below our 5-point action threshold but logged for the next quarterly review. Known limitation: the classifier underperforms on resumes in non-standard formats (design portfolios, video resumes) and returns a low-confidence flag rather than a score in that case, routing the applicant to full manual review. Foreseeable risk: a recruiter treating the fit score as a hard cutoff rather than a triage aid; mitigated by requiring recruiters to view at least one non-shortlisted resume per batch before final decisions (enforced in the ATS integration, not just written policy)."
Why the chosen metrics are the right ones for this system, not the raw numbers themselves.
"We measure accuracy, which is 94%."
"We report recall on the top-20% band rather than overall accuracy, because the cost of a false negative (a strong candidate scored low and never reviewed) is asymmetric with a false positive (a weak candidate reviewed and rejected by a human anyway). Overall accuracy would reward the system for correctly scoring the easy majority of clearly-unqualified resumes while hiding poor performance on the harder, higher-stakes top band. Current top-20% recall: 91%, measured against a held-out set of 500 resumes with human-panel consensus labels, re-measured quarterly against a refreshed sample to catch drift."
A structured record of identified risks, how they were assessed, and mitigations, tied to actual testing evidence.
"Risk assessment completed. No major risks found."
A risk register with rows like: "R-014, Proxy discrimination via graduation year as an age proxy, Likelihood: Medium, Impact: High, Evidence: fairness test EV-0091 (3.1pt gap), Mitigation: graduation year excluded from scoring inputs as of v3.1, monitored quarterly, Status: Mitigated, owner: Vera Quill, review date 2026-11-01." One row per identified risk, each with a testing citation and a status, not a paragraph asserting risk was "considered."
What changed, when, and why, kept current, not reconstructed at deadline.
"The system has been updated periodically to improve performance."
"v3.1 (2026-04-02): removed graduation year as a scoring input following fairness test EV-0087, which found it acted as an age proxy. v3.2 (2026-06-01): added low-confidence routing for non-standard resume formats after a customer-reported miss on a design-portfolio applicant; see incident log INC-2026-11." A dated, append-only log entry per change, referencing the evidence or incident that triggered it.
Which standard you followed, or what you did instead and why it's adequate if none applies yet.
"We follow industry best practices for AI safety."
"No harmonised standard under the Act currently covers resume-screening classifiers specifically. In its absence, we apply ISO/IEC 23894:2023 (AI risk management) as our risk-process framework and NIST AI RMF's Measure function for our testing cadence, both referenced by article and clause in our internal risk register (Section 5). We will adopt the relevant harmonised standard once published and update this section accordingly."
A signed legal attestation. No document generator or scan produces this; a person with legal authority signs it, referencing the rest of the technical file as its basis.
Leaving this section blank until "someone from legal gets to it," or an unsigned draft with no named signatory.
A signed statement identifying the provider, the system by name and version, the applicable Annex III category, and a named individual with the authority to attest conformity, dated, referencing the technical documentation file (this document) as its evidentiary basis. This is the one section where the "passing" version isn't about writing more, it's about actually getting the signature, which is a process step, not a documentation task.
How you'll monitor the system after deployment: what, how often, and what triggers a response.
"We will monitor the system's performance after launch."
"Fairness and accuracy re-tests run monthly against a refreshed 500-resume held-out sample (automated, results logged to the evidence pack with timestamp and diff against the prior run). A gap exceeding the 5-point action threshold from Section 3 auto-opens a review ticket assigned to the on-call ML owner, with a 5-business-day SLA to either remediate or document why the gap is acceptable. Customer-reported incidents (like INC-2026-11) feed the same register and are reviewed at the next scheduled risk-management meeting, logged in Section 6 if they result in a change."
Every passing example above has three things the thin version doesn't: a specific number or named artifact instead of a general claim, a date or version tying it to a point in time, and a named owner or process instead of a passive "is done." That's the actual bar. Legal language in Annex IV describes what to cover; it doesn't describe how specific the answer needs to be, which is exactly what auditors and procurement reviewers actually check for.
The four evidence-shaped sections (3, 5, 6, 9) are the ones where this specificity is hardest to fake after the fact, because it depends on having actually run the tests and logged the changes at the time, not reconstructed a plausible-sounding paragraph months later.
A real Sentinel Scan run, mapped to the sections above it feeds most directly (3, 5, 6, 9): dated findings, OWASP LLM Top 10 references, laid out as a sample report you can read end to end.
We're building a one-shot intake form that turns your system's answers to questions like the ones above into an Annex IV-structured draft, one-time purchase, no subscription. It doesn't exist yet; we're validating demand with a waitlist before writing a line of generator code.
Sources cited: EU AI Act Annex IV, artificialintelligenceact.eu (Regulation (EU) 2024/1689); Annex III for the employment high-risk category referenced in the example system.