Summative evaluation on final or near-final device with representative users covering all hazard-related use scenarios, with a usability evaluation report concluding safe use.
AMD1:2020 extends analysis to use difficulties — near-misses must be identified and root-caused when they could lead to a hazardous situation.
Use-Related Risk Analysis format, minimum 15 participants per user group, and post-evaluation risk table updates — three common FDA human factors deficiency areas.
Maps to
IEC 62366: §5.9 Perform SUMMATIVE EVALUATION of the USABILITY of the USER INTERFACE
ISO 13485: §7.3.7 Design and development validation
Pre-QMSR Part 820 (legacy QSR): §820.30(g) Design validation.
Requirement text
The manufacturer shall perform a summative usability evaluation on the final or near-final device to confirm that the user interface can be used safely and effectively by intended users for the intended use. All hazard-related use scenarios must be included. The manufacturer shall analyse the data of the summative evaluation and identify all use errors and use difficulties that occurred; root cause must be determined for any use error or use difficulty that could lead to a hazardous situation. Results including observed use errors, use difficulties, and encountered hazards must be documented in a Usability Evaluation Report.
Why this clause exists
Summative evaluation is the human-factors equivalent of design validation — it is the one point in the development process where the hypothesis that the UI design is safe for intended users must be tested against real user behavior rather than designer intent. Without a structured, protocol-driven evaluation on the final or near-final device with representative participants, the risk assessment for UI-related hazards rests entirely on analytical prediction, and analytical prediction has a poor track record for use-error modes that involve perceptual and cognitive failures under realistic conditions. The AED and defibrillator device class illustrates the failure mode at stake: devices whose operator interfaces generated confusion about energy-delivery readiness — confirmed by post-market complaint data — had typically passed internal design reviews that assessed the interface against engineering intent rather than against observed operator behavior under simulated emergency conditions. Regulators codified the summative evaluation requirement in IEC 62366-1:2015 clause 5.9 because no substitute for structured human-performance testing exists for validating safety-critical interface behaviors. AMD1:2020 extended the data analysis obligation to include use difficulties — near-misses that did not result in a confirmed error — because near-miss frequency is a leading indicator of interface fragility: a scenario where many participants nearly made an error but caught it in a controlled setting is reliably a scenario where field users under higher cognitive load will complete the error.
What changed
IEC 62366-1:2015 replaced the 2007 first edition with a major restructuring. The standard was split into Part 1 (normative requirements) and Part 2 (IEC TR 62366-2, informative guidance and methods). The scope broadened to include hazards of all types including psychological hazards, not just direct physical hazards.
The standard introduced a formative/summative evaluation framework not present in 2007. The 2007 requirement to identify primary operating functions was removed — instead, the 2015 version mandates identification and evaluation of hazard-related use scenarios.
Amendment 1 (IEC 62366-1:2015+AMD1:2020 CSV) added a normative requirement in §5.9: the manufacturer must now identify all USE ERRORS and use difficulties that occurred during the summative evaluation (not just use errors). Where a use difficulty can lead to a hazardous situation, root cause must be determined. A 'use difficulty' is defined as a near-miss or struggle encountered during use, typically momentary and overcome by the user — including 'close calls' where a use error was nearly committed but avoided. This extends the analysis obligation beyond confirmed errors. ISO 14971 cross-references updated from 2007 to 2019 edition throughout.
Common gaps (what we see in audits)
- Insufficient participant numbers and representativeness — FDA expects a minimum of 15 participants per user group for summative evaluation. Manufacturers frequently test with too few participants or with participants who do not adequately represent the intended user population's physical and cognitive characteristics.
- Inadequate Use-Related Risk Analysis (URRA) — FDA expects a URRA in a specific format that maps use errors to specific hazardous situations and harms. Many manufacturers submit risk analyses using standard FMEA or fault-tree analysis tools that do not map to FDA's expected human factors format.
- Training decay not accounted for in test design — Summative evaluations test users immediately after training without a 24-48 hour washout period to simulate real-world memory retention. Results appear better than actual field performance because participants remember training instructions that real users would have forgotten.
- No re-evaluation after post-summative interface changes — Any modification to the user interface after summative evaluation requires a new summative evaluation or documented rationale for why the change does not affect validated critical tasks. Manufacturers sometimes make post-evaluation changes under schedule pressure without reassessing.