Maps to
IEC 62366: §5.9 Perform SUMMATIVE EVALUATION of the USABILITY of the USER INTERFACE
ISO 13485: §7.3.7 Design and development validation
Pre-QMSR Part 820 (legacy QSR): §820.30(g) Design validation.
Requirement text
The manufacturer shall perform a summative usability evaluation on the final or near-final device to confirm that the user interface can be used safely and effectively by intended users for the intended use. All hazard-related use scenarios must be included. The manufacturer shall analyse the data of the summative evaluation and identify all use errors and use difficulties that occurred; root cause must be determined for any use error or use difficulty that could lead to a hazardous situation. Results including observed use errors, use difficulties, and encountered hazards must be documented in a Usability Evaluation Report.
What changed
IEC 62366-1:2015 replaced the 2007 first edition with a major restructuring. The standard was split into Part 1 (normative requirements) and Part 2 (IEC TR 62366-2, informative guidance and methods). The scope broadened to include hazards of all types including psychological hazards, not just direct physical hazards.
The standard introduced a formative/summative evaluation framework not present in 2007. The 2007 requirement to identify primary operating functions was removed — instead, the 2015 version mandates identification and evaluation of hazard-related use scenarios.
Amendment 1 (IEC 62366-1:2015+AMD1:2020 CSV) added a normative requirement in §5.9: the manufacturer must now identify all USE ERRORS and use difficulties that occurred during the summative evaluation (not just use errors). Where a use difficulty can lead to a hazardous situation, root cause must be determined. A 'use difficulty' is defined as a near-miss or struggle encountered during use, typically momentary and overcome by the user — including 'close calls' where a use error was nearly committed but avoided. This extends the analysis obligation beyond confirmed errors. ISO 14971 cross-references updated from 2007 to 2019 edition throughout.
Atomic constraints
- •A summative evaluation must be conducted on the final or near-final device (not an early prototype).
- •Test participants must match the intended user profile from the Use Specification.
- •At least a minimum number of participants must be used (the plan must specify this threshold).
- •All hazard-related use scenarios must be covered by summative test cases.
- •Test results including pass/fail counts, observed use errors, and any hazards encountered must be documented.
- •Use errors observed during the summative evaluation must be analyzed for risk impact.
- •New hazards identified during summative evaluation must feed back into the risk management file.
- •The Usability Evaluation Report must conclude whether the device can be used safely by intended users.
- •The summative evaluation data analysis must identify all use errors AND use difficulties that occurred; root cause must be determined for any use difficulty that could lead to a hazardous situation (per §5.9, AMD1:2020).
Common gaps
Insufficient participant numbers and representativeness
majorFDA expects a minimum of 15 participants per user group for summative evaluation. Manufacturers frequently test with too few participants or with participants who do not adequately represent the intended user population's physical and cognitive characteristics.
Inadequate Use-Related Risk Analysis (URRA)
majorFDA expects a URRA in a specific format that maps use errors to specific hazardous situations and harms. Many manufacturers submit risk analyses using standard FMEA or fault-tree analysis tools that do not map to FDA's expected human factors format.
Training decay not accounted for in test design
moderateSummative evaluations test users immediately after training without a 24-48 hour washout period to simulate real-world memory retention. Results appear better than actual field performance because participants remember training instructions that real users would have forgotten.
No re-evaluation after post-summative interface changes
majorAny modification to the user interface after summative evaluation requires a new summative evaluation or documented rationale for why the change does not affect validated critical tasks. Manufacturers sometimes make post-evaluation changes under schedule pressure without reassessing.
Evidence signals
- •
FILE_EXISTS
Usability.*Evaluation.*Report|Usability.*Report|Usability.*Test.*Result|Usability.*Protocol
- •
CONTENT_MATCH
Does this document report the results of a summative usability evaluation conducted with participants matching the intended user profile, covering all hazard-related use scenarios, with documented use errors and use difficulties analyzed for safety impact and a conclusion about whether the device can be used safely?
Audit defense
The Usability Evaluation Report for [your product] (Doc ID: [your document ID]) documents the summative evaluation results from testing with users matching our Use Specification profile. All hazard-related use scenarios were covered, observed use errors were analyzed for risk impact, and the report concludes that the device can be used safely and effectively by intended users.