Formative and summative evaluation planning, participant criteria, test setting, and hazard scenario coverage requirements remain the evaluation planning backbone.
AMD1:2020 added three summative plan obligations: use-difficulty data collection method, user-group justification for participant numbers, and defined correct use per scenario.
Summative evaluation plan coverage matrix — each hazard-related use scenario must map to a test case; missing entries are an immediate finding.
Maps to
IEC 62366: §5.7 Establish USER INTERFACE EVALUATION plan
ISO 13485: §7.3.6 Design and development verification
Pre-QMSR Part 820 (legacy QSR): §820.30(f) Design verification.
Requirement text
The manufacturer shall plan both formative and summative usability evaluation activities. The evaluation plan must specify the evaluation methods, participant criteria, settings, and how results will be analyzed. Per AMD1:2020 §5.7.3(e), summative evaluation planning must specify the method of collecting data for subsequent analysis of both use errors AND use difficulties, justify participant grouping into distinct user groups, and define correct use for each hazard-related use scenario. Summative evaluation planning must ensure coverage of all hazard-related use scenarios.
Why this clause exists
Summative evaluation conducted without a pre-approved plan produces results that regulators cannot accept as evidence of safe design because there is no pre-specified definition of what constitutes a pass — acceptance criteria set after testing is completed are suspect by construction. FDA's human factors deficiency data shows that missing or inadequate evaluation plans are among the top three reasons for deficiency letters in 510(k) submissions with human factors components. The three new AMD1:2020 obligations added to §5.7.3(e) — data collection method for use difficulties, user-group justification for participant numbers, and defined correct use per scenario — reflect a pattern regulators observed in submitted plans: manufacturers described the summative evaluation setting and participant criteria but left ambiguous what outcome would constitute a pass for each task, and what would constitute a near-miss requiring investigation even when the user ultimately completed the task. Without pre-specified definitions of use difficulty and correct use, post-test analysis becomes judgment rather than measurement, and the evaluation cannot distinguish a well-designed interface from a design that users managed to work around under controlled test conditions but will fail under field conditions involving higher cognitive load, time pressure, or reduced training recency.
What changed
IEC 62366-1:2015 replaced the 2007 first edition with a major restructuring. The standard was split into Part 1 (normative requirements) and Part 2 (IEC TR 62366-2, informative guidance and methods). The scope broadened to include hazards of all types including psychological hazards, not just direct physical hazards.
The standard introduced a formative/summative evaluation framework not present in 2007. The 2007 requirement to identify primary operating functions was removed — instead, the 2015 version mandates identification and evaluation of hazard-related use scenarios.
Amendment 1 (IEC 62366-1:2015+AMD1:2020 CSV) made three substantive normative additions to §5.7.3(e): (1) the summative evaluation plan must specify the method of collecting data during the usability test for subsequent analysis of observed use errors AND use difficulties (near-misses/close calls); (2) the plan must justify how test participants are grouped into distinct user groups for the purpose of determining participant numbers; and (3) the plan must define correct use for each hazard-related use scenario. In 2015, only use-error analysis and participant profile criteria were required; these three additions tightened the planning obligation. Also changed: 'USER PROFILE' terminology in plan requirements was updated to 'USER GROUP', and specimen wording in examples was adjusted accordingly (editorial). ISO 14971 cross-references updated from 2007 to 2019 edition.
Common gaps (what we see in audits)
- Formative evaluation used to skip summative testing — Companies mistakenly believe they can skip summative testing by only doing formative evaluations. While you can reduce scope by leveraging data from a predicate device (Clause 5.10), you must provide a technical rationale explaining why design changes do not affect previously validated critical tasks. Without this rationale, summative evaluation is required.