Formative and Summative Evaluation of Medical Devices
Are you planning the approval of your medical device or in-vitro diagnostic product and need to prove its usability? Do you want to ensure good usability through regular evaluations during development? Our experts conduct comprehensive and standard-compliant evaluations for you and advise you on all your questions.
How Can We Support You?
The evaluation of medical devices and in-vitro diagnostics presents a variety of methodological challenges and formal requirements. How many test participants are needed? What needs to be considered during the execution? How should the documentation be structured to be accepted by the notified bodies?
We advise you on all relevant questions and handle the complete implementation of the evaluation for you in accordance with IEC 62366-1 – from preparation and participant recruitment to professional execution, analysis, and documentation of the results. Discover a comprehensive overview of all our services in Medical UX.
Comprehensive Consulting
Do you have questions about the evaluation of your product? We are here to assist you with all aspects of the usability engineering process and provide you with the necessary documentation.
Planning the usability measures
We organize both formative and summative evaluations and advice you on the most suitable methods and procedures tailored to your budget and project plan. Additionally, our services include consulting on the required sample sizes that are accepted by the notified bodies.
Creating relevant documents
In collaboration with you, we determine which necessary documents are already available and identify any gaps. We create the missing usability documents based on your existing templates, or we can provide our own document templates as needed.
Preparation of the Evaluation
We take care of the entire content and organizational preparation of the formative or summative evaluation. You simply provide us with the necessary information about your product.
Recruitment and participant management
We handle the recruitment of test participants based on the defined user groups. Thanks to our extensive participant database and long-standing collaboration with specialized partners, we can recruit even highly specific target groups – from patient profiles to specialized surgeons for navigated spinal surgery.
We also manage all communication with the test participants. We ensure their commitment to confidentiality and, if necessary, organize the payment of compensation. The entire process is GDPR-compliant.
Content preparation
We create the test plan, including the test protocol, for you. This includes, among other things, the formulation of use scenarios, the identification of potential use errors, and the creation of the test script. Alternatively, we can conduct the evaluation using the test plan you have prepared.
Conducting the Evaluation
We conduct the formative or summative evaluation for you and ensure that the test setting accurately reflects the real context of use.
Testing in the usability lab
We adapt the evaluation to your schedule and ensure that it can be conducted in time for the completion of specific development phases.
In our fully equipped usability lab, we can implement a wide range of test settings – from a living room setup for home care applications to a simulated operating table. You can follow the tests on-site or remotely via live stream.
Analysis & Documentation
As a final result, you will receive a complete, standard-compliant documentation of the formative or summative evaluation of your product. We will work with you to prepare everything for approval.
Analysis and assessment of the results
During the test execution, we systematically record all use errors and use difficulties and determine the causes of the identified usability findings through a subsequent root cause analysis.
In collaboration with your risk management team, we assess the results and prepare the test report for integration into your quality management system. We are also happy to assist you in further developing the Usability Engineering File.
For formative evaluations, we additionally provide recommendations and suggest ways to address the identified use errors and use difficulties.
Would you like to learn more about how we can support you?
We are happy to advise you. Free of charge and without obligation.
Formative and Summative Evaluation – What's the Difference?
The IEC 62366-1 standard distinguishes between formative evaluation, which accompanies the development process, and summative evaluation at the end of the development process as evidence of usability in accordance with MDR and IVDR.
Formative Evaluation
Formative evaluation is ideally an integral part of the development process. It serves to continuously assess usability with the goal of identifying and correcting use errors early on. This approach results in a product that is well-adapted to the context of use and helps avoid surprises during the summative evaluation.
Goal of the formative evaluation
Identify undocumented use errors and risks to address them before the summative evaluation.
Methodological features
- Various methods are possible, such as user observations and interviews, prototype testing, heuristic evaluations.
- Multiple iterations with a small number of participants (typically five people per user group)
Summative Evaluation
The summative evaluation is a formal and necessary requirement for the approval of medical devices and in-vitro diagnostics. It verifies whether all established usability requirements have been fully met, serving as the basis for market approval or necessary final adjustments.
Goal of the summative evaluation
Final evaluation at the end of the development process to thoroughly review and validate the product's usability.
Methodological features
- Usability Test as the gold standard
- Larger number of participants required
- Formal documentation of usability tests according to IEC 62366-1 for inclusion in the Usability Engineering File
Why Is Formative Evaluation Valuable?
- Avoid costly repetitions of the summative evaluation
- Avoid project delays caused by necessary product adjustments and the repetition of the summative evaluation
- Save development costs by reducing the need for retrospective corrections through regular evaluations
- Develop a product that not only controls risks but also delights users with its intuitive operation
Selected References
Certified Partner for Medical Usability
Do you have specific questions concerning your project? We are happy to advise you. Free of charge and without obligation.
Your contact:
- Alexander Rösler
- Senior UX Consultant, Medical Usability Engineer