17 May Is Your Design on the Road to Success? (Part 2 of 4)

Do you know what you are building?

Working with our customers over the years, we’ve seen patterns that lead to better outcomes than others.  How your company approaches the design cycle can have a major impact on whether the project will be successful or not.  We’ve condensed our experience down into this short questionnaire.

1. Before you begin formal development, do you know exactly what your product will look like, how it will function, and how it will handle fault conditions?
2. Do your processes allow you to efficiently make changes to designs while meeting FDA and international regulations?
3. Do the tools your company uses for development, documentation, configuration management, verification, and complaint-handling work well with your processes?
4. Does your company have a partner that can help you anywhere in your development cycle, from concept through design transfer, from usability to risk management, from process development and improvement to test lab preparation?

If you answered “yes” to all of the questions above, then way to go!  You’ve got the right approach toward development.  If you answered “no” to any of the above questions, have no fear; this series of articles will address the impact of each of these questions.  In this article, we’ll focus on the first question, “Before you begin formal development, do you know exactly what your product will look like, how it will function, and how it will handle fault conditions?”

Knowing what the end goal is before you start is critical to having an efficient project.  You wouldn’t get in your car and start driving without knowing your destination, so why would you try to design a product that way?  Inevitably on almost every project, the same questions come up:

• How is the product supposed to work when everything is working right? Preparing a Use Specification (as referenced in IEC 62366 section 8) helps establish the way the user(s) will interact with the device.
• How is the product supposed to work when a fault occurs? Given fault conditions will occur, it’s better to identify the expected behavior upfront with the benefit of having the time to clearly think through the expected device behavior.  ISO 14971 Appendix C provides a long list of questions to consider when thinking through fault conditions.  This includes how the device alerts the user and what actions it takes to keep itself and its end users safe.  Many customers spend months trying to figure this out.  A helpful tip: keep it simple.  Error-handling should be intuitive and consistent.  Rather than improving patient safety, handling faults in a bunch of different ways actually makes the product more complicated, more likely to miss a corner case, and harder to test. You’ll thank yourself when your V&V is simple and consistent.
• Does the science work the way you think it should? Is there a clear definition of what is passing versus failing, or is there overlap?  How do you differentiate if it’s not obvious?  If the product uses algorithms, are they robust?  Do they hold up under all different kinds of scenarios, or do they have to be babied?
• Does the product have the right features? Too few features can leave the end user wanting, but too many features can confuse the user and also adds a lot of complexity when it comes time for V&V.

Not knowing the answers to all of these questions can lead to one or more of several undesirable outcomes, depending on your developers.  Conservative developers will hesitate and stop working until they know the answer because they don’t want to waste effort on something that might not be correct.  This leads to the project grinding to a halt until the question is answered.  That halt stops the momentum the project had going for it, and once interrupted, that momentum can be hard to regain.  Intrepid developers will make an educated guess as to what they think the product should do and develop accordingly, which is fine until the product comes out and people realize it’s not what was wanted.  All of that effort then has to be undone and redone, which is costly and time-consuming.

Note in the question raised above the emphasis on “formal”.  Nobody expects a device manufacturer to know everything about the product at its conception.  But before going under full design controls, producing full documentation, and following all of the mandated processes (software life-cycle, risk management, usability), get the answers to your questions when it’s cheap and easy to do so.  The time will come for full design controls, but not until the project is lined out with sufficient details you need to make it a success.  Here are some suggestions on how to get those answers.

Build prototypes.  They’re relatively inexpensive, quick-turn, and don’t require the amount of documentation and process obligatory for formal development.  Prototypes need not be of the entire product; in fact, smaller, focused prototypes can be more valuable than entire product prototypes because they let you really focus on the issue at hand rather than spending time developing support for the rest of the system.  A prototype might be as simple as a 3D-printed form factor for users to evaluate as part of an IEC 62366 formative study, a breadboarded circuit to test whether a circuit works the way you think it should, or a few lines of code to evaluate a microprocessor you’re considering.

Remember to keep your prototypes and their design outputs (schematics, drawings, source code) under configuration management so that you can reproduce them later.  Bear in mind that while configuration management is a part of design controls, it is not all of design controls.  There is a big difference between knowing how to reproduce something you built and producing formal documentation for it and following full software lifecycle and risk management processes.  The intent is to capture enough about the design outputs to be able to build the prototype again, without miring yourself in documentation.

The goals of your prototypes should be to answer technical questions about the product (what can it do, what can’t it do, does the science consistently work the way we expect), simulate the user interface for representative user groups (this will lend itself towards the usability process later), and identify potential pitfalls in the design that could cause problems or hazards later.  See our Why Prototype ? article for lots more good information on prototypes.

Ask your stakeholders and subject-matter experts.  Sometimes getting the answer is as easy as asking, yet many times the question never gets asked, and the question inevitably comes up over and over again during the design, introduces uncertainty and delays, and gets put off for later so the process can repeat.  Ask the question and document the answer with its rationale.  The answer frequently isn’t enough by itself; people will come back later and ask why that is the answer.  If you’ve got the answer and its rationale documented in a common, well-known place where people can access it easily, it will significantly reduce the overhead associated with not knowing.

Contact The Realtime Group today at 972 985-9100 and let us help you identify the questions you need to answer and suggest ways to find those answers.  We can also help you design your prototypes and produce the documentation you’ll need down the road.

Working with our customers over the years, we’ve seen patterns that lead to better outcomes than others.  How your company approaches the design cycle can have a major impact on whether the project will be successful or not.  We’ve condensed our experience down into this short questionnaire.

1. Before you begin formal development, do you know exactly what your product will look like, how it will function, and how it will handle fault conditions?
2. Do your processes allow you to efficiently make changes to designs while meeting FDA and international regulations?
3. Do the tools your company uses for development, documentation, configuration management, verification, and complaint-handling work well with your processes?
4. Does your company have a partner that can help you anywhere in your development cycle, from concept through design transfer, from usability to risk management, from process development and improvement to test lab preparation?

If you answered “yes” to all of the questions above, then way to go!  You’ve got the right approach toward development.  If you answered “no” to any of the above questions, have no fear; this series of articles will address the impact of each of these questions.  In this article, we’ll focus on the first question, “Before you begin formal development, do you know exactly what your product will look like, how it will function, and how it will handle fault conditions?”

Knowing what the end goal is before you start is critical to having an efficient project.  You wouldn’t get in your car and start driving without knowing your destination, so why would you try to design a product that way?  Inevitably on almost every project, the same questions come up:

• How is the product supposed to work when everything is working right? Preparing a Use Specification (as referenced in IEC 62366 section 8) helps establish the way the user(s) will interact with the device.
• How is the product supposed to work when a fault occurs? Given fault conditions will occur, it’s better to identify the expected behavior upfront with the benefit of having the time to clearly think through the expected device behavior.  ISO 14971 Appendix C provides a long list of questions to consider when thinking through fault conditions.  This includes how the device alerts the user and what actions it takes to keep itself and its end users safe.  Many customers spend months trying to figure this out.  A helpful tip: keep it simple.  Error-handling should be intuitive and consistent.  Rather than improving patient safety, handling faults in a bunch of different ways actually makes the product more complicated, more likely to miss a corner case, and harder to test. You’ll thank yourself when your V&V is simple and consistent.
• Does the science work the way you think it should? Is there a clear definition of what is passing versus failing, or is there overlap?  How do you differentiate if it’s not obvious?  If the product uses algorithms, are they robust?  Do they hold up under all different kinds of scenarios, or do they have to be babied?
• Does the product have the right features? Too few features can leave the end user wanting, but too many features can confuse the user and also adds a lot of complexity when it comes time for V&V.

Not knowing the answers to all of these questions can lead to one or more of several undesirable outcomes, depending on your developers.  Conservative developers will hesitate and stop working until they know the answer because they don’t want to waste effort on something that might not be correct.  This leads to the project grinding to a halt until the question is answered.  That halt stops the momentum the project had going for it, and once interrupted, that momentum can be hard to regain.  Intrepid developers will make an educated guess as to what they think the product should do and develop accordingly, which is fine until the product comes out and people realize it’s not what was wanted.  All of that effort then has to be undone and redone, which is costly and time-consuming.

Note in the question raised above the emphasis on “formal”.  Nobody expects a device manufacturer to know everything about the product at its conception.  But before going under full design controls, producing full documentation, and following all of the mandated processes (software life-cycle, risk management, usability), get the answers to your questions when it’s cheap and easy to do so.  The time will come for full design controls, but not until the project is lined out with sufficient details you need to make it a success.  Here are some suggestions on how to get those answers.

Build prototypes.  They’re relatively inexpensive, quick-turn, and don’t require the amount of documentation and process obligatory for formal development.  Prototypes need not be of the entire product; in fact, smaller, focused prototypes can be more valuable than entire product prototypes because they let you really focus on the issue at hand rather than spending time developing support for the rest of the system.  A prototype might be as simple as a 3D-printed form factor for users to evaluate as part of an IEC 62366 formative study, a breadboarded circuit to test whether a circuit works the way you think it should, or a few lines of code to evaluate a microprocessor you’re considering.

Remember to keep your prototypes and their design outputs (schematics, drawings, source code) under configuration management so that you can reproduce them later.  Bear in mind that while configuration management is a part of design controls, it is not all of design controls.  There is a big difference between knowing how to reproduce something you built and producing formal documentation for it and following full software lifecycle and risk management processes.  The intent is to capture enough about the design outputs to be able to build the prototype again, without miring yourself in documentation.

The goals of your prototypes should be to answer technical questions about the product (what can it do, what can’t it do, does the science consistently work the way we expect), simulate the user interface for representative user groups (this will lend itself towards the usability process later), and identify potential pitfalls in the design that could cause problems or hazards later.  See our Why Prototype ? article for lots more good information on prototypes.

Ask your stakeholders and subject-matter experts.  Sometimes getting the answer is as easy as asking, yet many times the question never gets asked, and the question inevitably comes up over and over again during the design, introduces uncertainty and delays, and gets put off for later so the process can repeat.  Ask the question and document the answer with its rationale.  The answer frequently isn’t enough by itself; people will come back later and ask why that is the answer.  If you’ve got the answer and its rationale documented in a common, well-known place where people can access it easily, it will significantly reduce the overhead associated with not knowing.

Contact The Realtime Group today at 972 985-9100 and let us help you identify the questions you need to answer and suggest ways to find those answers.  We can also help you design your prototypes and produce the documentation you’ll need down the road.