AN INCLUSIVE APPROACH TO THE DEVELOPMENT OF PLATFORM MEDICAL DEVICES

PREFILLED SYRINGES

Citation: Austin F, “An Inclusive Approach to the Development of Platform Medical Devices”. ONdrugDelivery, Issue 117 (Mar 2021), pp 78–80.

Finola Austin discusses the difficulties presented by designing a user testing programme for platform devices when the target patient population is as yet unknown, and presents the testing framework based on seven user groups that Owen Mumford uses to meet this challenge.

User-centred design can present a challenge to platform drug delivery device manufacturers in circumstances where the intended therapy area – and therefore intended patient characteristics – are not yet known. Applying an inclusive strategy to user evaluation studies helps to ensure device safety and effectiveness for a broad range of potential end-users. It is essential to comprehensively assess whether a device encourages adherence across various user groups and whether the needs of different patient groups have been addressed. Manufacturers therefore face the challenge of ensuring that study samples are sufficiently representative of the full range of potential end-users. Achieving this aspiration of representative sampling requires a realistic, carefully designed programme that makes the best use of company resources.

COMPREHENSIVE TESTING

“After five subjects have been tested, major usability problems will be observed repeatedly with successive subjects, and little additional usability information will be gained.”

Regulatory human factors guidance and international best practices advise that a medical device must be tested by the intended users to ensure that it is both safe and effective. User testing provides results that can be confidently considered representative of the wider user population. It is therefore important that the test participants correspond to the actual end users of the device.

However, this requires accurate identification of the intended user populations, which is not possible for platform devices where the intended therapy area has not yet been specified. For such devices, human factors sampling strategies for user testing must aim to encompass as wide a range of user capabilities as is practicably possible. This enables product designers to make informed decisions about design, whilst providing confidence to future business partners that all usability problems associated with the device’s user interface have been discovered during early-stage development, and won’t arise as an unwelcome surprise further down the line.

RECOMMENDED SAMPLE SIZE

Manufacturers must first determine an appropriate sample size to demonstrate a sound analysis of their device. Early in device development, it is generally accepted that usability tests require only five to eight participants per distinct user group.1 After five subjects have been tested, major usability problems will be observed repeatedly with successive subjects, and little additional usability information will be gained. For example, one study illustrated that doubling the number of participants from five to ten only increased the mean percentage of usability problems found from 85.6% to 94.7% (Table 1).2

Table 1: Percentage of total known usability problems found in 100 analysis samples (rounded to one decimal place).

“The US FDA states that caregivers, healthcare professionals, younger users and adults should be considered as distinct user types; these categories have been included as four groups in the sampling plan. The remaining three groups cover aspects of user/device interaction: perception, cognition and action.”

However, whilst early-stage studies might be feasible, and indeed effective, with participant numbers as low as five, it is also advantageous to ensure wide representation to guarantee timely identification of use issues and allow for the design of any relevant mitigations. Further, as development progresses, prospective pharmaceutical partners will understandably seek assurance that their intended user has been adequately considered throughout the design process and iterative user testing. For validation testing, having 15 to 20 participants per user group is recommended by US and UK regulators. This number of test participants should be large enough to reasonably reflect the heterogeneity of device users.

INCLUSIVE USER EVALUATION

The second factor to consider is that samples must encompass a range of user characteristics and needs. To this end, Owen Mumford has adopted a practical and robust framework based on seven user groups (Table 2). These groups cover the widest possible range of characteristics that are likely to influence how users use devices. The US FDA states that caregivers, healthcare professionals, younger users and adults should be considered as distinct user types;3 these categories have been included as four groups in the sampling plan. The remaining three groups cover aspects of user/device interaction: perception, cognition and action. These aspects encapsulate the user’s ability to perform the required task correctly.

Table 2: Human factors sampling strategy.

For the purposes of evaluation during development, it is useful to keep each use-impairment group mutually exclusive. This can be especially helpful in supporting the needs of prospective pharmaceutical partners, helping to illustrate the impact of different aspects of the user interface on a range of characteristics. A user evaluation strategy based on this sampling plan may be preceded by a less formal user evaluation with small groups of users that are easier to access, such as participants from device designer’s internal workforce. However, early objective user testing with more representative participants is recommended.

BROADENING REPRESENTATION

“As such, it is therefore imperative to develop an effective sampling strategy to manage potential risks and use errors across a broad a range of patients as possible, as well as anticipate the needs of prospective pharmaceutical partners.”

The user groups outlined above can be adjusted in size and makeup to incorporate a representative range of secondary characteristics, such as hand dominance, gender and ethnicity. Numbers per user group may also be increased to diversify the sample. For instance, the size of the “Action” group might be increased to accommodate a more in-depth examination of patients with biomechanical or neurological impairments as separate groups. This can help to ensure enough data is available to support root cause analysis where use difficulties and/or errors are identified. For commercial purposes, the size of each group can be adjusted in line with the projected needs of the business, such as by seeking to recruit a minimum representation of users with a specific diagnosis or comorbidity. This is especially useful where market trends and insights are available.

COVERING ALL BASES

The challenge with user evaluation is planning an effective number of studies, at the right time, and with the appropriate level of prototype fidelity. A further challenge with platform devices with an undefined intended therapy area is that the medical device developer must assume a hugely diverse target population. As such, it is therefore imperative to develop an effective sampling strategy to manage potential risks and use errors across as broad a range of patients as possible, as well as anticipate the needs of prospective pharmaceutical partners. To respond to these challenges, the human factors sampling strategy outlined in this article provides a framework for user evaluation planning in the absence of user data. This framework allows platform device manufacturers to satisfactorily assess a wide range of participants and meet best practices in a cost-effective manner.

REFERENCES

1. “ISO, IEC/TR 62366-2:2016, Medical devices – Part 2: Guidance on the application of usability engineering to medical devices”. ISO, published 2016.
2. Faulkner L, “Beyond the five-user assumption: Benefits of increased sample sizes in usability testing”. Behav Res Methods Instrum Comput, 2003, Vol 35, pp 379–383.
3. “Guidance for Food and Drug administration Staff: Applying Human Factors and Usability Engineering to Medical Devices”. US FDA, published 2016.

What's this?