Citation: Motruk R, Blazei N, “Understanding Your Process Reliability: Compliance Considerations for Emergency-Use Delivery System”. ONdrugDelivery, Issue 125 (Oct 2021), pp 52–54.
Richard Motruk and Nathan Blazei discuss the issues to be considered in light of the US FDA’s tightened reliability standards for emergency-use autoinjectors.
There are many different varieties of drug delivery systems, including oral, pulmonary, transdermal and parenteral. The use of nasal sprays for emergency-use drug delivery is increasing but, historically, autoinjectors have been the predominant method of delivery for therapeutics involved in emergency-use scenarios (e.g. adrenaline, naloxone and similar). Autoinjectors are designed for self-administration, whereby patients have the convenience of receiving the injections in a non-clinical setting (e.g. home or office).
“Since autoinjectors automatically carry out an injection cycle once actuated, patient compliance is improved compared with manual delivery devices, such as syringes.”
Since autoinjectors automatically carry out an injection cycle once actuated, patient compliance is improved compared with manual delivery devices, such as syringes. This capability is particularly important for emergency treatments, as an autoinjector can reliably administer the necessary dose in high-pressure situations, whereas more manual delivery devices could be prone to user error and result in drastic outcomes for the patient. Because autoinjectors are typically the delivery device of choice for emergency medications, they come with increased scrutiny from regulatory bodies regarding their safety and effectiveness.
In the US, an autoinjector is regulated as a combination product, which is a device and a drug (or biological product) assembled as a single entity or packaged together for assembly by the user. The increased regulatory scrutiny is necessary to ensure that patients receive their life-saving treatments without error or delay. Publicised recalls,1 in which the reliability of an autoinjector has come under examination due to failures in the field of use, have resulted in the FDA altering its expectations for manufacturers. The FDA has requested some manufacturers of emergency-use autoinjectors demonstrate overall system reliability of 99.999% at a 95% confidence level – or a 1:100,000 failure frequency. This requirement has been applied not only to new product applications but also to previously marketed products and is consistent with the FDA guidance document2 issued in April 2020.
This reliability requirement becomes a large cause of concern for developers and manufacturers when system reliability at or above 99.999% was not considered during the design stage of commercially approved products prior to the tightened standards. In some cases, reverse engineering of an autoinjector was required to improve its reliability, which led to shortages of critical therapies while improvements to the autoinjector were carried out. In other cases, manufacturing and inspection techniques required modification to obtain higher-precision components with improved quality levels. Although these modifications have improved reliability, they have also increased costs, which are met with resistance from payers and users alike. Thus, meeting the compliance requirements poses a significant challenge for developers and manufacturers.
Essential performance requirements (EPRs) are used to assess autoinjector reliability and to drive design efforts. Autoinjectors are assessed relative to the following four EPRs:
- Activation Force – the force required to trigger the device, extend the needle, and dispense the therapeutic dose through the needle cannula into the targeted tissue
- Extended Needle Length – the distance the needle travels beyond the protective sheath of the device following dose delivery
- Delivered Volume – the volume of drug dispensed during activation
- Dispense Time – the time to deliver the therapeutic dose.
Statistical tolerance intervals, or k-factor analyses, are widely accepted for estimating the reliability of data, as described in ISO 16269-6, Statistical Interpretation of Data: Determination of Statistical Tolerance Intervals. These calculations can be performed manually but industry-accepted statistical software packages, such as Minitab or JMP, make the calculations easier.
When considering tolerance intervals, there are two critical characteristics. First, the desired reliability or coverage is selected for each EPR. For the case of emergency-use devices, the FDA is mandating a minimum reliability/coverage of 99.999% at what it describes as the system level. If you consider a fault tree analysis (FTA) diagram, the system level would be the very top failure identified in that fault tree. However, one layer down from the system level within the fault tree is where the EPRs are listed, each with their own branches of contributing failure modes (faults). At this level, the FDA, for most applications, accepts 99.99% reliability.
The second characteristic of the tolerance interval estimation is the statistical confidence level. A 95% confidence level is the industry standard and is what the FDA will expect. For each EPR, a 99.99% tolerance interval with 95% confidence is calculated. The lower and upper limits of the tolerance interval calculation can then be compared with the applicable specification limits.
Consider an autoinjector with a specification for extended (deployed) needle length of 17.0–20.0 mm. A sub-sample of devices is triggered, and the resulting extended needle length is measured and recorded. From that sub-sample, the results are analysed using tolerance interval calculations at the 99.99% tolerance/95% confidence level and determined to be 17.6–19.4 mm. These tolerance interval limits fall inside the specification limits, and therefore the process is at least 99.99% reliable.
A commonly observed issue when retrospectively establishing reliability performance (e.g. for already marketed products) is that the tolerance interval range exceeds the specifications. Possible reasons for this excursion could be that the data were not evaluated at the appropriate intervals or that more data than necessary were compiled into the tolerance interval calculation. For example, a manufacturing process may use multiple tools to produce the moulded components of the device. The inter-tool variability is often larger than the intra-tool variability. This variability often leads to subtle (within tolerance) but statistically significant shifts in the data. If moulded components from unique, qualified tools are never mixed but the data are grouped without isolating the tool, an artificially inflated standard deviation used in the tolerance interval calculation may result. Thus, an overestimation of the tolerance interval width for the EPR is observed, leading to a false conclusion that the desired reliability is not being achieved.
“Manufacturers must adjust the sampling plans to the appropriate sizes, and their evaluation of the data sets must switch from a pass/fail evaluation to the use of variable data.”
Another common pitfall faced by manufacturers of emergency-use autoinjectors is that the device was not designed with these tightened reliability standards in mind. Historically, most design verification efforts targeted 99.9% as the reliability endpoint; 99.9% (1/1000) makes it logistically feasible to evaluate the performance of a batch of devices with pass/fail (or attribute) scoring and maintain reasonable certainty that the performance is meeting the reliability thresholds for each of the EPRs. Unfortunately, whether considering 99.99% or 99.999% as the new reliability target, pass/fail scoring requires a total number of samples that is entirely impractical to demonstrate the achievement of the required reliability. Consequently, manufacturers must adjust the sampling plans to the appropriate sizes, and their evaluation of the data sets must switch from a pass/fail evaluation to the use of variable data. This approach allows for the tolerance interval calculations to be carried out as previously discussed.
“All manufacturing processes are unique and need to be addressed as such.”
All manufacturing processes are unique and need to be addressed as such. A good rule of thumb to account for these issues would be to segment the data intelligently. For example, evaluating the reliability for each lot may not produce successful results, especially if the sample size for a product not originally designed with a 99.999% system reliability target is underpowered at that reliability threshold. Likewise, trying to evaluate multiple quarters or years of production into a single calculation may result in an artificially inflated tolerance interval that is not representative of the product. A frequency for evaluating the tolerance intervals needs to be established, and then the performance of the autoinjector relative to each EPR over time can be trended. Once this exercise is completed for each EPR, goals can be determined. Sub-teams can be created to investigate and remediate any areas that are not performing as expected.
One recommended approach to addressing identified reliability concerns with EPRs is to start with a reliability analysis tool such as FTA, which allows the visualisation of the reliability “equation” using component-level reliabilities to calculate the system-level reliability. The FTA can identify the primary cause(s) of reduced reliability to further target opportunities for improvement at the component level. Autoinjector manufacturing processes are often highly automated, with in-line or off-line automated inspections, creating “AND” gates in the FTA. For a defective component or defective sub-assembly to negatively impact the reliability calculation, two things need to happen:
- A defective component or sub-assembly must be presented to the inspection system
- The inspection system must fail to correctly reject (or falsely accept) the part.
Therefore, it may be necessary to execute intelligently designed studies to precisely characterise the false acceptance rates of the relevant inspection systems.
“The proper analysis tools and techniques will enable manufacturing teams to ensure that the analyses are meaningful, valid and appropriate for the application.”
The proper analysis tools and techniques will enable manufacturing teams to ensure that the analyses are meaningful, valid and appropriate for the application. Designers and manufacturers should plan accordingly during the design stage to select components and technologies that can meet these evolving reliability expectations, otherwise crippling regulatory field actions may result, which will ultimately limit patient access to life-saving treatment options. Understanding the manufacturing and inspection processes down to the component level is necessary to help organisations achieve the product reliability requirements that the FDA expects for emergency-use autoinjectors.
REFERENCES
- “FDA alerts consumers of nationwide voluntary recall of EpiPen and EpiPen Jr”. Press release, US FDA, March 31, 2017.
- “Draft Guidance for Industry: Technical Considerations for Demonstrating Reliability of Emergency-Use Injectors Submitted under a BLA, NDA or ANDA: Guidance for Industry and Food and Drug Administration Staff”. US FDA, April 2020.