Sign up for the Packaging Digest News & Insights newsletter.
Most medical device packaging developers often struggle to figure out how many samples to test. Here are some guidelines.
“How many packaging samples should I test?” For many medical device packaging professionals, this is a common question without an easy answer. Packaging test methods rarely contain sample size guidance, so it is left to the individual manufacturer to determine and justify an appropriate sample size.
Sample size justifications should be based on statistically valid rational and risk assessments. Unfortunately, there is no “magic number” that is right for every situation. For this article, we will use a commonly accepted approach to determine a sample size and discuss some special considerations to remember when doing sample size justifications.
Usually, the first step in selecting an adequate sample size is to calculate risk. Risk is the, “combination of occurrence of harm and the severity of that harm that can occur due to failure (ISO 14971).” A common approach to calculating risk is known as a Risk Priority Number (RPN). The RPN is a calculation based on an assigned severity, occurrence and detection value. Each category is assigned a value ranging from 1 to 10. It is important to recognize that other ranges are also acceptable (that is, 1 to 5) to use when establishing the risk value.
Table 1: RPN categories and definitions
The product is evaluated and a number (1-10) is assigned for each category. These values are multiplied together to calculate the RPN. You can then categorize the RPN as Low, Medium or High Risk. The RPN is not a measurement of the manufacturer’s risk; rather, it is an assignment of risk priority.
For example, in a product that has a high Severity level (such as a failure could end with catastrophic injury), a rating of 10 can be issued, which is worst case. The same product may also have a high Detection rating (where a failure is difficult to spot) so a value of 8 is assigned. Finally, it is assumed that a failure is rare and does not occur often. Therefore, an Occurrence level of 4 is assigned. These values are then multiplied together for an RPN of 320, which is a Medium Risk priority. In this example a 95% Confidence / 95% Reliability level would be assigned (Table 2).
Table 2: Example of correlating Risk (RPN) to Confidence and Reliability. ISO 14971:2007 (R2016) directs us to establish these values using a 3 x 3 risk matrix. The values are examples using a range of 1-10. These would change if the range was 1-5 or 1-3.
The Confidence Interval is an expression of uncertainty about an unknown constant. Reliability is how many units will successfully meet the pass/fail criteria.
For example, a 90% reliability means that 90 out of 100 units will successfully meet all pass/fail criteria, and a 95% Confidence Interval, indicates that a manufacturer is 95% confident that they will have less than or equal to 10 true failures.
A Method 1 Non-parametric Binomial Reliability chart (Table 3) can be used to determine a minimum sample size based on Confidence/Reliability. Non-parametric binomial reliability demonstration tests are used widely for test methods that generate attribute or qualitative data.
Table 3: This is just one example of Non-parametric Binomial Reliability chart with zero (0) allowable failures. Many options and processes exist for determining this. Search the internet for binomial parametric distribution and then identify how many failures are acceptable.
In our hypothetical example of an RPN of 320 and a 95%/95% Confidence/Reliability level, we would select a sample size of 59 with zero allowable failures. When incorporating allowable test failures into the equation, the sample size required to achieve the same confidence and reliability intervals increases significantly.
Finally, there are a number of additional factors that can also affect sample size.
• Product cost and availability can be prohibitive to larger sample sizes. Facsimile product (similar in shape, materials and weight) may need to be used when product is expensive or not readily available.
• More complex products or products that do not have a long manufacturing history may be considered a higher risk for defects and may require larger sample sets. Some products have higher inherent risks to patients. As patient risk increases, so should sample sizes.
• The test methods chosen can also affect the sample size. Generally, qualitative methods should have a larger sample set than quantitative methods. Also, depending on the type of device and packaging components, different test methods may or may not be appropriate.
Selecting an appropriate sample size can be complex as there are many factors to consider, and no single approach will be appropriate 100% of the time. There are multiple approaches to determining risk, and risk assessment is usually not simple and straight forward. It is important to have qualified people who understand your device, how the device will be used, how to appropriately determine risk, which test methods would be suitable for your device and how to build a statistically valid sampling plan. A packaging expert and a statistician can be valuable partners in helping you select and justify your sampling plan.
About the Author(s)
Consulting Manager, Nelson Laboratories LLC
Wendy Mach RM (NRCM), CQA (ASQ), serves as consulting manager for Nelson Laboratories LLC, a Sotera Health company, which is a leading provider of microbiological testing and consulting services for MedTech companies. Mach has more than 24 years of medical device manufacturing and laboratory experience. She worked in the medical manufacturing environment in Minnesota for seven years, performing ethylene oxide (EO) and gamma radiation sterilization validations. While at the manufacturing site, she also insourced the microbiology tests for sterility assurance and related equipment validation. Mach also spent four years as a lab supervisor, managing the sterility, microbial limits, bioburden, endotoxin and antimicrobial effectiveness services. In 2004, Mach accepted a position in the EO department at Nelson Laboratories and was responsible for sterilization D-value and packaging microbial aerosol challenge studies. She is a participating member on the Technical Committee ISO/TC 198 Sterilization of Healthcare Products, the subcommittee chairperson for ASTM F02.15 Chemical / Safety Properties and a member of the Packaging and Distribution Expert Committee for United States Pharmacopeia (USP).
Expert Technical Consultant, Nelson Laboratories
Jennifer Gygi is an Expert Technical Consultant at Nelson Laboratories; a microbiological testing company specializing in improving the quality of life by ensuring medical products are safe, sterile, and functional. Gygi has over 26 years of laboratory experience at Nelson Labs. She has worked in the Microbiology, Bioburden, Organism IDs, Packaging, and QC Validations groups. As one of the original members of the packaging group, Gygi has unique insights into how the packaging tests work, including pitfalls and failure points. She was also involved with all the packaging validations, new methods, and documents in her 14 years in the group.
In addition to packaging expertise, Gygi has a sound understanding of Microbiology tests and concepts and was certified as both a Registered Microbiolgist and Specialist Microbiologist by the American Society for Microbiology (ASM). Gygi serves on American Standards and Testing Materials (ASTM) packaging committees and gives frequent presentations on packaging test methodology, selection, and validation.
You May Also Like
Coffee's Still Hot in America, Especially When It’s ColdFeb 20, 2024|6 Min Read
What Does the EU MDR Extension Mean for Packaging?Feb 16, 2024|3 Min Read
Is Chlorine Dioxide the Answer to the 'EtO Problem'?Feb 15, 2024|4 Min Read
Best in New Food and Beverage PackagingFeb 15, 2024|2 Min Read