Roundtable Q & A: Testing
Jeffrey D. Lind (JL), President, Compliance West, USA
Brandon Tillman (BT), Sales Manager, Nelson Laboratories
Dave Kralovetz (DK), Medical Account Development Manager, Cincinnati Test Systems
Lisa Olson (LO), Vice President of Testing and Service Development, WuXi AppTec
Question 1: What is a common and easily remedied error medical device manufacturers commit when it comes to product testing? How can they avoid this?
JL: In many cases, the Standard requirements being met are not contained in just one paragraph of the Standard, or even in the Standard itself. If manufacturers took steps to ensure that the requirements were fully understood before a test plan was implemented, they would stand a much better chance of an accurate result. Another commonly overlooked step that helps to ensure testing reliability is to make certain there is complete understanding of all the ways the new product is going to be used, how all the features and options are integrated, and what they will be used for. With this information, the pretesting program can be tailored to embrace the entire product.
BT: Incorporating product testing into the initial design specifications. For example, MDMs save themselves a lot of time, money, and hassle when they choose materials in the design phase that are known or have already been proven to be biocompatible. The same principles apply to sterilization (choosing materials that can be sterilized), cleaning for reuseable devices (choosing materials and device designs that can be cleaned), and packaging (choosing packaging that can allow product to be sterilized and maintain aseptic presentation).
DK: Some manufacturers leak test using pressure-change specifications (e.g., -0.03 psig in 5.0 seconds @ 20 psig) as opposed to rejecting a true leak rate in volumetric flow (e.g., 1.0 sccm @ 20 psig). While often working well with a single tester, fixture, and device model, manufacturers often don’t realize that applying that pressure change spec to other similar products or test systems (tester and fixture) with differing volumes means they are in fact rejecting different leak rates (with a given leak, pressure change is heavily influenced by volume). The volumetric flow method compensates for volume differences and allows calibration and setup challenge using an NIST traceable leak standard so that each combination (tester, fixture, device model) is rejecting the same leak rate.
LO: Many manufacturers of combination products and medical devices start biocompatibility testing prior to final design freeze. This includes final packaging and production processes. Inadvertent contaminations and adulterations to the product can happen in development-type production that would otherwise have been well controlled during normal lot release production. These contaminations can cause failures in assays that have to be investigated and justified. Significant changes to design can also impact test results. Guidance documents are clear that it is final product that is to be evaluated, so changes to materials, formulations, and processes will require re-testing.
Careful planning early in the design and development stages, including mapping out when and what testing is needed, provides significant benefits to the manufacturer. This planning should also include good estimates of the number of test samples needed. For example, targeting necessary submission dates can avoid last minute scrambles as, in many cases, the experimental portion of the assay has fixed timepoints that cannot be changed. Early understanding of sample requirements allows time for production and avoids having to compromise on the sample preparation for testing. So, in general, good planning can make the difference between delaying submissions and having the final report available.
Question 2: What misstep do you see device manufacturers take when it comes to ensuring a device will pass through validation testing successfully?
JL: Erroneous results are often recorded by device manufacturers due to tester malfunction or errors in test setup, whereas a simple dry run of the tester in standalone mode would have pointed out the problems in advance. In addition, conducting the test with the product out of the circuit and checking the output for a reasonable result before pretests begin will certainly help ensure accurate validation testing. Medical device manufacturers should also make it a priority to establish relationships with their test equipment supplier, which will work to ensure open dialogue and expedite finding solutions to any testing challenges that might occur.
DK: Instead of validating the technology being used, some device manufacturers validate a single leak or flow tester model itself for use with a very wide variety of devices they manufacture, even when that model may be archaic or its design far from ideal for all the products they test. While seeming to save significant effort in validating new products (ability to reuse much of their current documentation/methodology), the pitfall is this concept locks the manufacturer into a tester/supplier that sometime later may prove to be anything but the preferred solution. Later, realizing this, they learn that the cost of re-validating these lines is prohibitive and they must “make do” with what they have, like it or not.
LO: In some cases, there is the thought that if a little is good, more is better. Many times, manufacturers will over-sterilize devices prior to testing, which can dramatically impact final results. In other cases, manufacturers are rushing and take samples from bench-top or R&D production runs that aren’t representative of what the patient will see nor are they well controlled and traceable.
Question 3: What tests are critical for medical device manufacturers to perform during the product design phase that are sometimes overlooked?
BT: Material compatibility both sterilization and biocompatibility. MDMs can ask themselves the following questions, “Can the materials I am using be sterilized by ethylene oxide, radiation, steam, or vapor hydrogen peroxide?” “Does the sterilization method I am using cause brittleness or discoloration?” “Am I using a sterilization method with limited availability?” “Do any of these sterilization methods affect the functionality of my device?” “Am I using raw materials that are not medical grade, or that do not have proper documentation to support biocompatibility claims?” “Have I added a dye to the device where leachable/extractables need to be explored?”
DK: Ideally, a device manufacturer will not only understand what conditions will cause their device to drastically fail in operation but will also take the time to create a matrix of marginal or partial failures and thoroughly test them with air, vacuum, or tracer gas test, followed by testing empirically in a clinical manner and correlate the results. Almost always, the testing results from these “manufactured” defects can hasten the determination of reject limits that should be used in production. This prevents the manufacturer from making their specifications more rigid or lax than necessary.
LO: Combination product and medical device manufacturers frequently overlook performing basic materials characterization. It is instrumental in trouble shooting, change control, and supplier management to have a good, basic understanding of materials that are in those initial promising devices.
Final Word: Any thoughts/comments on testing or validation that you wish you could tell medical device manufacturers to aid them with this process?
JL: Because of the general nature of the Standard, interpretation of requirements are necessary to test many products. If pretesting is being done, a good relationship with the product testing regulatory agency can be very helpful when there are new interpretations. Although the interpretation of the Standard by the manufacturer may be correct, if the agency engineer’s opinion differs, you may not win the day. If for no other reason than to educate the agency in the actual use of the product, a close relationship and good communication with your agency engineer may pay off in the long run.
BT: To aid MDMs with this process there are capable CROs and consultants. Some CROs, such as Nelson Laboratories, offer complimentary advice to help medical device manufacturers avoid common product testing and validation pitfalls.
DK: While the initial cost may be higher, the cost of testing without the operator involved in the accept/reject decision making process is always less expensive in the long run. Removing the operator from the decision making process and replacing his influence with digital dry-testing instrumentation that not only makes the decision, but can also deliver quantified results (grading of leak or flow), makes the process far easier to validate and later system failures easier to diagnose. In addition, poke-yoke features can be incorporated by the instrument, which prevent the loading/unloading system (manual or automatic) from missing a reject result, adding robustness to the test process.
LO: Combination product and device manufacturers should really understand their product, as well as the materials and manufacturing processes involved in making the product. It sounds like a simple statement, but in reality, it involves a lot of areas of investigation and expertise. Also, manufacturers should carefully review the testing they are considering. The manufacturer should be requiring the test laboratory to help them design studies and study panels that are relevant to their product specifically.