Guidance for Industry and Food and Drug Administration Staff - Establishing the Performance Characteristics of In Vitro Diagnostic Devices for the Detection or Detection and D...
For questions regarding this document contact Kate Simon, Ph.D., at (301)796-6204 or by email at kathleen.simon@fda.hhs.gov [1] or Marina V. Kondratovich, Ph.D., at (301)796-6036 or by email at marina.kondratovich@fda.hhs.gov [2].
U.S. Department of Health and Human Services Food and Drug Administration Center for Devices and Radiological Health Office of In Vitro Diagnostic Device Evaluation and Safety Division of Microbiology Devices |
Contains Nonbinding Recommendations
Preface
Public Comment
You may submit written comments and suggestions at any time for Agency consideration to the Division of Dockets Management, Food and Drug Administration, 5630 Fishers Lane,
rm. 1061, (HFA-305), Rockville, MD, 20852. Submit electronic comments to http://www.regulations.gov [3]. Identify all comments with the docket number listed in the notice of availability that publishes in the Federal Register. Comments may not be acted upon by the Agency until the document is next revised or updated.
Additional Copies
Additional copies are available from the Internet. You may also send an e-mail request to dsmica@fda.hhs.gov [4] to receive an electronic copy of the guidance or send a fax request to 301-827-8149 to receive a hard copy. Please use the document number 1740 to identify the guidance you are requesting.
Table of Contents
- Introduction [5]
- Background [6]
- Scope [7]
- Risks to Health [8]
- Establishing Performance Characteristics [9]
- Analytical Studies [10]
- Limit of Detection [11]
- Within-Laboratory Precision/Repeatability [12]
- Reproducibility [13]
- Cross-Reactivity [14]
- Interference [15]
- Carry-Over and Cross-Contamination Studies (for devices with automated liquid handling systems) [16]
- Specimen Storage and Shipping Conditions [17]
- Reagent Storage and Shipping Conditions [18]
- Clinical Performance Studies [19]
- Consideration of Guidelines [20]
- Intended Use [21]
- General study design considerations common to ASC-US triage and adjunct intended uses (and likely any other intended uses) [22]
- ASC-US Triage Claim – High Risk HPV Tests [23]
- ASC-US Population – HPV Genotyping Tests [24]
- Adjunct Claim – High Risk HPV Tests [25]
- Adjunct Claim – HPV Genotyping Tests [26]
- Controls [27]
- External Controls [28]
- Internal Control [29]
- Analytical Studies [10]
- References [30]
- Statistical Analysis Appendix [31]
Guidance for Industry and Food and Drug Administration Staff
Establishing the Performance Characteristics of In Vitro Diagnostic Devices for the Detection or Detection and Differentiation of Human Papillomaviruses
This guidance represents the Food and Drug Administration's (FDA's) current thinking on this topic. It does not create or confer any rights for or on any person and does not operate to bind FDA or the public. You can use an alternative approach if the approach satisfies the requirements of the applicable statutes and regulations. If you want to discuss an alternative approach, contact the FDA staff responsible for implementing this guidance. If you cannot identify the appropriate FDA staff, call the appropriate number listed on the title page of this guidance. |
Calculating Score Confidence Intervals for Percentages and Proportions
The following are additional recommendations for performing statistical analyses of percentages or proportions. There are several different methods available. We suggest that either a score method described by Altman, et al. (Altman D.A., Machin D., Bryant T.N., Gardner M.J. eds. Statistics with Confidence. 2 nd ed. British Medical Journal; 2000) or a Clopper-Pearson Method (Clopper CJ, Pearson E . The use of confidence or fiducial limits illustrated in the case of binomial. Biometrika 1934; 26:404-413) be used. The advantages with the score method are that it has better statistical properties and it can be calculated directly. Score confidence limits tend to yield narrower confidence intervals than Clopper-Pearson confidence intervals, resulting in a larger lower confidence limit. Thus when n=70 samples and 65/70=92.9%, the score lower limit of two-sided 95% confidence interval is 84.3%. In contrast, the Clopper-Pearson lower confidence limit is 84.1%. In this document, we have illustrated the reporting of confidence intervals using the score approach. For convenience, we provide the formulas for the score confidence interval for a percentage.
A two-sided 95% score confidence interval for the proportion of A/B is calculated as: [100%(Q_{1}-Q_{2}) / Q_{3}, 100%(Q_{1}+Q_{2}) / Q_{3}], where the quantities Q_{1}, Q_{2}, and Q_{3} are computed from the data using the formulas below. For the proportion of A/B:
Q_{1} = 2 • A + 1.96^{2 }= 2 • A + 3.84
Q_{2} = 1.96√(1.96^{2} + 4 • A • (B - A) / B) = 1.96√(3.84 + 4 • A • (B - A) / B)
Q_{3} = 2 • (B + 1.96^{2}) = 2 • B + 7.68
In the formulas above, 1.96 is the quantile from the standard normal distribution that corresponds to 95% confidence.
For an example of proportion if (65/70), Q_{1}=133.84, Q_{2}=9.28, and Q_{3}=147.68, then the two-sided 95% score confidence interval is 84.3% to 96.9%
Calculation of Confidence Intervals for Positive Predictive Value (PPV) and Negative Predictive Value (NPV) based on Confidence Intervals for Likelihood Ratios (Prevalence is Constant)
PPV is (1+PLR -1*(1-π)/π) -1, where PLR is positive likelihood ratio (PLR=se/(1-sp)); NPV is (1+NLR*π/(1-π)) -1 , where NLR is negative likelihood ratio (NLR=(1-se)/sp)) and π is prevalence. For the calculation of 95% confidence intervals for the likelihood ratios, use calculation of confidence intervals for the ratio of two independent proportions (the estimate of Se and the estimate of (1-Sp) for PLR and the estimate of (1-Se) and the estimate of Sp for NLR). There are several different methods available for calculation of the confidence intervals for the likelihood ratios (see Altman D.A., Machin D., Bryant T.N., Gardner M.J. eds. Statistics with Confidence. 2 nd ed. British Medical Journal; 2000, pages 18-110). We suggest that a score method described in paper by Nam (Nam J. Confidence limits for the ratio of two binomial proportions based on likelihood scores: non-iterative method. Biom J 1995; 37:375-9) be used. Using the 95% confidence interval for the corresponding likelihood ratio, it is easy to calculate the 95% CI for the corresponding predictive value where π (prevalence) is a constant.
Note:
Suppose that [L, U] is a 1-r level confidence interval for b and suppose that G is a function defined on the parameter space.
If G is increasing, then [G(L), G(U)] is 1-r level confidence interval for G(b).
If G is decreasing, then [G(U), G(L)] is 1-r level confidence interval for G(b).
(Functions (1+x -1*(1-π)/π) -1 and(1+x*π/(1-π)) -1 are monotonic functions when π is a constant.)
^{1} If the standard deviations (SD) in the precision studies for concentrations around the cutoff value are almost constant, then: C_{95} = C_{50} + 1.645 x SD, and C_{5} = C_{50} – 1.645 x SD. If the coefficient of variation (CV) in the precision studies for concentrations around the cutoff value are almost constant, then C_{95} = C_{50} + 1.645 x CV x C_{95} and C_{5} = C_{50} – 1.645 x CV x C_{5}. From here, C_{95} = C_{50} / (1 – 1.645 x CV) and C_{5} = C_{50} / (1 + 1.645 x CV).
^{2} An exception would be if a woman was twice cytology negative and HPV positive (at consecutive yearly visits) – in this scenario she should be sent to colposcopy per the 2006 consensus guidelines [Ref 13]. The bias created in this situation is unavoidable as patient health is paramount.
^{3} Procedures for Handling Post-Approval Studies Imposed by PMA Order [32]
SOURCE [33]