1223

VALIDATION OF ALTERNATIVE MICROBIOLOGICAL METHODS

Change to read:

INTRODUCTION

This chapter provides guidance on the selection, evaluation, and use of microbiological methods as alternatives to compendial methods. To properly implement alternative methods, one must consider a number of important issues before selecting the analytical technology and qualifying that method with the actual product. These issues include, but are not limited to, identification of suitable alternative methodology, development of user specifications for equipment selection, demonstration of the applicability of the method as a replacement for a standard compendial method, and qualification of the method in the laboratory.

This chapter outlines:

User requirements

The limitations of the use of CFU as a standard signal for microbiological methods.

Four novel options for demonstrating equivalence

Application of the concept of non-inferiority to method validation

Guidance on statistical methods that may be employed in method validation

A glossary of the terms used in the chapter is provided at the end of this chapter.

Microbiological methods, other than microbial identification and strain typing methods (discussed in Microbial Characterization, Identification, and Strain Typing

1113

), described in the compendia fall into two general categories:

Qualitative methods (not enumerative) that are used to assess the general microbial quality of compendial articles. This category includes assays that are intended to demonstrate the presence or absence of microorganisms in a compendial article.
Quantitative methods that yield a numerical (enumerative) result in terms of the microbial content of a compendial article.

There are inherent analytical factors that must be considered in the implementation of microbiological methods and in the comparison of a candidate alternative method to an existing compendial method. With respect to qualitative (“absence of”) analysis, it is critical to consider that in microbiology, the finding of “no microorganisms present” does not mean in absolute terms that zero cells are actually present in the compendial article. A result of “no growth” in a current compendial method is properly interpreted as “no growth was detected in the test sample from the compendial article under the specified conditions”.

The actual limits of detection of compendial microbiological methods have never been established quantitatively, and it is understood that many variables can affect the recovery of microorganisms. These variables include selection of growth media, incubation conditions, nutritional requirements of microorganisms that may be present, physical condition of microorganisms, and characteristics of the compendial article under test. Studies on the recovery of microorganisms from potable and environmental waters have demonstrated that traditional plate-count methods reporting cell count estimates as colony-forming units (cfu) may recover 0.1%–1% of the actual microbial cells present in a sample (1), in comparison to alternative methods that use flow cytometry and therefore yield a different signal (cell count). The presence of a greater number of cells based on an alternative method with a signal other than cfu has not correlated with more user risk or a higher likelihood of pathogens being present when there is an established safety record. These results do indicate that in some types of samples, the mean estimated cell count recorded using a growth-based compendial assay may result in a very different mean value than a cell count estimate derived from an alternative microbiological method that relies on a signal other than the cfu. Also, one must consider that in analytical microbiology the concept of false positive or false negative results are both scientifically and conceptually difficult. It would not be appropriate, for example to consider in a comparison between a standard compendial method and a candidate alternative method that a negative result in the standard method meant that positives observed in alternative method were false positives. It is a normal characteristic of a conventional growth-based method to recover some species well and yet to be unable to recover others. While it is not necessary for an alternative method and the conventional method to produce a match in terms of result, what is important is that the candidate method be capable of allowing a microbiologist to make an equivalent decision regarding product quality consistently.

It is extremely important in the application of this chapter that users take into account that microbiology is a logarithmic science. While we can distinguish between 100 and 1000 cfu (a difference of 1 log10), it may be not possible to discern smaller differences (less than 0.3–0.5 log10). The inherent variability of these methods is substantially greater than analytical chemistry methods. This inherent analytical variability must always be considered in the selection, development, and validation of alternative methods. The expectation of a degree of agreement between alternate microbiological methods and traditional growth-based methods beyond what is technically feasible could complicate the implementation of newer analytical technologies regardless of their specific mode of analysis.

Achieving the level of characterization (variability of the method) that is possible using modern chemical methods (e.g., high-performance liquid chromatography with a precision of 1%–2% relative standard deviation) is not possible in microbiology. It is reasonable to consider that the typical level of precision will typically be on the order of 15%–35% relative standard deviation, although results outside this range both on the high and low sides are certainly possible. Also, the enormous numbers and diversity of potential microorganisms as well as the inherent variability of metabolic activity levels in nature can complicate recovery. The advent of alternative microbiological methods, which in some cases may recover higher cell counts than typically observed using existing compendial methods, should not be taken to mean that new patient risks now exist that had not been heretofore recognized.

USP Perspective on Implementation of Alternative Methods or Procedures

The U.S. Pharmacopeia (USP) has long provided mechanisms for the implementation of alternative assay methods or procedures to analyze compendial articles. General Notices and Requirements in the USP states, “Alternative methods and/or procedures may be used if they provide advantages in terms of accuracy, sensitivity, precision, selectivity, or adaptability to automation or computerized data reduction, or in other special circumstances.” This statement allows considerable user latitude in the decision to use an alternative procedure for routine product release, provided that proper technical and scientific attention is paid to the selection, qualification, and implementation of the method. If a product has proven safe in widespread use when released or controlled using current methods, the implementation of an alternative method which can be well-correlated to the existing method should be straightforward.

USP Methods and Procedures as Referee Tests

All methods and procedures described in the USP general chapters on microbiological tests, which are numbered below

1000

, are intended to be referee tests for any product legally marketed in the United States. This means that in the event a dispute should occur for any reason, only the result obtained using the method or procedure published in USP is conclusive. Thus, alternative methods or procedures implemented and qualified by a user will not serve as a legal replacement for the official USP method, which will continue to serve as the referee test in the case of a dispute.

General Considerations Regarding Quality Control Product-Release Assays

Although the methods and procedures described in the USP general chapters are intended to be official referee test methods, it is recognized that these tests may not function optimally as quality control tests for specific compendial articles without modification. It is expected that the official USP methods will require evaluations to determine their suitability for use when applied to specific products, and that such evaluations will be conducted on the basis of a user's specific product knowledge. Procedures for demonstration of this method suitability (verification; inhibition/enhancement) are provided in the relevant compendial test chapters. For example, some compendial articles may have inherent antimicrobial properties that could, if not modified (or neutralized), adversely affect the suitability of a given compendial method or procedure. When these methods or procedures are included in regulatory filings and the product is approved, they will be used as product-release and shelf-life tests.

Harmonized Chapters

Some USP microbiological methods or procedures contain a statement that they have been harmonized with corresponding methods or procedures found in the European Pharmacopoeia and the Japanese Pharmacopoeia. In chapters containing this statement, the text describing the method or procedure is considered interchangeable (for additional information, see Pharmacopeial Harmonization

1196

). If a compendial method or procedure is conducted using harmonized text as written in the USP, European Pharmacopoeia, or Japanese Pharmacopoeia, it is considered legally interchangeable. However, the implementation of an alternative method as a quality control test to replace a method described in the USP does not mean this method or procedure would be interchangeable from the perspective of all relevant jurisdictions unless it meets the criteria for alternative methods as specified by the other compendia.

Submission of Alternative Methods or Procedures to USP

In the USP, it is stipulated by General Notices and Requirements that alternate methods should be submitted to the USP for evaluation, if an organization wishes to have the alternative method to be considered for inclusion as a compendial method. This opportunity to advance microbiological testing is often overlooked by USP stakeholders. Submission of alternative methods or procedures allows USP to consider any such method as an addition to or replacement for an existing, standard method or procedure. A submission of an alternative method must include complete analytical and equipment details, as well as detailed analytical data from relevant qualification trials.

For any method or procedure to be considered as a replacement or additional referee method, it must not be a patented method or procedure with reagents or instrumentation available from only a single source. Also, any candidate replacement or additional method must have broad applicability, suitable for routine use and must be compatible with a broad spectrum of relevant compendial articles.

Additional USP Chapters Germane to the Implementation of Alternative Microbiological Methods

Several USP general information chapters provide usual information regarding the implementation of alternative methods and procedures. Validation of Compendial Procedures

1225

provides detailed information on submissions of alternative methods to the compendia as well as general guidance on validation and the evaluation of analytical performance characteristics. Useful information regarding the validation of methods used for the recovery of viable microorganisms from compendial articles (neutralization of antimicrobial properties of articles) appears in Validation of Microbial Recovery from Pharmacopeial Articles

1227

. Information on the qualification of microbial identification methods may be found in

1113

. Finally, information on the qualification of analytical equipment can be found in Analytical Instrument Qualification

1058

Although Analysis of Biological Assays

1034

is primarily concerned with the determination of product potency by utilizing bioassays, it also discusses fundamental principles regarding variance, error, and biometrics that may be informative to those who are developing and validating alternative microbiological methods or procedures.

Other Information and Regulation Regarding the Use of Alternative Methods

In the U.S. Food and Drug Administration (FDA) Current Good Manufacturing Practices regulations, 21 CFR Part 211.194 describes requirements for test methods utilized to assess the compliance of pharmaceutical articles with approved specifications. The regulations state that test methods must have suitable capability regarding accuracy and reliability. This subsection of the regulations also recognizes the legal basis of USP and the National Formulary (NF) standards and makes it clear that it is the responsibility of the user to validate methods or procedures that differ from those standardized in the compendia.

The International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) document Validation of Analytical Procedures: Text and Methodology [Q2(R1)] also may be a useful reference in the development and validation of alternative methods or procedures. This ICH document provides information useful in the submission of analytical procedures in association with product registration applications submitted within the United States as well as Japan and Europe.

USER REQUIREMENTS

Determining the precise technical requirements necessary for an alternative microbiological method is essential before one can select the appropriate technology and equipment that meet the relevant assay requirements. Therefore, it is suggested that organizations wishing to develop and validate a candidate alternative method produce a user requirement specification (URS) document. This document should include all critical functions of the technology, critical user interface requirements, space requirements, environmental requirements, operational requirements, and all other important characteristics of an alternative method for the intended use. These requirements will be specific to the company or organization, as well as to the alternative method's intended use, and therefore the requirements should be generated by the user.

In generating this URS document, three separate components of the alternative microbiological method validation must be considered:

Instrument qualification. Most alternative microbiological methods will depend on specific equipment. This analytical equipment is subject to industry standard instrument qualification requirements (see 1058 for further information).
Validation of alternate technologies. The basic rationale for using an alternative methodology is to improve on some aspect of the existing technology of the current compendial method without sacrificing essential characteristics of that technology (e.g., plate count and membrane filtration). The current technology for compendial microbiology methods consists of detection of the growth of viable microorganisms on (or in) a nutrient medium. The alternative technology must be at least equivalent to the current technology in terms of performance for the intended use. Much of the technical support for equivalence may come from the peer-reviewed scientific literature or from a prior regulatory submission (e.g., a vendor submitted the Drug Master File to the FDA, or prior submission from a company on this technology), but this must be confirmed, as appropriate for the intended use.
Method suitability. This consideration must address both the technology's suitability to the specific test and the lack of product inhibition and enhancement on the test results.

Components of Data Quality

General information chapter

1058

describes four different components of data quality. The most fundamental component is qualification of the instrument; that is, a demonstration that the instrument is functioning as designed. Next in significance is the method validation; a demonstration that the technology is functioning as expected. For instance, this might be a demonstration that an alternative microbiological method is at least as suitable for its role in the test method as was the traditional plate count or recovery in nutrient broth. Next in importance is the inclusion of relevant controls in the test to demonstrate the ongoing suitability of the test system. The final component of data quality is the use of quality control samples, a practice not commonly used in microbiology because analysts strive to exclude live cultures from a product testing area. These different components of data quality are an important consideration in validating an alternative microbiological test as they help frame the URS.

Classical Microbiological Methods

The cfu has been in use for about 125 years and continues to be specified as the unit of microbial enumeration in all current USP monographs. However, it is important to understand that the cfu has always been an estimation of microorganisms present, rather than an actual count. The conceptualization of cfu as a signal requires a fundamental grasp of the process of plating bacteria, yeast, or mold on solid media, as well as knowledge of what is required to produce a single colony.

The plate count method provides an estimate of the number of microorganisms present based on the growth of discrete countable colonies on an individual plate; thus, the plate count is not a true cell count. Although it is theoretically possible for a single viable cell to give rise to a cfu, a single cell growing into a colony on a plate is unlikely to happen in nature. “Viability of a cell” is defined as the ability to multiply by binary fission such that a colony appears. For a colony to appear, viable cells must find specific conditions of nutrient growth medium, incubation, and time. Individual cells, however, are a rarity in nature, and it is far more likely that any colony growing on solid media arose from a clump, chain, or mass of cells deposited together. The cfu signal then is prone to underestimate the actual number of cells present in a sample. The extent of underestimation will vary, depending on the nature of the microorganism and the way in which the sample was prepared.

The cfu signal is also completely dependent upon recovery of microorganisms from environmental conditions, which produce stress on the organism's ability to survive. It is important to note that all organisms that are outside their preferred environmental niche will be to one degree or another stressed. Furthermore, outside laboratory culture microorganisms are unlikely to be in the exponential growth phase most optimal for transfer onto microbiological media for recovery and growth. In a very real sense most microorganisms present in compendial articles, in dry, nutrient-free environments, at elevated temperatures, high ionic strength, pH extremes, or in the presence of antimicrobial chemicals are severely stressed and may prove difficult or impossible to recover. Clearly these stress factors play a role in the plate count anomaly mentioned above. If the growth, nutritional, or incubation conditions presented to microorganisms are not sufficient to result in recovery and the growth of colonies, the signal may be 0 cfu, or no growth, even when viable cells are present. These stress factors may not be important considerations with nongrowth-based alternate methods and such methods may produce signals from cells that will not grow on media. Precision can be compromised further when organisms are present in large clumps, often associated with organic material, and are broken into smaller units during preparation. In this case, depending on the processing or handling of the sample, a clump could appear as a single colony or multiple colonies. Furthermore, the number of cfus on a plate must be in a countable range, for example bacteria, 25–250, for reasonably reliable enumeration.

Thus, the methods of growth-based microbiology represent a logarithmic science with a signal of enumeration (cfu) that is truly an estimate rather than a precise cell count. Understanding the strengths and weaknesses of the cfu as a signal is vital in the validation of an alternative method that uses an alternative signal. Therefore the cfu cannot be considered the only unit of microbiological enumeration.

Signals from Alternate Microbiological Methods

Rapid or modern microbiological methods typically produce signals in units other than cfu for microbial estimation and enumeration. These signals are often processed via instruments rather than visually. Extensive studies have been conducted on the capabilities of the various methods that can be applied to microbial assessment of compendial articles, and in most cases the prospective user will know the characteristics of the method and the signal it produces before selecting that method as an alternative. Guidance on method selection is provided in the section on Validation of Alternate Technologies and in peer-reviewed scientific publications.

Most of the rapid microbiological methods are, to some extent, direct cell count methods. They, therefore, may provide a higher cell count estimate than the cfu method for a given sample, depending on how the method is used and which compendial article is under evaluation.

Some alternate or rapid methods detect and estimate cell counts on the basis of metabolic activity, which gives rise to a signal that can be measured instrumentally. Examples of these types of signals include adenosine triphosphate (ATP) content (bioluminescence), laser-induced fluorescence, enzymatic activity, and physiochemical changes to the composition of a nutritional broth or the headspace above the broth.

Alternative methods may also be based on vital staining, in which cells are stained or exhibit autofluorescence (based on cell components) and then are directly counted, either microscopically or instrumentally. To increase the probability that only living cells will be counted, multiple stains may be used, which can (1) increase sensitivity based on cell membrane function, (2) enhance reaction with nucleic acids, or (3) improve detection of metabolic activity.

There are also nucleic acid based methods that can be used, as well as a range of other physicochemical methods of analysis that have been utilized in pharmaceutical, biopharmaceutical, clinical, and food microbiology. These methods may target, amplify, detect, or quantify a nucleic acid sequence, and it is important to understand the type of signal that results from the analytical method. In addition, one should understand the physiological characteristic of the microorganism that gives rise to the signal, which then makes it possible to enumerate the cells.

success criteria

Alternative methods for obtaining a cell count may provide higher or lower cell counts than those provided by traditional compendial methods during the enumerative analysis of compendial articles. However, whether the cell counts are higher or lower with the alternative method, it is generally possible to detect adverse trends in comparison with the estimates obtained using a compendial method.

Observations of cell counts that differ from cfu results are not a concern if the different methods and their different signals of cell presence are equivalent to or are non-inferior to referee methods in terms of assessing the microbiological safety of an article. Higher cell counts must not be considered as necessarily indicative of greater risk given the inherent variability of standard growth methods and the physical and chemical nature of compendial articles subject to analysis. This is especially true when enumerating microorganisms in articles that have a long history of safe and effective use. In such cases the discovery that an article contains a higher cell count then previously known does not mean that its safety has deteriorated.

With qualitative methods, i.e., presence or absence, comparisons of false negative and positive results obtained in controlled studies with the compendial and alternative microbiological method may be a measure of equivalence. There are commercially available enhancements to growth-based methods that allow colonies on solid media to be read more quickly, with substantially less incubation time, than is possible using only the unaided eye. These methods still require the growth of organisms on or in media. Therefore, many of these methods are not in the strictest sense different from the existing compendial methods, but are instead merely enhancements providing a more rapid detection of colonies. In the implementation of these enhanced methods for the detection of colony growth only the detection capability of the method requires verification.

sample size

Any alternative microbiological test method (within its intended purpose) may use any sample size and number of tests that is sufficient to produce an equivalent decision (or better) regarding microbiological quality as compared to the reference method. It may be simpler, for many if not most alternative methods, to comply with the sampling instructions that are provided in the official compendial method. In case the sampling approach defined in the official compendial method is utilized no justification of the sample size is required.

Statistics and Alternative Methods

Attempts to use statistics to compare the cfu results to signals arising from biochemical, physiological, or genetic methods of analysis may have limited value. Given the differences among these methods, they cannot be expected to yield signals that could be compared statistically in terms of mean values and variability. Thus, the enumerative values, given as cfu results in association with reference methods, typically cannot be used as acceptance criteria for the assessment of articles via candidate alternative methods. Instead, it is the users' responsibility to propose values, supported where necessary by scientific literature, that they can demonstrate are appropriate for the method that they have chosen and validated. This can be done independently of existing standards expressed in terms of cfu.

INSTRUMENT QUALIFICATION

Instrument qualification should follow, at least in general terms, the discussion in

1058

. The instrument qualification for equipment critical to the functioning of an alternative microbiological method involves four distinct phases:

Establishment of User Requirement Specifications for critical method attributes which should be formalized in a URS document.
Installation qualification—Was the instrument installed correctly?
Operational qualification—Does the instrument meet the manufacturer's specifications for correct operation?
Performance qualification—Does the instrument meet the URS for performance?

VALIDATION OF ALTERNATE TECHNOLOGIES

User Requirements Specification

Preparation of the URS document should involve input from all stakeholders for the microbiological test method. These stakeholders may include representatives from the Microbiology, Quality, Regulatory Affairs, and Operations groups, as well as others. The time spent on this step should be considered an investment in reaching a clear understanding of the company's needs before equipment is purchased, which will drive the performance qualification. At minimum, this document should include the following:

Purpose and intended use (defined need for instrument)
Description of who will use the equipment
Operational requirements (data format, user interfaces, and operating environment)
Constraints (timetables, downtime, maintenance, user skill levels, product compatibility, limit of detection, accuracy, and rapidity)
Life cycle (development, testing, delivery, validation, training, and obsolesce)
Capability (turnaround time, test capacity and throughput, and labor requirements)
Sustainability (consumables, calibration, validation, and preventative maintenance)

See

1058

for information on the qualification of analytical instrumentation. The principles outlined in

1058

are generally applicable to the qualification of instruments used to conduct alternative microbiological analysis. The user may need to tailor the specific recommendations in

1058

to their particular instrument qualification specifications.

Validation Criteria

The validation parameters generally recommended for qualitative and quantitative microbiological tests are shown in Table 1. Examples of qualitative tests are the sterility test and the test for absence of specified microorganisms. A quantitative test would be microbial enumeration. Note that qualitative testing is binary, and for this reason there is generally no need to define equivalency of units of measure, only equivalency of outcome.

Table 1. Validation Parameters by Type of Microbiological Test

Validation Parameter	Qualitative Tests	Quantitative Tests
Accuracy	No	Yes
Precision	No	Yes
Specificity	Yes	Yes
Limit of detection	Yes	Yes
Limit of quantification	No	Yes
Linearity	No	Yes
Operational (dynamic) range	No	Yes
Robustness	Yes	Yes
Repeatability	Yes	Yes
Ruggedness	Yes	Yes
Equivalency	Yes	Yes

specificity

Definition: The specificity of an alternate qualitative microbiological method is defined as its ability to detect a range of challenge microorganisms specific to the technology. “Range of microorganisms” may be defined as a limited number of microorganisms representing risk to patient or product, microorganisms found in the manufacturing environment and product failures, microorganisms that are appropriate for measuring the effectiveness of the alternative method, and microorganisms that are representative in terms of morphological and physiological attributes appropriate for the method and the product.

Demonstration: Specificity is demonstrated by comparable recovery of the challenge panel of microorganisms in both the compendial and alternate methods. The microbial challenge is above the limit of detection or quantification but at a level that provides a measure of efficacy of the methods.

Growth based—Add low numbers (around 100 cfu) of each microorganism on the panel and perform both the compendial and alternative methods to demonstrate recovery of the microorganism.

Nongrowth based—Use suitable negative and positive controls to demonstrate that extraneous matter that may be in the system (e.g., extracellular ATP, DNA, or inhibition and enhancement factors) does not interfere with the detection of the defined range of microorganisms.

All challenge microorganisms should be recovered and identified in growth-based methods. For nongrowth-based methods microorganisms should be recovered and identified where possible.

limit of detection

Definition: The limit of detection (LOD) of an alternate microbiological method is defined as lowest number of microorganisms in a defined volume of sample that can be detected, but not necessarily quantified, under the stated experimental conditions. This should be conducted with the quality control organisms cited in Antimicrobial Effectiveness Testing

, Microbiological Examination of Nonsterile Products: Microbial Enumeration Tests

, Microbiological Examination of Nonsterile Products: Tests for Specified Microorganisms

, Mycoplasma Tests

, and Sterility Tests

as appropriate to the alternative method.

Demonstration

Method 1

Inoculate a suitable diluent solution with a serial dilution range of each challenge microorganism, appropriate for the intended use of the method and the technology. In most cases, the compendial media growth promotion test panel may be sufficient.
The level of inoculation should be adjusted to a target of 50% of the dilution samples that show growth in the compendial test.
Perform both the compendial and alternate tests.
Tests should be repeated a sufficient number of times (statistically significant alpha risk: 0.05; beta risk: 0.20) for both the compendial and alternate tests.

— Statistics: Use the chi-square test or another appropriate approach to demonstrate equivalent recovery of the microorganism challenges.

— Alternately, use Method 2.

Method 2 (MPN Method)

Create a dilution series of the challenge organisms in a suitable diluent solution to include at least the range of 101 cfu to 102 cfu (for a 10-fold series) or 5 cfu to 101 cfu/inocula volume (for a 2-fold dilution series).
Perform both the compendial and alternate tests with at least 5 simultaneous replicates of each dilution from the chosen series.
Determine the most probable number (MPN) from three dilutions in series that provide both positive and negative growth (or signal).
Statistics: Use the chi-square test or another appropriate approach to demonstrate equivalent recovery of the microorganism challenges.

robustness

Definition: A capacity of the method to remain unaffected by small but deliberate variations in method parameters, e.g., reagent volume, incubation time, or ambient temperature providing an indication of its reliability during normal usage. A measure of robustness is not a comparison between the compendial and alternate methods; rather, it is a necessary component of validation of the alternate method so that the user understands the limits of the operating parameters of the method. The user may rely on data supplied by test method manufacturer.

ruggedness

Definition: The degree of precision of test results obtained by the analysis of the same samples under a variety of typical test conditions such as different analysts (for example, three), instruments, and reagent lots (the method for demonstration may follow instrument or materials supplier recommendations, or it could be based solely on data supplied by test method manufacturer).

For the definition of other validation parameters see Glossary.

METHOD SUITABILITY

For each new product to be tested using the validated alternate microbiological method, perform the suitability test as described in general test methods (see

, and

), using the number of unit and quantities prescribed and the sample preparation appropriate for the product and the required test sensitivity to determine the absence of a product effect that would obscure the signal of the method.

Method suitability may be demonstrated using three independent tests. Only the accuracy and precision validation parameters are required for quantitative methods. For qualitative methods, recovery of challenge organisms as indicated in

, and

1227

is sufficient.

After an alternative method has been shown to be equivalent to the compendial test with one product, it is not necessary to repeat the equivalency parameters for every new product; it is merely necessary to verify the method suitability for each additional product. For example, when employing a nucleic acid based method, with each new product, one must demonstrate that residual product does not interfere with the concentration, extraction, purification, and recovery of the target nucleic acid, or the polymerase chain reaction (PCR) amplification and chemical probe detection of the target ribosomal ribonucleic acid (rRNA) gene sequence.

EQUIVALENCY

All microbiological tests are performed to enable informed decision making regarding the microbiological quality of a product, raw material, component, or process step. In this respect, the intended purpose of microbiological tests may be to either evaluate for the presence or absence of microorganisms (as in the sterility test) or to estimate the number of organisms present. The technological means by which microbiological test methods assess microbiological quality and enable a product-quality decision may differ from the growth-based means typical of reference methods. The units of measurement (signal) of a microbiological quality assessment performed using alternative microbiological test methods will generally not be a cfu, but rather a different approach to obtaining a cell count estimate. Therefore, the validation of alternative microbial methods should involve two components: (1) equivalence demonstration and (2) analytical method and equipment qualification.

Equivalence Demonstration

General Notices and Requirements states that alternate methods may be used if they provide advantages in terms of accuracy, sensitivity, precision, selectivity, or adaptability to automation or computerized data reduction. It further stipulates that alternative methods can be implemented in other special circumstances. Such alternate methods shall be validated as described in

1225

and must be shown to produce equivalent or better results than the referee method for any given quality attribute. When comparing two test procedures to show equivalent or better performance, statistical evidence is assembled to show equivalence or, in statistical terms, non-inferiority. For example, with microbial enumeration, equivalency may be shown if there is no statistically significant difference between the two means generated when enumerating with the compendial and alternative methods. However, this may not be possible when the two methods yield different signals. Examples of this situation are when the microbial enumeration method uses vital staining of microbial cells or measurement of genomic material in place of cfus.

Similarly, the FDA Guidance for Industry document Analytical Procedures and Methods Validation: Chemistry, Manufacturing, and Controls Documentation states that a validated alternative analytical procedure should be submitted only if it is shown to have performance equal to or better than the regulatory analytical procedure. Also, section 2.7 of the ICH document Test Procedures and Acceptance Criteria for New Drug Substances and New Drug Products: Chemical Substances (Q6A) states that alternative procedures are those that may be used to measure an attribute when such a procedure controls the quality of a drug substance or drug product to an extent that is comparable to or superior to the official method. However, other options to demonstrate equivalence are available and are discussed in Demonstration of Equivalency.

Demonstration of Equivalency

Four options are available to establish the equivalence of a candidate alternative analytical method: (1) acceptable procedures (i.e., merely meeting a minimum performance or acceptance requirement without a need to demonstrate equivalence to the compendial method); (2) performance equivalence to the compendial method; (3) results equivalence to the compendial method; and (4) decision equivalence to the compendial method (1). A comparison of these four equivalence options is given in Table 2. The multiple equivalence options reflect the diversity in the technology and applications of the alternative test methodologies and may be viewed as a paradigm shift in Equivalence Demonstration.

Table 2. Equivalence Option Matrix

Option	Demonstration	Comparison to Official Compendial Method	Based on Numerical Results or Conclusion	Number of Characteristics
1. Acceptable procedures	Acceptable	No	Results	Multiple
2. Performance equivalence	Equivalent	Yes	Results	Multiple
3. Results equivalence	Equivalent	Yes	Results	Single
4. Decision equivalence	Equivalent	Yes	Conclusions	Single

option 1: acceptable procedure

This is not strictly an equivalence option that requires direct comparison between the candidate alternative method and an official compendial method. With this option, a reference material with known properties may be used, such as a standard inoculum of a specific microorganism, a quantity of highly purified bacterial genome, an ATP level or another appropriate signal. In some cases, it could be required that the alternative method measure the signal in the presence of the test sample, with validation criteria that are consistent with the capability of the technology, as described in the scientific literature.

option 2: performance equivalence

Performance equivalence requires the demonstration of equivalent or better results with respect to validation criteria—such as accuracy, precision, specificity, limit of detection, limit of qualification, robustness, and ruggedness—that may be appropriate for the intended use of the alternative qualitative or quantitative method. It is possible that the alternative method may not conform to some of the validation parameters listed compared with the official method and still be acceptable because of the advantages of the alternative method. This may be the case if the alternative method has any of the advantages stated in the General Notices and Requirements (methods “may be used if they provide advantages in terms of accuracy, sensitivity, precision, selectivity, or adaptability to automation or computerized data reduction, or in other special circumstances”). Other special circumstances would include improvements in time to obtain a result or the cost of running the test. If a candidate alternative method is suitable for assessing the quality of the material tested, it may be still acceptable, even if it differs from the official method in one or more validation parameters. The final analytical qualification criteria should reflect only the criteria that the microbiologist deems necessary to achieve performance equivalence.

option 3: results equivalence

When results equivalence is required, the hypothesis to be tested is that the alternative and compendial test methods give equivalent numerical results. This contrasts with the evaluation of the validation parameters, as is done in performance equivalence. Because the same sample cannot be tested in microbiology, typically a tolerance interval is established when comparing the two methods, with the alternative method determined to be numerically superior or non-inferior. Reports on the use of alternative non-growth-based methods have shown that they may produce significantly higher cell count estimates than a growth method that reports outcomes in cfu. In this case, the analyst could use a calibration curve showing a correlation between the two methods in the product specification range.

option 4: decision equivalence

A decision equivalence is similar to results equivalence but differs in that a numerical result is not generated; instead a pass/fail result is obtained. With this approach, the frequency of positive and negative results generated should be no worse than with the compendial method. This non-inferiority requirement is based on the long history of product quality tested by and released with the referee compendial test. For the purposes of qualification, laboratory studies involving spiking low levels of microorganisms may be considered. The following sections provide suggested approaches for demonstrating that the alternative procedure is equivalent to or better than the compendial procedure. Users may use other valid methods for demonstration of equivalence with supporting scientific justification.

Equivalence Demonstration for Alternative Qualitative Microbiological Procedures

Results obtained by procedures in

, and

are indicative of the presence or absence of microorganisms in the sample tested. These tests do provide a decision (i.e., the compendial article either passes or fails the test). This type of data fits in the decision equivalence category as described in the Stimuli article (2). Approach 1 (see below) is based on demonstrating decision equivalence. Approach 2 (see below) is an alternative that converts the qualitative results to quantitative ones by using the most probable number (MPN) procedure. Both approaches use a non-inferiority hypothesis (3).

To demonstrate the acceptability of the alternate procedure relative to the current microbiological procedure, the laboratory must demonstrate that the new procedure is as good as or better than the current procedure in terms of the ability to detect the presence of microorganisms. In general, a recommended approach for comparing the alternate procedure to the compendial procedure is to use a non-inferiority test (one-sided, as in non-inferiority tests conducted in clinical trials for new drug products) (4) rather than two-sided equivalence [as in bioequivalence (5)]. Non-inferiority is an appropriate approach for two reasons. First, from a patient perspective, it is beneficial to promote an alternative procedure that is potentially more sensitive than the referee procedure. In contrast, a two-sided approach penalizes better recovery of microorganisms. It is important to note that those implementing the alternate method will need to assess the risk associated with the change in procedure, because a more sensitive procedure may generate more positive results. Second, the alternate procedure has benefits (principally, reduced time to a result) that make it preferable to the compendial procedure, even if it is not as sensitive, as long as it allows for a quality decision on the product that is non-inferior to the compendial method.

approach 1: use presence and absence results

The non-inferiority hypothesis for this approach is that the proportion of samples that produce a signal for the new procedure (PN) is NMT some amount (D > 0) less than the proportion for the current, compendial procedure (PC) (6):

Result = PN

The D is the non-inferiority margin. Unless the laboratory requires a tighter margin, use D = 0.20 in the experiments described below. Calculate a one-sided 90% confidence interval for PN

PC (7). Non-inferiority is concluded if the lower confidence limit exceeds

0.20. If the experiment is able to conclude in favor of the non-inferiority hypothesis, then it can be stated, with 95% confidence, that PN

0.20 at the bioburden level studied.

This evaluation should be conducted using types of microorganisms selected by the laboratory as representative of the general types of microorganisms encountered. The choices can follow

or appropriate suitability test organisms, organisms recovered from product testing and/or microorganisms representative of those that may convey risk to patients given a product's route of administration.

The laboratory should conduct an evaluation to determine whether the alternate procedure can be shown to be non-inferior to the microbiological procedure in terms of sensitivity as measured by the proportion of samples returning a positive result for microbial recovery and growth. For each organism in a qualitative test, conduct three evaluations. The first uses samples prepared by serial dilution to be at or around 100, i.e., 1 cfu, where no growth is likely to be observed (hence no signal will be detected by the growth-based microbiological procedure) to characterize the sensitivity of the new procedure at this level. The second uses samples at or around 102 (100–200 cfu), where the microbiological procedure would be expected to detect growth at a relatively high percentage of about 75% or greater, to determine the acceptability of the new procedure. The third is a comparison of the two procedures at a burden where 50%–75% of samples would be expected to grow colonies [often a serial dilution to around 101 (10–50 cfu)] to test the non-inferiority hypothesis as described above. In the non-inferiority experiment for qualitative microbiology tests, a minimum of 75 samples should be tested on each procedure. Using 75 samples provides approximately 80% power. Should the laboratory find that a higher statistical power is necessary given the requirements of the analysis for a given product, increasing the number of samples analyzed to 100 will result in a power of approximately 90%. These procedures are appropriate for the purposes of concluding non-inferiority if the two procedures are actually equally sensitive using D = 0.20. If the laboratory concludes their new procedure is less sensitive than the compendial procedure, a larger number of samples will be required to maintain these power levels.

Independent samples: Suppose that NA samples have been tested with the candidate alternative procedure, of which XA samples are positive, and that NC samples (not the same as those tested with the candidate) have been tested with the compendial procedure and that XC samples are positive. Calculate the following:

where R is the ratio of variances at which to determine power.

Conclude non-inferiority if Z > z

where z

is the upper

percentage point of a standard normal distribution.

Paired samples: Suppose that N samples have been tested by both the candidate alternative and compendial procedures. The results can be displayed in a 2 × 2 table (see Table 3).

Table 3. Results for Paired Sample

Alternative Procedure	Compendial Procedure		Row Totals
Alternative Procedure	Positive	Negative	Row Totals
Positive	X11	X10	XA
Negative	X01	X00	N XA
Column totals	XC	N XC	N

Compute the following:

Conclude non-inferiority if Z > z

where z

is the upper

percentage point of a standard normal distribution.

approach 2: compare mpn results

For the compendial reference and the alternative procedures, conduct an MPN comparative study using standard procedures for MPN for each of the N samples. Ideally, the same samples are used for the two procedures, but this is not a necessity.

For Approach 2, the non-inferiority hypothesis is

µA

µC

log(R) or antilog(µA

µC)

where µA and µC are the means in the log scale for the alternative and compendial procedures, respectively.

Independent samples: Determine MPN for NA samples by the alternative procedure, convert all values to logs, and determine the sample mean of the log values (xA) and sample variance of the log values (S2A). Similarly, determine xC and S2C from the logs of NC samples tested with the compendial procedure. Determine the following:

where t

, df is the upper

percentage point of the t distribution with df degrees of freedom and

If using software that only allows for integer degrees of freedom (e.g., Excel), use linear interpolation to obtain the t value. Conclude non-inferiority if antilog(Llow)

Paired data: Determine MPN for N samples by the alternative procedure and for the same N samples by the compendial procedure, convert all 2N values to logs, and determine the sample mean (x) and variance (S2) of the differences of log alternative value minus log compendial value. Determine the following:

Llow = x

1S/

where t

1 is the upper

percentage point of the t distribution with N

1 degrees of freedom. Conclude non-inferiority if antilog(Llow)

Equivalence Demonstration for Alternative Quantitative Microbiological Procedures

A key characteristic of some alternative quantitative procedures is that their signal may differ significantly from the cfu of the compendial microbiological procedure. As a consequence, equivalence as it is typically understood cannot be shown; that is, the numerical results are expected to differ in magnitude and units. Instead, this chapter suggest two criteria for the verification of candidate alternative quantitative procedures:

Results from the candidate procedure have at least acceptable precision (repeatability).
The results from the candidate procedure are highly correlated with those from the compendial procedure. A high correlation is taken to indicate that quantitative acceptance criteria expressed in cfu can be calibrated to criteria in the units of the alternative procedure.

precision

Prepare a minimum of six samples at a minimum of two bioburden levels near specification limits relevant to the laboratory. Run the candidate alternative procedure for the prepared samples. [Note—This is to correspond to repeatability conditions; see

1225

] At each level, determine the sample variance (S2) of the logarithms (log10) of sample results. Calculate the following:

Where n is the number of samples (n

6) and

2.05,N

1 is the lower 5% value of a chi-square distribution with n

1 degrees of freedom. Precision is acceptable if *UL

, where

is the predetermined maximal acceptable repeatability percent geometric coefficient of variation, %GCV. (For small values, the %GCV will be approximately the %RSD.)

The greater the number of samples (n) the greater the likelihood (power, in statistical terms) that a procedure, the precision of which is actually acceptable, will yield data that meet this criterion and thus be declared acceptable. The laboratory may use prior data to determine a value of n that meets their needs.

Example: For the data for the alternative procedure in Table 4, calculate the following:

n = 10

S2 = 0.000241, and

= 3.325113, so

UL = 6.06%

This alternative procedure thus has acceptable repeatability precision as long as the prespecified success criterion

had been at least 6.06%.

correlation (linearity)

Prepare a minimum of two samples at each of four different bioburden levels covering the range from near limit of qualification (LOQ) to one log above the specification limit defined in the standard compendial assay to which the candidate alternative method is being compared. Determine the activity for all samples using both the candidate alternative and compendial procedures. Plot and determine the correlation between the log of values from the candidate alternative procedure (y) and the log of values from the compendial procedure (x). The correlation is acceptable if at least 0.95 (or R2 at least 0.9025).

Although a linear relationship between the two sets of results is typically expected, a nonlinear relationship can be acceptable. In the case of a nonlinear relationship, use the Spearman (nonparametric) correlation instead of the Pearson correlation.

Table 4

Compendial (cfu)	Alternative (cell count)
70	970
71	965
75	950
92	990
100	1000
105	1051
116	1046
123	1039
127	985
130	1020

Figure 1 shows the plot of these data after conversion to base 10 logs. Because R2 does not meet the stated requirement, the results from these two procedures are not sufficiently correlated.

Figure 1.

Candidate alternative procedures that may be suitable for making decisions about the microbiological quality of a sample (as in Figure 1) may not correlate well enough with the compendial procedure to meet the above correlation requirement. In that case, the other option is to apply the decision equivalence approach as described earlier for qualitative tests. During procedure development, the laboratory should determine a specification for the alternative method to correspond to the compendial specification for the required level of microbiological quality. For example, if the required level of microbiological quality is NMT 102 cfu, for which the compendial maximum acceptable count is 200 cfu, the laboratory will need to determine an acceptance criterion for the candidate alternative procedure that will match that value from the perspective of making a decision regarding microbial quality. Then, the validation experiment to confirm this choice proceeds as described earlier for qualitative tests.

GLOSSARY

Accuracy: Closeness of the test results obtained by the alternative test method to the value obtained by the compendial method, to be demonstrated across the dynamic (operational) range of the method.

Alternative microbiological method: A modern or rapid microbiological test procedure (MMM or RMM) that is different from the traditional growth-based method, such as the plate count or recovery in liquid broth. The alternative or rapid method may use different technologies, instrumentation and software to manage the testing and analyses of data and may provide quantitative (enumeration) or qualitative (detection) microbial test results.

Colony-forming unit (cfu): An estimate of the number of microorganisms obtained by traditional plate count methods. The enumeration is dependent on the ability of the microorganisms in the sample to grow on the microbiological culture media under the conditions of incubation. Because it is uncertain whether a colony was derived from the growth of one or even one thousand cells, the results are reported as cfu/mL (for a liquid) or cfu/g (for a solid) and not as cells/mL or cells/g.

Conventional microbiological method: A classical or traditional growth-based method, such as enumeration on an agar plate or detection in a liquid broth when incubated for a specified time and temperature. These methods are used in

, and

Equivalence: When the test results from two procedures are sufficiently close for the intended use of the procedures. Demonstration of equivalence requires a prespecified measure of how similar the test results need to be.

False negative: A test result that is incorrectly determined as negative (e.g., the absence of a viable microbial detection result when viable microorganisms are present). A type II error, also known as an error of the second kind, occurs when the null hypothesis is false but erroneously fails to be rejected. It is failing to assert what is, in fact, present—a miss. A type II error may be compared with a so-called false negative in a test (and seen as a “miss”) that is checking for a single condition with a definitive result of true or false. The rate of the type II error is denoted by the Greek letter

and is related to the power of a test (which equals 1

False positive: A test result that is incorrectly determined as positive (e.g., a viable microbial detection result when viable microorganisms are not present). In statistical test theory, the idea of a statistical error is an integral part of hypothesis testing. These are described as type I and type II errors. A type I error, also known as an error of the first kind, occurs when the null hypothesis (H0) is true but is rejected. It is asserting something exists that is, in fact, absent (i.e., a false hit). A type I error may be compared with a so-called false positive (a result that indicates that a given condition is present when it actually is not present) in tests where a single condition is tested. The rate of the type I error is called the “size” of the test and denoted by the Greek letter

. It usually equals the significance level of a test. In the case of a simple null hypothesis,

is the probability of a type I error.

Independent samples: Samples selected from the same population or different populations that have no effect on one another. That is, no correlation exists between the samples.

Limit of detection (LOD): The lowest concentration of microorganisms in a test sample that can be detected, but not necessarily quantified, under defined experimental conditions.

Limit of quantification (LOQ): The lowest number of microorganisms in a test sample that can be enumerated with acceptable accuracy and precision under defined experimental conditions.

Linearity: The ability to produce results that are proportional to the concentration of microorganisms present in the sample within a given range.

Method suitability: Demonstration of lack of enhancement or inhibition by the product on the signal generated by the method.

Non-inferiority: Demonstration that the alternate method is not worse than the compendial method by more than a small prespecified amount. This amount is known as the non-inferiority margin or

. Non-inferiority is different from equivalence in that in an equivalence trial, the desired conclusion is that two microbiological methods are not unacceptably different from each other. In a non-inferiority test the objective is to demonstrate that a new product is not unacceptably worse than an older one.

Paired samples: A sample of matched pairs of similar units.

Range-Dynamic or Operational: The interval between the upper and lower levels of microorganisms that have been demonstrated to be determined with specified accuracy, precision, and linearity.

Repeatability precision: The degree of agreement among individual test results when the procedure is applied repeatedly to multiple samplings of the same suspension of microorganisms and uses different suspensions across the range of the test. Also known as “repeatability”.

Robustness: A method's capacity to remain unaffected by small but deliberate variations in method parameters, such as, reagent volume, time or temperature of incubation providing an indication of its reliability during normal usage.

Ruggedness: Intermediate (within laboratory) precision associated with changes in operating conditions. Factors contributing to intermediate precision involve anything that can change within a given laboratory and that may affect the assay (e.g., different days, different analysts, different equipment).

Specificity: The ability to detect a range of microorganisms, which demonstrate that the method is fit for its intended use. These microorganisms may include a limited number of microorganisms representing risk to patient or product, microorganisms found in the manufacturing environment and product failures, microorganisms that are appropriate for measuring the effectiveness of the alternative method, and microorganisms that are specified in the relevant compendial tests, that are appropriate for the method and the product.

User Responsibility: The responsibility for the installation and operational, and performance qualification may be a joint responsibility of the instrument manufacturer and the user of the alternative microbiological method. The method validation may be conducted by the instrument manufacturer, when justified. The method suitability testing conducted for each specific product must be solely the user's responsibility.

Validation: The process of demonstrating and documenting that the performance characteristics of a procedure and its underlying method meet the requirements for the intended application and that the procedure is thereby suitable for its intended use. Formal validations are conducted prospectively according to a written plan that includes justifiable acceptance criteria on validation procedures.

REFERENCES

Staley, JT and Konopka, A. Measurement of in situ activities of non-photosynthetic microorganisms in aquatic and terrestrial habitats. Ann. Rev. Microbiol. 1985;39:321–346
Hauck WW, DeStefano AJ, Cecil TL, Abernethy DR, Koch WF, Williams RL. Acceptable, equivalent, or better: approaches for alternatives to official compendial procedures. Stimuli to the Revision Process. Pharmaceutical Forum. 2009;35(3).
ReChen JJ, Tsong Y, Kang S-H. Tests for equivalence or noninferiority between two proportions. Drug Info J. 2000;34:569–578.
FDA. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. E9 Statistical principles for clinical trials. Guidance for Industry. Sep 1998.Washington: U.S. Food and Drug Administration.
Schuirmann DJ. A comparison of the two one-sided tests procedure and the power approach for assessing the equivalence of average bioavailability. J Pharmacokinet Biopharm. 1987;15(6):657–680.
Farrington CP, Manning G. Test statistics and sample size formulae for comparative binomial trials with null hypothesis of non-zero risk difference or non-unity relative risk. Stat Med. 1990;9(12):1447–1454.
Lachenbruch PA, Lynch CJ. Assessing screening tests: extensions of McNemar's test. Stat Med. 1998;17(19):2207–2217.

2S (USP38)

Auxiliary Information— Please check for your question in the FAQs before contacting USP.

Topic/Question	Contact	Expert Committee
General Chapter	Radhakrishna S Tirumalai, Ph.D. Principal Scientific Liaison (301) 816-8339	(GCM2010) General Chapters - Microbiology

USP38–NF33 Page 1439

USP38–NF33 Supplement : No. 2 Page 7667

Pharmacopeial Forum: Volume No. 40(4)