A bioassay validation protocol should include the number and types of samples that will be studied in the validation; the study design, including inter-run and intra-run factors; the replication strategy; the intended validation parameters and justified target acceptance criteria for each parameter; and a proposed data-analysis plan. Note that in regard to satisfying acceptance criteria, failure to find a statistically significant effect is not an appropriate basis for defining acceptable performance in a bioassay; conformance to acceptance criteria may be better evaluated using an equivalence approach.

In addition, assay, run, and sample acceptance criteria such as system suitability and similarity should be specified before performing the validation. Depending on the extent of development of the bioassay, these may be proposed as tentative and can be updated with data from the validation. Assay, run, or sample failures may be reassessed according to criteria which have been defined in the validation protocol and, with sound justification, included in the overall validation assessment. Additional validation trials may be required in order to support changes to the method.

The bioassay validation protocol should include target acceptance criteria for the proposed validation parameters. Steps to be taken upon failure to meet a target acceptance criterion should be specified in the validation protocol, and may result in a limit on the range of potencies that can be measured in the bioassay or a modification to the replication strategy in the bioassay procedure.

Bioassay validation results should be documented in a bioassay validation report. The validation report should support the conclusion that the method is fit for use or should indicate corrective action (such as an increase in the replication strategy) that will be undertaken to generate sufficiently reliable results to achieve fitness for use. The report could include the raw data and intermediate results (e.g., variance component estimates should be provided in addition to overall intermediate precision) which would facilitate reproduction of the bioassay validation analysis by an independent reviewer. Estimates of validation parameters should be reported at each level and overall as appropriate. Deviations from the validation protocol should be documented with justification. The conclusions from the study should be clearly described with references to follow-up action as necessary. Follow-up action can include amendment of system or sample suitability criteria or modification of the bioassay replication strategy. Reference to prevalidation experiments may be included as part of the validation study report. Prevalidation experiments may include robustness experiments, where bioassay parameters have been identified and ranges have been established for significant parameters, and also may include qualification experiments, where the final procedure has been performed to confirm satisfactory performance in routine operation. Conclusions from prevalidation and qualification experiments performed during development contribute to the description of the operating characteristics of the bioassay procedure.

The biological assay validation should include samples that are representative of materials that will be tested in the bioassay and should effectively establish the performance characteristics of the procedure. For relative accuracy, sample relative potency levels that bracket the range of potencies that may be tested in the bioassay should be used. Thus samples that span a wide range of potencies might be studied for a drug or biological with a wide specification range or for a product that is inherently unstable, but a narrower range can be used for a more durable product. A minimum of three potency levels is required, and five are recommended for a reliable assessment. If the validation criteria for relative accuracy and IP are satisfied, the potency levels chosen will constitute the range of the bioassay. A limited range will result from levels that fail to meet their target acceptance criteria. Samples may also be generated for the bioassay validation by stressing a sample to a level that might be observed in routine practice (i.e., stability investigations). Additionally, the influences of the sample matrix (excipients, process constituents, or combination components) can be studied strategically by intentionally varying these together with the target analyte, using a multifactorial approach. Often this will have been done during development, prior to generating release and stability data.

The bioassay validation design should consider all facets of the measurement process. Sources of bioassay measurement variability include sample preparation, intra-run factors, and inter-run factors. Representative estimation of bioassay variability necessitates consideration of these factors. Test sample and Standard preparation should be performed independently during each validation run.

The replication strategy used in the validation should reflect knowledge of the factors that might influence the measurement of potency. Intra-run variability may be affected by bioassay operating factors that are usually set during development (temperature, pH, incubation times, etc.); by the bioassay design (number of animals, number of dilutions, replicates per dilution, dilution spacing, etc.); by the assay acceptance and sample acceptance criteria; and by the statistical analysis (where the primary endpoints are the similarity assessment for each sample and potency estimates for the reference samples). Operating restrictions and bioassay design (intra- and inter-run formulae that result in a reportable value for a test material) are usually specified during development and may become a part of the bioassay operating procedure. IP is studied by independent runs of the procedure, perhaps using an experimental design that alters those factors that may have an impact on the performance of the procedure. Experiments (including those that implement formalized design of experiments [DOE]) with nested or crossed design structure can reveal important sources of variability in the procedure, as well as ensure a representative estimate of long-term variability. During the validation it is not necessary to employ the format required to achieve the reportable value for a Test sample. A well-designed validation experiment that combines both intra-run and inter-run sources of variability provides estimates of independent components of the bioassay variability. These components can be used to verify or forecast the variability of the bioassay format.

A thorough analysis of the validation data should include graphical and statistical summaries of the validation parameters’ results and their conformance to target acceptance criteria. The analysis should follow the specifics of the data-analysis plan outlined in the validation protocol. In most cases, log relative potency should be analyzed in order to satisfy the assumptions of the statistical methods (see section 2.7 Statistical Considerations, Scale of Analysis,). Those assumptions include normality of the distribution from which the data were sampled and homogeneity of variability across the range of results observed in the validation. These assumptions can be explored using graphical techniques such as box plots and probability plots. The assumption of normality can be investigated using statistical tests of normality across a suitably sized collection of historical results. Alternative methods of analysis should be sought when the assumptions can be challenged. Confidence intervals should be calculated for the validation parameters, using methods described here and in general chapter Analytical Data—Interpretation and Treatment 1010.

Parameters that should be verified in a bioassay are relative accuracy, specificity, IP (which incorporates repeatability), and range. Other parameters discussed in general chapter 1225 and ICH Q2(R1) such as detection limit and quantitation limit have not been included because they are usually not relevant to a bioassay that reports relative potency. These may be relevant, however, to the validation of an ancillary assay such as one used to score responders or measure response in conjunction with an in vivo potency assay. Likewise linearity is not part of bioassay validation, except as it relates to relative accuracy (dilutional linearity). There follow strategies for addressing bioassay validation parameters.

The validation target acceptance criteria should be chosen to minimize the risks inherent in making decisions from bioassay measurements and to be reasonable in terms of the capability of the art. When there is an existing product specification, acceptance criteria can be justified on the basis of the risk that measurements may fall outside of the product specification. Considerations from a process capability (Cp) index can be used to inform bounds on the relative bias (RB) and the IP of the bioassay. This chapter uses the following Cpm index:

where USL and LSL are the upper and lower release specification, RB is a bound on the degree of relative bias in the bioassay, and

and

are target product variance (i.e., lot-to-lot variability) and release assay variance (with associated format) respectively. (See section 3 A Bioassay Validation Example for an example of determination of

and Cpm.) This formulation requires prior knowledge regarding target product variability, or the inclusion of a random selection of lots to estimate this characteristic as part of the validation. Given limited understanding of assay performance, manufacturing history, and final specifications during development, this approach may be used simply as a guide for defining validation acceptance criteria.

The choice of a bound on Cpm is a business decision. The proportion of lots that are predicted to be outside their specification limits is a function of Cpm. Some laboratories require process capability corresponding to Cpm greater than or equal to 1.3. This corresponds to approximately a 1 in 10,000 chance that a lot with potency at the center of the specification range will be outside the specification limits.

When specifications have yet to be established for a product, a restriction on relative bias or IP can be formulated on the basis of the capability of the art of the bioassay methodology. For example, although chemical assays and immunoassays are often capable of achieving near single digit percent coefficient of variation (%CV, or percent relative standard deviation, %RSD), a more liberal restriction might be placed on bioassays, such as animal potency bioassays, that operate with much larger variability (measured as %GCV which can be compared to %CV; see Appendix). In this case the validation goal might be to characterize the method, using the validation results to establish an assay format that is predicted to yield reliable product measurements. A sound justification for target acceptance criteria or use of characterization should be included in the validation protocol.

Once a bioassay has been validated it can be implemented. However, it is important to monitor its behavior over time. This is most easily accomplished by maintaining statistical process control (SPC) charts for suitable parameters of the Standard response curve and potency of assay QC samples. The purpose of these charts is to identify at an early stage any shift or drift in the bioassay. If a trend is observed in any SPC chart, the reason for the trend should be identified. If the resolution requires a modification to the bioassay or if a serious modification of the bioassay has occurred for other reasons (for example, a major technology change), the modified bioassay should be revalidated or linked to the original bioassay by an adequately designed bridging study with acceptance criteria that use equivalence testing.

Several statistical considerations are associated with designing a bioassay validation and analyzing the data. These relate to the properties of bioassay measurements as well as the statistical tools that can be used to summarize and interpret bioassay validation results.

Data at each level can be analyzed using variance component analysis. With balanced data, as in this example, variance components can be determined from a standard one-way ANOVA. An example of the calculation performed at a single level (0.50) is presented in Table 5.

The top of the table represents a standard ANOVA analysis. Analyst and media lot have not been included because of the small number of levels (2 levels) for each factor. The factor “Run” in this analysis represents the combined runs across the analyst by media lot combinations. The Expected Mean Square is the linear combination of variance components that generates the measured mean square for each source. The variance component estimates are derived by solving the equation “Expected Mean Square = Mean Square” for each component. To start, the mean square for Error estimates Var(Error), the within-run component of variability, is

The between-run component of variability, Var(Run), is subsequently calculated by setting the mean square for Run to the mathematical expression for the expected mean square, then solving the equation for Var(Run) as follows:

These variance component estimates are combined to establish the overall IP of the bioassay at 0.50:

The same analysis was performed at each level of the validation, and is presented in Table 6.

A combined analysis can be performed if the variance components are similar across levels. Typically a heuristic method is used for this assessment. One might hold the ratio of the maximum variance to the minimum variance to no greater than 10 (10 is used because of the limited number of runs performed in the validation). Here the ratios associated with the between-run variance component, 0.003639/0.000648 = 5.6, and the within-run component, 0.004303/0.000577 = 7.5, meet the 10-fold criterion. Had the ratio exceeded 10 and if this was due to excess variability in one or the other of the extremes in the levels tested, that extreme would be eliminated from further analysis and the range would be limited to exclude that level.

The analysis might proceed using statistical software that is capable of applying a mixed effects model to the validation results. That analysis should account for any imbalance in the design, random effects such as analyst and media lot, and fixed effects such as level (see section 2.7 Statistical Considerations, Modeling Validation Results Using Mixed Effects Models). Variance components can be determined for analyst and media lot separately in order to characterize their contributions to the overall variability of the bioassay.

In the example, variance components can be averaged across levels to report the IP of the bioassay. This method of combining estimates is exact only if a balanced design has been employed in the validation (i.e., the same replication strategy at each level). A balanced design was employed for the example validation, so the IP can be reported as 7.2% GCV.

Because of the recommendation to report validation results with some measure of uncertainty, a one-sided 95% upper confidence bound can be calculated for the IP of the bioassay. The literature contains methods for calculating confidence bounds for variance components. The upper bound on IP for the bioassay example is 11.8% GCV. The upper confidence bound was not calculated at each level separately because of the limited data at an individual level relative to the overall study design.

The analysis might proceed with an assessment of relative accuracy at each level. Table 7 shows the average and 90% confidence interval of validation results in the log scale, as well as corresponding potency and relative bias.

The analysis has been performed on the average of the duplicates from each run (n = 8 runs) because duplicate measurements are correlated within a run by shared IP factors (analyst, media lot, and run in this case). A plot of relative bias versus level can be used to examine patterns in the experimental results and to establish conformance to the target acceptance criterion for relative bias (12%).

Figure 4 shows an average positive bias across sample levels (i.e., the average relative bias is positive at all levels). This consistency is due in part to the lack of independence of bioassay results across levels. In addition there does not appear to be a trend in relative bias across levels. The latter would indicate that a comparison of samples with different measured relative potency (such as stability samples) is biased, resulting perhaps in an erroneous conclusion. Trend analysis can be performed using a regression of log relative potency versus log level. Introduction during the development of the bioassay validation protocol of an acceptance criterion on a trend in relative accuracy across the range can be considered.

After establishing that there is no meaningful trend across levels, the analysis proceeds with an assessment of the relative accuracy at each level. The bioassay has acceptable relative bias at levels from 0.50 to 1.41, yielding 90% confidence bounds (equivalent to a two one-sided t-test) that fall within the acceptance region of 11% to 12% relative bias. The 90% confidence interval at 2.0 falls outside the acceptance region, indicating that the relative bias may exceed 12%.

A combined analysis can be performed utilizing statistical software that is capable of applying a mixed effects model to the validation results. That analysis accurately accounts for the validation study design. The analysis also accommodates random effects such as analyst, media lot, and run (see section 2.7 Statistical Considerations, Modeling Validation Results Using Mixed Effects Models).

The conclusions derived from the assessment of IP and relative accuracy can be used to establish the bioassay’s range that demonstrates satisfactory performance. Based on the acceptance criterion for IP equal to 8% GCV (see Table 6) and for relative bias equal to 12% (see Table 7), the range of the bioassay is 0.50 to 1.41. In this range, level 1.0 has a slightly higher than acceptable estimate of IP (8.5% versus the target acceptance criterion 8.0%), which may be due to the variability of the estimate that results from a small dataset. Because of this and other results in Table 6, one may conclude that satisfactory IP was demonstrated across the range.

When the study has been performed to estimate the characteristics of the bioassay (characterization), the variance component estimates can also be used to predict the variability for different bioassay formats and thereby can determine a format that has a desired level of precision. The predicted variability for k independent runs, with n individual dilution series of the test preparation within a run, is given by the following formula for format variability:

Using estimates of intra-run and inter-run variance components from Table 6 [Var(Run) = 0.002723 and Var(Error) = 0.002172], if the bioassay is performed in three independent runs, the predicted variability of the reportable value (geometric mean of the relative potency results) is equal to:

This calculation can be expanded to include various combinations of runs and minimal sets (assuming that the numbers of samples, dilutions, and replicates in the minimal sets are held constant) within runs as shown in Table 8.

Clearly the most effective means of reducing the variability of the reportable value (the geometric mean potency across runs and minimal sets) is by independent runs of the bioassay procedure. In addition, confidence bounds on the variance components used to derive IP can be utilized to establish the bioassay’s format variability.

Significant sources of variability must be incorporated into runs in order to effect variance reduction. A more thorough analysis of the bioassay validation example would include analyst and media lot as factors in the statistical model. Variance component estimates obtained from such an analysis are presented in Table 9.

Identification of analyst as a significant bioassay factor should ideally be addressed during bioassay development. Nonetheless the laboratory may choose to address the apparent contribution of analyst-to-analyst variability through improved training or by using multiple analysts in formatting the assay for routine performance of the bioassay.

Estimates of intra-run and inter-run variability can also be used to determine the sizes of differences (fold difference) that can be distinguished between samples tested in the bioassay. For k runs, with n minimal sets within each run, using an approximate two-sided critical value from the standard normal distribution with z = 2, the critical fold difference between reportable values for two samples that are tested in the same runs of the bioassay is given by:

When samples have been tested in different runs of the bioassay (such as long-term stability samples), the critical fold difference is given by (assuming the same format is used to test the two series of samples):

For comparison of samples the laboratory can choose a design (bioassay format) that has suitable precision to detect a practically meaningful fold difference between samples.

The estimate of IP from the validation is highly uncertain because of the small number of runs performed. After the laboratory gains suitable experience with the bioassay, the estimate can be confirmed or updated by analysis of control sample measurements such as the variability of a positive control. This analysis can be done with the control prepared and tested like a Test sample (i.e., same or similar dilution series and replication strategy). This assessment should be made after sufficient assays have been performed to obtain an alternative estimate of the bioassay’s intermediate precision, including implementation of changes (e.g., different analysts, different key reagent lots, and different cell preparations) associated with the standardized assay protocol. The reported IP of the bioassay should be modified as an amendment to the validation report if the assessment reveals a substantial disparity of results.

The bioassay should be revalidated whenever a substantial change is made to the method. This includes but is not limited to a change in technology or a change in readout. The revalidation may consist of a complete re-enactment of the bioassay validation or a bridging study that compares the current and the modified methods.