1 BRIEFING 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 <1033> Biological Assay Validation. General Chapter Biological Assay Validation <1033> is a companion chapter to three other proposed USP chapters pertaining to bioassay: Design and Development of Biological Assays <1032>; Analysis of Biological Assays <1034>; and a “roadmap” chapter (as yet unnumbered) that will include a glossary applicable to <1032>, <1033> and <1034>. Taken together, these General Chapters update and expand on information provided in General Chapter <111>, which will continue to be available with modifications. A draft of <1033> previously appeared in Pharmacopeial Forum [2009;35(2):349–367]. This revision incorporates responses to comments received regarding the chapter, as well as further sections to describe bioassay validation. USP encourages input from all interested parties regarding <1033> and its companion chapters. USP’s intent is to reflect the best contemporary thought regarding bioassay validation. This will be achieved when the bioassay community takes advantage of this opportunity to engage in the chapter’s development by responding to the material in this In-Process Revision. Comments regarding this material should be sent to Tina S. Morris, PhD (
[email protected]). 17 18 <1033> BIOLOGICAL ASSAY VALIDATION 19 20 INTRODUCTION 21 22 23 24 25 26 27 28 Biological assays (also called bioassays) are an integral part of the quality assessment required for the manufacturing and marketing of many biological and some nonbiological drug products. Bioassays commonly used for drug potency estimation can be distinguished from chemical tests by their reliance on a biological substrate (e.g., animals, living cells, or functional complexes of target receptors). Because of multiple operational and biological factors arising from this reliance on biology, they typically exhibit a greater variability than do chemically-based tests. 29 30 31 32 33 34 35 36 37 38 39 40 Bioassays are one of several physicochemical and biologic tests with procedures and acceptance criteria that control critical quality attributes (measurands) of a biological drug product. As described in the ICH Guideline entitled Specifications: Test Procedures And Acceptance Criteria For Biotechnological /Biological Products (Q6B), section 2.1.2, bioassay techniques may measure an organism’s biological response to the product; a biochemical or physiological response at the cellular level; enzymatic reaction rates or biological responses induced by immunological interactions; or ligandand receptor-binding. As new biological drug products and new technologies emerge, the scope of bioassay approaches is likely to expand. Therefore, General Chapter Biological Assay Validation <1033> emphasizes validation approaches that provide flexibility to adopt new bioassay methods, new biological drug products, or both in conjunction for the assessment of drug potency. © 2010 The United States Pharmacopeial Convention All Rights Reserved 1 41 42 43 44 45 46 47 48 49 50 51 52 Good Manufacturing Practice requires that test methods used for assessing compliance of pharmaceutical products should meet appropriate standards for accuracy and reliability. Assay validation is the process of demonstrating and documenting that the performance characteristics of the procedure and its underlying method meet the requirements for the intended application and that the assay is thereby suitable for its intended use. USP General Chapter Validation of Compendial Procedures <1225> describes the assay performance characteristics that should be evaluated for procedures supporting small-molecule pharmaceuticals and is broadly based on their use for lot release, marketplace surveillance, and similar applications. Although application of these validation parameters is straightforward for many types of analytical procedures for well-characterized, chemically-based drug products, their interpretation and applicability for some types of bioassays has not been clearly delineated. 53 54 55 56 57 58 59 60 Assessment of bioassay performance is a continuous process, but bioassay validation should be performed when development has been completed and the bioassay standard operating procedure (SOP) has been approved. Bioassay validation is guided by a validation protocol describing the goals and design of the validation study. General Chapter <1033> provides validation goals pertaining to relative potency bioassays. Relative potency bioassays are based on a comparison of bioassay responses for a Test sample and a designated Standard, allowing a judgment regarding the activity of the Test relative to that of the Standard. 61 62 63 64 65 66 67 68 69 70 71 Validation parameters discussed include relative accuracy, specificity, intermediate precision, and range. Laboratories may use dilutional linearity to verify the relative accuracy and range of the method. Although robustness is not a requirement for validation, General Chapter <1033> recommends that a bioassay’s robustness be assessed either pre-validation or with additional experiments soon after the validation. In addition, <1033> describes approaches for validation design (sample selection and replication strategy), validation acceptance criteria, data analysis and interpretation, and finally bioassay performance monitoring through quality control. Documentation of bioassay validation results is also discussed, with reference to pre-validation experiments performed to optimize bioassay performance. In the remainder of General Chapter <1033> the term bioassay is used to indicate a relative potency bioassay. 72 73 FUNDAMENTALS OF BIOASSAY VALIDATION 74 75 76 77 78 79 80 81 82 83 84 The goal of validation for a bioassay is to confirm the operating characteristics of the procedure for its intended use. Multiple dilutions (concentrations) of one or more Test samples and the Standard sample are included in a single bioassay. These dilutions are herein termed a replicate set, which contains a single organism type, e.g., group of animals or vessel of cells, at each dilution for each sample [Test(s) and Standard]. A run consists of performing assay work during a period when the accuracy (trueness) and precision in the assay system can reasonably be expected to be stable. In practice, a run frequently consists of the work performed by a single analyst in one lab, with one set of equipment, in a short period of time (typically a day). An assay is the body of data used to assess similarity and estimate potency relative to Standard for each Test © 2010 The United States Pharmacopeial Convention All Rights Reserved 2 In some cases the Standard sample is assigned a value according to another property such as protein concentration. key factors in these studies such as incubation time and temperature and. more detailed discussion is found in <1032>.. A run may contain multiple assays. curves generated for Standard sample and Test article should be linear. each at 10 dilutions. instruments. 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 In order to establish the relative accuracy and range of the bioassay. Multiple assays may be combined to yield a reportable value for a sample. Determining what will be an assay is an important strategic decision. cell passage number and cell number may be included in the validation. The variability of potency from these combined elements defines the intermediate precision (IP) of the bioassay. Information about how to assess parallelism is provided in General Chapters <1032> and <1034>. and distinguished only by their slopes. In that case the potency of the Test sample is reported as specific activity. For sloperatio bioassay. in the nonedge wells of each of several 96-well cell culture plates) the groups (plates) constitute statistical blocks that should be elements in the assay and validation analyses (blocks are discussed in <1032>).g. © 2010 The United States Pharmacopeial Convention All Rights Reserved 3 . An appropriate study of the variability of potency.. four-parameter logistic) bioassay is that the dose–response curves that are generated using a Standard sample and a Test sample have similar (parallel) curve shape distinguished only by a horizontal shift in the log dose. 6 samples. can help the laboratory confirm an adequate testing strategy. the validation study should be directed toward obtaining a representative estimate of the variability of the relative potency determination.0 or 100%. Although robustness studies are usually performed during bioassay development. pass through a common intercept. thereby forecasting the inherent variability of the reportable value (which may be the average of multiple potency determinations). the design of the bioassay validation should include consideration of these factors. The reportable value is the value that is compared to a product specification. Within-block replicates for Test samples are rarely cost-effective. for cell-based bioassays. 97 98 99 100 101 102 103 104 105 106 107 108 109 110 The amount of activity (potency) of the Standard sample is typically assigned 1. Variability estimates can also be utilized to establish the sizes of differences (fold difference) that can be distinguished between samples tested in the bioassay. In addition. An assumption of parallelline or parallel-curve (e. which is the relative potency of the Test sample in reference to the potency of the Standard sample. This results in a unitless measure. a single assay. which is the relative potency times the assigned value of the Standard sample.g. or reagent sources. including the impact of intra-assay and inter-run factors. Because of potential influences within a single laboratory on the bioassay from inter-run factors such as multiple analysts. 91 92 93 94 95 96 In assays where the replication set involves groups at each dilution (e. and the potency of the Test sample is calculated by comparing the concentration–response curves for the Test sample and Standard sample pair. Blocks will not be further discussed in this chapter.85 86 87 88 89 90 sample in the assay. The factors involved in that decision are described in greater detail in General Chapter <1032> and are assumed to be resolved by the time a bioassay is in validation. or part of an assay. Test samples may be constructed for validation by a dilution series of the Standard sample to assess dilutional linearity (linearity of the relationship between determined and constructed relative potency). the study design. Depending on the extent of development of the bioassay. Results of intermediate measures should be reported when they add clarity and facilitate reproduction of validation conclusions (e. 134 135 Bioassay Validation Protocol 136 137 138 139 140 141 142 143 144 A bioassay validation protocol should include the number and types of samples that will be studied in the validation.. or instrument measurements such as OD) and the results of statistical calculations. assay (possibly run) and sample acceptance criteria such as system suitability and similarity should be specified before performing the validation. failure to find a statistically significant effect is not an appropriate basis for defining acceptable performance in a bioassay. 156 157 Documentation of Bioassay Validation Results 158 159 160 161 162 163 164 165 166 167 168 169 170 Bioassay validation results should be documented in a bioassay validation report. The conclusions from the study should be clearly described with references to follow-up action as necessary. the replication strategy. 145 146 147 148 149 150 151 In addition. Deviations from the validation protocol should be documented with justification. run. The report should include the raw data from the study (bioassay response measurements such as titers. and a proposed data-analysis plan.g. Other analytical methods may complement a bioassay in measuring or identifying other components in a sample. variance component estimates should be provided in addition to overall intermediate precision). or sample failures may be reassessed in an appropriate fashion and. with sound justification. conformance to acceptance criteria may be better evaluated using an equivalence approach. these may be proposed as tentative and can be updated with data from the validation.129 130 131 132 133 Demonstrating specificity requires evidence of lack of interference (also known as selectivity) from matrix components such as manufacturing process components or degradation products so that measurements describe the target molecule only. the intended validation parameters and justified target acceptance criteria for each parameter. Failure to meet a target acceptance criterion may result in a limit on the range of potencies that can be measured in the bioassay or a modification to the replication strategy in the bioassay procedure. Additional validation trials may be required to support changes to the method. included in the overall validation assessment. 152 153 154 155 The bioassay validation protocol should include target acceptance criteria for the proposed validation parameters. Assay. Estimates of validation parameters should be reported at each level and overall as appropriate. including inter-run and intra-run factors. Follow-up action can © 2010 The United States Pharmacopeial Convention All Rights Reserved 4 . Note that in regard to satisfying acceptance criteria. The validation report should support the conclusion that the method is fit for use or should indicate corrective action (such as change in bioassay format) that will be undertaken to generate reliable results. and inter-run factors. pH. by the bioassay design (number of animals. and 5 are recommended for a reliable assessment. Operating restrictions and bioassay design (intra. and by the statistical analysis (where the primary endpoints are the similarity assessment for each sample and potency estimates for the reference samples). and also may include qualification experiments. For relative accuracy. A limited range will result from levels that fail to meet their target acceptance criteria. and sample preparation should be performed independently during each validation run. A minimum of 3 potency levels is required.171 172 173 174 175 176 177 178 179 include amendment of system or sample suitability criteria or modification of the bioassay replication strategy. sample preparation. replicates per dilution. Representative estimation of bioassay variability necessitates consideration of these factors. Samples may also be generated for the bioassay validation by stressing a sample to a level that might be observed in routine practice (i. intra-run. Conclusions from prevalidation and qualification experiments performed during development contribute to the description of the operating characteristics of the bioassay procedure. The potency levels chosen will constitute the range of the bioassay if the criteria for relative accuracy and IP are met during the validation. 199 200 201 202 203 204 The bioassay validation design should consider all facets of the measurement process. etc). Sources of bioassay measurement variability include sampling. by the assay acceptance and sample acceptance criteria. Validation samples should be randomly selected from the sources from which test articles will be obtained during routine use. Thus samples that span a wide range of potencies might be studied for a drug or biological with a wide specification range or for a product that is inherently unstable. incubation times. dilution spacing.. stability investigations). where bioassay parameters have been identified and ranges have been established for significant parameters. sample relative potency levels that bracket the range of potencies that may be tested in the bioassay should be used. Intra-run variability may be affected by bioassay operating factors that are usually set during development (temperature. 180 181 Bioassay Validation Design 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 The bioassay validation should include samples that are representative of materials that will be tested in the bioassay and should effectively establish the performance characteristics of the procedure. Additionally.). the influences of the sample matrix (excipients. where the final procedure has been performed to confirm satisfactory performance in routine operation. Prevalidation experiments may include robustness experiments.and inter-run formulae that result in a reportable value for a test material) are usually specified during development and may become a part of the bioassay operating procedure. number of dilutions. Reference to prevalidation experiments may be included as part of the validation study report. IP is studied by © 2010 The United States Pharmacopeial Convention All Rights Reserved 5 .e. etc. or combination components) can be studied strategically by intentionally varying these together with the target analyte using a multifactorial approach. 205 206 207 208 209 210 211 212 213 214 The replication strategy used in the validation should reflect knowledge of the factors that might influence the measurement of potency. process constituents. but a narrower range can be used for a more durable product. perhaps using an experimental design that alters those factors that may have an impact on the performance of the procedure. specificity. In most cases. During the validation it is not necessary to employ the format required to achieve the reportable value for a Test sample. The relative bias at individual levels is calculated as follows: © 2010 The United States Pharmacopeial Convention All Rights Reserved 6 . Relative accuracy in bioassay refers to a unit slope between log measured relative potency vs. and range. Relative accuracy. 247 248 249 250 251 252 253 254 255 256 257 1. This type of study is often referred to as a dilutional linearity study. IP. 225 226 227 228 229 230 231 232 233 234 235 236 237 238 A thorough analysis of the validation data should include graphical and statistical summaries that address the validation parameters and their conformance to target acceptance criteria. Following are strategies for addressing bioassay validation parameters. log level when levels are known. These components can be used to verify or forecast the variability of the bioassay format. The results from a dilutional linearity study should be assessed using the estimated relative bias at individual levels and via a trend in relative bias across levels. log relative potency should be analyzed in order to satisfy the assumptions of the statistical methods (see Statistical Considerations. The most common approach to demonstrating relative accuracy for relative potency bioassays is by construction of target potencies by dilution of the standard material or a Test sample with known potency. 239 240 Validation Strategies for Bioassay Performance Characteristics 241 242 243 244 245 246 Parameters that should be verified in a bioassay are relative accuracy.g. as well as ensure a representative estimate of long-term variability. below).. The analysis should follow the specifics of the data-analysis plan outlined in the validation protocol. design of experiments (DOE) for treatment design] with nested or crossed design structure can reveal important sources of variability in the procedure. Other parameters discussed in Chapter <1225> and ICH Q2(R1) such as detection limit and quantitation limit have not been included because they are usually not relevant to a bioassay that reports relative potency. The assumption of normality can be investigated using statistical tests of normality across a suitably sized collection of historical results. A well-designed validation experiment that combines both intra-run and inter-run sources of variability provides estimates of independent components of the bioassay variability. Confidence intervals should be calculated for the validation parameters using methods described here and in Chapter Analytical Data— Interpretation and Treatment <1010>. Scale of Analysis. Alternative methods of analysis should be sought when the assumptions can be challenged. Those assumptions include normality of the distribution from which the data were sampled and homogeneity of variability across the range of results observed in the validation. These assumptions can be explored using graphical techniques such as box plots and probability plots. The relative accuracy of a relative potency bioassay is the relationship between measured relative potency and known relative potency. Experiments [e.215 216 217 218 219 220 221 222 223 224 independent runs of the procedure. can help the laboratory confirm an adequate testing strategy. The overall variability from measurements taken under a variety of normal test conditions within one laboratory defines the IP of the bioassay. below). the bioassay is specific against the compound. 280 281 282 283 284 285 286 287 288 289 IP measures the influence of factors that will vary over time after the bioassay is implemented. 264 265 266 267 268 269 270 271 272 2. the impact of each factor can first be explored graphically to establish important contributions to potency variability. etc. or reagent lots. key intra-run factors such as incubation time and temperature may be included in the validation using multifactor design of experiments (DOE). receipt of new reagent lots. thereby forecasting the inherent variability of the reportable value as well as the critical fold difference (see Use of Validation Results for Characterization. For these assessments both similarity and potency may be assessed using appropriate equivalence tests. the design of the bioassay validation should include evaluation of these factors. which should be held to a target acceptance criterion. below). Variance component analysis is best carried out using a statistical software package © 2010 The United States Pharmacopeial Convention All Rights Reserved 7 . instruments. IP is the ICH and USP term for what is also commonly referred to as interrun variability. For products or intermediates associated with complex matrices. Specificity. These influences are generally unavoidable and include factors like change in personnel (new analysts). the estimated relative bias at each level can be held to a prespecified target acceptance criterion that has been defined in the validation protocol (see Bioassay Validation Example. An appropriate study of the variability of the bioassay. If the curves are similar and the potency conforms to expectations of a Standard-to-Standard comparison. 296 297 298 Contributions of validation study factors to the overall IP of the bioassay can be determined by performing a variance component analysis on the validation results. including the impact of intra-run and inter-run factors. If there is no trend in relative bias across levels. Because of potential influences on the bioassay by factors such as analysts. Although robustness studies are normally conducted during bioassay development. Intermediate precision. 290 291 292 293 294 295 When the validation has been planned using multifactor DOE. 273 274 275 276 277 278 279 3. specificity (sometimes called selectivity) involves demonstrating lack of interference from matrix components or product-related components that can be expected to be present. such as further restrictions on intra-assay operating conditions or strategic qualification procedures on inter-run factors such as analysts. and reagent lots.258 ⎛ Measured Potency ⎞ Relative Bias = 100 ⋅ ⎜ − 1⎟ % ⎝ Target Potency ⎠ 259 260 261 262 263 The trend in bias is measured by the estimated slope of log measured potency vs log target potency. This can be assessed via parallel dilution of the Standard sample with and without a spike addition of the potentially interfering compound. The identification of important factors should lead to procedures that seek to control their effects. instruments. 301 302 A variance component analysis yields variance component estimates such as 2 2 . The range is normally derived from the dilutional linearity study and minimally should cover the product specification range for potency.299 300 that is capable of performing a mixed-model analysis with restricted maximum likelihood estimation (REML).or hypo-potent test articles into the bioassay range.g. These can be used to σˆ Intra and σˆ Inter estimate the IP of the bioassay. When there is an existing product specification. release testing and stability evaluation). For stability testing and to minimize having to dilute or concentrate hyper. there is value in validating the bioassay over a broader range. acceptance criteria can be justified on the basis of the risk that measurements may fall outside of the product specification. IP expressed as percent geometric standard deviation (%GSD) is given by using the natural log of the relative potency in the analysis (see Statistical Considerations. 321 322 Validation Target Acceptance Criteria 323 324 325 326 327 328 329 330 331 The validation target acceptance criteria should be chosen to minimize the risks inherent in making decisions from bioassay measurements and to be reasonable in terms of the capability of the art. Considerations from a process capability index (Cpk) can be used to inform bounds on the relative bias (RB) and the IP of the bioassay: Cpk = USL − LSL 2 2 2 6 ⋅ σPr oduct + RB + σRA © 2010 The United States Pharmacopeial Convention All Rights Reserved 8 . Range. corresponding to intra-run and inter-run variation. 313 314 315 316 317 318 319 320 4. The range of the bioassay is defined as the true or known potencies for which it has been demonstrated that the analytical procedure has a suitable level of relative accuracy and IP. Scale of Analysis): 303 304 305 306 307 308 309 310 ( Intermediate Precision = 100 ⋅ e 2 2 σˆ Inter +σˆ Intra ) −1 The variability of the reportable value from testing performed with n replicate sets in each of k runs (format variability) is equal to: ( Format Variability = 100 ⋅ e 2 2 σˆ Inter k +σˆ Intra n⋅k ) −1 311 312 This formulation can be used to determine a testing format suitable for various uses of the bioassay (e.. as well as the variability of the reportable value for different bioassay formats (format variability). the reason for the trend should be identified. These relate to the properties of bioassay measurements as well as the statistical tools that can be used to summarize and interpret bioassay validation results. If a trend is observed in any SPC chart. the modified bioassay should be revalidated or linked to the original bioassay by an adequately designed bridging study with acceptance criteria that use equivalence testing. The choice of a bound on Cpk is a business decision and is related to the risk of obtaining an out-of-specification (OOS) result. a more liberal restriction might be placed on bioassays.334 335 336 337 338 339 340 341 where USL and LSL are the upper and lower specification acceptance criteria. and σ 2Pr oduct and σ2RA are target product and release assay (with associated format) variances respectively. it is important to monitor its behavior over time. or the inclusion of a random selection of lots to estimate this characteristic as part of the validation. A sound justification for target acceptance criteria or use of characterization should be included in the validation protocol. although chemical assays and immunoassays are often capable of achieving near single digit percent geometric standard deviation (%GSD). However. a restriction on relative bias or IP can be formulated on the basis of the capability of the art of the bioassay methodology. Some laboratories require process capability corresponding to Cpk greater than or equal to 1. using the validation results to establish an assay format that is predicted to yield reliable product measurements. If the resolution requires a modification to the bioassay or if a serious modification of the bioassay has occurred for other reasons (for example. 365 366 Statistical Considerations 367 368 369 370 371 Several statistical considerations are associated with designing a bioassay validation and analyzing the data. The proportion of lots that are predicted to be OOS is a function of Cpk. For example. a major technology change). This corresponds to approximately a 1 in 10. The purpose of these charts is to identify at an early stage any shift or drift in the bioassay. RB is a bound on the degree of relative bias in the bioassay.3. 342 343 344 345 346 347 348 349 350 351 When specifications have yet to be established for a product. In this case the validation goal might be to characterize the method. This formulation requires prior knowledge regarding target product variability. such as animal potency bioassays. 332 333 352 353 Assay Maintenance 354 355 356 357 358 359 360 361 362 363 364 Once a bioassay has been validated it can be implemented. that operate with much larger %GSD. This is most easily accomplished by maintaining statistical process control (SPC) charts for suitable parameters of the Standard sample response curve and potency of assay QC samples.000 chance that a lot with potency at the center of the specification range will be OOS. 372 © 2010 The United States Pharmacopeial Convention All Rights Reserved 9 . slog. The log base e is used in the illustration.0 = 0.00 level (taken from an example that follows). Relative potency measurements are typically nearly log normally distributed.0 is derived as follows: 399 GM = 0. of log transformed relative potency measurements. An example with five levels would be 0. Intermediate levels are obtained as the geometric mean of two adjacent levels. GSD is calculated as the anti-log of the standard deviation. Thus. 1. if the natural log (log to the base e) is used to transform relative potency measurements. Predicted response should be reported as the geometric mean of individual relative potency measurements.71. 413 414 © 2010 The United States Pharmacopeial Convention All Rights Reserved 10 . the mid-level between 0. This chapter assumes that appropriate methods are already in place to reduce the raw bioassay response data to relative potency (as described in USP <1034>). Intervals that might be calculated from GSD will be consistent with intervals calculated from mean and standard deviation of log transformed data. Table 1 presents an example of the calculation of geometric mean (GM) and associated RB. analysis of potency after log transformation generates data that more closely fulfill both of these requirements. and variability expressed as %GSD. 0. where the standard deviation is proportional to the level of response.0. summary results are converted back to the bioassay scale utilizing base e.50 ⋅ 1. Thus for example. 393 394 395 396 397 398 As a consequence of the usual (for parallel-line assays) log transformation of relative potency measurements. the validation can be carried out using methods described in USP Validation of Analytical Methods <1225>. The base of the log transformation does not matter as long as a consistent base is maintained throughout the analysis. The scale of analysis of bioassay validation. Typically.4 and 2. Scale of analysis.50. The formula is given by: 405 GSD = antilog(slog)-1.0. with %GSD for a series of relative potency measurements performed on samples tested at the 1. 400 401 402 403 404 Likewise.373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 1. If it is determined that potency measurements are normally distributed. Log normally distributed measurements are skewed and are characterized by heterogeneity of variability.71. summary measures of the validation are influenced by the log-normal scale. for example. there are advantages (balance and representativeness) if the levels selected for the validation study are evenly spaced on the log scale. 1. but some of the procedures require homogeneity of variability in measurements across the potency range. must be considered in order to obtain reliable conclusions from the study. approximating a normal distribution.50 and 1. 389 390 391 392 The distribution of potency measurements should be assessed as part of bioassay development (as described in <1032>). The statistical methods outlined in this chapter require that the data be symmetric. 406 407 408 409 410 411 412 Variability is expressed as GSD rather than RSD of the log normal distribution in order to preserve continuity using the log transformation. where data are the relative potencies of samples in the validation study. 0953 0.0000 1.9287 1. This illustration utilizes the average of within-run replicates.1388 -0.8% 416 417 418 419 420 Here the GM of the relative potency measurements is calculated as the antilog of the average log relative potency measurements and then expressed as relative bias.89 ⋅ 0.0000 0.1489 1. the percent deviation from the target potency: GM = e Average = e0.0000 1.0065. CIln.0060 0.0755 GM = 1.0441 SD 0.8% 423 424 425 426 Note that the GSD calculated for this illustration is not equal to the IP determined in the bioassay validation example for the 100% level (9.0740 0.09462) © 2010 The United States Pharmacopeial Convention All Rights Reserved 11 . A confidence interval’s most common interpretation is as the likely range of the true value of the parameter.0060 1.415 Table 1.0451 RB = 4.0488 Average ln RP 0. 0.00 ⎠ ⎝ ⎠ 421 and the percent geometric standard deviation (%GSD) is calculated as: 422 %GSD = 100 ⋅ ( eSD − 1) % = 100 ⋅ ( e0. Estimates of bioassay validation parameters should be presented as a point estimate together with a confidence interval. see Table 4.0441 = 1.0476 0.1000 1.0000 0.0755 / 8 = ( −0.0451 ⎞ ⎛ GM ⎛ 1.2%). such as the GM or %GSD.1388 0. 427 428 429 430 431 432 433 2.0755 − 1) % = 7.0451 ⎞ RB = 100 ⋅ ⎜ − 1⎟ = 100 ⋅ ⎜ − 1⎟ % = 4. Illustration of calculations of GM and %GSD RP 1. but the IP in the validation example represents the variability of individual replicates.1489 0.51% %GSD = 7. as follows: 434 CIln = x ± t df ⋅ s / n = 0. The previous example determines a 90% confidence interval for average log relative potency. Reporting validation results using confidence intervals. A point estimate is the numerical value obtained for the parameter.0441 ± 1.51% Target ⎝ 1. 92%) ⎝ 1.. 100 ⋅ ⎜ − 1⎟ = ( −0. This is not the same as concluding that the parameter conforms to its target value.92% with 90% confidence. for the uncertainty in the estimated validation parameter. It should not be confused with the practice of performing a significance test.00 ⎠ ⎠ 437 438 439 440 The statistical constant (1. Additionally.05 (equivalent to a confidence interval that includes the target value for the parameter) indicates that there is insufficient evidence to conclude that the parameter is different from the target value.00 ⎝ 1. however. Bioassay validation results are compared to target acceptance criteria in order to demonstrate that the bioassay is fit for use. 463 464 © 2010 The United States Pharmacopeial Convention All Rights Reserved 12 .435 For percent relative bias this becomes: 436 ⎞ ⎞ ⎛ e −0. 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 A common practice is to apply acceptance criteria to the estimated validation parameter.65%. These scenarios are illustrated in Figure 1. a significance test may detect a small deviation from target that is practically insignificant. A confidence interval for IP or format variability can be formulated using methods for variance components. with degrees of freedom (df) equal to the number of measurements minus one (df = 8 – 1 = 7).89) is from a t-table. Product specifications should inform the process of setting validation acceptance criteria. 0% relative bias).09462 CIRB = 100 ⋅ ⎜ − 1⎟ . such as a t-test. 9.65% and 9.g. A significance test associated with a P value > 0. This does not account. 441 442 443 444 445 446 447 3. or the validation data may be too variable to discover a meaningful difference from target. Thus the true relative bias falls between –0. The process of establishing conformance of validation parameters to validation acceptance criteria should not be confused with establishing conformance of relative potency measurements to product specifications. The study design may have too few replicates. This is a standard statistical approach used to demonstrate conformance to expectation and is called an equivalence test. which seeks to establish a difference from some target value (e. A solution is to hold the confidence interval on the validation parameter to the acceptance criterion.0065 ⎛ e0. Assessing conformance to acceptance criteria. is nevertheless acceptable because the confidence interval falls within the target acceptance range 479 480 481 482 483 484 485 486 487 Using the 90% confidence interval calculated previously. 100*[(1. although the point estimate (the solid diamond) falls within the acceptance range. the converse.12) – 1]%. the confidence bound includes the target. although it is statistically significantly different from the target. another. Thus scenario c. signifying conformance to the acceptance criterion. which signifies that the true relative bias may be outside the acceptable range. and the dashed lines form the lower (LAL) and upper (UAL) acceptance limits. © 2010 The United States Pharmacopeial Convention All Rights Reserved 13 . 12%). In scenario b.0 level. Use of confidence intervals to establish that validation results conform to an acceptance criterion U AL 0 LAL a b c 467 468 469 470 471 472 473 474 475 476 477 478 The solid horizontal line represents the target value (perhaps 0% relative bias). The interval in scenario c also falls within the acceptance range butt excludes the target. we can establish whether the bioassay has acceptable relative bias at the 1. for example.465 466 Figure 1. involves risks. 9. In scenario a. 488 489 490 491 492 493 494 495 4.65%. the interval falls within the acceptance range. A consideration related to these risks is sample size. Risks in decision-making and number of validation runs.00 level compared to a target acceptance criterion of no greater than + 12%. Because the 90% confidence interval for percent relative bias (– 0. the interval extends outside the range. One risk is that the parameter does not meet its acceptance criterion although the property associated with that parameter is satisfactory. including the assessment of conformance of a validation parameter to its acceptance criteria. This is common practice and is the same as the two 1-sided test (TOST) approach used in pharmaceutical bioequivalence testing. Note that a 90% confidence interval is used in an equivalence test rather than a conventional 95% confidence interval. and thus one could conclude there is insufficient evidence to conclude a difference from target (the significance test approach). However. The application of statistical tests.12/1) – 1]%) = (– 11%. we conclude that there is acceptable relative bias at the 1.92%) falls within the interval (100*[(1/1. is that the parameter meets its acceptance criterion although the procedure is truly unsatisfactory. 0762 ≈ 8 runs 508 509 510 511 512 513 514 Note that this formulation of sample size assumes no intrinsic bias in the bioassay.11 log (i. If variances across levels can be pooled.g. The primary goal of the analysis is to estimate critical parameters rather than establish the significance of an effect. © 2010 The United States Pharmacopeial Convention All Rights Reserved 14 .89 + 2.df and tβ/2. This results in a greater sample size to offset the impact of the bias.11). and replicate).076 log. Note also that this calculation represents a recursive solution (because the degrees of freedom depends on n) requiring statistical software or an algorithm that employs iterative methodology. potency level). Specifically..df ) σˆ IP 2 α.36 ) n≥ 0.. if the acceptance criterion for relative bias is ± 0. The results of the analysis may be summarized in an analysis of variance (ANOVA) table or a table of variance component estimates. In the current example the sample size increases to 10 runs if one assumes an intrinsic bias equal to 2%. Many analyses associated with bioassay validation must account for multiple design factors such as fixed effects (e.g. statistical modeling can also determine the overall relative bias and IP by combining information across levels performed in the validation. run. θ = 0.. 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 5. the minimum number of runs needed to establish conformance to an acceptance criterion for relative bias is given by: (t n≥ 2 + t β 2. Similarly.496 497 498 499 500 The two types of risk can be simultaneously controlled by strategic design. the bioassay variability is σˆ IP = 0.df θ2 504 where tα. as well as random effects (e. mixed effects models can be used to obtain variance components for validation study factors and to combine results across validation study samples and levels. including consideration of the number of runs that will be conducted in the validation. A more conservative solution includes some nonzero bias in the determination of sample size.e. 501 502 503 507 (1.112 2 0. and θ is the acceptable deviation (target acceptance criterion). respectively. These compose a confidence interval that is compared to the acceptance criterion as described above. The modeling output provides parameter estimates together with their standard errors of estimates that can be utilized to establish conformance of a validation parameter to its acceptance criterion. α and β are the one-sided type I and type II errors.df are distributional points from a Student’s t-distribution. and α = β = 0.05. df is the degrees of freedom 2 associated with the study design (usually n – 1). Modeling validation results using mixed effects models. operator. Statistical models composed of both fixed and random effects are called mixed effects models and usually require sophisticated statistical software for analysis. Thus the average relative bias at each level is obtained as a portion of the analysis together with its associated variability. 505 506 For example. σˆ IP is a preliminary estimate of IP. Statistical design.534 535 536 537 538 539 540 6. Validation runs should be randomized whenever possible to mitigate the potential influences of run order or time. 543 Table 2: Example of a multifactor DOE with three factors Run 1 2 3 4 5 6 7 8 Analyst 1 1 1 1 2 2 2 2 Cell Prep 1 1 2 2 1 1 2 2 Reagent Lot 1 2 1 2 1 2 1 2 544 545 546 547 548 549 550 551 552 553 554 In this design each analyst performs the bioassay with both cell preparations and both reagent lots. plate nested within analyst). Unlike screening experiments. This is an example of a full factorial design because all combinations of the factors are performed in the validation study. and multiple reagent lots into the validation plan. multiple cell culture preparations. the validation design should incorporate as many factors at as many levels as possible in order to obtain a representative estimate of IP. 555 556 Figure 2 illustrates an example of a validation using nesting (replicate sets nested within plate. 557 558 © 2010 The United States Pharmacopeial Convention All Rights Reserved 15 . To reduce the number of runs in the study. fractional factorial designs may be employed when more than three factors have been identified. More than two levels of a factor should be employed in the design whenever possible. Such designs are useful to incorporate factors that are believed to influence the bioassay and that vary during long-term use of the procedure. such as multifactor DOE or nesting can be used to organize runs in a bioassay validation. Statistical design. These are also used to better characterize the sources of variability and to develop a strategic test plan that can be used to manage the variability of the bioassay. 541 542 Table 2 shows an example of a multifactor DOE that incorporates multiple analysts. Validation designs do not need to regard the resolution requirements of typical screening designs. For examplle. Rounding to too many or o too few significant s figures will result in nonrepresentative esstimates of precision. Rou und potenciies in the va alidation to an appropriate numbe er of significant figuress when an nalyzing the e results. a spe ecification acceptance a e criterion might m be 0.8 80 to 1. The bioasssay will be used to support a specification s n range of 0. com mponents off variabilitty can be estimated fro om the valid dation resu ults. Cpkk is calculate ed on the basis b of the variability of o a reporta able value using u three independe ent runs of the t bioassa ay (see disccussion of fo ormat varia ability. The nu umber of siignificant fig gures in a reported r ressult from a bioassayy is related to the latte er’s precisio on. f The e number of o significan nt figures should not n be confu used with th he number of decimal places—re eported valu ues equal to t 1.12 have the same number n (two o) of significcant figuress. abovve).71 0 to 1. a table e is derived showing g the projec cted rate of OOS resultts for variou us restrictio ons on RB and a IP. Likewise. a bioassay with %GSD betwe een 2% and 20% will su upport two significant figures. more ant figures should s be used u in that portion of the t analysiss.2 and 0. specificatio on acceptance criteria should s be stated s with the t approprriate numbe er of significcant figuress. These componentts of variability can be used u to iden ntify significcant source es of variability as well as to derivve a bioassa ay format th hat meets the t procedu ure’s require ements for precision. Signiificant figu ures. The la aboratory may m wish to include tarrget © 201 10 The United States S Pharma acopeial Conve ention All Rightss Reserved 16 .559 Fig gure 2: Exam mple of a nested desig gn using tw wo analysts 560 561 562 563 564 565 For both h of these ty ypes of dessign as welll as combin nations of th he two. This stan ndard of rounding g is appropriate for log g normally scaled s mea asurementss that have proportiona al rather th han additive e variability. if two fig gures are warrante ed for a parrticular bioa assay. p In general the laborato ory should initially round r poten ncies to two o significantt figures wh hen assesssing variability. In generral. Becausse the laboratory woulld like to ob btain a preccise estimatte of relativve accuracyy.4 for the prod duct. Using g the Cpk describe ed in the se ection on Va alidation Ta arget Accep ptance Crite eria. Note that rounding occurs at the e end of a series s of calculatiions when the t final me easurementt is reported d and used d for decisio on making such s as confo ormance to specificatio on acceptance criteria a. Product variability is assumed to be equ ual to 0 in th he calculations.2. 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 7. significa 586 587 A BIO OASSAY VA ALIDATION EXAMPL LE 588 589 590 591 592 593 594 An exam mple illustra ates the prin nciples described in th his chapter. The num mber of signifiicant figure es in a speccification should be con nsistent witth the numb ber of figure es supporte ed by the precision of the reporta able value. 1.5 0.4) − ln(0.4 20 20 0. The example dataset is presented in Table 4. acceptable performance (less than 1% chance of obtaining an OOS result due to bias and variability of the bioassay) can be expected if the IP is ≤ 8% and relative bias is ≤ 12%.54 10. 0.4 10 5 1.71–1.0. A run consists of a full titration curve of the Standard sample. 1.93 601 Pr ob(OOS) = 2 ⋅ NORMINV( −3 ⋅ 0.89 + 2.53%) 602 603 604 605 606 607 608 From Table 3. 611 612 613 614 615 616 617 618 619 620 621 Five levels of the target analyte are studied in the validation: 0.93) = 0.53 0.0004 598 599 600 The calculation is illustrated for IP equal to 8% and relative bias equal to 12% (n = 3 runs): Cpk = ln(1. manufactured by a similar process. Runs are generated by two trained analysts using two cell preparations. The laboratory should strive to design the validation with as many levels of each factor as possible in order to best model the long-term performance of the bioassay.08)2 ln(1.4 8 12 0.36 ) n≥ 2 ln(1. Cell preparations must be independent and must represent those elements of cell culture preparation that vary during long-term performance of the bioassay.12)2 ≈ 8 runs.50. Other factors may be considered and incorporated into the design using a fractional factorial layout. The sample size formula given in the section on Risks in decision making and number of runs can be used to derive the number of runs required to establish conformance to an acceptance criterion for relative bias equal to 12% (using σIP = 8%.0053 (0. An estimate of target product variability can be obtained from data from a product.93 0. This yields duplicate measurements of relative potency at each level in each validation run. together with two full titration curves of the Test sample. 622 © 2010 The United States Pharmacopeial Convention All Rights Reserved 17 .71–1. α = β = 0. 609 610 Thus 8 runs would be needed to have a 95% chance of passing the target acceptance criterion for relative bias if the true relative bias is zero.0.71. 597 Table 3: Cpk and probability of OOS for various restrictions on RB and IP LSL–USL IP (%) RB (%) Cpk Prob(OOS) (%) 0.4. In this example each analyst performs two runs using each cell preparation.12)2 = 0.08)2 3 + ln(1. and 2.71) 6 ⋅ ln(1.595 596 product variability. for example.54 0.05): (1.71–1. 51 0.50 0.0 1.0 2.62 0.51 0.4 1.5 1.51 0.0 1 .5 1.4 1.1 2. In particular.50 0. 2 .4 2. a properly prepared plot can reveal a failure in agreement of validation results with © 2010 The United States Pharmacopeial Convention All Rights Reserved 18 .52 0. and 2 runs Analyst/Cell Prep 1/1 1/1 1/2 1/2 2/1 2/1 2/2 2/2 Run 1 2 1 2 1 2 1 2 0.2 1.3 2.0 2.0 1.69 0. A plot of the validation results vs the sample levels.1 0.2 0.45 0.1 1.67 0.70 0.76 0. Figure 3.74 0.52 0.0 2.5 0 0 .71 0.7 1 0 .0 2.4 2 .74 1. 2 cell preparations.77 0.98 1.5 0 0 .0 1.2 2.4 2.88 1.1 1.0 0.1 2.57 0.52 0.0 Analyst 2 0 .3 1.4 1.1 1.50 0.71 0.9 2.0 1.3 1.54 0.54 0.70 0.7 1 1 .2 2.Table 4: Example of bioassay validation with 2 analysts.50 0.4 Analyst 1 1 .5 1.3 1.4 1.56 0.69 0.5 2.0 627 628 629 630 The plot is used to reveal irregularities in the experimental results.4 1.4 1.64 0.0 1 .0 1.4 2.5 1.0 1.6 1.4 1.68 0.2 2.1 1.2 2.92 1.5 1.45 0.71 0.1 1.3 Level 623 624 625 626 Note: Levels and potencies are rounded to 2 significant figures following the section on significant figures.75 0.5 1.71 0.82 0.1 2.4 1.55 0.53 0. 000818 653 © 2010 The United States Pharmacopeial Convention All Rights Reserved 19 .50) is presented in Table 5.006542 0. These steps are demonstrated using the example validation data.059004 0.5 level Sum of Mean Source df Squares Square Expected Mean Square Run 7 0. The analyst 1 and analyst 2 data are deliberately offset with respect to the expected potency to allow clear visualization and comparison of the data sets from each analyst. 652 Table 5. 636 637 638 639 640 641 642 643 644 645 646 A formal analysis of the validation data might be undertaken in the following steps: (1) an assessment of variability (IP) should precede an assessment of relative accuracy or specificity in order to establish conformance to the assumption that variances across sample levels can be pooled. along with some details of the calculations for illustrative purposes. Note that the calculations illustrated in the following sections are appropriate only with a balanced dataset. passing through the origin). depending on how well the data across levels can be pooled.. The example plot in Figure 3 includes the unit line (line with slope equal to 1. Imbalanced designs or datasets with missing relative potency measurements should be analyzed using a mixed model analysis with restricted maximum likelihood estimation (REML). and (2) relative accuracy is assessed either at separate levels or by a combined analysis. Variance component analysis performed on log relative potency measurements at the 0.631 632 633 634 635 validation levels.008429 Var(Error) + 2Var(Run) Error 8 0.065546 Variance Component Estimates Var(Run) 0.000818 Var(Error) Corrected Total 15 0.003806 Var(Error) 0. 647 648 Intermediate Precision 649 650 651 Data at each level can be analyzed using variance component analysis. An example of the calculation performed at a single level (0. as well as heterogeneity of variability across levels (see discussion of the log transformation in Statistical Considerations). The Expected Mean Square is the linear combination of variance components that generates the measured mean square for each source.50 0.000818 0.002590 0.7% 7.0033806 + 0.003579 0. The factor “Run” in this analysis represents the combined runs across the analyst by cell preparation combinations.0 Average Var(Run) 0.0% −1 The same analysis was performed at each level of the validation and is presented in Table 6.3% 9.008429 − 0.0% 7.000460 0. then solving the equation for Var(Run) as follows: MS(Run) = Var(Error) + 2 ⋅ Var(Run) 666 Var(Run) = = 667 668 MS(Run) − Var(Error) 2 0.003806 2 These variance component estimates are combined to establish the overall IP of the bioassay at 0. Typically a heuristic is used for this assessment.2% 6.50: ( = 100 ⋅ ( e IP = 100 ⋅ e 669 670 671 Var(Run) + Var(Error) 0. To start. the within-run component of variability is 662 Var(Error) = MS(Error) = 000818 663 664 665 The between-run component of variability.003490 0. The variance component estimates are derived by solving the equation "Expected Mean Square = Mean Square" for each component.6% 673 674 675 A combined analysis can be performed if the variance components are similar across levels.000818 ) − 1) = 7. the mean square for Error estimates Var(Error).000818 = 0.002960 0.002561 Overall 7.002859 Var(Error) 0. Var(Run).71 1.654 655 656 657 658 659 660 661 The top of the table represents a standard ANOVA analysis. is subsequently calculated by setting the mean square for Run to the mathematical expression for the expected mean square. Variance component estimates and overall variability for each validation level and the average Level Component 0.000604 0.6% 7.0 1.004234 0. 672 Table 6.4 2.003806 0. and cell preparation have not been included because of the small number of levels (2 levels) for each factor.004557 0. One might hold the ratio of the © 2010 The United States Pharmacopeial Convention All Rights Reserved 20 . 000460 = 8. 6.10.3772. 0. 1. 683 684 685 686 687 688 The analysis might proceed using statistical software that is capable of applying a mixed effects model to the validation results. 0.52 0. © 2010 The United States Pharmacopeial Convention All Rights Reserved 21 .29) 3.41 8 8 8 8 – 0.000604 = 7.54) (0. 0.65.38.03% 4. –0. Scale of Analysis. The upper confidence bound was not calculated at each level separately because of the limited data at an individual level relative to the overall study design.99. 1.7423.6% GSD.29% 0.3611 2.2% GSD.88) (– 3. 0.10. 3.3071) (– 0.7860 (– 0.00 8 0.35.63) a Analyses performed on averages of duplicates from each run.43 2.3.3422 0. and the within-run component. Had the ratio exceeded 10 and if this was due to excess variability in one or the other of the extremes in the levels tested.69.92) (– 2. 0. 7.0065.6608 – 0.50 0. as well as corresponding potency and relative bias. 700 701 Relative Accuracy 702 703 704 705 The analysis might proceed with an assessment of relative accuracy at each level.05) (5. A balanced design was employed for the example validation.7042.003806/0.49.6173) (– 0.3199. The literature contains methods for calculating confidence bounds for variance components. 9. Here the ratios associated with the between-run variance component.05 1.51% 1. as well as other random effects such as analyst and cell preparation (see Modeling Validation Results Using Mixed Effects Models in Statistical Considerations).5. Variance components can be determined for Analyst and Cell Preparation separately in order to characterize their contributions to the overall variability of the bioassay 689 690 691 692 693 In the example variance components can be averaged across levels to report the IP of the bioassay. 694 695 696 697 698 699 Because of the recommendation to report validation results with some measure of uncertainty. 2. the same replication strategy at each level). 0.04. Average Potency and Relative Bias at Individual Levels log Potency 707 708 Potency Relative Bias Level n a Average (90% CI) Average (90% CI) Average (90% CI) 0. 14.8297) 0. The upper bound on IP for the bioassay example is 12. Table 7 shows the average and 90% confidence interval of validation results in the log scale. meet the 10-fold criterion..676 677 678 679 680 681 682 maximum variance to the minimum variance to no greater than 10 (10 is used because of the limited number of runs performed in the validation). 0. b Calculation illustrated in Statistical Considerations.74) (0.4024) (0.73% (– 1. This method of combining estimates is exact only if a balanced design has been employed in the validation (i.50) (2.00b 1.10) (1. so the IP can be reported as 7.60) (– 0.77% 9. 706 Table 7. That analysis should account for any imbalance in the design.19 (0.71 1. that extreme would be eliminated from further analysis and the range would be limited to exclude that level.71 1.0946) (0.e. –0.42. a one-sided 95% upper confidence bound can be calculated for the IP of the bioassay.004557/0.0441 0. resulting perhaps in an erroneous conclusion. Trend analysis can be performed using a regression of log relative potency vs log level. cell preparation. 740 741 Range 742 © 2010 The United States Pharmacopeial Convention All Rights Reserved 22 . yielding 90% confidence bounds (equivalent to a two one-sided t-test) that fall within the acceptance region of – 11% to 12% relative bias.0 0 1 . cell preparation. The analysis also accommodates random effects such as analyst.e. This consistency is due in part to the lack of independence of bioassay results across levels. the analysis proceeds with an assessment of the relative accuracy at each level. The 90% confidence interval at 2.5 0 0 .7 1 1 .. and run in this case). indicating that the relative bias may exceed 12%.4 1 2 . That analysis accurately accounts for the validation study design. In addition there does not appear to be a trend in relative bias across levels. Introduction of an acceptance criterion on a trend in relative accuracy across the range can be considered and may be introduced during development of the bioassay validation protocol.709 710 711 712 713 714 The analysis has been performed on the average of the duplicates from each run (n = 8 runs) because duplicate measurements are correlated within a run by shared IP factors (analyst.4.0 0 717 718 Note: lower acceptance criterion equal to 100·[(1/1. 719 720 721 722 723 724 725 726 727 728 Figure 4 shows an average positive bias across sample levels (i. A plot of relative bias vs level can be used to examine patterns in the experimental results and to establish conformance to the target acceptance criterion for relative bias (12%). 735 736 737 738 739 A combined analysis can be performed using statistical software that is capable of applying a mixed effects model to the validation results.12) – 1] = – 11%. Plot of 90% confidence intervals for relative bias vs the acceptance criterion 1 2 % 0 % -1 1 % 0 . the average relative bias is positive at all levels). 715 716 Figure 4. 729 730 731 732 733 734 After establishing that there is no meaningful trend across levels. The bioassay has acceptable relative bias at levels from 0. The latter would indicate that a comparison of samples with different measured relative potency (such as stability samples) is biased.0 falls outside the acceptance region. and run (see Modeling Validation Results Using Mixed Effects Models in Statistical Considerations).50 to 1. 8% 2. Use of Validation Results for Bioassay Characterization When the study has been performed to estimate the characteristics of the bioassay (characterization). In this range.2% versus the target acceptance criterion ≤ 8. level 1.1% 2 6.002561/3 ) − 1 = 4.002561].3% This calculation can be expanded to include various combinations of runs and replicate sets within runs as shown in Table 8.6% 4.9% 4.7% 3 6.3% 4.3% 3. 769 Table 8.41. one may conclude that satisfactory IP was demonstrated across the range. Because of this and other results in Table 6. the range of the bioassay is 0.4% 770 © 2010 The United States Pharmacopeial Convention All Rights Reserved 23 . which may be due to the variability of the estimate that results from a small dataset.4% 2.0%).3% 4.6% 2. with n individual dilution series of the test preparation within a run is given by the following formula for format variability: ( Format Variability = 100 ⋅ e Var(Run)/k + Var(Error)/n⋅k ) −1 Thus using estimates of intra-run and inter-run variance components from Table 6 [Var(Run) = 0.002859/3 + 0.4% 3. the variance component estimates can also be used to predict the variability for different bioassay formats and thereby can determine a format that has a desired level of precision. the predicted variability of the reportable value (geometric mean of the relative potency results) is equal to: ( Format Variability = 100 ⋅ e 0.50 to 1.1% 3.743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 The conclusions from the assessment of IP and relative accuracy can be used to establish the bioassay’s range that demonstrates satisfactory performance. if the bioassay is performed in 3 independent runs with a single dilution series of the test within each run.7% 3.002859 and Var(Error) = 0.0 has a slightly higher than acceptable estimate of IP (9.5% 6 5. Based on the acceptance criterion for IP equal to 8 %GSD (see Table 6) and for relative bias equal to 12% (see Table 7).6% 5. Format variability for different combinations of number of runs (k) and number of replicate sets within run (n) Number of Runs (k) Reps (n) 1 2 3 6 1 7. The predicted variability for k independent runs. 0000 V a r(C e llP re p ) 0.0000 V a r(R u n (A n a ly s is * C e llP re p )) 0. a n d R u n Com ponent V a ria n c e E s t im a t e V a r(A n a ly s t ) 0. confidence bounds on the variance components used to derive IP can be utilized to establish the bioassay’s format variability. 775 776 777 778 Significant sources of variability must be incorporated into runs in order to effect variance reduction. 799 © 2010 The United States Pharmacopeial Convention All Rights Reserved 24 . multiple cell preparations should be utilized to moderate the inter-run variability of the bioassay. n replicate sets within each run.0021 V a r(E rro r) 0. Variance component estimates obtained from such an analysis are presented in Table 9. In addition. 785 786 787 788 789 790 Estimates of intra-run and inter-run variability can also be used to determine the sizes of differences (fold difference) that can be distinguished between samples tested in the bioassay.771 772 773 774 Clearly the most effective means of reducing the variability of the reportable value (the geometric mean potency across runs and replicate sets) is by independent runs of the bioassay procedure. 779 780 T a b le 9 : R E M L E s t im a t e s o f V a ria n c e C o m p o n e n t s A s s o c a t e d w it h A n a ly s t .0026 781 782 783 784 Because cell preparation is a significant component of the overall potency variability. z = 2): 791 792 793 794 795 796 797 798 Critical Fold Difference = e2⋅ Var(Run)/k + Var(Error)/(n⋅k) When samples have been tested in different runs of the bioassay (such as long-term stability samples). The critical fold difference between reportable values for two samples that are tested in the same runs of the bioassay is given by (for k runs.0014 V a r(A n a ly s t * C e llP re p ) 0. using an approximate 2-sided critical value from the standard normal distribution. C e ll P re p a ra t io n . A more thorough analysis of the bioassay validation example would include analyst and cell preparation as factors in the statistical model. the critical fold difference is given by (assuming the same format is used to test the two series of samples): Critical Fold Difference = e 2⋅ 2⋅[ Var(Run)/k + Var(Error)/(n⋅k)] For comparison of samples the laboratory can choose a design (bioassay format) that has suitable precision to detect a practically meaningful fold difference between samples. Stat Sci. The revalidation may consist of a complete re-enactment of the bioassay validation or a bridging study that compares the current and the modified methods. 816 817 LITERATURE 818 819 Additional information and alternative methods can be found in the references listed below..e. 820 821 822 ASTM. Statistical Principles in Experimental Design. After the laboratory gains suitable experience with the bioassay.800 Confirmation of Intermediate Precision and Revalidation 801 802 803 804 805 806 807 808 809 810 811 The estimate of IP from the validation is highly uncertain because of the small number of runs performed. 2008. 2003. 829 830 Schofield TL. intersection-union tests and equivalence confidence intervals.11(4):283–319. In: Chow SC. Hsu J. New York: Marcel Dekker. 827 828 Haaland P. different key reagent lots. 2nd ed. and different cell preparations) have taken place. 2nd ed. ASTM E29-08. 1989:64– 66. In: Chow SC. Experimental Design in Biotechnology. New York: Marcel Dekker. Assay validation. ed. Standard Practice for Using Significant Digits in Test Data to Determine Conformance with Specifications. Encyclopedia of Biopharmaceutical Statistics. 2nd ed. ed. 825 826 Burdick R. Conshohocken. © 2010 The United States Pharmacopeial Convention All Rights Reserved 25 . The variability of a positive control can be used when the control is prepared and tested as is a Test sample (i. 823 824 Berger R. Assay development. PA: ASTM. 1996. 812 813 814 815 The bioassay should be revalidated whenever a substantial change is made to the method. different analysts. This includes but is not limited to a change in technology or a change in readout. New York: Marcel Dekker.. 831 832 Schofield TL. 1992:28–39. The reported IP of the bioassay should be modified as an amendment to the validation report if the assessment reveals a substantial disparity of results. 2003. Graybill F. New York: McGraw-Hill. New York: Marcel Dekker. Bioequivalence trials. same or similar dilution series and replication strategy). the estimate can be confirmed or updated by analysis of control sample measurements. Confidence Intervals on Variance Components.g. Encyclopedia of Biopharmaceutical Statistics. The assessment should be made after sufficient runs have been performed and after routine changes in bioassay factors (e. 1971:244–251. 833 834 Winer B.