Warner-lambert Company, Petitioner, v. Margaret M. Heckler, Secretary of Health and Human Services,and Frank E. Young, Commissioner of Food Anddrugs, Respondents.william H. Rorer, Inc., and Wallace Laboratories, Divisionof Carter-wallace, Inc. and Armour Pharmaceuticalcompany, Petitioners, v. United States Food and Drug Administration, Respondent, 787 F.2d 147 (3d Cir. 1986)

Several novel issues regarding the required showing of efficacy of new drugs are presented in this appeal by four drug manufacturers from the Order of the Commissioner of the Food and Drug Administration (FDA) withdrawing approval for the oral proteolytic enzymes (OPEs) that they manufacture. The Commissioner withdrew approval of the new drug applications for these drugs after concluding that there was a lack of substantial evidence that the OPEs will have the effects they are purported or represented to have for their intended conditions of use. Petitioners challenge the scope of the Commissioner's authority, the evidentiary rulings made in the proceedings, the Commissioner's evaluation of the evidence, and the conclusion reached. We have jurisdiction pursuant to 21 U.S.C. § 355(h).

Factual Background

The petitioners, who manufacture and market oral proteolytic enzymes, are Warner-Lambert (Warner), who produces the OPE Papase, and Armour Pharmaceutical Company, William H. Rorer, Inc., and Wallace Laboratories (collectively ARW), who presented their case jointly before the FDA and before this court, and who produce, respectively, the OPEs Chymoral, Ananase, and Avazyme. A fifth OPE that was also before the Commissioner is not before us on this appeal.¹ 

OPEs are prescription medications that the manufacturers claim are effective for use as adjunctive relief in alleviating swelling and inflammation associated with accidental trauma, surgical, obstetrical, and dental procedures, and infections and allergic reactions. Some of the drug companies claim that their OPEs will ease pain and accelerate tissue repair. See Joint Appendix (JA) at 17.

The manufacturers obtained FDA approval for the OPEs prior to 1962. At the time approval was granted, the Food, Drug, and Cosmetic Act required the FDA to determine only that a drug was safe for human use. In 1962, Congress amended the Act to require drug manufacturers to prove to the FDA that drugs they wished to market were effective as well as safe. Drug Amendments of 1962, Pub. L. No. 87-781, 76 Stat. 780 (codified in scattered sections of 21 U.S.C.). The 1962 amendments also required the FDA to reevaluate drugs that it had previously approved.² 

The Supreme Court opinion in Weinberger v. Hynson, Westcott & Dunning, Inc., 412 U.S. 609, 93 S. Ct. 2469, 37 L. Ed. 2d 207 (1973), reviews the legislative history leading to the 1962 amendments, the major provisions enacted, and the subsequent administrative actions to enforce the new statutory requirements. As the Court pointed out, substantial numbers of drugs were being marketed for which efficacy could not be shown. The panels evaluating 16,500 claims made on behalf of 4,000 drugs found that seventy percent of the claims were not supported by substantial evidence of effectiveness. Id. at 621. Only 434 drugs were found effective for all of their clinical uses. Id.

Pursuant to the statutory mandate, the FDA requested the OPE manufacturers to submit documentation showing the effectiveness of their drugs. Manufacturers of nine OPEs submitted data for evaluation. Based on these initial submissions, the Commissioner of the FDA found in 1970 that the OPEs were "possibly effective" for "diagnostic gastric lavage" and for relieving symptoms associated with the obstetrical procedure episiotomy, and that the OPEs lacked substantial evidence of effectiveness for all other claims. See 35 Fed.Reg. 10,393-94 (1970).³ 

In the same notice, the OPE manufacturers were required to delete from their labeling the indications for which the drug has been classified as lacking substantial evidence. Id. at 10,394. They were also required to provide additional data which would constitute substantial evidence of the effectiveness of OPEs for those indications for which the drugs had been classified as "possibly effective." See id. Five years later, the FDA published an extensive review of the additional data submitted in which the agency concluded that the manufacturers had not supported any of their claims of effectiveness for any indication. Therefore, the FDA announced that it proposed to withdraw approval for the OPEs. See Notice of Opportunity for Hearing, 40 Fed.Reg. 30,995-31,014 (1975). Seven OPE manufacturers were granted a formal hearing on the question whether the OPEs were effective, the other manufacturers having withdrawn or having failed to submit data. See 44 Fed.Reg. 75,718-19 (1979). Before the hearing, two more OPE manufacturers withdrew. See JA at 10 n. 4.

After prehearing conferences in February and March, 1980, a hearing before an administrative law judge was held in July and August of that year. Warner and ARW submitted a total of 31 studies, 13 on behalf of Warner and 18 on behalf of ARW, to demonstrate the effectiveness of their OPEs.⁴  In addition, the manufacturers presented the testimony of many of the investigators who conducted the studies. The manufacturers sought to bolster this evidence by the testimony of experts who testified that, based on their review of the submitted studies, the drugs had been shown to be effective. In opposition, the FDA produced the testimony of several employees of the agency's Bureau of Drugs and several outside experts.

In April 1981, the ALJ issued his initial decision in the matter. In a lengthy opinion that reviewed each of the studies submitted by the sponsoring manufacturer, the ALJ concluded that each study failed to meet the requirements for adequate and well-controlled clinical investigations. The ALJ thus found that the drug manufacturers had not met their statutory burden of producing evidence demonstrating that the OPEs were effective. The ALJ therefore ordered that approval for the drugs be withdrawn. See JA at 683.

The parties filed extensive exceptions to the ALJ's decision with the Commissioner. Four years later, the Commissioner issued his decision withdrawing approval for the OPEs. The Commissioner rejected all the manufacturers' challenges to the conduct of the hearing. In a detailed opinion, the Commissioner reviewed each of the studies submitted by the manufacturers in support of the efficacy of the OPEs and found each one methodologically inadequate, although in some cases for reasons different than those upon which the ALJ had relied. The Commissioner also found that there was a lack of substantial evidence that the five OPEs have the effects represented, and, accordingly, he withdrew approval. The Commissioner stayed this decision pending judicial review. See 50 Fed.Reg. 31,245-46 (1985).

The Statutory and Regulatory Framework

Under the Federal Food, Drug, and Cosmetic Act, 21 U.S.C. §§ 301-392, a drug manufacturer or sponsor cannot market a new drug⁵  unless the FDA first approves the drug. See id. Sec. 355. To obtain approval, a drug's sponsor must submit to the FDA

full reports of investigations which have been made to show whether or not such drug is safe for use and whether such drug is effective in use.

Id. Sec. 355(b) (1).

The Act requires drug manufacturers to show not only that the drug is safe but also to show by "substantial evidence" that the submitted drug "will have the effect it purports or is represented to have under the conditions of use prescribed, recommended, or suggested." Id. Sec. 355(d) (5). Substantial evidence is defined in the Act as follows:

evidence consisting of adequate and well-controlled investigations, including clinical investigations, by experts qualified by scientific training and experience to evaluate the effectiveness of the drug involved, on the basis of which it could fairly and responsibly be concluded by such experts that the drug will have the effect it purports or is represented to have under the conditions of use prescribed.

Id. Sec. 355(d). The statute mandates the same standard when the Commissioner is deciding whether to withdraw approval for a drug that previously received FDA approval. See id. Sec. 355(e) (3).

Because the Act uses the plural "investigations," the FDA requires drug manufacturers to submit at least two "adequate and well-controlled" studies showing the effectiveness of the drug. See JA at 27.⁶  The FDA has issued regulations defining "adequate and well-controlled" by several substantive criteria: (1) the study must select subjects in a way that assures that they are suitable for the study, that assigns subjects to test groups in a way designed to minimize bias, and that reduces the effect of other individual factors, such as the subjects' age, sex, or use of other drugs; (2) the study must explain its methodology, including the steps taken to minimize investigator and subject bias; (3) the study must use a method of control that permits "quantitative evaluation"; (4) the drug tested must be standardized for identity, strength, purity, quality, and dosage form. See 21 C.F.R. Sec. 314.111(a) (5) (ii) (a) (2), (3), (4) & (b) (1984). The regulations also require that a study provide a clear statement of its objectives and a summary of the methods of analysis, and an evaluation of the data derived from the study. Id. Sec. 314.111(a) (5) (ii) (a) (l) & (5). Uncontrolled studies alone are not sufficient to show effectiveness, but the FDA will consider them as corroborative support. See id. Sec. 314.111(a) (ii) (c). A failure to meet any one of these criteria renders the study unacceptable for meeting the statutory requirement of substantial evidence.⁷ 

Judicial review of the Commissioner's decision is pursuant to 21 U.S.C. § 355(h), which provides that the Secretary's findings of fact "if supported by substantial evidence, shall be conclusive." In addition, under the Administrative Procedure Act, a reviewing court shall "set aside agency action, findings, and conclusions found to be ... arbitrary, capricious, an abuse of discretion, or otherwise not in accordance with law." 5 U.S.C. § 706(2) (A). Although our review of the Commissioner's legal decision is plenary, it is clear that to the extent the Commissioner's decision involves interpretation of the Food, Drug, and Cosmetic Act, we must give "considerable weight" to his construction. See Chevron, U.S.A., Inc. v. Natural Resources Defense Council, Inc., 467 U.S. 837, 104 S. Ct. 2778, 2782, 81 L. Ed. 2d 694 (1984).

Discussion

The drug companies have filed extensive challenges to the Commissioner's decision. They attack the Commissioner's determination that each of the 31 investigations that they submitted failed to meet the regulatory criteria for an adequate and well-controlled investigation. The Commissioner filed an exhaustive opinion of 585 pages, in which he painstakingly reviewed each investigation and gave at least two reasons why each was rejected. We will not discuss each investigation, but will refer to particular studies when appropriate or as illustrative.

In addition to challenging the Commissioner's rejection of the individual studies, the drug companies present certain general challenges to the Commissioner's decision. All argue that the Commissioner gave inadequate weight to the testimony and opinions of their expert witnesses, ARW argues that the Commissioner improperly required that the clinical investigations show therapeutically significant evidence of improvement, and all claim that the Commissioner erred by failing to find that they had established effectiveness under the statutory standard. Warner also raises two procedural issues relating to the ALJ's conduct of the proceedings specific to it. It contends that the ALJ impermissibly allowed the hearing to extend beyond the issues outlined in the prehearing stipulation and erred by excluding the direct testimony of three of Warner's experts.

We will consider first petitioners' argument that the Commissioner improperly substituted his view for that of the experts. If petitioners are correct, then, of course, the decision could not stand.

Commissioner's Authority to Assess Studies

Warner and ARW make the novel and potentially far-reaching argument that it is the opinions and conclusions of their experts, rather than the opinion of the Commissioner, that must determine whether studies submitted by the drug companies were adequate and well-controlled and whether they proved the effectiveness of the drugs under consideration. Thus, Warner argues that the statutory language "obligates the Commissioner to base any decision on the adequate and well-controlled character of a clinical study, and on the determinations of a drug's effectiveness drawn therefrom, solely on the conclusion of qualified experts." Warner's brief at 27 (emphasis added). Warner stresses that the views of the experts on these issues were intended to "be paramount and controlling." Id.

We believe that this argument and the comparable one made by ARW run counter not only to the statutory scheme and the legislative history, but also to the policy and rationale underlying use of administrative agencies. In establishing a regulatory scheme such as that administered by the FDA, Congress has chosen to place rulemaking, administrative control, and enforcement authority in an expert agency. Its chief officer, the Commissioner of Food and Drugs to whom the Secretary of HHS has delegated the statutory responsibility, 21 C.F.R. Sec. 5.10 (1985), has the obligation of effectuating Congress' intent that drugs on the marketplace be both safe and effective. The Commissioner's decision that studies submitted by drug manufacturers should be evaluated pursuant to "well-established principles of scientific investigations," as set forth in the regulations promulgated for that purpose, was approved by the Supreme Court in Weinberger v. Hynson, Westcott & Dunning, Inc., 412 U.S. 609, 93 S. Ct. 2469, 37 L. Ed. 2d 207 (1973). See also Pharmaceutical Manufacturers Association v. Richardson, 318 F. Supp. 301, 306-10 (D. Del. 1970), approvingly cited in Hynson, 412 U.S. at 619 n. 13, 93 S. Ct. at 2478 n. 13.

If, as Warner and ARW suggest, the Commissioner were required to defer to the opinions of experts testifying with respect to the validity of the methodology used in particular studies and the ultimate question of effectiveness, the Commissioner's statutory function would be significantly curtailed. Moreover, he would not even be able to insure that his own regulations were interpreted uniformly and consistently. Patently, the drug companies' argument would undercut the traditional discretion such statutory schemes relegate to the administrator.

The drug companies argue, however, that the interpretation they proffer is compelled by the statutory definition of "substantial evidence" which refers to conclusions reached "by ... experts." The language of the statutory definition of "substantial evidence" as consisting of "adequate and well-controlled investigations ... on the basis of which it could fairly and responsibly be concluded by such experts that the drug will have the effect it purports ... to have," 21 U.S.C. § 355(d) (emphasis added), hardly supports the drug companies' startling claim that the experts' view must be binding on the Commissioner. The plain import of the reference to experts in the definition of "substantial evidence" is that the investigations submitted be of such quality that the investigating experts could responsibly rely on them for a conclusion. The definition does not suggest any allocation of responsibility for decisionmaking between the Commissioner and the experts, and it is unlikely that Congress would use a definition for that purpose. Elsewhere, the statute makes clear that withdrawal of new drug approval is the prerogative of the Secretary "if the Secretary finds ... that there is a lack of substantial evidence that the drug will have the effect it purports or is represented to have...." 21 U.S.C. § 355(e) (3) (emphasis added).

The petitioners also claim that their interpretation is supported by Congress' decision to permit the drug manufacturers to prove the effectiveness of their drugs by substantial evidence, rather than the higher standard of preponderance of the evidence. See S.Rep. No. 1744, Pt. 2, 87th Cong., 2d Sess. 6 (1962), 1962 U.S.Code Cong. & Ad.News 2884. That fact, while relevant to the issue of the quantity of evidence, also does not go to the allocation of decisionmaking responsibility. The legislative history hardly suggests that Congress intended that the Commissioner have no role in determining whether substantial evidence exists as to a particular drug. The Senate Report, in its explanation of the statutory definition of substantial evidence, makes clear that the Commissioner may reject the experts' conclusions. The Report states:

[A] claim could be rejected if it were found (a) that the investigations were not "adequate"; (b) that they were not "well controlled"; (c) that they had been conducted by experts not qualified to evaluate the effectiveness of the drug for which the application is made; or (d) that the conclusions drawn by such experts could not fairly and responsibly be derived from their investigations.

S.Rep. No. 1744 Part 2, 87th Cong., 2d Sess. 6 (1962), 1962 U.S.Code Cong. & Admin.News 2884, quoted in Weinberger v. Hynson, Westcott & Dunning, Inc., 412 U.S. at 619-20, 93 S. Ct. at 2478.

Any doubt as to Congress' intention to vest in the Secretary, and not the drug company experts, the responsibility of making the ultimate decision called for in the statute is plainly dispelled by the unambiguous language of the Senate Report:

What the Secretary is required to do is evaluate the claims of effectiveness in the light of the information submitted to him in the new drug application and any other information before him with respect to such drug and to decide whether there is substantial evidence, as defined above, to support the claim. The question of whether the claim would or would not be allowed would be determined by his evaluation of whether the claim had been supported by substantial evidence as defined above. If on the basis of his evaluation the Secretary finds that the claim is not supported by substantial evidence, as defined above, it will not be allowed; if he finds that it is so supported, it will be allowed....

S.Rep. No. 1744 Part 2, 87th Cong., 2d Sess 6, 1962 U.S.Code Cong. & Admin.News 2884 (emphasis added).

It would defeat the clear purposes of the Act to transfer the Commissioner's authority to the manufacturers' experts as petitioners propose. We thus conclude that Congress' concern with the quality of the expert submissions leads to the conclusion that it is the Commissioner who must determine, after giving full consideration to all of the evidence that has been submitted, including expert opinions, if the studies meet the regulatory criteria and show effectiveness.⁸ 

Required Showing of Effectiveness

ARW contends that the Commissioner erred in reviewing each study to determine if the results were therapeutically significant, interchangeably referred to as clinically significant. Therapeutic or clinical significance is contrasted for this purpose with statistical significance, which ARW accurately characterizes as based on a premise of the elimination of chance results. ARW Reply Brief at 6-7. ARW argues that the Commissioner must look only to whether any of the submitted studies show statistically significant differences between the test groups given the OPE and the control groups. If so, the Commissioner must find the drug effective, and any determination of therapeutic value must be left to the individual prescribing physician. In essence, ARW's claim is that "effectiveness" as used in the Act means only that the drug will have the effect the manufacturer claims for it; the amount of the effect is irrelevant, except to the treating physician.

To support its view, ARW points to the statute which requires the FDA to find only that the submitted drug "will have the effect it purports or is represented to have under the conditions of use prescribed, recommended, or suggested in the labeling thereof ..." 21 U.S.C. § 355(e) (3). ARW also points to statements in the legislative history that indicate that Congress believed that effectiveness would be measured by the claims made for the drug by the manufacturer. See, e.g., S.Rep. No. 1744, 87th Cong., 2d Sess. reprinted in 1962 U.S.Code Cong. & Ad.News 2884, 2892; Drug Industry Antitrust Act, Hearings on H.R. 6245 before Subcommittee on Antitrust and Monopoly, 87th Cong., 2d Sess. 143 (1962). These isolated sentences, however, do not support ARW's inference which is contrary to the policy behind the Act.

The Act does not define "effectiveness," thus leaving the task of deciding how effective a new drug must be to the agency to which Congress delegated enforcement. It is important to note that the Commissioner does not contend that the effectiveness shown must amount to a "medical breakthrough", as ARW complains, but contends in his brief that he would be satisfied with even a modest clinical or therapeutic effect. Respondents' Brief at 23.

The Commissioner's interpretation of the statute as requiring a showing of clinical significance, rather than merely statistical significance, is persuasive for several reasons. First, in requiring evidence of therapeutic effectiveness, the FDA is interpreting the language of the Food, Drug, and Cosmetic Act, which Congress has entrusted to that agency's supervision. As the Supreme Court has frequently noted, "the construction of a statute by those charged with its administration is entitled to substantial deference." United States v. Rutherford, 442 U.S. 544, 553, 99 S. Ct. 2470, 61 L. Ed. 2d 68 (1979). See Chevron, U.S.A., Inc. v. Natural Resources Defense Council, Inc., 467 U.S. 837, 104 S. Ct. 2778, 2782, 81 L. Ed. 2d 694 (1984). We may not substitute ARW's interpretation of effectiveness for that of the FDA, unless the Commissioner's interpretation is arbitrary or irrational, or unless it is established that Congress meant to foreclose the requirement of therapeutic effectiveness.

The fact that the drug, not chance, can be assumed to have contributed to the factor measured does not necessarily establish that patients will receive a benefit from the drug. The Commissioner has consistently required a showing of some benefit as an element of the statutory requirement of effectiveness. See Benylin; Final Order, 44 Fed.Reg. 51512, 51521 (1979); Luxtrexin; Withdrawal of Approval of New Drug Application, 41 Fed.Reg. 14406, 14419 (1976).

It is difficult to conceive of any good reason why the Commissioner should be required to find a drug effective if there has been no showing of its therapeutic effect. For example, in the Soule, Wasserman, and Burstein study of the effect of the OPE Chymoral (marketed by Armour) on symptoms associated with episiotomy, the investigators made 240 comparisons between the study and control group on factors such as edema (swelling), erythema (skin redness), bruising, and the subject's pain while resting, sitting, and walking. These were studied at different time periods after delivery. There were no statistically significant results in favor of Chymoral for reduction of swelling or bruising. Of the 240 tests, only six provided statistical results indicating that Chymoral had some effectiveness in the reduction of pain, but these results were shown only after stratification of the subjects into subgroups that the Commissioner found had no scientific basis. For example, statistically significant results were reached showing effectiveness of Chymoral for reduction of pain "on sitting on day four in subjects with labors over eight hours"; for reduction of pain "on sitting, on post-partum days two and three, in the 20 subjects less than 21 years old"; and for reduction of pain in multipara subjects (those who had earlier pregnancies) "on ambulation on post-partum days three, four and five," but not in primiparas, first pregnancies subjects, as a group. JA at 399.

ARW argues that these positive results in six of 240 comparisons constitute the required statutory showing of effectiveness. The Commissioner stated that the most reliable test of effectiveness is the comparison of the total drug group to the total placebo group, a comparison never made in this study. JA at 405. Since there was expert evidence to support the Commissioner's conclusion that the post-hoc stratification was inappropriate, the Commissioner's conclusion that six statistically significant findings cannot support a finding of effectiveness, JA at 407, was supported by substantial evidence.

Another illustration of the ARW position is afforded by the Lee, Larsen, and Posch Study No. 1, which was proffered to show Chymoral's effectiveness in reducing edema in subjects with hand surgery. The largest difference in measurements between the drug group and the placebo group in absolute figures (rather than percentages) was approximately 1/4 inch. Although the percentage difference was statistically significant, the Commissioner rejected the study, in part based on expert testimony that the difference in measurements was "therapeutically trivial". JA at 366-67.

The Commissioner's construction that clinically significant evidence of effectiveness must be shown accords with the policies behind the 1962 amendments to the Act. These amendements were passed to increase the protection accorded to the consuming public. In a somewhat related context, the Supreme Court recognized that the requirement of effectiveness added by these amendments necessarily entails a showing of some benefit to the patient. The Court stated, "in the treatment of any illness, terminal or otherwise, a drug is effective if it fulfills, by objective indices, its sponsor's claim of prolonged life, improved physical condition, or reduced pain. See 42 Fed.Reg. 39776-39786 (1977)." United States v. Rutherford, 442 U.S. at 555, 99 S. Ct. at 2477.⁹ 

Congress rejected the notion, asserted here by ARW, that individual physicians should be left to decide whether particular drugs were effective. For example, in separate concurring remarks accompanying the Senate Report, Senator Kefauver, the leading figure behind the legislation, joined by four other members of the Senate Judiciary Committee, made the following observations:

Leading physicians testified that it is impossible to keep currently informed of the state of medical knowledge to be found scattered in hundreds of medical journals on the 400 new drugs introduced each year. Moreover, they stressed that the marketing of a safe but ineffective drug may well be positively injurious to the public health. When an ineffective drug is prescribed, it is usually in place of an older but effective drug. The problem is compounded by the fact that usually a considerable period elapses between the time when a highly-advertised new drug is put on the market and when knowledge becomes widely disseminated among the medical profession that its performance falls seriously short of its claims.

S.Rep. No. 1744, 87th Cong., 2d Sess, 1962 U.S.Code Cong. & Ad.News 2902 (views of Senators Kefauver, Carroll, Hart, Dodd & Long) (footnotes omitted).

Congress wanted to ensure that prescribing physicians received sufficient and accurate information concerning the effectiveness of FDA-approved drugs, see Weinberger v. Hynson, Westcott & Dunning, Inc., 412 U.S. at 619, 93 S. Ct. at 2478, a task that had become impossible due to the sheer volume of new drugs and to the heavy and persistent drug company advertising. See 108 Cong.Rec. 21070 (1962) (remarks of Representative Fogarty) ("medical practitioners cannot keep abreast [of new information] on each drug"); 108 Cong.Rec. 17382 (1962) (colloquy between Senators Kefauver and Hart) (discussing drug company advertising). See also Upjohn Company v. Finch, 422 F.2d 944, 952-54 (6th Cir. 1970) (rejecting contention that public is protected because a physician must prescribe the drugs).

Given the strength of the congressional concern with the protection to the public underlying the Drug Amendments of 1962, it would be anomalous to hold that drug manufacturers may demonstrate effectiveness merely by showing statistical significance. Therefore, we hold that the Commissioner did not err in requiring petitioners to show that OPEs are therapeutically effective. This interpretation accords with long-standing FDA policy and with the policies embodied in the Food, Drug and Cosmetic Act.C.

Substantial Evidence

The pivotal issue on this appeal is whether there is any basis for us to overturn the Commissioner's conclusion that petitioners had not produced substantial evidence that the OPEs will have the purported or represented effects. Central to this conclusion was the Commissioner's determination that all of the investigations submitted by the petitioners did not meet the requirements of adequate and well-controlled studies.

As we previously commented, the Commissioner's review of each of the submitted studies was extensive. For each study, the Commissioner first reviewed the factual data of the investigation, including information concerning the subjects, the condition tested, the method of reporting results, and the claimed results. Second, the Commissioner reviewed the administration of the OPE and stated whether the subjects had received concomitant medication. Finally, the Commissioner reviewed the parties' exceptions to the ALJ's findings on each study, the information contained in each study, and the expert testimony supporting the study. Neither ARW nor Warner challenges the accuracy of the Commissioner's descriptions of the studies.

Instead, petitioners challenge the Commissioner's bases for the rejections. They argue that all of the studies they submitted demonstrate the effectiveness of the particular drug at issue, but Warner directs our attention particularly to the Commissioner's rejection of four studies it submitted and ARW focuses on the Commissioner's rejection of six of its studies, two supporting each OPE.

Because the Commissioner's findings in this case were made after a hearing, we review them under the substantial evidence test. See 21 U.S.C. § 355(h). See also Weinberger v. Hynson, Westcott & Dunning, Inc., 412 U.S. at 622 n. 19, 93 S. Ct. at 2479. That review standard does not require that we resolve scientific disputes over the evidence, even though some expert witnesses may have drawn conclusions different from those of the experts on whom the Commissioner relied. See Rhone-Poulenc, Inc., Hess & Clark Division v. FDA, 636 F.2d 750, 753 (D.C. Cir. 1980). Nor does the requirement of substantial evidence require a finding of preponderant or conclusive evidence. See United States v. An Article of Drug Consisting of 4680 Pails, 725 F.2d 976, 985 (5th Cir. 1984). Rather, we must determine if the Commissioner's decision has a reasonable basis. See Pfizer, Inc. v. Richardson, 434 F.2d 536, 546-47 (2d Cir. 1970).

Although courts frequently express reluctance to intrude into complex scientific or medical issues on which we have inadequate technical background, see, e.g., Cooper Laboratories, Inc. v. Commissioner, Federal FDA, 501 F.2d 772, 788 (D.C. Cir. 1974) (Leventhal, J., dissenting) (understanding and applying "the drug law and regulations [require] technical knowledge that is not part of this court's working kit"), we cannot completely abdicate the review function imposed on us by Congress and inherent in a democratic society which looks to the judiciary to check excesses from other branches of the government. Fortunately, the issues in this case involve in the main those relating to scientific methodology cognizable even to laymen. See Pfizer, Inc. v. Richardson, 434 F.2d at 546-47.

The basis given by the Commissioner most frequently in rejecting the submitted studies was that the investigators' use of concomitant medication was uncontrolled. In 16 studies, the Commissioner found that the case report forms were missing information as to the dosage, frequency, or length of administration of the concomitant medication, and that the investigation could therefore not be used as evidence in support of the claims. It is unclear whether the petitioners are challenging the Commissioner's position that accurate and complete details about the administration of concomitant medication is a sine qua non of an acceptable clinical investigation or whether they are merely challenging the Commissioner's requirement that dosage, frequency, and length of administration must have been contemporaneously recorded on the case report form.

In his opinion the Commissioner fully explained the importance of reviewing the use of concomitant medication and cited to the supporting expert opinion in that connection. He stated that the uncontrolled use of concomitant medication "violates several of the most basic scientific principles governing clinical explanations." JA at 51. These principles are embodied in the Commissioner's regulations. See, e.g., 21 C.F.R. Sec. 314.111(a) (5) (ii) (a) (b) & (c). Because many concomitant medications, such as analgesics and anti-inflammatory drugs, and concomitant treatments, such as ice packs, are active treatments for the symptoms which the OPEs purport to help alleviate, the use of concomitant medications in both the test and the control groups makes it difficult to state what results in the test group can be attributed to the OPE treatment. JA at 52-53. However, the Commissioner rejected the position that a per se rule applies to the use of concomitant medication, and instead opted for "the more flexible scientific approach" which entailed examination of the case report forms to determine if the actual effect of the concomitant medications can be determined. JA at 56, 148. The Commissioner concluded that if the concomitant medication is uncontrolled, and the reports of the studies do not provide the details necessary to permit a scientific evaluation of the use of concomitant medication, then the study cannot be considered as part of the basis for approval of effectiveness claims. JA at 54.

Ample precedent supports the Commissioner's insistence on "well-controlled investigations". In Weinberger v. Hynson, Westcott & Dunning, the Court stated: "The standard of 'well-controlled investigations' particularized by the regulations is a protective measure designed to ferret out those drugs for which there is no affirmative, reliable evidence of effectiveness." 412 U.S. at 622, 93 S. Ct. at 2479. Moreover, the expert evidence in the record supports the Commissioner's conclusion that use of concomitant medication flaws a clinical study unless it can be determined whether such use was controlled.

ARW raises three related challenges to the Commissioner's analysis of concomitant medication. First, ARW argues that the Commissioner in effect held that concomitant medication could never be used because of the stringent restrictions he required to constitute control of their use. According to ARW, to meet these requirements, clinical investigators would have had to administer the OPEs in isolation. Because OPEs are adjunctive therapy, not primarily intended to reduce pain or to be used alone, it would be unconscionable to withhold the concomitant medication to subjects who were suffering painful post-surgical trauma symptoms.

We recognize that clinical investigation on adjunctive therapy raises research difficulties that may not be presented when studying other types of drugs. However, the Commissioner cited to expert testimony in this record to show that means of controlling concomitant medication existed. See JA at 57 n. 45. Indeed, the adjunctive use of OPEs would seem to indicate that more, rather than less, control over comcomitant medication would be necessary to demonstrate effectiveness. Cf. Cooper Laboratories v. Commissioner, Federal FDA, 501 F.2d at 779 (rejecting argument that controlled tests are impossible for drug designed to relieve pain). Furthermore, the petitioners would be in a more sympathetic posture to complain about the difficulties of showing effectiveness of adjunctive therapy if their studies had reported the use of concomitant medication in accordance with settled scientific research principles. As the Commissioner commented, "In no OPE study criticized by the ALJ because of the use of concomitant medication did all subjects receive the same type and amount of concomitant medication or receive it in a blinded manner." JA at 57 n. 45. A holding that the Commissioner cannot require clinical investigations to be strictly controlled would undermine Congress' intent to have effectiveness proven by a "strict and demanding" test. See Weinberger v. Hynson, Westcott & Dunning, Inc., 412 U.S. at 619, 93 S. Ct. at 2478. Moreover, we find it significant that Congress has not accepted proposals to delete the statutory requirement of a showing of effectiveness. See, e.g., H.R. 12573, 94 Cong., 2d Sess (1976).

ARW's second argument is that " [s]o long as patients in control groups and test groups receive at random, concomitant medications, the groups should be roughly equal." ARW brief at 29 (emphasis added). The Commissioner rejected this argument because it would essentially require him to disregard the use of concomitant medication. See JA at 60. He concluded he could not be assured, as the drug companies argued, that randomization balanced out the amounts of concomitant medications used in the studies. JA at 61. In addition, based on expert testimony, the Commissioner concluded that while random assignment of subjects could help minimize the effects of unknown and uncontrollable factors, it should not be used to minimize the effect of factors that are known and can be otherwise controlled. See JA at 60-61.

Finally, ARW argues that the FDA has approved two other drugs which were supported by studies in which concomitant medication was used. This argument is irrelevant. We have already held that the Commissioner may insist on strict controls on the use of concomitant medication. That he approved other drugs that were tested with concomitant medications does not bear on the decision in this case, especially when no showing has been made that the studies supporting the other drugs lacked sufficient controls. We decline to overturn the Commissioner's decision in this case on the basis of unsupported allegations about the decision in another.

2. Other Bases for Rejection of Individual Studies

ARW and Warner object to the Commissioner's other reasons for rejecting numerous studies. We have reviewed the Commissioner's decision and the studies and testimony upon which he relied and we are unable to agree with petitioners. The Commissioner fully met the requirement set forth in Cotter v. Harris, 642 F.2d 700, 706 (3d Cir. 1981), cited by Warner, regarding the need for an adequate discussion of the evidence in an adjudicatory decision. The Commissioner painstakingly reviewed the petitioners' submissions, devoting 370 pages to analysis of the evidence submitted in support of petitioners' OPEs. Each study was rejected for at least two violations of the regulatory criteria, and, of the studies petitioners particularly focus on in this court, six were rejected for three or more reasons. It would serve little purpose here to review each of these reasons. Illustrative of the reasons given by the Commissioner for finding particular studies were not adequate and well-controlled were the failure to show that adequate steps were taken to minimize bias and that blinding was maintained, JA at 150, the failure to record the investigator's initial expectations in studies using the "comparison with expectations" method of analysis, JA 540-42, 544-46, the absence of assurance that the drug and placebo groups were comparable for severity of disease, JA at 514-15, and the use of an inappropriate statistical method to evaluate data. JA at 137-40. Our review convinces us that the Commissioner disqualified each study "in light of the pertinent regulations." Weinberger v. Hynson, Westcott & Dunning, Inc., 412 U.S. at 622, 93 S. Ct. at 2479.

We emphasize that it is not the court's role to substitute its judgment for that of the Commissioner. Warner argues that there is a facial implausibility to the Commissioner's rejection of 31 studies, but we cannot find that he did not have a reasonable basis for disqualifying each submission. Petitioners suggest that the Commissioner's rejection of all of the studies stems from his prejudicial bias that OPEs cannot possibly work. We find no evidence of such a bias. Moreover, the rejection of all of the studies is just as consistent with the possibility that, in fact, OPEs do not work.

Our task is not to resolve the scientific question of the methological correctness of the submitted studies, but the legal question whether the Commissioner's rejection of the studies is supported by substantial evidence. See Smithkline Corporation v. FDA, 587 F.2d 1107, 1119 (D.C. Cir. 1978). Were we to reverse the Commissioner's order merely because of an asserted improbability in his reasoning, we would be discarding the Commissioner's expertise in favor of our own inexpert judgment. Such a course would contravene a clear congressional judgment to leave to the agency's expertise the primary decision as to drug effectiveness.

Procedural Issues Raised by Warner

Warner contends that the Commissioner could not properly reject any of its studies on the basis of its use of concomitant medication because the issue of concomitant medication was not preserved in the prehearing stipulation entered into between it and the FDA. It argues that the Commissioner's reliance on that ground therefore violated the requirement of the Food, Drug, and Cosmetic Act that a drug manufacturer be given due notice prior to withdrawal of FDA approval. See 21 U.S.C. § 355(e).¹⁰ 

The prehearing stipulation entered into between the FDA and Warner provided that the issues set forth therein "constitute the only issues in this hearing with respect to Papase tablets, unless either of the parties can show good cause why other issues should be considered." JA at 2779. The first issue listed was

1. Whether there are adequate and well-controlled investigations, including clinical investigations, by experts qualified by scientific training and experience to evaluate the effectiveness of Papase, on the basis of which it can fairly and responsibly be concluded by such experts that Papase is effective for any of its labeled conditions of use.

JA at 2777. The remaining twelve issues identify specific bases for rejection of the studies Warner submitted in support of Papase. For example, the second issue listed was:

2. Whether the method of blinding used in the studies by Magnes, Metro, Protzel, Royer, Vallis, Weinstock, Levy, and Pollack was adequate to minimize bias on the part of the investigator.

JA at 2777. The remainder of the issues are similarly phrased. None of them mentions the use of concomitant medication.

The FDA argues, nevertheless, that the first issue relating to "adequate and well-controlled investigations" encompasses the issue of concomitant medication because, under the regulatory criteria, a study cannot be adequate and well-controlled if concomitant medication is used in uncontrolled fashion. We reject this argument as contrary to the language and apparent design of the stipulation. Each of the twelve issues specifically defined deals with a necessary element of an adequate and well-controlled investigation. If the first issue were construed to be as broad as the Commissioner now suggests, there would have been no reason to specify each of the remaining twelve issues separately. Although it is possible that experienced counsel should have known that administration of concomitant medication would generally be an issue when the drug is marketed as adjunctive therapy, we cannot construe the stipulation as giving specific notice of this issue.

Nonetheless, under the circumstances of this proceeding, the failure to include the issue of concomitant medication in the stipulation did not preclude the Commissioner's reliance on that ground. Defects in notice in administrative hearings may be cured if the actual conduct of the proceedings provides notice to the participants. See National Steel & Shipbuilding Company v. Director, Office of Workers' Compensation Programs, 616 F.2d 420, 421 (9th Cir. 1980); Common Carrier Conference-Irregular Route v. United States, 534 F.2d 981, 982-83 (D.C. Cir.), cert. denied, 429 U.S. 921, 97 S. Ct. 317, 50 L. Ed. 2d 288 (1976). The test is whether the parties to the proceeding understood the issue and had a full opportunity to litigate it. See Golden Grain Macaroni Company v. FTC, 472 F.2d 882, 885-86 (9th Cir. 1972), cert. denied, 412 U.S. 918, 93 S. Ct. 2780, 37 L. Ed. 2d 144 (1973).

Prior to the oral phase of the hearing in this case, the FDA introduced written testimony relating to the uncontrolled use of concomitant medication. ARW, whose prehearing stipulation was almost identical to Warner's, see JA at 718, moved to strike all FDA testimony on this issue on the ground that introduction of this testimony exceeded the scope of the stipulation. The ALJ denied this motion over a month before the oral phase of the hearing began. Warner offers no explanation why it did not file a motion to strike even though the proceedings had been consolidated for the hearing and, as counsel for Warner conceded at oral argument, Warner knew of ARW's motion. Transcript of oral argument at 42. Thus, Warner was aware of the FDA's position on the uncontrolled use of concomitant medication.

As the Commissioner stated in rejecting Warner's claim of inadequate notice, the drug companies had an opportunity to request additional oral direct examination on issues on which they claimed surprise. JA at 49. Certainly they could have, and the Commissioner stated that they did, cross-examine FDA witnesses on the issue of concomitant medication. Since Warner had notice that concomitant medication would be a hearing issue in sufficient time to prepare and submit additional materials on that issue, it may have chosen not to do so as trial strategy. Although we believe it important to reiterate the principle that an agency should abide by its stipulations, in the circumstances of this case showing actual notice to Warner, we cannot find that the Commissioner erred in considering the use of concomitant medication in rejecting Warner's Papase studies.¹¹ 

Warner's remaining procedural challenge is that the Commissioner erred in upholding the ALJ's exclusion of the direct testimony of three of its experts who conducted studies supporting the efficacy of Papase. The ALJ excluded this testimony because Warner did not make the experts available for cross-examination during the period scheduled for oral testimony.

The schedule for the period of oral cross-examination was originally set at the March 4, 1980 prehearing conference. At that time, the ALJ stressed to the parties that their witnesses had to be available during that period. See JA at 3386. On July 17, 1980, the first day of the hearing, Warner informed the ALJ that three of its witnesses, Drs. Protzel, Royer and Vallis, would not be able to testify on the date scheduled because of "their heavy professional schedules." JA at 3412. It provided no further explanation to the ALJ. Warner proposed that the FDA cross-examine these doctors by written questions or telephone. The ALJ denied this request and told Warner that if the witnesses could not be available for cross-examination on the appointed day, their direct testimony would be stricken. After two interlocutory appeals to the Commissioner, and subsequent consideration by the ALJ of the alternative means proposed by Warner, including having one of the experts, Dr. Royer, cross-examined on Saturday or in his hometown, Detroit, the ALJ persisted in his position that the witnesses must be produced. Because the witnesses did not appear for oral cross-examination, their written testimony was stricken.

Warner alleges that the ALJ has no discretion to exclude relevant evidence in an adjudicatory proceeding, and that it was prejudiced because the excluded testimony might have provided the FDA with the necessary evidence of effectiveness for three of the studies Warner submitted. It asks us to remand the proceeding to the agency for additional testimony.

The Administrative Procedure Act (APA) requires agencies to exclude "irrelevant, immaterial, or unduly repetitious evidence." 5 U.S.C. § 556(d). In this case, however, the ALJ's decision to exclude the testimony of Warner's experts was not based on evidentiary grounds. In general, "the formulation of administrative procedures is a matter left to the discretion of the administrative agency." Laird v. Interstate Commerce Commission, 691 F.2d 147, 154 (3d Cir. 1982), cert. denied, 461 U.S. 927, 103 S. Ct. 2086, 77 L. Ed. 2d 297 (1983). The ALJ's decision in this case was within his discretion to guide the course of the hearing. See Montgomery v. Commissioner of Internal Revenue, 367 F.2d 917, 920 (9th Cir. 1966). This discretion includes the power to make reasonable, nonarbitrary decisions regarding the admission or exclusion of evidence for procedural reasons. See Second Taxing District of the City of Norwalk v. Federal Energy Regulatory Commission, 683 F.2d 477, 485 (D.C. Cir. 1982). In challenging a decision of this kind, the burden is on the objecting party to make a "strong showing" that the ALJ has abused his discretion. Montgomery v. C.I.R., 367 F.2d at 920.

Warner has not met that burden here. Warner has never given a satisfactory explanation for the inability of Dr. Protzel, a retired physician, to attend. It has suggested nothing unusual about Dr. Vallis' schedule. Its contention that Dr. Royer had to cover the caseload of his partner, who had lost the sight in one eye, was raised only belatedly in its brief supporting its second interlocutory appeal to the Commissioner. The FDA argues, apparently accurately, that this contention was never otherwise presented to the ALJ.

Warner was given ample notice of the dates for the hearing. The ALJ particularly stressed the necessity of supplying experts for oral cross-examination during this period. Although we do not deprecate the importance of a physician's responsibilities to his or her patients, the physician's willingness to undertake research of this type necessarily implies a similar undertaking to appear when necessary in support of the results reached. The ALJ was not obliged to travel to the physician's place of residence to suit his convenience, although the ALJ was willing to do so for an ill expert witness. Nor was he obliged to arrange the hearing for a Saturday, when the government building was not ordinarily open for that purpose. Obviously, he was not obliged to forego the benefit of oral cross-examination in favor of the less potent written interrogatories.

Given the prior notice of the hearing and the length of time provided for cross-examination (over three weeks), we cannot find that the ALJ abused his discretion in this respect. Just as agencies may make reasonable determinations as to the need for cross-examination in adjudicative proceedings, see Seacoast Anti-Pollution League v. Costle, 572 F.2d 872, 880 (1st Cir.), cert. denied, 439 U.S. 824, 99 S. Ct. 94, 58 L. Ed. 2d 117 (1978), agency decisionmakers must have reasonable latitude to structure cross-examination when it is provided.

Conclusion

In summary, we conclude that the policies behind delegation to administrative agencies, and in particular the policy underlying the 1962 amendments to the Food, Drug and Cosmetic Act requiring that drug manufacturers produce substantial evidence of the effectiveness of their drugs, support the Commissioner's position that he had authority to make the requisite determinations. We also conclude that the Commissioner did not err as a legal matter in ruling that each study could be evaluated to determine if the results were therapeutically significant rather than merely statistically significant. The Commissioner's conclusion that petitioners had not made the requisite showing that their drugs will have the purported or represented effect is supported by substantial evidence in the record. Finally, we conclude that the Commissioner was not precluded by the stipulation with Warner from considering the use of concomitant medicine in rejecting the studies proffered by Warner in support of its drug and that the Commissioner did not abuse his discretion in upholding the ALJ's ruling excluding the expert testimony of the Warner witnesses who failed to appear for oral cross-examination.

For the reasons set forth herein, we will affirm the order of the Commissioner withdrawing approval for the new drugs at issue and deny the drug manufacturers' petition for review.