This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Michaelis, L. C. Articles by Ratain, M. J. Search for Related Content PubMed PubMed Citation Articles by Michaelis, L. C. Articles by Ratain, M. J. Related Collections Related Article

Phase II Trials Published in 2002: A Cross-Specialty Comparison Showing Significant Design Differences between Oncology Trials and Other Medical Specialties

Authors' Affiliation: University of Chicago, Department of Medicine, Section of Hematology/Oncology, Chicago, Illinois

Requests for reprints: Mark J. Ratain, Section of Hematology/Oncology, 5841 S. Maryland Ave., MC2115 Chicago, IL 60637-1470. Phone: 773-702-4400; Fax: 773-702-3969; E-mail: MRatain{at}medicine.bsd.uchicago.edu.

	Abstract

Top Abstract Materials and Methods Results Discussion References

 
Purpose: Phase II trials play an essential role in drug development pathway, and their conclusions often impact the decision to embark on large, pivotal trials. However, the determination of agent activity is highly dependent on trial design. Formal comparisons of phase II trial designs across medical specialties are uncommon. We hypothesized that there are significant differences in the design of trials conducted by oncologists and those conducted by other medical and surgical specialties.

Experimental Design: We screened MEDLINE for the abstracts of phase II trials published in 2002. All abstracts were analyzed and classified by a priori defined variables, including study type, intervention, subspecialty, journal impact factor, method of control, and study conclusions.

Results: Our search yielded 703 abstracts of phase II trials published in 2002. A total of 586/703 (83%) were trials on antineoplastic agents. Twenty percent (143/703) of the trials included explicit control subjects. Oncology trials, as compared with all trials done by other specialties, were significantly less likely to use control subjects (13% versus 56%, P < 0.001) and were less likely to conclude that the investigational intervention was safe and efficacious and/or worthy of additional investigation (76% versus 89%, P < 0.01).

Conclusions: There are significant differences in the phase II trials published in oncology compared with those conducted by other medical and surgical specialties. The impact that such differences have on the efficiency of drug development should be investigated.

In oncology, allocating development resources appropriately is a complex endeavor. There are more than 1,100 antineoplastic agents in preclinical development and nearly 500 antineoplastic agents undergoing phase I or phase II testing (3, 4). However, many oncology drugs fail to prove efficacy in phase III trials despite putative activity in earlier clinical investigation (5). The costs of drug development and the increasing number of investigational agents underscore the need to rigorously evaluate the efficiency and performance of the drug development process and the phase II trial in particular. Although there has been considerable attention paid to the statistical design of phase II trials in oncology (6–10), there are few studies evaluating trends in trial design and implementation across medical specialties.

We hypothesized that significant quantifiable differences exist between phase II trials conducted by oncologists and those conducted by other medical subspecialties. An analysis of these differences might illuminate the components of phase II design and implementation that best identify the agents that will eventually prove efficacious and cost-effective. We analyzed phase II trials published in a single year, 2002, and represented in the MEDLINE database. Our aim was to compare trial design, characteristics, and conclusions and to identify trends that distinguish the trials conducted in oncology from those conducted by other medical subspecialties.

	Materials and Methods

Top Abstract Materials and Methods Results Discussion References

 
Identification and selection of studies. We conducted a computerized literature search for all 2002 published phase II trials. We searched MEDLINE using the commercially available SilverPlatter search engine. The search strategy first pulled records that included the term "Clinical-Trial-Phase II." This set was then limited to publication year 2002. The final set included 859 publications.

An algorithm was developed to determine which publications would be included in the analysis (Table 1 ). Publications were included if they (a) described an original trial explicitly labeled as phase I/II, phase II, or phase II/III; or (b) described a trial that seemed to be a first attempt to test activity of an experimental intervention or combination of interventions. Publications in languages other than English and trials involving nonhuman subjects were excluded. It is likely that our search may have missed some legitimate phase II trials because neither our search criteria nor the MEDLINE database are likely to be exhaustive. Additionally, our criteria included publications from journal supplements. As such, it includes some phase II trials that did not undergo formal peer review.

Classification system. The inclusion algorithm yielded 703 publications of phase I/II, II, or II/III trials. The following information was recorded for each publication: author; title; journal and citation; brief description of experimental intervention; medical or surgical subspecialty conducting the trial; number of subjects; trial design; and trial conclusions (Table 2 ). All classification decisions were made solely by the reader based only on the information in the published abstracts. All data were recorded, sorted, and tallied using Microsoft Excel spreadsheet software.

	Results

Top Abstract Materials and Methods Results Discussion References

 
Fields of study. Our search yielded 703 publications included in MEDLINE for the year 2002, which could be classified, according to our inclusion algorithm, as phase II trials. A total of 586/703 (83%) were trials conducted in the field of oncology. Non-oncology trials accounted for the remaining 117 publications conducted by 26 other medical and surgical subspecialties. Of these, more than a quarter (30/117, 26%) tested drugs or interventions in the subspecialty of infectious disease. The majority of the latter (16/30, 53%) were phase II trials of HIV interventions (Fig. 1 ).

View larger version (10K): [in this window] [in a new window]   Fig. 1. Phase II abstracts: other medical specialties.

View larger version (18K): [in this window] [in a new window]   Fig. 2. Classification of oncology abstracts by subspecialty.

In addition, we found that oncology trials examining novel targeted agents were unlikely to use control subjects or even historical controls in their studies. Of the trials looking at monotherapy treatment regimens, 47/228 were trials on novel targeted agents. In this subset, 9/47 or 19% were controlled trials of any kind. Of the 101 trials where cytotoxic chemotherapy was tested as monotherapy, six trials, or 17%, used control agents. All but 3 of the 289 oncology trials looking at combination chemotherapy included cytotoxic therapies, rather than agents thought to work through cytostatic or targeted mechanisms. Of the three oncology trials testing combinations of targeted agents, one included control subjects, and one randomized subjects to varied dosing schedules.

Trial conclusions. We attempted to tabulate the overall conclusions of the abstracts. We did this by developing a sorting system that labeled each abstract according to its concluding claim. The following three labels were used:

1. The abstract claims that the intervention is active, that the data support phase III trials of the intervention, or that the data warrant further study of the intervention.
   
2. The abstract claims that the data do not support additional study, that the intervention is not active, or that the intervention is dangerous.
   
3. The abstract does not make any claims about activity of the intervention.
   

We found that 668/703 (95%) abstracts made a concluding claim about the efficacy of their intervention. In 520 of those 668 abstracts (78%), the claim was that the experimental intervention warranted further development or study. Approximately 76% (422/558) of the trials conducted by oncologists made that claim, whereas 98/110 (89%) of trials conducted by other specialties made similar claims about their results. Overall, published phase II trials seem to overwhelmingly advocate additional study of tested interventions. That said, oncology trials were significantly less likely to conclude that the intervention was safe and active and/or worthy of additional study (76% versus 89%; P < 0.01).

	Discussion

Top Abstract Materials and Methods Results Discussion References

 
Our data show that phase II trials in oncology are much more prevalent in the medical literature than are phase II trials in other medical and surgical subspecialties and, importantly, very different in design. The difference in the number of published phase II trials is one of this analysis' most striking findings and accounts for the fact that the comparator set in our analysis has many fewer trials. A number of factors may contribute to this finding. First, it is possible that oncologists are conducting phase II trials at the same frequency as other specialties, but that oncology journals are more likely to publish these uncontrolled phase II trials than are journals that focus on other fields. Second, the large number of published phase II oncology trials may reflect the current state of drug development; if there are many more novel clinical entities and untested interventions to investigate in the treatment of malignant diseases, there would be more phase II trials done and published. The skewing in the number of published trials might also be due to the current paradigm of cancer treatment, which largely relies on combination chemotherapy. Therefore, new entities undergo phase II testing as monotherapy and also as additions to combination regimens. The number of combinations of interventions potentially leads to a greater number of phase II trials. One issue that we are not able to answer with this study is the impact of this high number of published phase II trials. Does this speed drug development in the specialty? Does it mean that more phase III trials are undertaken, and if so, is that beneficial or harmful?

Our study also found that, overall, relatively few phase II trials used any explicit control subjects in their design, and that phase II trials conducted with oncology subjects were significantly less likely to use controls than were other medical and surgical subspecialties. Is there a fundamental reason why phase II trials should not include controls? Not according to the FDA, which, in their monograph General Considerations for the Clinical Evaluation of Drugs, state that the majority of investigational drugs should be evaluated using placebo and/or active drug controls (11). In describing the requirements for phase II trials, the U.S. Code of Federal Regulations spells out the following criteria: "Phase 2 studies are typically well controlled, closely monitored, and conducted in a relatively small number of patients, usually involving no more than several hundred subjects."² As noted earlier, traditionally, phase II trials in the practice of oncology have been designed as single-arm trials with historical control data incorporated into the thresholds for response rate. In one commonly used design, the null hypothesis is that the drug's effect is less than some minimum response rate (12). Thus, failure to reject the null hypothesis suggests that the drug is inactive and should not be tested further for that indication. However, rejection of the null hypothesis does not prove the alternative hypothesis (a response rate much greater than the threshold used for the null hypothesis). Not surprisingly, the positive predictive value is low, which may be due to the lack of concurrent controls (8, 13). It is reasonable for trials of cytotoxics, as single agents, to be conducted as uncontrolled trials, assuming the response rate is the end point. However, we found that even when novel agents, thought to act by targeted mechanisms, were tested as monotherapy, the great majority of trials, 81%, did not control for the intervention.

Other medical specialties, according to our data, are not as narrow in their designs. For example, the FDA guidelines for drugs developed for the treatment of rheumatoid arthritis spell out that: "Once a reasonably safe range of doses has been established, randomized, parallel-arm dose comparison trials are ordinarily recommended... Active controlled designs that specify an arm with a well-characterized, known therapy can also be very useful" (14).

Is it ethically difficult for oncology trials to include controls? It has been suggested that, because physicians always have a treatment preference and their primary duty is to their patient, all randomized trials are unethical (15). In life-threatening diseases such as cancer, this may be even more problematic for some. In addition, it can be particularly difficult to randomize oncology patients to placebo, particularly in the United States. That said, many medical conditions (including, for example, HIV infection, sepsis, critical care illnesses, and acute coronary syndromes) conform to these same restrictions. Notably, controlling for a new intervention does necessitate a placebo arm; rather, it merely implies that one group will not be exposed to the experimental intervention. The use of randomized discontinuation, dose ranging, or active controls (i.e., randomization to a standard treatment arm) have all been proposed as ethically acceptable control methods for phase II trials (16, 17). In addition, the use of crossover designs can also reduce the potential exposure of subjects to inactive therapies (18). At a minimum, the use of explicitly described historical controls may allow for improved identification of bias and limitations of generalizability. As one author writing on the development of pharmaceuticals for congestive heart failure wrote, "Even the most accomplished physicians cannot predict the natural course of patients and thus cannot interpret clinical responses in the absence of a control group" (19). Other alternative methods for controlling the effect of an intervention have been widely proposed and advocated for adoption in oncology trials and other specialties (20, 21). It is beyond the scope of this article to review the variety of designs available for phase II trials. However, our data do show remarkable monotony in the designs chosen for oncology trials as opposed to those chosen in other specialties.

Our study also found that non-oncology trials were more likely to identify interventions that they claim warranted advancement in the development process or additional investigation. Explanations for this discrepancy may well be due to publication bias, and that journals in medical specialties other than oncology are not as likely to publish phase II trials without findings of significant efficacy. Understanding whether publication bias accounts for these findings would require that we investigate the total number of trials conducted rather than just those that are published in the literature. Our survey does not allow for such a determination.

It is clear that a meta-analysis such as ours is too blunt an instrument to tease out many details about trends in phase II design. Among the obvious limitations is that it represents only 1 year's worth of abstracts, and more time points may be necessary to define whether the identified trends are durable. In addition, there is limitation in using only the abstracts as the source data because many abstracts may not suitably record the details of the trial and the trial design. There may be cases where classification decisions were made based on information in the abstracts that would have differed if the full published study had been examined. However, we believe that any bias in classification decision making was spread out among oncology and non-oncology trials. Overall, we felt that the benefit of surveying such a wide number of trials merited the technique. It is, of course, beyond the scope of an analysis like this one to prove that randomized trials are the most efficient or valuable method of testing drugs at this early phase of development. We do, however, think that this analysis illustrates the remarkable homogeneity of phase II testing strategies in oncology, as opposed to other medical and surgical subspecialties.

Phase II trials serve as a crucial filter point in the pathway of bringing a drug to market. If the trial design is inadequate, three major errors can happen: patients can be needlessly exposed to risk, resources can be wasted by initiating phase III trials on drugs that were poorly evaluated in phase II testing, and new interventions that may actually have utility can be considered failures. We would argue that the ideal phase II trial is a randomized, blinded, and controlled design, whether or not the intervention is expected to work via a cytotoxic or a cytostatic mechanism. We acknowledge that such an ideal may be difficult to achieve; however, it is still a worthy goal. There are at least several examples of well-designed phase II trials that have led to early and efficient phase III approval even in the absence of meeting traditional objective response rate outcomes. For example, a randomized discontinuation phase II trial of the kinase inhibitor sorafenib found only a 4% objective response rate, but nonetheless showed a failure-free survival advantage after randomization, which was later confirmed in a phase III trial (8, 22). The pivotal phase II trial of bevacizumab and 5-fluorouracil for metastatic colorectal cancer used a randomized, placebo-controlled dose-ranging design, and results garnered from that study led the way to successful phase III studies (23).

We see this survey as a starting point for investigations into whether our current paradigm of trial design in oncology is the optimal one for a specialty that needs to accommodate an increasing number of therapies with greater and greater heterogeneity of mechanisms. Finding ways to make the current process more efficient, while also ensuring that good drugs are not missed and that patients are protected, remains the paramount goal. Given the massive societal costs of pharmaceuticals (particularly for oncology), more efficient use of resources for drug development is critical.

	Footnotes

 
The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

Note: L.C. Michaelis' current address: Department of Medicine, Division of Hematology/Oncology, Loyola University Chicago, Chicago, IL.

¹ Tufts Center for the Study of Drug Development (homepage on the Internet). Boston: TCSDD (updated October 20, 2006; cited November 7, 2006). Backgrounder: How new drugs move through the development and approval process. Available from: http://csdd.tufts.edu/NewsEvents/RecentNews.asp?newsid=4.

² U.S. Government Printing Office-NARA (homepage on the Internet). FDA DoHaHS, Code of Federal Regulations. 21CFR312.21-Sec. 312.21 Phases of an investigation. pp. 63 2000. Available from: http://www.access.gpo.gov/cgi-bin/cfrassemble.cgi?title=200021.

Received 6/23/06; revised 10/12/06; accepted 11/16/06.

	References

Top Abstract Materials and Methods Results Discussion References

 

1. Michaelis LC, Ratain MJ. Measuring response in a post-RECIST world: from black and white to shades of grey. Nat Rev Cancer 2006;6:409–14.[CrossRef][Medline]
2. Berry DA. General keynote: clinical trial design. Gynecol Oncol 2003;88:S114–6.[CrossRef][Medline]
3. Mathieu MP, editor. PAREXEL's Pharmaceutical R&D Statistical Sourcebook 2004/2005. Waltham: PAREXEL International Corporation; 2004.
4. Booth B, Glassman R, Ma P. Oncology's trials. Nat Rev Drug Discov 2003;2:609–10.[CrossRef][Medline]
5. Zia MI, Siu LL, Pond GR, Chen EX. Comparison of outcomes of phase II studies and subsequent randomized control studies using identical chemotherapeutic regimens. J Clin Oncol 2005;23:6982–91.[Abstract/Free Full Text]
6. Hoekstra R, Verweij J, Eskens FA. Clinical trial design for target specific anticancer agents. Invest New Drugs 2003;21:243–50.[CrossRef][Medline]
7. Mariani L, Marubini E. Content and quality of currently published phase II cancer trials. J Clin Oncol 2000;18:429–36.[Abstract/Free Full Text]
8. Ratain MJ. Phase II oncology trials: let's be positive. Clin Cancer Res 2005;11:5661–2.[Free Full Text]
9. Rosner GL, Stadler W, Ratain MJ. Randomized discontinuation design: application to cytostatic antineoplastic agents. J Clin Oncol 2002;20:4478–84.[Abstract/Free Full Text]
10. Stadler W. New trial designs to assess antitumor and antiproliferative agents in prostate cancer. Invest New Drugs 2002;20:201–8.[CrossRef][Medline]
11. U S. Department of Health and Human Services, Food and Drug Administration. General considerations for the clinical evaluation of drugs, HEW Publication No. (FDA) 77–3040. Rockville (MD): Center for Drug Evaluation and Research; 1977.
12. Simon R. Optimal two-stage designs for phase II clinical trials. Control Clin Trials 1989;10:1–10.[Medline]
13. Ratain MJ, Mick R, Schilsky RL, Siegler M. Statistical and ethical issues in the design and conduct of phase I and II clinical trials of new anticancer agents. J Natl Cancer Inst 1993;85:1637–43.[Free Full Text]
14. U S. Department of Health and Human Services, Food and Drug Administration. Guidance for industry: clinical development programs for drugs, devices, and biological products for the treatment of rheumatoid arthritis (RA). Rockville (MD): Office of Training and Communications, Division of Communications Management, Drug Information Branch; 1999.
15. Hellman S, Hellman DS. Of mice but not men. Problems of the randomized clinical trial. N Engl J Med 1991;324:1585–9.[Medline]
16. Simon R, Wittes RE, Ellenberg SS. Randomized phase II clinical trials. Cancer Treat Rep 1985;69:1375–81.[Medline]
17. Scher HI, Heller G. Picking the winners in a sea of plenty. Clin Cancer Res 2002;8:400–4.[Abstract/Free Full Text]
18. Daugherty CK, Ratain MJ, Emanuel EZ, Farrell AT. Research, ethical, and regulatory perspectives regarding the use of placebos for terminally ill patients with cancer. In: Perry MC, editor. American Society of Clinical Oncology education book. Alexandria (VA): American Society of Clinical Oncology; 2006. p. 165–73.
19. Packer M. Current perspectives on the design of phase II trials of new drugs for the treatment of heart failure. Am Heart J 2000;139:S202–6.[CrossRef][Medline]
20. Lee JJ, Feng L. Randomized phase II designs in cancer clinical trials: current status and future directions. J Clin Oncol 2005;23:4450–7.[Abstract/Free Full Text]
21. DeMets DL. Design of phase II trials in congestive heart failure. Am Heart J 2000;139:S207–10.[CrossRef][Medline]
22. Ratain MJ, Eisen T, Stadler WM, et al. Phase II placebo-controlled randomized discontinuation trial of sorafenib in patients with metastatic renal cell carcinoma. J Clin Oncol 2006;24:2505–12.[Abstract/Free Full Text]
23. Kabbinavar F, Hurwitz HI, Fehrenbacher L, et al. Phase II, randomized trial comparing bevacizumab plus fluorouracil (FU)/leucovorin (LV) with FU/LV alone in patients with metastatic colorectal cancer. J Clin Oncol 2003;21:60–5.[Abstract/Free Full Text]

This article has been cited by other articles:

				J. H. Doroshow Commentary: Publishing Cancer Clinical Trial Results: A Scientific and Ethical Imperative Oncologist, September 1, 2008; 13(9): 930 - 932. [Full Text] [PDF]

				S. Kopetz, M. Overman, D. Z. Chang, K. Y. Glover, I. Shureiqi, R. A. Wolff, J. L. Abbruzzese, and C. Eng Systematic Survey of Therapeutic Trials for Metastatic Colorectal Cancer: Room for Improvement in the Critical Pathway J. Clin. Oncol., April 20, 2008; 26(12): 2000 - 2005. [Abstract] [Full Text] [PDF]

				W. M. Stadler Tumor Burden Endpoints and Phase II Clinical Trial Design ASCO Educational Book, January 1, 2008; 2008(1): 89 - 93. [Abstract] [Full Text] [PDF]

				M. J. Ratain and R. H. Glassman Biomarkers in Phase I Oncology Trials: Signal, Noise, or Expensive Distraction? Clin. Cancer Res., November 15, 2007; 13(22): 6545 - 6548. [Full Text] [PDF]

				B. A. Chabner and M. J. Ratain Clarification Regarding "Phase II Trials Published in 2002: A Cross-Specialty Comparison Showing Significant Design Differences between Oncology Trials and Other Medical Specialties" and the Accompanying Commentary, "Phase II Cancer Trials: Out of Control?" Clin. Cancer Res., November 1, 2007; 13(21): 6540 - 6540. [Full Text] [PDF]

				Design of Phase II Oncology Trials Journal Watch Oncology and Hematology, May 14, 2007; 2007(514): 6 - 6. [Full Text]

				B. Chabner Phase II Cancer Trials: Out of Control? Clin. Cancer Res., April 15, 2007; 13(8): 2307 - 2308. [Full Text] [PDF]

				M. R. Tomblyn and J. D. Rizzo Are There Circumstances in Which Phase 2 Study Results Should Be Practice-Changing? Hematology, January 1, 2007; 2007(1): 489 - 492. [Abstract] [Full Text] [PDF]

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Michaelis, L. C. Articles by Ratain, M. J. Search for Related Content PubMed PubMed Citation Articles by Michaelis, L. C. Articles by Ratain, M. J. Related Collections Related Article