This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Dilts, D. M. Articles by Comis, R. Search for Related Content PubMed PubMed Citation Articles by Dilts, D. M. Articles by Comis, R.

Development of Clinical Trials in a Cooperative Group Setting: The Eastern Cooperative Oncology Group

Authors' Affiliations: ¹ Owen Graduate School of Management, and Engineering Management program and Center for Management Research in Healthcare, Vanderbilt University, Nashville, Tennessee; ² Vanderbilt-Ingram Cancer Center, Vanderbilt University, Nashville, Tennessee; ³ Center for Management Research in Healthcare, Vanderbilt University, Nashville, Tennessee; and ⁴ Eastern Cooperative Oncology Group, Philadelphia, Pennsylvania

Requests for reprints: David M. Dilts, Owen Graduate School of Management and Engineering Management Program, School of Engineering, Vanderbilt University, 401 21st Avenue South, Nashville, TN 37203. Phone: 615-322-2259; Fax: 615-322-7996; E-mail: David.dilts{at}vanderbilt.edu.

	Abstract

Top Abstract Materials and Methods Results Discussion Disclosure of Potential... References

 
Purpose: We examine the processes and document the calendar time required to activate phase II and III clinical trials by an oncology group: the Eastern Cooperative Oncology Group (ECOG).

Methods: Setup steps were documented by (a) interviewing ECOG headquarters and statistical center staff, and committee chairs, (b) reviewing standard operating procedure manuals, and (c) inspecting study records, documents, and e-mails to identify additional steps. Calendar time was collected for each major process for each study in this set.

Results: Twenty-eight phase III studies were activated by ECOG during the January 2000 to July 2006 study period. We examined a sample from 16 of those studies in detail. More than 481 distinct processes were required for study activation: 420 working steps, 61 major decision points, 26 processing loops, and 13 stopping points. Median calendar days to activate a trial in the phase III subset was 783 days (range, 285-1,542 days) from executive approval and 808 days (range, 435-1,604 days) from initial conception of the study. Data were collected for all phase II and phase III trials activated and completed during this time period (n = 52) for which development time represented 43.9% and 54.1% of the total trial time, respectively.

Conclusion: The steps required to develop and activate a clinical trial may require as much or more time than the actual completion of a trial. The data shows that to improve the activation process, research should to be directed toward streamlining both internal and external groups and processes.

There are four types of administrative or setup barriers to activation processes at cooperative oncology groups: procedural, structural, infrastructural, and synchronicity. The first three of these types of barriers are common among cooperative oncology groups, comprehensive cancer centers, and community oncology practices (2). Although synchronicity is an issue for any clinical trial, it is exacerbated in cooperative groups as a result of dealing with multiple external agencies (1).

Procedural barriers are policies (formal or informal) that are inherent to the process, or a series of steps necessary to activate a study, which may inhibit other problem-solving actions. For example, a procedural delay for a cooperative oncology group occurs when, after a concept has been internally approved, the concept must then be reviewed by an outside agency before investigators at ECOG can continue to develop it. Our findings show that such procedural barriers occur throughout the activation process and do so both within ECOG and at the external interface with other participants in the activation process such as the National Cancer Institute (NCI, NIH, Bethesda, MD), the NCI Cancer Therapy Evaluation Program (CTEP), the NCI Centralized Institutional Review Board (CIRB), the U.S. Food and Drug Administration (FDA), pharmaceutical firms, and trial study chairs who volunteer their services while typically holding positions at universities and academic medical centers.

The second type of barrier, structural (i.e., barriers created by the design of the organization or the interface between organizations), typically arises whenever different participants in the process follow a different ordering of steps, often leading to miscommunication and misunderstanding. This barrier is represented by the circular mismatch loop that arises when one participant sends information to another, who then replies back to the sender—a correspondence that must take place before the process can proceed. For example, study chairs and cooperative groups run on different timelines with respect to responding to changes. This can lead to a circular, often protracted, situation: one group cannot collect the required information until it approves a condition, but it will not approve the condition without the required information.

Infrastructural barriers are those barriers that arise from the design of the underlying system and the interconnection between various system aspects. Consider the CTEP infrastructure as an example: protocol review, drug distribution, and NCI common data element and CIRB review are each a basic part of the required infrastructure (3, 4). Each of these aspects is individually productive but could cause delays in activation as studies wait for final approval from multiple parts of the system—especially because each aspect may function according to different timelines, support system, or oversight.

The final type of barrier, synchronicity, is characterized as the need for compilation of various components of a concept or a protocol before it can be submitted to other participants in the process. In other words, synchronicity results from a myriad of steps having to come together prior to a study advancing toward activation. At ECOG, for example, protocol assembly, financial affairs, forms/data management, and drug acquisition/distribution—each handled by a different functional group—must all be completed and compiled or synchronized before a study can be activated.

Although these are the four main types of hurdles faced by cooperative oncology groups, each can be exacerbated by simultaneous occurrence with other barriers. This kind of interactive effect can be observed in the overlap of administrative and synchronicity barriers which occur whenever CTEP, pharmaceutical company, FDA, and CIRB reviews are completed sequentially. External reviews may be fragmented, and sometimes, contradictory because they collect information at various times from various group or individuals. Thus, these represent both timing and synchronicity barriers.

	Materials and Methods

Top Abstract Materials and Methods Results Discussion Disclosure of Potential... References

 
Study settings and time frame. The setting for this research is the ECOG, a national clinical research group sponsored by the NCI, with its Group Chair's office located in Philadelphia, PA and the Coordinating Center located in Boston, MA. ECOG was established in 1955 as one of the first cooperative groups launched to perform multicenter cancer clinical trials. A cooperative group is a network of researchers, physicians, and health care professionals at public and private institutions across the country that are members of the group. Funded primarily by the NCI, ECOG has evolved from a five-member consortium of institutions on the East Coast to one of the largest clinical cancer research organizations in the United States with almost 6,000 physicians, nurses, pharmacists, statisticians, and clinical research associates from the United States, Canada, Australia, Peru, Israel, and South Africa. Institutional members include universities, medical centers, Community Clinical Oncology Programs, and Cooperative Group Outreach Programs. These institutions work toward the common goal of controlling, effectively treating, and ultimately curing cancer (5).

A sample of ECOG phase III studies activated in a 6-year period (January 2000 to July 2006) was investigated (n = 16). Sixteen studies were selected to provide a sufficiently broad set of studies. Phase III studies were chosen because they are the most complex, i.e., they contain all the relevant steps in study activation needed for all types of treatment trials.

Because studies activated in the year 2000 may have begun the process much earlier, the actual scope of time investigated was a period of 8 years (February 1998 to July 2006) to acquire all relevant timing data. All trials investigated were therapeutic studies; intergroup studies were excluded so as to concentrate on identifying where ECOG could reduce internal process steps and times.

To study development versus operation time in a clinical trial, data were collected for all ECOG phase II (n = 37) and phase III (n = 16) therapeutic studies activated in the study period. One of the 16 phase III studies was still open to accrual during the study period and was excluded from this part of the analysis.

Process mapping. There are two primary aspects of the Dilts and Sandler (2) method used: process mapping and timing analysis. Process mapping consists of a graphical representation of the flow of inputs, resources, steps, and processes (both work and decision) required to create an output—in this case, to activate a study (6).

To generate the process map, a team of experts from the Center for Management Research in Healthcare was engaged. Individuals from the Vanderbilt-Ingram Cancer Center (Nashville, TN), the Vanderbilt University (Nashville) School of Engineering, and the Vanderbilt University Owen Graduate School of Management comprise the Center for Management Research in Healthcare. Data were collected by means of initial on-site personnel interviews, additional follow-up e-mail correspondence, and a series of clarification teleconferences with members of ECOG. Using these information sources, the team developed initial process flows needed for all major activities required to activate a phase III study, including concept development, protocol development, forms and data management, financial affairs (if applicable), and regulatory affairs. Additional on-site clarification interview sessions were conducted prior to the final results presentation to verify the flows with relevant participants and to validate the steps shown by consulting individual study documentation.

Timing analysis. For this aspect of the method, the calendar time needed for each of the major processes required to activate a study was collected. It is important to note that work time (i.e., the amount of effort in writing a study) was not evaluated, but that attention was focused only on calendar time (i.e., the days from initiation to completion of a process step). These data were contained in various forms, including written documentation, formal electronic formats, and informal e-mail communication.

Detailed analysis of two studies. In addition to process mapping and timing data analysis for the full sample of studies, detailed analyses were conducted for two studies that represented relatively "fast" and "slow" activation processes, respectively. For both studies, the duration, in calendar days, of various elements of the system was identified and tracked (e.g., when and for how many days the study was with the study chair, with ECOG, with CTEP, and with CIRB respectively). For the slower study, a detailed process map was created explicitly showing the number of process loops and timing data for each step of the process. A similar analytic tool was created for the faster of the two case studies, showing the time between each step as well as the overall time taken to activate.

	Results

Top Abstract Materials and Methods Results Discussion Disclosure of Potential... References

 
Process mapping. Process maps can be created by (a) listing all process steps on one large complete diagram, or (b) creating a hierarchy of diagrams, each of which provides additional detail from a master, or level 0, diagram (2, 7). Because of its size (605 x 60 inches in 8-point font), the complete process diagram is impossible to reproduce here. Figure 1 presents an overview of a level 0 diagram.⁵

View larger version (17K): [in this window] [in a new window] [Download PPT slide]   Fig. 1. Level 0 process flow map for activating a phase III study at ECOG. Days are the median calendar days from receipt to acceptance by the process. Concept development refers to the time the concept was developed by the principal investigator when data was available (n = 5). Brown boxes are joint efforts between ECOG and external governmental agencies. Grant development is necessary only if additional funding was requested from a pharmaceutical firm; regulatory affairs development and FDA are necessary only if ECOG is involved with IND filing process. Although initial FDA review is within 30 to 45 d, studies required multiple loops to attain final FDA approval.

1. Concept development by members both internal and external to ECOG
   
2. ECOG executive review
   
3. Concept consensus review by an external agency
   
4. Protocol development
   
5. Protocol consensus review by an outside agency
   
6. Forms development and data management development
   
7. Common data element compliance review by an outside agency
   
8. Grant development, which includes internal and external groups to ECOG (i.e., the NCI and/or pharmaceutical firms)
   
9. Regulatory affairs
   
10. FDA review
    
11. CIRB review
    
12. Final external approval
    
13. Study activation activities
    

At the initial stages, concept development and executive review, a study chair presents a potential phase III concept study to an ECOG disease or modality committee chair. It is important to note that these individuals, although members of ECOG, are volunteers in the process. Each concept is discussed with the committee chair and, if time is sufficient, the idea is presented before the disease or modality committee. If approved by the committee, the concept is presented for ECOG executive committee concept review. If the executive committee sends the concept back for revision and resubmission, the first of 26 processing loops occurs. A loop is any time the concept or protocol is returned to a previous process step for change. If the concept is approved by the executive committee, it is sent to CTEP for concept review and approval. Frequently, this review process may require an additional loop in the process.

Once approved by the executive committee, the ECOG study team begins work on the study protocol. Protocol development is the most resource-intensive and time-intensive aspect of the activation process, involving multiple process steps which include interaction between the study chair, ECOG, and CTEP. The process of protocol development begins with efforts by the study chair and ECOG protocol developers to craft the details of the protocol from the concept. Once a protocol is sufficiently advanced, it is sent to CTEP for a protocol consensus review. As with concept review, this may initiate a series of process loops between the study chair, ECOG, and CTEP due to requested changes. Such changes can be minor (addressed in a local process loop) or major (requiring the study to return to an earlier step in the process, potentially returning to additional executive and concept review). This aspect of the overall process continues until the protocol is conditionally approved by CTEP pending CIRB review.

Concurrent to the protocol development and review processes are the steps required for forms and data management. These processes begin within ECOG after the initial protocol has been sufficiently developed so that forms can be created. Once the forms are completed within ECOG, they are submitted for NCI common data element compliance review. Process loops also occur during these processes.

If additional funding is needed for a particular study, ECOG begins grant development or contract negotiations with a potential sponsor immediately upon concept acceptance by the executive committee. It is important to note that, whereas grant/contract development seems to take extensive time, in most cases, the final contract must await the final study budget, which is dependent on the final study protocol. Hence, the majority of effort may be completed early on the budget and contract but the "clock" continues to "tick" because the final statement of work and contract has not been agreed upon and signed.

The final major process flow that runs concurrently with protocol development is regulatory affairs and drug acquisition (if necessary). This identifies the source of the drug to be studied, establishes a drug distribution plan, and determines Investigational New Drug requirements. Once these aspects are completed, the Investigational New Drug application (if required) is sent for FDA review.

Once the protocol has been accepted, the CIRB commences its review. Again, this process may require looping to early steps in the process. With the final CIRB approval, there is one final CTEP approval (the "final-final" review) before the last activation activities can take place.

One often overlooked feature of the process and timing requirements of these steps is the need for synchronicity. This is the idea that various elements of the protocol need to merge at the proper time for the study to be approved by internal and external organizations. For illustration, compare a similar phenomenon in the automotive industry: the correct engine must be matched with the correct body in the assembly line before any additional work can be done on the vehicle. If the correct engine is not ready, the assembly process stops until the engine is ready and the line restarts. In study activation, synchronicity is required between protocol development and forms/data management before the study can be sent for CIRB review. Additionally, all processes (i.e., CTEP, CIRB, FDA, and sponsor negotiations) must be synchronized prior to activation. This means that there are several potential areas of developmental lag, created when one group or step in the process is held up because a prior step has not yet been completed.

Although informative for overall flow of processes, the level 0 process map does not indicate the number of steps, decision points, processing loops, or stopping points. The ECOG phase III study trial process requires at least 481 processing steps composed of:

• at least 420 working steps
  
• 61 major decision points
  
• 26 processing loops
  
• 13 stopping points (9 external to ECOG and 4 internal)
  

As a loop may return a protocol for reprocessing, the 481 steps identified represent the minimum set of steps possible. In actuality, due to the high likelihood of looping, the actual number for a study can be considerably greater. Examples of these loops are discussed below (see Detailed analysis of two studies).

Timing analysis. During the comprehensive chart review, timing data were collected on 16 sample phase III studies. To ensure the accuracy of data collected, data were checked for completeness and consistency by cross-referencing electronic records with paper records. Those studies that had missing dates were not included in summary timing analysis shown in Table 1 .

For further analysis, we expanded the timing research to include additional sets of timing data. First, the time from executive committee approval to activation for all phase III therapeutic studies with concept submission and activation dates from January 2000 to July 2006 was collected. For this data, the median time was 755.5 days (n = 28; range, 330-2,801). This helps assure us that the studies sampled in detail are representative.

To investigate the question of how much time concept and protocol development requires of the total time to complete an oncology clinical trial, data were collected for all ECOG phase II and III therapeutic studies activated between January 2000 and July 2006, and closed to accrual by July 2006. A total of 52 studies were identified. For phase II studies, 43.9% of total study time (development plus operation) was spent in activities required to activate the study (n = 37, development median time = 562 days, operation median time = 717 days; see Fig. 2 ). For phase III trials, development time exceed operation time and required 54.1% of the total study time (n = 15, development median = 877, operation median = 743). With development time requiring such a large percentage of the total study time, further investigation is required in the important area of research into the development of oncology clinical trials.

View larger version (24K): [in this window] [in a new window] [Download PPT slide]   Fig. 2. Median number of development and study open days for phase II and phase III ECOG clinical trials opened for accrual and closed from January 2000 to July 2006.

View larger version (10K): [in this window] [in a new window] [Download PPT slide]   Fig. 3. Median days for development of therapeutic studies activated from January 2000 to July 2006. A, phase II studies (n = 71); B, phase III studies (n = 28).

1. Concept development and review: the time from ECOG's initial receipt of a concept to CTEP concept approval.
   
2. Initial protocol development: the time from CTEP concept approval to initial protocol submission to CTEP (it should be noted that the actual total protocol development time includes this time plus time spent working on the protocol from executive approval, and is therefore longer than indicated here).
   
3. Protocol review: the time from initial protocol submission to CTEP to CTEP approval pending CIRB approval.
   
4. CIRB review: the time from CTEP approval pending CIRB to CIRB approval of the protocol.
   
5. CTEP final review and study activation: time from CIRB approval of the protocol to study activation.
   
6. Total time is the calendar days from ECOG's initial receipt of a concept to study activation.
   

The faster of the two studies required a total time of 599 calendar days from receipt of concept to study activation (see Table 2 ). Within that time period, there were a total of eight reviews conducted.

1. Concept development and review: 281 days, one executive, one initial CTEP, and one revision review.
   
2. Initial protocol development: 71 days (total actual protocol development time was 331 for this study), no revisions.
   
3. Protocol review: 62 days, one initial and one revision review.
   
4. CIRB review: 133 days, one initial and one revision review.
   
5. CTEP final review and study activation: 52 days, one review.
   

The slower study required 17 reviews and a calendar time of 975 days from executive committee receipt to study activation.

1. Concept development and review: 206 days, one executive, one initial CTEP, and one revision review.
   
2. Initial protocol development: 189 days (total actual protocol development time was 339 for this study), no revisions.
   
3. Protocol review: 84 days, one initial and one revision review.
   
4. CIRB review: 494 days, one initial and four revision reviews.
   
5. CTEP final review and study activation: 2 days, one review.
   

Of the 17 reviews, 6 were triggered by preactivation amendments. Broken down by individual entity or organization, study chair time was 171 days, ECOG time was 399 days (not counting additional time spent on protocol development), CTEP time was 282 days, and CIRB time was 123 days.

Clearly, one significant difference between the slow and the fast study is the number of revisions required (8 versus 17). Again, it should be mentioned that these are calendar days and not actual working days. It should also be noted that in some instances, responsibility for longer time includes the initiator of an activity. In other words, it may take one entity or agency only 3 days to make a decision that another entity or agency must engage in an activity that takes 60 days to complete.

	Discussion

Top Abstract Materials and Methods Results Discussion Disclosure of Potential... References

 
Our focus is on assessing the barriers that can impede the activation of a clinical trial study at a major NCI-sponsored cooperative oncology group, ECOG. Given that, with more than 481 process steps, it is clear that the process for activating a study at the ECOG is highly complex and involves input from multiple individuals and agencies. As shown in the detailed flow, a study must pass through multiple hands, multiple times before gaining final acceptance for activation. As has been shown in numerous industries, the more frequently work is handed-off, the greater the likelihood for error and adding additional inspection points only exacerbates the problem.

The results presented here are consistent with research which analyzed the development process of another NCI-sponsored cooperative group, the CALGB trials (1). Table 3A presents various process count information concerning work steps, decision points, processing loops, and stopping points for ECOG and CALGB, respectively. What is most important to clinical research is time to study activation; comparison data for time to activate phase III clinical trials at ECOG and CALGB are shown in Table 3B. It is important to note that these data were from a sample of phase III studies opened by each group that had complete initiation data. Actual study development time is highly variable as seen in the broad range between the minimum and maximum time for both cooperative groups.

One of the most striking findings in our current analysis relates to the discovery that the development time equals or exceeds the time required to complete the accrual of patients onto a study. Accordingly, the issues raised here must be addressed in order to speed the time to the decision-making point on the benefits, or lack thereof, of a specific clinical trial—by all parties involved in the process.

Our findings indicate that a multipronged approach is necessary to decrease calendar time for activating phase III trials. First, there are internal process changes that should occur within the cooperative groups. With the all the processes identified in a process map, it is possible to determine those steps that do not add value to the resulting protocol. Such steps should be eliminated. Second, it is important to establish study selection guidelines that include study operational complexity in addition to scientific merit. Third, it is important to acknowledge that even simple changes to a concept or protocol may require an extensive looping process thus greatly adding to time to activation.

It is important to note that improving processes solely within cooperative groups, without addressing their need to interface with external organizations, may result in a minimal effect on the time to activate a phase III study. Not only is it important to streamline the intramural processes at cooperative oncology groups, it is also essential that interagency processes be made more efficient. Currently, we are in the process of investigating such interagency interactions.

	Disclosure of Potential Conflicts of Interest

Top Abstract Materials and Methods Results Discussion Disclosure of Potential... References

 
No potential conflicts of interest were disclosed.

	Footnotes

 
Grant support: NIH grant 3U10-CA21115-32.

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

⁵ The complete process map is available online at http://www.cmrhc.org/processmaps.htm.

⁶ Two studies were excluded due to incomplete data.

Received 12/ 4/07; revised 2/20/08; accepted 2/27/08.

	References

Top Abstract Materials and Methods Results Discussion Disclosure of Potential... References

 

1. Dilts DM, Sandler AB, Baker M, et al. Processes to activate phase III clinical trials in a Cooperative Oncology Group: the case of Cancer and Leukemia Group B. J Clin Oncol 2006;24:4553–7.[Abstract/Free Full Text]
2. Dilts DM, Sandler AB. Invisible barriers to clinical trials: the impact of structural, infrastructural, and procedural barriers to opening oncology clinical trials. J Clin Oncol 2006;24:4545–52.[Abstract/Free Full Text]
3. Fukuda H. Infrastructure of cancer clinical trial cooperative groups in western countries. Gan To Kagaku Ryoho 2000;27:1144–51.[Medline]
4. National Cancer Institute. Cancer Therapy Evaluation Program: NCI Clinical Trials Cooperative Group program guidelines. Bethesda (MD): Division of Cancer Treatment and Diagnosis, National Cancer Institute; 2005.
5. Eastern Cooperative Oncology Group. ECOG homepage. [Available from: http://ecog.dfci.harvard.edu/general/intro.html.
6. Harrington HJ. Business process improvement: McGraw-Hill New York, NY; 1991.
7. Gane CP, Sarson T. Structured systems analysis: tools and techniques. Prentice Hall Professional Technical Reference; 1979.
8. Clinical Cancer Advances 2005. Major research advances in cancer treatment, prevention, and screening—a report from the American Society of Clinical Oncology. J Clin Oncol 2006;24:190–205.[Abstract/Free Full Text]
9. Clinical Cancer Advances 2006. Major research advances in cancer treatment, prevention, and screening—a report from the American Society of Clinical Oncology. J Clin Oncol 2007;25:146–62.[Abstract/Free Full Text]

This article has been cited by other articles:

				D. M. Dilts, A. B. Sandler, S. K. Cheng, J. S. Crites, L. B. Ferranti, A. Y. Wu, S. Finnigan, S. Friedman, M. Mooney, and J. Abrams Steps and Time to Process Clinical Trials at the Cancer Therapy Evaluation Program J. Clin. Oncol., April 10, 2009; 27(11): 1761 - 1766. [Abstract] [Full Text] [PDF]

				D. P. Steensma The Ordinary Miracle of Cancer Clinical Trials J. Clin. Oncol., April 10, 2009; 27(11): 1737 - 1739. [Full Text] [PDF]

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Dilts, D. M. Articles by Comis, R. Search for Related Content PubMed PubMed Citation Articles by Dilts, D. M. Articles by Comis, R.