Calculating exact confidence interval for binomial proportion within each group using the Clopper-Pearson method

Clopper-Pearson confidence interval is commonly used in calculating the exact confidence interval for binomial proportion, incidence rate,... The confidence interval is calculated for a single group, therefore Clopper-Pearson method is not for calculating the confidence interval for the difference between two groups. 

In many oncology studies where there is no concurrent control group. For response rate, The exact confidence interval will be constructed (usually through Clopper-Pearson method) and then the lower limit of the 95% confidence interval is compared with the historical rate to determine if there is a treatment effect. 

Here are some examples that Clopper-Pearson method was used to calculate the exact confidence interval: 

Medical and statistical review for Venetoclax NDA:

"For the primary efficacy analyses, statistical significance was determined by a two-sided p value less than 0.05 (one-sided less than 0.025). The assessment of ORR was performed once 70 subjects in the main cohort completed the scheduled 36-week disease assessment, progressed prior to the 36-week disease assessment, discontinued study drug for any reason, or after all treated subjects discontinued venetoclax, whichever was earlier. The ORR for venetoclax was tested to reject the null hypothesis of 40%. If the null hypothesis is rejected and the ORR is higher than 40%, then venetoclax has been shown to have an ORR significantly higher than 40%. The ninety-five percent (95%) confidence interval for ORR was based on binomial distribution (Clopper-Pearson exact method). "

Motzer et al (2015) Nivolumab versus Everolimus in Advanced Renal-Cell Carcinoma

"If superiority with regard to the primary end point was demonstrated, a hierarchical statistical testing procedure was followed for the objective response rate (estimated along with the exact 95% confidence interval with the use of the Clopper–Pearson method)"

Foster et al (2015) Sofosbuvir and Velpatasvir for HCV Genotype 2 and 3 Infection

"Point estimates and two-sided 95% exact confidence intervals that are based on the Clopper–Pearson method are provided for rates of sustained virologic response for all treatment groups, as well as selected sub-groups."

Cicardi et al (2010) Icatibant, a New Bradykinin-Receptor Antagonist, in Hereditary Angioedema

"Fisher’s exact test, with 95% confidence intervals calculated for each group by means of the Clopper–Pearson method, was used to compare the percentage of patients with clinically significant relief of the index symptom at 4 hours after the start of the study drug. Two-sided 95% confidence intervals for the difference in proportions were calculated with the use of the Anderson–Hauck correction."

According to SAS manual, the Clopper-Pearson confidence interval is described as below:

The confidence interval using Clopper-Pearson method can be easily calculated with SAS Proc Freq procedure. Alternatively, it can also be calculated directly using the formula or using R function. 

Using Venetoclax NDA as an example, the primary efficacy endpoint ORR (overall response rate) is calculated as 85 / 107 = 79.4. 95% confidence interval can be calculated using Clopper-Pearson method as following: 

Using SAS Proc Freq:  

With proc freq, we should get 95% confidence interval of 70.5 – 88.6.

data test2;

  input orr $ count @@;

datalines;

have 85

no 22

;

ods output BinomialCLs=test;

proc freq data=test2 order=data;

  weight count;

  tables orr/binomial(exact) alpha=0.05 ;

run;

            

Using formula:

data test;

  input n n1 alpha;

  phat = n1/n;

  fvalue1 = finv( (alpha/2), 2*n1, 2*(n-n1));

  fvalue2 = finv( (1-alpha/2), 2*(n1+1), 2*(n-n1));

  pL =  (1+   ((n-n1+1)/(n1*fvalue1) ))**(-1);

  pU =  (1+   ((n-n1)/((n1+1)*fvalue2) ))**(-1);

datalines;

107 85 0.05

;

proc print;

      run;

Using R: 

n=107
n1=85
alpha=0.05
f1=qf(1-alpha/2, 2*n1, 2*(n-n1+1), lower.tail=FALSE)
f2=qf(alpha/2, 2*(n1+1), 2*(n-n1), lower.tail=FALSE)
pl=(1+(n-n1+1)/(n1*f1))^(-1)
pu=(1+(n-n1)/((n1+1)*f2))^(-1)

f1
f2
pl
pu