Validation Files for risksetROC Package

Set up/Background

Our goal here is to validate the accuracy of the risksetROC package. Specifically, we want to see if we have a "true" AUC trace at a given set of time of interest, how close is the estimated AUC trace if we use the risksetROC package to produce the AUCs. We here describe the background and compare the "true" and simulated AUC traces. For reference, we include the simulation file, two output files and two pdf files. The first output file contains AUCs at time points of interest averaged over simulations and produced using different method of estimation (MLE, Cox, LocalCox, Schoenfeld with order = 0 and order = 1). The second output file contains the integrated AUC (iAUC) traces over simulations. We also provide the link for the documentation file for risksetROC package.

We start with the setup for simulation. We simulate (log) survival times and marker values from a bivariate normal distribution with m = (0, 10)^T, r = -0.70, and variance of 1 (please note that time is in log scale if not mentioned otherwise). As per the convention, r is taken to be negative so that higher marker value indicates smaller event time. Censoring time is generated independently as normal, so that, about 40% of the observations are censored. Using this data we estimate ROC curves, AUCs at each of TargetTimes ranging from 7.5 to 12.5. Let n denote the sample size and Nsim denote the number of repetitions. For each of Nsim number of repetitions, we generate data for n individuals and repeat this procedure Nsim times, to get Nsim set of AUCs values at each of TargetTimes. Finally, we estimate the integrated AUC (iAUC) from the estimated AUC and the Kaplan-Meier estimate of the survival time.

As discussed in the paper by Heagerty and Zheng (2005), Biometrics, 61, 92 -- 105, the true positive (TP) and false positive based on a bivariate normal distribution of Marker(M) and Time(T) can be expressed as follows:

The "true" AUC trace and concordance index is obtained as follows. At each of the TargetTimes, we evaluated the TP and FP using the known parametric values and obtained AUC by numerically integrating the area under ROC curve. True concordance index is evaluated at the known r.

The AUC curves are estimates using five methods. We first use MLEs of the parameter along with the known parametric form of TP and FP to estimate the bivariate normal AUC. The iAUC is estimated using the MLE of the correlation coefficient of marker and event time. Estimation for MLEs is discussed below while the details of the rest of the methods (Cox, LocalCox, Schoenfeld with order = 0 and order = 1) are discussed in the aforementioned paper and the risksetROC package is used to estimate AUC traces at TargetTimes as well as iAUC.

The MLEs are estimated as follows. First note that we have observed marker values and (censored) event times. Observed marker mean and variance is used as MLEs for marker distribution. To estimate the expectation and variance from the censored event time we fit a parametric survival model for event time with normal error and marker as the only covariate:

Once the estimates of b₀, b₁ and error variance are obtained, we estimated the mean and variance of event time and correlation using the above relations. R package survival and function survreg was used to estimate the regression coefficients.

Two output files are out1.txt and out2.txt, the first contains the average AUC values at each TargetTimes (averaged over simulations) while the second contains the iAUC values over simulations. The plot associated with out1.txt is figure 1 while figure 2 plots the iAUC traces over simulations. Please see the documentation file for further details of the functions and more background.