1. Basic Receiver Operating Characteristic (ROC) Analysis

Authors: Bansal A, Morris DM, Longton G, Pepe MS, Janes H

The pcvsuite R package can be downloaded here:
[Windows (R 2.10.1)] [Apple OS (R 2.9.0)] [Unbuilt Source Directory]

The package contains four main functions:

• roccurve: Plots an estimate of the ROC curve for one or more diagnostic tests (or biomarkers). Confidence intervals can be displayed for the true positive fraction corresponding to a specified false positive fraction. Confidence intervals are calculated using the bootstrap. Accommodates covariate adjustment. [R help file]
• comproc: Calculates summary ROC indices for two tests along with confidence intervals for each and for the difference. A p-value for testing equality of the ROCs based on the summary indices is output. Accommodates covariate adjustment. [R help file]
• rocreg: Fits an ROC-GLM regression model. Accommodates covariate adjustment. [R help file]
• predcurve: Uses the risk distribution associated with a marker or model to evaluate marker utility when applied to the population. The classification performance is optionally included in an integrated display of predictiveness and classification measures. Alternate graphical outputs include CDFs and densities of the risk estimation. Support for nested models, and for testing differences between two models is provided. [Stata help file]

Additional Documentation:

• roccurve, comproc, rocreg: The following articles describe the functions and their options in more detail. The syntax in the articles is Stata-specific; however, the implementation of the functions in R remain the same. The arguments are also the same between R and Stata.
  • Pepe MS, Longton G, Janes H (2009). Estimation and comparison of receiver operating characteristic curves. The Stata Journal, 1: 1-16. [pdf]
  • Janes H, Longton G, Pepe MS (2009). Accommodating covariates in receiver operating characteristic analysis. The Stata Journal, 1: 17-39. [pdf]
• predcurve:
  • Pepe MS, Feng Z, Huang Y, Longton G, Prentice R, Thompson IM, Zheng Y (2008). Integrating the predictiveness of a marker with its performance as a classifier. American Journal of Epidemiology, 167(3): 362-368. [pdf]

2. ROC Analysis with Time-Dependent Data

Authors: Heagerty PJ, Bansal A

The meanrankROC R package can be downloaded here: [meanrankROC]

The package contains three main functions:

• dynamicTP: Accommodates updated marker values by using time-dependent data as above, and appropriately specifying start and stop times for intervals with updated marker values. dynamicTP(), along with nne_TPR() provides a smooth curve over time of sensitivity (or TPF) or ROC_t^I/D(p) for a fixed specificity 1-p.
• MeanRank: Accommodates updated marker values by using time-dependent data as above, and appropriately specifying start and stop times for intervals with updated marker values. MeanRank(), along with nne.Crossvalidate() provides a smooth curve of AUC^I/D(t) over time.
• dynamicIntegrateAUC: Estimates the c-index. Confidence intervals can be computed using bootstrapping

Example:

• See the following tutorial for an illustration of the meanrankROC package applied to assessing time-dependent discrimination accuracy of both baseline and time-varying markers.
  • Bansal A, Heagerty PJ (2018). A tutorial on evaluating the time-varying discrimination accuracy of survival models used in dynamic decision-making. To appear in Medical Decision Making.
  • Example code: [runme-example.R]