Reproducibly Run Various Kinds of Cross-Validation

Enables doing classification schemes such as ordinary 10-fold, 100 permutations 5-fold, and leave one out cross-validation. Processing in parallel is possible by leveraging the package BiocParallel.

Usage

# S4 method for matrix
runTests(measurements, outcome, ...)

# S4 method for DataFrame
runTests(
  measurements,
  outcome,
  crossValParams = CrossValParams(),
  modellingParams = ModellingParams(),
  characteristics = S4Vectors::DataFrame(),
  ...,
  verbose = 1
)

# S4 method for MultiAssayExperiment
runTests(measurements, outcome, ...)

Arguments

measurements: Either a matrix, DataFrame or MultiAssayExperiment containing all of the data. For a matrix or DataFrame, the rows are samples, and the columns are features.
...: Variables not used by the matrix nor the MultiAssayExperiment method which are passed into and used by the DataFrame method or passed onwards to prepareData.
outcome: Either a factor vector of classes, a Surv object, or a character string, or vector of such strings, containing column name(s) of column(s) containing either classes or time and event information about survival. If measurements is a MultiAssayExperiment, the names of the column (class) or columns (survival) in the table extracted by colData(data) that contain(s) the samples' outcome to use for prediction. If column names of survival information, time must be in first column and event status in the second.
crossValParams: An object of class CrossValParams, specifying the kind of cross-validation to be done.
modellingParams: An object of class ModellingParams, specifying the class rebalancing, transformation (if any), feature selection (if any), training and prediction to be done on the data set.
characteristics: A DataFrame describing the characteristics of the classification used. First column must be named "charateristic" and second column must be named "value". Useful for automated plot annotation by plotting functions within this package. Transformation, selection and prediction functions provided by this package will cause the characteristics to be automatically determined and this can be left blank.
verbose: Default: 1. A number between 0 and 3 for the amount of progress messages to give. A higher number will produce more messages as more lower-level functions print messages.

Value

An object of class ClassifyResult.

Author

Dario Strbenac

Examples


  #if(require(sparsediscrim))
  #{
    data(asthma)
    
    CVparams <- CrossValParams(permutations = 5)
    tuneList <- list(nFeatures = seq(5, 25, 5), performanceType = "Balanced Error")
    selectParams <- SelectParams("t-test", tuneParams = tuneList)
    modellingParams <- ModellingParams(selectParams = selectParams)
    runTests(measurements, classes, CVparams, modellingParams,
             DataFrame(characteristic = c("Assay Name", "Classifier Name"),
                       value = c("Asthma", "Different Means"))
             )
#> Processing sample set 10.
#> Processing sample set 20.
#> An object of class 'ClassifyResult'.
#> Characteristics:
#>    characteristic                   value
#>        Assay Name                  Asthma
#>   Classifier Name         Different Means
#>    Selection Name     Difference in Means
#>  Cross-validation 5 Permutations, 5 Folds
#> Features: List of length 25 of feature identifiers.
#> Predictions: A data frame of 950 rows.
#> Performance Measures: None calculated yet.
  #}