Parameters for Cross-validation Specification — CrossValParams • ClassifyR

Collects and checks necessary parameters required for cross-validation by runTests.

Usage

CrossValParams(
  samplesSplits = c("Permute k-Fold", "Permute Percentage Split", "Leave-k-Out",
    "k-Fold"),
  permutations = 100,
  percentTest = 25,
  folds = 5,
  leave = 2,
  tuneMode = c("Resubstitution", "Nested CV", "none"),
  adaptiveResamplingDelta = NULL,
  parallelParams = bpparam()
)

Arguments

samplesSplits: Default: "Permute k-Fold". A character value specifying what kind of sample splitting to do.
permutations: Default: 100. Number of times to permute the data set before it is split into training and test sets. Only relevant if samplesSplits is either "Permute k-Fold" or "Permute Percentage Split".
percentTest: The percentage of the data set to assign to the test set, with the remainder of the samples belonging to the training set. Only relevant if samplesSplits is "Permute Percentage Split".
folds: The number of approximately equal-sized folds to partition the samples into. Only relevant if samplesSplits is "Permute k-Fold" or "k-Fold".
leave: The number of samples to generate all possible combination of and use as the test set. Only relevant if samplesSplits is "Leave-k-Out". If set to 1, it is the traditional leave-one-out cross-validation, sometimes written as LOOCV.
tuneMode: Default: Resubstitution. The scheme to use for selecting any tuning parameters.
adaptiveResamplingDelta: Default: NULL. If not null, adaptive resampling of training samples is performed and this number is the difference in consecutive iterations that the class probability or risk of all samples must change less than for the iterative process to stop. 0.01 was used in the original publication.
parallelParams: An instance of BiocParallelParam specifying the kind of parallelisation to use. Default is to use two cores less than the total number of cores the computer has, if it has four or more cores, otherwise one core, as is the default of bpparam. To make results fully reproducible, please choose a specific back-end depending on your operating system and also set RNGseed to a number.

Author

Dario Strbenac

Examples


  CrossValParams() # Default is 100 permutations and 5 folds of each.
#> An object of class "CrossValParams"
#> Slot "samplesSplits":
#> [1] "Permute k-Fold"
#> 
#> Slot "permutations":
#> [1] 100
#> 
#> Slot "percentTest":
#> NULL
#> 
#> Slot "folds":
#> [1] 5
#> 
#> Slot "leave":
#> NULL
#> 
#> Slot "tuneMode":
#> [1] "Resubstitution"
#> 
#> Slot "adaptiveResamplingDelta":
#> NULL
#> 
#> Slot "parallelParams":
#> class: MulticoreParam
#>   bpisup: FALSE; bpnworkers: 78; bptasks: 0; bpjobname: BPJOB
#>   bplog: FALSE; bpthreshold: INFO; bpstopOnError: TRUE
#>   bpRNGseed: ; bptimeout: NA; bpprogressbar: FALSE
#>   bpexportglobals: TRUE; bpexportvariables: FALSE; bpforceGC: FALSE
#>   bpfallback: TRUE
#>   bplogdir: NA
#>   bpresultdir: NA
#>   cluster type: FORK
#> 
  snow <- SnowParam(workers = 2, RNGseed = 999)
  CrossValParams("Leave-k-Out", leave = 2, parallelParams = snow)
#> An object of class "CrossValParams"
#> Slot "samplesSplits":
#> [1] "Leave-k-Out"
#> 
#> Slot "permutations":
#> NULL
#> 
#> Slot "percentTest":
#> NULL
#> 
#> Slot "folds":
#> NULL
#> 
#> Slot "leave":
#> [1] 2
#> 
#> Slot "tuneMode":
#> [1] "Resubstitution"
#> 
#> Slot "adaptiveResamplingDelta":
#> NULL
#> 
#> Slot "parallelParams":
#> class: SnowParam
#>   bpisup: FALSE; bpnworkers: 2; bptasks: 0; bpjobname: BPJOB
#>   bplog: FALSE; bpthreshold: INFO; bpstopOnError: TRUE
#>   bpRNGseed: 999; bptimeout: NA; bpprogressbar: FALSE
#>   bpexportglobals: TRUE; bpexportvariables: TRUE; bpforceGC: FALSE
#>   bpfallback: TRUE
#>   bplogdir: NA
#>   bpresultdir: NA
#>   cluster type: SOCK
#> 
  # Fully reproducible Leave-2-out cross-validation on 4 cores,
  # even if feature selection or classifier use random sampling.