Function for Nested Resampling

Source: R/fselect_nested.R

Function to conduct nested resampling.

fselect_nested(
  method,
  task,
  learner,
  inner_resampling,
  outer_resampling,
  measure,
  term_evals = NULL,
  term_time = NULL,
  ...
)

Arguments

method

(character(1))
Key to retrieve fselector from mlr_fselectors dictionary.
task

(mlr3::Task)
Task to operate on.
learner

(mlr3::Learner).
inner_resampling

(mlr3::Resampling)
Resampling used for the inner loop.
outer_resampling

mlr3::Resampling)
Resampling used for the outer loop.
measure

(mlr3::Measure)
Measure to optimize.
term_evals

(integer(1))
Number of allowed evaluations.
term_time

(integer(1))
Maximum allowed time in seconds.
...

(named list())
Named arguments to be set as parameters of the fselector.

Value

mlr3::ResampleResult

Examples

rr = fselect_nested(
  method = "random_search",
  task = tsk("pima"),
  learner = lrn("classif.rpart"),
  inner_resampling = rsmp ("holdout"),
  outer_resampling = rsmp("cv", folds = 2),
  measure = msr("classif.ce"),
  term_evals = 4)

# performance scores estimated on the outer resampling
rr$score()

#>                 task task_id             learner              learner_id
#> 1: <TaskClassif[47]>    pima <AutoFSelector[40]> classif.rpart.fselector
#> 2: <TaskClassif[47]>    pima <AutoFSelector[40]> classif.rpart.fselector
#>            resampling resampling_id iteration              prediction
#> 1: <ResamplingCV[19]>            cv         1 <PredictionClassif[19]>
#> 2: <ResamplingCV[19]>            cv         2 <PredictionClassif[19]>
#>    classif.ce
#> 1:  0.2343750
#> 2:  0.2552083

# unbiased performance of the final model trained on the full data set
rr$aggregate()

#> classif.ce 
#>  0.2447917