Function to conduct nested resampling.
Usage
fselect_nested(
method,
task,
learner,
inner_resampling,
outer_resampling,
measure,
term_evals = NULL,
term_time = NULL,
...
)
Arguments
- method
(
character(1)
)
Key to retrieve fselector from mlr_fselectors dictionary.- task
(mlr3::Task)
Task to operate on.- learner
(mlr3::Learner)
Learner to optimize the feature subset for.- inner_resampling
(mlr3::Resampling)
Resampling used for the inner loop.- outer_resampling
mlr3::Resampling)
Resampling used for the outer loop.- measure
(mlr3::Measure)
Measure to optimize. IfNULL
, default measure is used.- term_evals
(
integer(1)
)
Number of allowed evaluations.- term_time
(
integer(1)
)
Maximum allowed time in seconds.- ...
(named
list()
)
Named arguments to be set as parameters of the fselector.
Examples
# Nested resampling on Palmer Penguins data set
rr = fselect_nested(
method = "random_search",
task = tsk("penguins"),
learner = lrn("classif.rpart"),
inner_resampling = rsmp ("holdout"),
outer_resampling = rsmp("cv", folds = 2),
measure = msr("classif.ce"),
term_evals = 4)
# Performance scores estimated on the outer resampling
rr$score()
#> task task_id learner learner_id
#> 1: <TaskClassif[50]> penguins <AutoFSelector[46]> classif.rpart.fselector
#> 2: <TaskClassif[50]> penguins <AutoFSelector[46]> classif.rpart.fselector
#> resampling resampling_id iteration prediction
#> 1: <ResamplingCV[20]> cv 1 <PredictionClassif[20]>
#> 2: <ResamplingCV[20]> cv 2 <PredictionClassif[20]>
#> classif.ce
#> 1: 0.0755814
#> 2: 0.1569767
# Unbiased performance of the final model trained on the full data set
rr$aggregate()
#> classif.ce
#> 0.1162791