![](../logo.png)
Feature Selection with Random Search
Source:R/FSelectorBatchRandomSearch.R
mlr_fselectors_random_search.Rd
Feature selection using Random Search Algorithm.
Source
Bergstra J, Bengio Y (2012). “Random Search for Hyper-Parameter Optimization.” Journal of Machine Learning Research, 13(10), 281–305. https://jmlr.csail.mit.edu/papers/v13/bergstra12a.html.
Details
The feature sets are randomly drawn.
The sets are evaluated in batches of size batch_size
.
Larger batches mean we can parallelize more, smaller batches imply a more fine-grained checking of termination criteria.
Control Parameters
max_features
integer(1)
Maximum number of features. By default, number of features in mlr3::Task.batch_size
integer(1)
Maximum number of feature sets to try in a batch.
Super classes
mlr3fselect::FSelector
-> mlr3fselect::FSelectorBatch
-> FSelectorBatchRandomSearch
Examples
# Feature Selection
# \donttest{
# retrieve task and load learner
task = tsk("penguins")
learner = lrn("classif.rpart")
# run feature selection on the Palmer Penguins data set
instance = fselect(
fselector = fs("random_search"),
task = task,
learner = learner,
resampling = rsmp("holdout"),
measure = msr("classif.ce"),
term_evals = 10
)
# best performing feature subset
instance$result
#> bill_depth bill_length body_mass flipper_length island sex year
#> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl>
#> 1: FALSE TRUE FALSE TRUE TRUE FALSE FALSE
#> features n_features classif.ce
#> <list> <int> <num>
#> 1: bill_length,flipper_length,island 3 0.08695652
# all evaluated feature subsets
as.data.table(instance$archive)
#> bill_depth bill_length body_mass flipper_length island sex year
#> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl>
#> 1: TRUE TRUE TRUE TRUE TRUE TRUE TRUE
#> 2: FALSE FALSE TRUE FALSE FALSE TRUE TRUE
#> 3: FALSE TRUE FALSE TRUE TRUE FALSE FALSE
#> 4: TRUE TRUE TRUE TRUE TRUE TRUE TRUE
#> 5: TRUE FALSE FALSE FALSE TRUE TRUE FALSE
#> 6: TRUE TRUE TRUE TRUE TRUE TRUE FALSE
#> 7: FALSE FALSE FALSE TRUE FALSE FALSE FALSE
#> 8: FALSE TRUE FALSE FALSE FALSE FALSE FALSE
#> 9: TRUE TRUE TRUE TRUE TRUE TRUE TRUE
#> 10: FALSE FALSE FALSE TRUE FALSE FALSE FALSE
#> classif.ce runtime_learners timestamp batch_nr warnings errors
#> <num> <num> <POSc> <int> <int> <int>
#> 1: 0.08695652 0.006 2024-07-24 12:01:15 1 0 0
#> 2: 0.27826087 0.004 2024-07-24 12:01:15 1 0 0
#> 3: 0.08695652 0.005 2024-07-24 12:01:15 1 0 0
#> 4: 0.08695652 0.005 2024-07-24 12:01:15 1 0 0
#> 5: 0.21739130 0.005 2024-07-24 12:01:15 1 0 0
#> 6: 0.08695652 0.005 2024-07-24 12:01:15 1 0 0
#> 7: 0.23478261 0.003 2024-07-24 12:01:15 1 0 0
#> 8: 0.23478261 0.004 2024-07-24 12:01:15 1 0 0
#> 9: 0.08695652 0.006 2024-07-24 12:01:15 1 0 0
#> 10: 0.23478261 0.004 2024-07-24 12:01:15 1 0 0
#> features n_features
#> <list> <list>
#> 1: bill_depth,bill_length,body_mass,flipper_length,island,sex,... 7
#> 2: body_mass,sex,year 3
#> 3: bill_length,flipper_length,island 3
#> 4: bill_depth,bill_length,body_mass,flipper_length,island,sex,... 7
#> 5: bill_depth,island,sex 3
#> 6: bill_depth,bill_length,body_mass,flipper_length,island,sex 6
#> 7: flipper_length 1
#> 8: bill_length 1
#> 9: bill_depth,bill_length,body_mass,flipper_length,island,sex,... 7
#> 10: flipper_length 1
#> resample_result
#> <list>
#> 1: <ResampleResult>
#> 2: <ResampleResult>
#> 3: <ResampleResult>
#> 4: <ResampleResult>
#> 5: <ResampleResult>
#> 6: <ResampleResult>
#> 7: <ResampleResult>
#> 8: <ResampleResult>
#> 9: <ResampleResult>
#> 10: <ResampleResult>
# subset the task and fit the final model
task$select(instance$result_feature_set)
learner$train(task)
# }