Feature Selection with Random Search

Feature selection using Random Search Algorithm.

Source

Bergstra J, Bengio Y (2012). “Random Search for Hyper-Parameter Optimization.” Journal of Machine Learning Research, 13(10), 281–305. https://jmlr.csail.mit.edu/papers/v13/bergstra12a.html.

Details

The feature sets are randomly drawn. The sets are evaluated in batches of size batch_size. Larger batches mean we can parallelize more, smaller batches imply a more fine-grained checking of termination criteria.

Dictionary

This FSelector can be instantiated with the associated sugar function fs():

fs("random_search")

Control Parameters

max_features

integer(1)
Maximum number of features. By default, number of features in mlr3::Task.

batch_size

integer(1)
Maximum number of feature sets to try in a batch.

See also

Other FSelector: FSelector, mlr_fselectors, mlr_fselectors_design_points, mlr_fselectors_exhaustive_search, mlr_fselectors_genetic_search, mlr_fselectors_rfe, mlr_fselectors_rfecv, mlr_fselectors_sequential, mlr_fselectors_shadow_variable_search

Super classes

mlr3fselect::FSelector -> mlr3fselect::FSelectorBatch -> FSelectorBatchRandomSearch

Methods

Public methods

• FSelectorBatchRandomSearch$new()

• FSelectorBatchRandomSearch$clone()

Inherited methods

• mlr3fselect::FSelector$format()
• mlr3fselect::FSelector$help()
• mlr3fselect::FSelector$print()
• mlr3fselect::FSelectorBatch$optimize()

---

Method new()

Creates a new instance of this R6 class.

Usage

FSelectorBatchRandomSearch$new()

---

Method clone()

The objects of this class are cloneable with this method.

Usage

FSelectorBatchRandomSearch$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

Examples

# Feature Selection
# \donttest{

# retrieve task and load learner
task = tsk("penguins")
learner = lrn("classif.rpart")

# run feature selection on the Palmer Penguins data set
instance = fselect(
  fselector = fs("random_search"),
  task = task,
  learner = learner,
  resampling = rsmp("holdout"),
  measure = msr("classif.ce"),
  term_evals = 10
)

# best performing feature subset
instance$result
#>    bill_depth bill_length body_mass flipper_length island    sex   year
#>        <lgcl>      <lgcl>    <lgcl>         <lgcl> <lgcl> <lgcl> <lgcl>
#> 1:       TRUE        TRUE      TRUE           TRUE   TRUE  FALSE  FALSE
#>                                                  features n_features classif.ce
#>                                                    <list>      <int>      <num>
#> 1: bill_depth,bill_length,body_mass,flipper_length,island          5 0.08695652

# all evaluated feature subsets
as.data.table(instance$archive)
#>     bill_depth bill_length body_mass flipper_length island    sex   year
#>         <lgcl>      <lgcl>    <lgcl>         <lgcl> <lgcl> <lgcl> <lgcl>
#>  1:      FALSE       FALSE     FALSE          FALSE  FALSE   TRUE  FALSE
#>  2:       TRUE        TRUE      TRUE           TRUE   TRUE  FALSE  FALSE
#>  3:       TRUE        TRUE     FALSE           TRUE   TRUE   TRUE   TRUE
#>  4:       TRUE        TRUE      TRUE           TRUE   TRUE   TRUE   TRUE
#>  5:      FALSE       FALSE      TRUE           TRUE   TRUE   TRUE  FALSE
#>  6:      FALSE        TRUE      TRUE           TRUE   TRUE   TRUE  FALSE
#>  7:      FALSE       FALSE      TRUE          FALSE   TRUE   TRUE   TRUE
#>  8:       TRUE       FALSE      TRUE          FALSE  FALSE  FALSE  FALSE
#>  9:       TRUE       FALSE      TRUE           TRUE   TRUE   TRUE   TRUE
#> 10:       TRUE        TRUE      TRUE           TRUE   TRUE  FALSE   TRUE
#>     classif.ce runtime_learners           timestamp batch_nr warnings errors
#>          <num>            <num>              <POSc>    <int>    <int>  <int>
#>  1: 0.54782609            0.004 2025-06-06 07:46:36        1        0      0
#>  2: 0.08695652            0.005 2025-06-06 07:46:36        1        0      0
#>  3: 0.08695652            0.005 2025-06-06 07:46:36        1        0      0
#>  4: 0.08695652            0.006 2025-06-06 07:46:36        1        0      0
#>  5: 0.25217391            0.005 2025-06-06 07:46:36        1        0      0
#>  6: 0.08695652            0.005 2025-06-06 07:46:36        1        0      0
#>  7: 0.31304348            0.005 2025-06-06 07:46:36        1        0      0
#>  8: 0.26956522            0.004 2025-06-06 07:46:36        1        0      0
#>  9: 0.25217391            0.006 2025-06-06 07:46:36        1        0      0
#> 10: 0.08695652            0.005 2025-06-06 07:46:36        1        0      0
#>                                                           features n_features
#>                                                             <list>     <list>
#>  1:                                                            sex          1
#>  2:         bill_depth,bill_length,body_mass,flipper_length,island          5
#>  3:          bill_depth,bill_length,flipper_length,island,sex,year          6
#>  4: bill_depth,bill_length,body_mass,flipper_length,island,sex,...          7
#>  5:                            body_mass,flipper_length,island,sex          4
#>  6:                bill_length,body_mass,flipper_length,island,sex          5
#>  7:                                      body_mass,island,sex,year          4
#>  8:                                           bill_depth,body_mass          2
#>  9:            bill_depth,body_mass,flipper_length,island,sex,year          6
#> 10:    bill_depth,bill_length,body_mass,flipper_length,island,year          6
#>      resample_result
#>               <list>
#>  1: <ResampleResult>
#>  2: <ResampleResult>
#>  3: <ResampleResult>
#>  4: <ResampleResult>
#>  5: <ResampleResult>
#>  6: <ResampleResult>
#>  7: <ResampleResult>
#>  8: <ResampleResult>
#>  9: <ResampleResult>
#> 10: <ResampleResult>

# subset the task and fit the final model
task$select(instance$result_feature_set)
learner$train(task)
# }