Skip to contents

Feature selection using Random Search Algorithm.

Source

Bergstra J, Bengio Y (2012). “Random Search for Hyper-Parameter Optimization.” Journal of Machine Learning Research, 13(10), 281--305. https://jmlr.csail.mit.edu/papers/v13/bergstra12a.html.

Details

The feature sets are randomly drawn. The sets are evaluated in batches of size batch_size. Larger batches mean we can parallelize more, smaller batches imply a more fine-grained checking of termination criteria.

Dictionary

This FSelector can be instantiated with the associated sugar function fs():

fs("random_search")

Control Parameters

max_features

integer(1)
Maximum number of features. By default, number of features in mlr3::Task.

batch_size

integer(1)
Maximum number of feature sets to try in a batch.

Super class

mlr3fselect::FSelector -> FSelectorRandomSearch

Methods

Inherited methods


Method new()

Creates a new instance of this R6 class.

Usage


Method clone()

The objects of this class are cloneable with this method.

Usage

FSelectorRandomSearch$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

Examples

# Feature Selection
# \donttest{

# retrieve task and load learner
task = tsk("penguins")
learner = lrn("classif.rpart")

# run feature selection on the Palmer Penguins data set
instance = fselect(
  method = fs("random_search"),
  task = task,
  learner = learner,
  resampling = rsmp("holdout"),
  measure = msr("classif.ce"),
  term_evals = 10
)

# best performing feature subset
instance$result
#>    bill_depth bill_length body_mass flipper_length island  sex year
#> 1:       TRUE        TRUE      TRUE           TRUE   TRUE TRUE TRUE
#>                                                          features classif.ce
#> 1: bill_depth,bill_length,body_mass,flipper_length,island,sex,... 0.06956522

# all evaluated feature subsets
as.data.table(instance$archive)
#>     bill_depth bill_length body_mass flipper_length island   sex  year
#>  1:       TRUE        TRUE      TRUE           TRUE   TRUE  TRUE  TRUE
#>  2:       TRUE       FALSE     FALSE          FALSE  FALSE FALSE FALSE
#>  3:       TRUE        TRUE      TRUE           TRUE   TRUE  TRUE  TRUE
#>  4:       TRUE        TRUE      TRUE          FALSE   TRUE  TRUE  TRUE
#>  5:       TRUE        TRUE     FALSE          FALSE   TRUE FALSE  TRUE
#>  6:       TRUE       FALSE     FALSE          FALSE   TRUE FALSE FALSE
#>  7:       TRUE       FALSE      TRUE           TRUE   TRUE  TRUE FALSE
#>  8:       TRUE        TRUE      TRUE           TRUE   TRUE FALSE  TRUE
#>  9:       TRUE        TRUE     FALSE           TRUE   TRUE FALSE FALSE
#> 10:       TRUE       FALSE     FALSE           TRUE   TRUE  TRUE  TRUE
#>     classif.ce runtime_learners           timestamp batch_nr warnings errors
#>  1: 0.06956522            0.009 2023-01-26 18:34:01        1        0      0
#>  2: 0.24347826            0.027 2023-01-26 18:34:01        2        0      0
#>  3: 0.06956522            0.009 2023-01-26 18:34:01        3        0      0
#>  4: 0.07826087            0.008 2023-01-26 18:34:01        4        0      0
#>  5: 0.07826087            0.007 2023-01-26 18:34:02        5        0      0
#>  6: 0.22608696            0.007 2023-01-26 18:34:02        6        0      0
#>  7: 0.17391304            0.009 2023-01-26 18:34:02        7        0      0
#>  8: 0.06956522            0.007 2023-01-26 18:34:02        8        0      0
#>  9: 0.06956522            0.007 2023-01-26 18:34:02        9        0      0
#> 10: 0.15652174            0.007 2023-01-26 18:34:02       10        0      0
#>          resample_result
#>  1: <ResampleResult[21]>
#>  2: <ResampleResult[21]>
#>  3: <ResampleResult[21]>
#>  4: <ResampleResult[21]>
#>  5: <ResampleResult[21]>
#>  6: <ResampleResult[21]>
#>  7: <ResampleResult[21]>
#>  8: <ResampleResult[21]>
#>  9: <ResampleResult[21]>
#> 10: <ResampleResult[21]>

# subset the task and fit the final model
task$select(instance$result_feature_set)
learner$train(task)
# }