Feature Selection with Sequential Search

Feature selection using Sequential Search Algorithm.

Details

Sequential forward selection (strategy = fsf) extends the feature set in each iteration with the feature that increases the model's performance the most. Sequential backward selection (strategy = fsb) follows the same idea but starts with all features and removes features from the set.

The feature selection terminates itself when min_features or max_features is reached. It is not necessary to set a termination criterion.

Dictionary

This FSelector can be instantiated with the associated sugar function fs():

fs("sequential")

Control Parameters

min_features

integer(1)
Minimum number of features. By default, 1.

max_features

integer(1)
Maximum number of features. By default, number of features in mlr3::Task.

strategy

character(1)
Search method sfs (forward search) or sbs (backward search).

See also

Other FSelector: mlr_fselectors_design_points, mlr_fselectors_exhaustive_search, mlr_fselectors_genetic_search, mlr_fselectors_random_search, mlr_fselectors_rfecv, mlr_fselectors_rfe, mlr_fselectors_shadow_variable_search, mlr_fselectors

Super class

mlr3fselect::FSelector -> FSelectorSequential

Methods

Public methods

• FSelectorSequential$new()

• FSelectorSequential$optimization_path()

• FSelectorSequential$clone()

Inherited methods

• mlr3fselect::FSelector$format()
• mlr3fselect::FSelector$help()
• mlr3fselect::FSelector$optimize()
• mlr3fselect::FSelector$print()

---

Method new()

Creates a new instance of this R6 class.`

Usage

FSelectorSequential$new()

---

Method optimization_path()

Returns the optimization path.

Usage

FSelectorSequential$optimization_path(inst, include_uhash = FALSE)

Arguments

inst

(FSelectInstanceSingleCrit)
Instance optimized with FSelectorSequential.

include_uhash

(logical(1))
Include uhash column?

Returns

data.table::data.table()

---

Method clone()

The objects of this class are cloneable with this method.

Usage

FSelectorSequential$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

Examples

# Feature Selection
# \donttest{

# retrieve task and load learner
task = tsk("penguins")
learner = lrn("classif.rpart")

# run feature selection on the Palmer Penguins data set
instance = fselect(
  fselector = fs("sequential"),
  task = task,
  learner = learner,
  resampling = rsmp("holdout"),
  measure = msr("classif.ce"),
  term_evals = 10
)

# best performing feature set
instance$result
#>    bill_depth bill_length body_mass flipper_length island   sex  year
#> 1:      FALSE        TRUE     FALSE           TRUE  FALSE FALSE FALSE
#>                      features classif.ce
#> 1: bill_length,flipper_length 0.07826087

# all evaluated feature sets
as.data.table(instance$archive)
#>     bill_depth bill_length body_mass flipper_length island   sex  year
#>  1:       TRUE       FALSE     FALSE          FALSE  FALSE FALSE FALSE
#>  2:      FALSE        TRUE     FALSE          FALSE  FALSE FALSE FALSE
#>  3:      FALSE       FALSE      TRUE          FALSE  FALSE FALSE FALSE
#>  4:      FALSE       FALSE     FALSE           TRUE  FALSE FALSE FALSE
#>  5:      FALSE       FALSE     FALSE          FALSE   TRUE FALSE FALSE
#>  6:      FALSE       FALSE     FALSE          FALSE  FALSE  TRUE FALSE
#>  7:      FALSE       FALSE     FALSE          FALSE  FALSE FALSE  TRUE
#>  8:       TRUE       FALSE     FALSE           TRUE  FALSE FALSE FALSE
#>  9:      FALSE        TRUE     FALSE           TRUE  FALSE FALSE FALSE
#> 10:      FALSE       FALSE      TRUE           TRUE  FALSE FALSE FALSE
#> 11:      FALSE       FALSE     FALSE           TRUE   TRUE FALSE FALSE
#> 12:      FALSE       FALSE     FALSE           TRUE  FALSE  TRUE FALSE
#> 13:      FALSE       FALSE     FALSE           TRUE  FALSE FALSE  TRUE
#>     classif.ce runtime_learners           timestamp batch_nr warnings errors
#>  1: 0.26086957            0.008 2023-03-02 12:42:40        1        0      0
#>  2: 0.20869565            0.007 2023-03-02 12:42:40        1        0      0
#>  3: 0.32173913            0.031 2023-03-02 12:42:40        1        0      0
#>  4: 0.16521739            0.007 2023-03-02 12:42:40        1        0      0
#>  5: 0.32173913            0.006 2023-03-02 12:42:40        1        0      0
#>  6: 0.49565217            0.007 2023-03-02 12:42:40        1        0      0
#>  7: 0.49565217            0.006 2023-03-02 12:42:40        1        0      0
#>  8: 0.20869565            0.008 2023-03-02 12:42:40        2        0      0
#>  9: 0.07826087            0.011 2023-03-02 12:42:40        2        0      0
#> 10: 0.15652174            0.008 2023-03-02 12:42:40        2        0      0
#> 11: 0.15652174            0.007 2023-03-02 12:42:40        2        0      0
#> 12: 0.17391304            0.009 2023-03-02 12:42:40        2        0      0
#> 13: 0.23478261            0.008 2023-03-02 12:42:40        2        0      0
#>                       features      resample_result
#>  1:                 bill_depth <ResampleResult[21]>
#>  2:                bill_length <ResampleResult[21]>
#>  3:                  body_mass <ResampleResult[21]>
#>  4:             flipper_length <ResampleResult[21]>
#>  5:                     island <ResampleResult[21]>
#>  6:                        sex <ResampleResult[21]>
#>  7:                       year <ResampleResult[21]>
#>  8:  bill_depth,flipper_length <ResampleResult[21]>
#>  9: bill_length,flipper_length <ResampleResult[21]>
#> 10:   body_mass,flipper_length <ResampleResult[21]>
#> 11:      flipper_length,island <ResampleResult[21]>
#> 12:         flipper_length,sex <ResampleResult[21]>
#> 13:        flipper_length,year <ResampleResult[21]>

# subset the task and fit the final model
task$select(instance$result_feature_set)
learner$train(task)
# }