Skip to contents

Feature selection using the Shadow Variable Search Algorithm. Shadow variable search creates for each feature a permutated copy and stops when one of them is selected.

Source

Thomas J, Hepp T, Mayr A, Bischl B (2017). “Probing for Sparse and Fast Variable Selection with Model-Based Boosting.” Computational and Mathematical Methods in Medicine, 2017, 1--8. doi:10.1155/2017/1421409 .

Wu Y, Boos DD, Stefanski LA (2007). “Controlling Variable Selection by the Addition of Pseudovariables.” Journal of the American Statistical Association, 102(477), 235--243. doi:10.1198/016214506000000843 .

Details

The feature selection terminates itself when the first shadow variable is selected. It is not necessary to set a termination criterion.

Resources

The gallery features a collection of case studies and demos about optimization.

Dictionary

This FSelector can be instantiated with the associated sugar function fs():

fs("shadow_variable_search")

Super class

mlr3fselect::FSelector -> FSelectorShadowVariableSearch

Methods

Inherited methods


Method new()

Creates a new instance of this R6 class.`


Method optimization_path()

Returns the optimization path.

Usage

FSelectorShadowVariableSearch$optimization_path(inst)

Arguments

inst

(FSelectInstanceSingleCrit)
Instance optimized with FSelectorShadowVariableSearch.


Method clone()

The objects of this class are cloneable with this method.

Usage

FSelectorShadowVariableSearch$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

Examples

# Feature Selection
# \donttest{

# retrieve task and load learner
task = tsk("penguins")
learner = lrn("classif.rpart")

# run feature selection on the Palmer Penguins data set
instance = fselect(
  fselector = fs("shadow_variable_search"),
  task = task,
  learner = learner,
  resampling = rsmp("holdout"),
  measure = msr("classif.ce"),
)

# best performing feature subset
instance$result
#>    bill_depth bill_length body_mass flipper_length island   sex  year
#> 1:      FALSE        TRUE     FALSE           TRUE   TRUE FALSE FALSE
#>                             features classif.ce
#> 1: bill_length,flipper_length,island 0.07826087

# all evaluated feature subsets
as.data.table(instance$archive)
#>     bill_depth bill_length body_mass flipper_length island   sex  year
#>  1:       TRUE       FALSE     FALSE          FALSE  FALSE FALSE FALSE
#>  2:      FALSE        TRUE     FALSE          FALSE  FALSE FALSE FALSE
#>  3:      FALSE       FALSE      TRUE          FALSE  FALSE FALSE FALSE
#>  4:      FALSE       FALSE     FALSE           TRUE  FALSE FALSE FALSE
#>  5:      FALSE       FALSE     FALSE          FALSE   TRUE FALSE FALSE
#>  6:      FALSE       FALSE     FALSE          FALSE  FALSE  TRUE FALSE
#>  7:      FALSE       FALSE     FALSE          FALSE  FALSE FALSE  TRUE
#>  8:      FALSE       FALSE     FALSE          FALSE  FALSE FALSE FALSE
#>  9:      FALSE       FALSE     FALSE          FALSE  FALSE FALSE FALSE
#> 10:      FALSE       FALSE     FALSE          FALSE  FALSE FALSE FALSE
#> 11:      FALSE       FALSE     FALSE          FALSE  FALSE FALSE FALSE
#> 12:      FALSE       FALSE     FALSE          FALSE  FALSE FALSE FALSE
#> 13:      FALSE       FALSE     FALSE          FALSE  FALSE FALSE FALSE
#> 14:      FALSE       FALSE     FALSE          FALSE  FALSE FALSE FALSE
#> 15:       TRUE       FALSE     FALSE           TRUE  FALSE FALSE FALSE
#> 16:      FALSE        TRUE     FALSE           TRUE  FALSE FALSE FALSE
#> 17:      FALSE       FALSE      TRUE           TRUE  FALSE FALSE FALSE
#> 18:      FALSE       FALSE     FALSE           TRUE   TRUE FALSE FALSE
#> 19:      FALSE       FALSE     FALSE           TRUE  FALSE  TRUE FALSE
#> 20:      FALSE       FALSE     FALSE           TRUE  FALSE FALSE  TRUE
#> 21:      FALSE       FALSE     FALSE           TRUE  FALSE FALSE FALSE
#> 22:      FALSE       FALSE     FALSE           TRUE  FALSE FALSE FALSE
#> 23:      FALSE       FALSE     FALSE           TRUE  FALSE FALSE FALSE
#> 24:      FALSE       FALSE     FALSE           TRUE  FALSE FALSE FALSE
#> 25:      FALSE       FALSE     FALSE           TRUE  FALSE FALSE FALSE
#> 26:      FALSE       FALSE     FALSE           TRUE  FALSE FALSE FALSE
#> 27:      FALSE       FALSE     FALSE           TRUE  FALSE FALSE FALSE
#> 28:       TRUE        TRUE     FALSE           TRUE  FALSE FALSE FALSE
#> 29:      FALSE        TRUE      TRUE           TRUE  FALSE FALSE FALSE
#> 30:      FALSE        TRUE     FALSE           TRUE   TRUE FALSE FALSE
#> 31:      FALSE        TRUE     FALSE           TRUE  FALSE  TRUE FALSE
#> 32:      FALSE        TRUE     FALSE           TRUE  FALSE FALSE  TRUE
#> 33:      FALSE        TRUE     FALSE           TRUE  FALSE FALSE FALSE
#> 34:      FALSE        TRUE     FALSE           TRUE  FALSE FALSE FALSE
#> 35:      FALSE        TRUE     FALSE           TRUE  FALSE FALSE FALSE
#> 36:      FALSE        TRUE     FALSE           TRUE  FALSE FALSE FALSE
#> 37:      FALSE        TRUE     FALSE           TRUE  FALSE FALSE FALSE
#> 38:      FALSE        TRUE     FALSE           TRUE  FALSE FALSE FALSE
#> 39:      FALSE        TRUE     FALSE           TRUE  FALSE FALSE FALSE
#> 40:       TRUE        TRUE     FALSE           TRUE   TRUE FALSE FALSE
#> 41:      FALSE        TRUE      TRUE           TRUE   TRUE FALSE FALSE
#> 42:      FALSE        TRUE     FALSE           TRUE   TRUE  TRUE FALSE
#> 43:      FALSE        TRUE     FALSE           TRUE   TRUE FALSE  TRUE
#> 44:      FALSE        TRUE     FALSE           TRUE   TRUE FALSE FALSE
#> 45:      FALSE        TRUE     FALSE           TRUE   TRUE FALSE FALSE
#> 46:      FALSE        TRUE     FALSE           TRUE   TRUE FALSE FALSE
#> 47:      FALSE        TRUE     FALSE           TRUE   TRUE FALSE FALSE
#> 48:      FALSE        TRUE     FALSE           TRUE   TRUE FALSE FALSE
#> 49:      FALSE        TRUE     FALSE           TRUE   TRUE FALSE FALSE
#> 50:      FALSE        TRUE     FALSE           TRUE   TRUE FALSE FALSE
#> 51:       TRUE        TRUE      TRUE           TRUE   TRUE FALSE FALSE
#> 52:       TRUE        TRUE     FALSE           TRUE   TRUE  TRUE FALSE
#> 53:       TRUE        TRUE     FALSE           TRUE   TRUE FALSE  TRUE
#> 54:       TRUE        TRUE     FALSE           TRUE   TRUE FALSE FALSE
#> 55:       TRUE        TRUE     FALSE           TRUE   TRUE FALSE FALSE
#> 56:       TRUE        TRUE     FALSE           TRUE   TRUE FALSE FALSE
#> 57:       TRUE        TRUE     FALSE           TRUE   TRUE FALSE FALSE
#> 58:       TRUE        TRUE     FALSE           TRUE   TRUE FALSE FALSE
#> 59:       TRUE        TRUE     FALSE           TRUE   TRUE FALSE FALSE
#> 60:       TRUE        TRUE     FALSE           TRUE   TRUE FALSE FALSE
#> 61:       TRUE        TRUE      TRUE           TRUE   TRUE  TRUE FALSE
#> 62:       TRUE        TRUE      TRUE           TRUE   TRUE FALSE  TRUE
#> 63:       TRUE        TRUE      TRUE           TRUE   TRUE FALSE FALSE
#> 64:       TRUE        TRUE      TRUE           TRUE   TRUE FALSE FALSE
#> 65:       TRUE        TRUE      TRUE           TRUE   TRUE FALSE FALSE
#> 66:       TRUE        TRUE      TRUE           TRUE   TRUE FALSE FALSE
#> 67:       TRUE        TRUE      TRUE           TRUE   TRUE FALSE FALSE
#> 68:       TRUE        TRUE      TRUE           TRUE   TRUE FALSE FALSE
#> 69:       TRUE        TRUE      TRUE           TRUE   TRUE FALSE FALSE
#> 70:       TRUE        TRUE      TRUE           TRUE   TRUE  TRUE  TRUE
#> 71:       TRUE        TRUE      TRUE           TRUE   TRUE  TRUE FALSE
#> 72:       TRUE        TRUE      TRUE           TRUE   TRUE  TRUE FALSE
#> 73:       TRUE        TRUE      TRUE           TRUE   TRUE  TRUE FALSE
#> 74:       TRUE        TRUE      TRUE           TRUE   TRUE  TRUE FALSE
#> 75:       TRUE        TRUE      TRUE           TRUE   TRUE  TRUE FALSE
#> 76:       TRUE        TRUE      TRUE           TRUE   TRUE  TRUE FALSE
#> 77:       TRUE        TRUE      TRUE           TRUE   TRUE  TRUE FALSE
#>     bill_depth bill_length body_mass flipper_length island   sex  year
#>     classif.ce runtime_learners           timestamp batch_nr
#>  1: 0.20000000            0.011 2023-11-17 12:02:40        1
#>  2: 0.21739130            0.011 2023-11-17 12:02:40        1
#>  3: 0.27826087            0.015 2023-11-17 12:02:40        1
#>  4: 0.15652174            0.012 2023-11-17 12:02:40        1
#>  5: 0.31304348            0.012 2023-11-17 12:02:40        1
#>  6: 0.51304348            0.011 2023-11-17 12:02:40        1
#>  7: 0.51304348            0.011 2023-11-17 12:02:40        1
#>  8: 0.56521739            0.010 2023-11-17 12:02:40        1
#>  9: 0.55652174            0.010 2023-11-17 12:02:40        1
#> 10: 0.60869565            0.010 2023-11-17 12:02:40        1
#> 11: 0.59130435            0.010 2023-11-17 12:02:40        1
#> 12: 0.51304348            0.011 2023-11-17 12:02:40        1
#> 13: 0.51304348            0.025 2023-11-17 12:02:40        1
#> 14: 0.51304348            0.011 2023-11-17 12:02:40        1
#> 15: 0.23478261            0.029 2023-11-17 12:02:41        2
#> 16: 0.08695652            0.011 2023-11-17 12:02:41        2
#> 17: 0.13043478            0.012 2023-11-17 12:02:41        2
#> 18: 0.20000000            0.012 2023-11-17 12:02:41        2
#> 19: 0.14782609            0.012 2023-11-17 12:02:41        2
#> 20: 0.18260870            0.011 2023-11-17 12:02:41        2
#> 21: 0.19130435            0.012 2023-11-17 12:02:41        2
#> 22: 0.24347826            0.012 2023-11-17 12:02:41        2
#> 23: 0.20000000            0.011 2023-11-17 12:02:41        2
#> 24: 0.21739130            0.012 2023-11-17 12:02:41        2
#> 25: 0.15652174            0.015 2023-11-17 12:02:41        2
#> 26: 0.24347826            0.013 2023-11-17 12:02:41        2
#> 27: 0.15652174            0.011 2023-11-17 12:02:41        2
#> 28: 0.08695652            0.013 2023-11-17 12:02:41        3
#> 29: 0.08695652            0.013 2023-11-17 12:02:41        3
#> 30: 0.07826087            0.012 2023-11-17 12:02:41        3
#> 31: 0.08695652            0.012 2023-11-17 12:02:41        3
#> 32: 0.08695652            0.012 2023-11-17 12:02:41        3
#> 33: 0.08695652            0.012 2023-11-17 12:02:41        3
#> 34: 0.08695652            0.013 2023-11-17 12:02:41        3
#> 35: 0.08695652            0.013 2023-11-17 12:02:41        3
#> 36: 0.08695652            0.042 2023-11-17 12:02:41        3
#> 37: 0.08695652            0.016 2023-11-17 12:02:41        3
#> 38: 0.08695652            0.012 2023-11-17 12:02:41        3
#> 39: 0.08695652            0.012 2023-11-17 12:02:41        3
#> 40: 0.07826087            0.014 2023-11-17 12:02:42        4
#> 41: 0.07826087            0.012 2023-11-17 12:02:42        4
#> 42: 0.07826087            0.012 2023-11-17 12:02:42        4
#> 43: 0.07826087            0.012 2023-11-17 12:02:42        4
#> 44: 0.07826087            0.013 2023-11-17 12:02:42        4
#> 45: 0.07826087            0.013 2023-11-17 12:02:42        4
#> 46: 0.07826087            0.012 2023-11-17 12:02:42        4
#> 47: 0.07826087            0.013 2023-11-17 12:02:42        4
#> 48: 0.07826087            0.013 2023-11-17 12:02:42        4
#> 49: 0.07826087            0.030 2023-11-17 12:02:42        4
#> 50: 0.07826087            0.013 2023-11-17 12:02:42        4
#> 51: 0.07826087            0.013 2023-11-17 12:02:42        5
#> 52: 0.07826087            0.016 2023-11-17 12:02:42        5
#> 53: 0.07826087            0.013 2023-11-17 12:02:42        5
#> 54: 0.07826087            0.013 2023-11-17 12:02:42        5
#> 55: 0.07826087            0.012 2023-11-17 12:02:42        5
#> 56: 0.07826087            0.012 2023-11-17 12:02:42        5
#> 57: 0.07826087            0.013 2023-11-17 12:02:42        5
#> 58: 0.07826087            0.014 2023-11-17 12:02:42        5
#> 59: 0.07826087            0.012 2023-11-17 12:02:42        5
#> 60: 0.07826087            0.013 2023-11-17 12:02:42        5
#> 61: 0.07826087            0.014 2023-11-17 12:02:42        6
#> 62: 0.07826087            0.014 2023-11-17 12:02:42        6
#> 63: 0.07826087            0.014 2023-11-17 12:02:42        6
#> 64: 0.07826087            0.013 2023-11-17 12:02:42        6
#> 65: 0.07826087            0.031 2023-11-17 12:02:42        6
#> 66: 0.07826087            0.014 2023-11-17 12:02:42        6
#> 67: 0.07826087            0.014 2023-11-17 12:02:42        6
#> 68: 0.07826087            0.013 2023-11-17 12:02:42        6
#> 69: 0.07826087            0.012 2023-11-17 12:02:42        6
#> 70: 0.07826087            0.014 2023-11-17 12:02:43        7
#> 71: 0.07826087            0.014 2023-11-17 12:02:43        7
#> 72: 0.07826087            0.013 2023-11-17 12:02:43        7
#> 73: 0.07826087            0.013 2023-11-17 12:02:43        7
#> 74: 0.07826087            0.012 2023-11-17 12:02:43        7
#> 75: 0.07826087            0.014 2023-11-17 12:02:43        7
#> 76: 0.07826087            0.014 2023-11-17 12:02:43        7
#> 77: 0.07826087            0.014 2023-11-17 12:02:43        7
#>     classif.ce runtime_learners           timestamp batch_nr
#>     permuted__bill_depth permuted__bill_length permuted__body_mass
#>  1:                FALSE                 FALSE               FALSE
#>  2:                FALSE                 FALSE               FALSE
#>  3:                FALSE                 FALSE               FALSE
#>  4:                FALSE                 FALSE               FALSE
#>  5:                FALSE                 FALSE               FALSE
#>  6:                FALSE                 FALSE               FALSE
#>  7:                FALSE                 FALSE               FALSE
#>  8:                 TRUE                 FALSE               FALSE
#>  9:                FALSE                  TRUE               FALSE
#> 10:                FALSE                 FALSE                TRUE
#> 11:                FALSE                 FALSE               FALSE
#> 12:                FALSE                 FALSE               FALSE
#> 13:                FALSE                 FALSE               FALSE
#> 14:                FALSE                 FALSE               FALSE
#> 15:                FALSE                 FALSE               FALSE
#> 16:                FALSE                 FALSE               FALSE
#> 17:                FALSE                 FALSE               FALSE
#> 18:                FALSE                 FALSE               FALSE
#> 19:                FALSE                 FALSE               FALSE
#> 20:                FALSE                 FALSE               FALSE
#> 21:                 TRUE                 FALSE               FALSE
#> 22:                FALSE                  TRUE               FALSE
#> 23:                FALSE                 FALSE                TRUE
#> 24:                FALSE                 FALSE               FALSE
#> 25:                FALSE                 FALSE               FALSE
#> 26:                FALSE                 FALSE               FALSE
#> 27:                FALSE                 FALSE               FALSE
#> 28:                FALSE                 FALSE               FALSE
#> 29:                FALSE                 FALSE               FALSE
#> 30:                FALSE                 FALSE               FALSE
#> 31:                FALSE                 FALSE               FALSE
#> 32:                FALSE                 FALSE               FALSE
#> 33:                 TRUE                 FALSE               FALSE
#> 34:                FALSE                  TRUE               FALSE
#> 35:                FALSE                 FALSE                TRUE
#> 36:                FALSE                 FALSE               FALSE
#> 37:                FALSE                 FALSE               FALSE
#> 38:                FALSE                 FALSE               FALSE
#> 39:                FALSE                 FALSE               FALSE
#> 40:                FALSE                 FALSE               FALSE
#> 41:                FALSE                 FALSE               FALSE
#> 42:                FALSE                 FALSE               FALSE
#> 43:                FALSE                 FALSE               FALSE
#> 44:                 TRUE                 FALSE               FALSE
#> 45:                FALSE                  TRUE               FALSE
#> 46:                FALSE                 FALSE                TRUE
#> 47:                FALSE                 FALSE               FALSE
#> 48:                FALSE                 FALSE               FALSE
#> 49:                FALSE                 FALSE               FALSE
#> 50:                FALSE                 FALSE               FALSE
#> 51:                FALSE                 FALSE               FALSE
#> 52:                FALSE                 FALSE               FALSE
#> 53:                FALSE                 FALSE               FALSE
#> 54:                 TRUE                 FALSE               FALSE
#> 55:                FALSE                  TRUE               FALSE
#> 56:                FALSE                 FALSE                TRUE
#> 57:                FALSE                 FALSE               FALSE
#> 58:                FALSE                 FALSE               FALSE
#> 59:                FALSE                 FALSE               FALSE
#> 60:                FALSE                 FALSE               FALSE
#> 61:                FALSE                 FALSE               FALSE
#> 62:                FALSE                 FALSE               FALSE
#> 63:                 TRUE                 FALSE               FALSE
#> 64:                FALSE                  TRUE               FALSE
#> 65:                FALSE                 FALSE                TRUE
#> 66:                FALSE                 FALSE               FALSE
#> 67:                FALSE                 FALSE               FALSE
#> 68:                FALSE                 FALSE               FALSE
#> 69:                FALSE                 FALSE               FALSE
#> 70:                FALSE                 FALSE               FALSE
#> 71:                 TRUE                 FALSE               FALSE
#> 72:                FALSE                  TRUE               FALSE
#> 73:                FALSE                 FALSE                TRUE
#> 74:                FALSE                 FALSE               FALSE
#> 75:                FALSE                 FALSE               FALSE
#> 76:                FALSE                 FALSE               FALSE
#> 77:                FALSE                 FALSE               FALSE
#>     permuted__bill_depth permuted__bill_length permuted__body_mass
#>     permuted__flipper_length permuted__island permuted__sex permuted__year
#>  1:                    FALSE            FALSE         FALSE          FALSE
#>  2:                    FALSE            FALSE         FALSE          FALSE
#>  3:                    FALSE            FALSE         FALSE          FALSE
#>  4:                    FALSE            FALSE         FALSE          FALSE
#>  5:                    FALSE            FALSE         FALSE          FALSE
#>  6:                    FALSE            FALSE         FALSE          FALSE
#>  7:                    FALSE            FALSE         FALSE          FALSE
#>  8:                    FALSE            FALSE         FALSE          FALSE
#>  9:                    FALSE            FALSE         FALSE          FALSE
#> 10:                    FALSE            FALSE         FALSE          FALSE
#> 11:                     TRUE            FALSE         FALSE          FALSE
#> 12:                    FALSE             TRUE         FALSE          FALSE
#> 13:                    FALSE            FALSE          TRUE          FALSE
#> 14:                    FALSE            FALSE         FALSE           TRUE
#> 15:                    FALSE            FALSE         FALSE          FALSE
#> 16:                    FALSE            FALSE         FALSE          FALSE
#> 17:                    FALSE            FALSE         FALSE          FALSE
#> 18:                    FALSE            FALSE         FALSE          FALSE
#> 19:                    FALSE            FALSE         FALSE          FALSE
#> 20:                    FALSE            FALSE         FALSE          FALSE
#> 21:                    FALSE            FALSE         FALSE          FALSE
#> 22:                    FALSE            FALSE         FALSE          FALSE
#> 23:                    FALSE            FALSE         FALSE          FALSE
#> 24:                     TRUE            FALSE         FALSE          FALSE
#> 25:                    FALSE             TRUE         FALSE          FALSE
#> 26:                    FALSE            FALSE          TRUE          FALSE
#> 27:                    FALSE            FALSE         FALSE           TRUE
#> 28:                    FALSE            FALSE         FALSE          FALSE
#> 29:                    FALSE            FALSE         FALSE          FALSE
#> 30:                    FALSE            FALSE         FALSE          FALSE
#> 31:                    FALSE            FALSE         FALSE          FALSE
#> 32:                    FALSE            FALSE         FALSE          FALSE
#> 33:                    FALSE            FALSE         FALSE          FALSE
#> 34:                    FALSE            FALSE         FALSE          FALSE
#> 35:                    FALSE            FALSE         FALSE          FALSE
#> 36:                     TRUE            FALSE         FALSE          FALSE
#> 37:                    FALSE             TRUE         FALSE          FALSE
#> 38:                    FALSE            FALSE          TRUE          FALSE
#> 39:                    FALSE            FALSE         FALSE           TRUE
#> 40:                    FALSE            FALSE         FALSE          FALSE
#> 41:                    FALSE            FALSE         FALSE          FALSE
#> 42:                    FALSE            FALSE         FALSE          FALSE
#> 43:                    FALSE            FALSE         FALSE          FALSE
#> 44:                    FALSE            FALSE         FALSE          FALSE
#> 45:                    FALSE            FALSE         FALSE          FALSE
#> 46:                    FALSE            FALSE         FALSE          FALSE
#> 47:                     TRUE            FALSE         FALSE          FALSE
#> 48:                    FALSE             TRUE         FALSE          FALSE
#> 49:                    FALSE            FALSE          TRUE          FALSE
#> 50:                    FALSE            FALSE         FALSE           TRUE
#> 51:                    FALSE            FALSE         FALSE          FALSE
#> 52:                    FALSE            FALSE         FALSE          FALSE
#> 53:                    FALSE            FALSE         FALSE          FALSE
#> 54:                    FALSE            FALSE         FALSE          FALSE
#> 55:                    FALSE            FALSE         FALSE          FALSE
#> 56:                    FALSE            FALSE         FALSE          FALSE
#> 57:                     TRUE            FALSE         FALSE          FALSE
#> 58:                    FALSE             TRUE         FALSE          FALSE
#> 59:                    FALSE            FALSE          TRUE          FALSE
#> 60:                    FALSE            FALSE         FALSE           TRUE
#> 61:                    FALSE            FALSE         FALSE          FALSE
#> 62:                    FALSE            FALSE         FALSE          FALSE
#> 63:                    FALSE            FALSE         FALSE          FALSE
#> 64:                    FALSE            FALSE         FALSE          FALSE
#> 65:                    FALSE            FALSE         FALSE          FALSE
#> 66:                     TRUE            FALSE         FALSE          FALSE
#> 67:                    FALSE             TRUE         FALSE          FALSE
#> 68:                    FALSE            FALSE          TRUE          FALSE
#> 69:                    FALSE            FALSE         FALSE           TRUE
#> 70:                    FALSE            FALSE         FALSE          FALSE
#> 71:                    FALSE            FALSE         FALSE          FALSE
#> 72:                    FALSE            FALSE         FALSE          FALSE
#> 73:                    FALSE            FALSE         FALSE          FALSE
#> 74:                     TRUE            FALSE         FALSE          FALSE
#> 75:                    FALSE             TRUE         FALSE          FALSE
#> 76:                    FALSE            FALSE          TRUE          FALSE
#> 77:                    FALSE            FALSE         FALSE           TRUE
#>     permuted__flipper_length permuted__island permuted__sex permuted__year
#>     warnings errors
#>  1:        0      0
#>  2:        0      0
#>  3:        0      0
#>  4:        0      0
#>  5:        0      0
#>  6:        0      0
#>  7:        0      0
#>  8:        0      0
#>  9:        0      0
#> 10:        0      0
#> 11:        0      0
#> 12:        0      0
#> 13:        0      0
#> 14:        0      0
#> 15:        0      0
#> 16:        0      0
#> 17:        0      0
#> 18:        0      0
#> 19:        0      0
#> 20:        0      0
#> 21:        0      0
#> 22:        0      0
#> 23:        0      0
#> 24:        0      0
#> 25:        0      0
#> 26:        0      0
#> 27:        0      0
#> 28:        0      0
#> 29:        0      0
#> 30:        0      0
#> 31:        0      0
#> 32:        0      0
#> 33:        0      0
#> 34:        0      0
#> 35:        0      0
#> 36:        0      0
#> 37:        0      0
#> 38:        0      0
#> 39:        0      0
#> 40:        0      0
#> 41:        0      0
#> 42:        0      0
#> 43:        0      0
#> 44:        0      0
#> 45:        0      0
#> 46:        0      0
#> 47:        0      0
#> 48:        0      0
#> 49:        0      0
#> 50:        0      0
#> 51:        0      0
#> 52:        0      0
#> 53:        0      0
#> 54:        0      0
#> 55:        0      0
#> 56:        0      0
#> 57:        0      0
#> 58:        0      0
#> 59:        0      0
#> 60:        0      0
#> 61:        0      0
#> 62:        0      0
#> 63:        0      0
#> 64:        0      0
#> 65:        0      0
#> 66:        0      0
#> 67:        0      0
#> 68:        0      0
#> 69:        0      0
#> 70:        0      0
#> 71:        0      0
#> 72:        0      0
#> 73:        0      0
#> 74:        0      0
#> 75:        0      0
#> 76:        0      0
#> 77:        0      0
#>     warnings errors
#>                                                           features
#>  1:                                                     bill_depth
#>  2:                                                    bill_length
#>  3:                                                      body_mass
#>  4:                                                 flipper_length
#>  5:                                                         island
#>  6:                                                            sex
#>  7:                                                           year
#>  8:                                                               
#>  9:                                                               
#> 10:                                                               
#> 11:                                                               
#> 12:                                                               
#> 13:                                                               
#> 14:                                                               
#> 15:                                      bill_depth,flipper_length
#> 16:                                     bill_length,flipper_length
#> 17:                                       body_mass,flipper_length
#> 18:                                          flipper_length,island
#> 19:                                             flipper_length,sex
#> 20:                                            flipper_length,year
#> 21:                                                 flipper_length
#> 22:                                                 flipper_length
#> 23:                                                 flipper_length
#> 24:                                                 flipper_length
#> 25:                                                 flipper_length
#> 26:                                                 flipper_length
#> 27:                                                 flipper_length
#> 28:                          bill_depth,bill_length,flipper_length
#> 29:                           bill_length,body_mass,flipper_length
#> 30:                              bill_length,flipper_length,island
#> 31:                                 bill_length,flipper_length,sex
#> 32:                                bill_length,flipper_length,year
#> 33:                                     bill_length,flipper_length
#> 34:                                     bill_length,flipper_length
#> 35:                                     bill_length,flipper_length
#> 36:                                     bill_length,flipper_length
#> 37:                                     bill_length,flipper_length
#> 38:                                     bill_length,flipper_length
#> 39:                                     bill_length,flipper_length
#> 40:                   bill_depth,bill_length,flipper_length,island
#> 41:                    bill_length,body_mass,flipper_length,island
#> 42:                          bill_length,flipper_length,island,sex
#> 43:                         bill_length,flipper_length,island,year
#> 44:                              bill_length,flipper_length,island
#> 45:                              bill_length,flipper_length,island
#> 46:                              bill_length,flipper_length,island
#> 47:                              bill_length,flipper_length,island
#> 48:                              bill_length,flipper_length,island
#> 49:                              bill_length,flipper_length,island
#> 50:                              bill_length,flipper_length,island
#> 51:         bill_depth,bill_length,body_mass,flipper_length,island
#> 52:               bill_depth,bill_length,flipper_length,island,sex
#> 53:              bill_depth,bill_length,flipper_length,island,year
#> 54:                   bill_depth,bill_length,flipper_length,island
#> 55:                   bill_depth,bill_length,flipper_length,island
#> 56:                   bill_depth,bill_length,flipper_length,island
#> 57:                   bill_depth,bill_length,flipper_length,island
#> 58:                   bill_depth,bill_length,flipper_length,island
#> 59:                   bill_depth,bill_length,flipper_length,island
#> 60:                   bill_depth,bill_length,flipper_length,island
#> 61:     bill_depth,bill_length,body_mass,flipper_length,island,sex
#> 62:    bill_depth,bill_length,body_mass,flipper_length,island,year
#> 63:         bill_depth,bill_length,body_mass,flipper_length,island
#> 64:         bill_depth,bill_length,body_mass,flipper_length,island
#> 65:         bill_depth,bill_length,body_mass,flipper_length,island
#> 66:         bill_depth,bill_length,body_mass,flipper_length,island
#> 67:         bill_depth,bill_length,body_mass,flipper_length,island
#> 68:         bill_depth,bill_length,body_mass,flipper_length,island
#> 69:         bill_depth,bill_length,body_mass,flipper_length,island
#> 70: bill_depth,bill_length,body_mass,flipper_length,island,sex,...
#> 71:     bill_depth,bill_length,body_mass,flipper_length,island,sex
#> 72:     bill_depth,bill_length,body_mass,flipper_length,island,sex
#> 73:     bill_depth,bill_length,body_mass,flipper_length,island,sex
#> 74:     bill_depth,bill_length,body_mass,flipper_length,island,sex
#> 75:     bill_depth,bill_length,body_mass,flipper_length,island,sex
#> 76:     bill_depth,bill_length,body_mass,flipper_length,island,sex
#> 77:     bill_depth,bill_length,body_mass,flipper_length,island,sex
#>                                                           features
#>          resample_result
#>  1: <ResampleResult[21]>
#>  2: <ResampleResult[21]>
#>  3: <ResampleResult[21]>
#>  4: <ResampleResult[21]>
#>  5: <ResampleResult[21]>
#>  6: <ResampleResult[21]>
#>  7: <ResampleResult[21]>
#>  8: <ResampleResult[21]>
#>  9: <ResampleResult[21]>
#> 10: <ResampleResult[21]>
#> 11: <ResampleResult[21]>
#> 12: <ResampleResult[21]>
#> 13: <ResampleResult[21]>
#> 14: <ResampleResult[21]>
#> 15: <ResampleResult[21]>
#> 16: <ResampleResult[21]>
#> 17: <ResampleResult[21]>
#> 18: <ResampleResult[21]>
#> 19: <ResampleResult[21]>
#> 20: <ResampleResult[21]>
#> 21: <ResampleResult[21]>
#> 22: <ResampleResult[21]>
#> 23: <ResampleResult[21]>
#> 24: <ResampleResult[21]>
#> 25: <ResampleResult[21]>
#> 26: <ResampleResult[21]>
#> 27: <ResampleResult[21]>
#> 28: <ResampleResult[21]>
#> 29: <ResampleResult[21]>
#> 30: <ResampleResult[21]>
#> 31: <ResampleResult[21]>
#> 32: <ResampleResult[21]>
#> 33: <ResampleResult[21]>
#> 34: <ResampleResult[21]>
#> 35: <ResampleResult[21]>
#> 36: <ResampleResult[21]>
#> 37: <ResampleResult[21]>
#> 38: <ResampleResult[21]>
#> 39: <ResampleResult[21]>
#> 40: <ResampleResult[21]>
#> 41: <ResampleResult[21]>
#> 42: <ResampleResult[21]>
#> 43: <ResampleResult[21]>
#> 44: <ResampleResult[21]>
#> 45: <ResampleResult[21]>
#> 46: <ResampleResult[21]>
#> 47: <ResampleResult[21]>
#> 48: <ResampleResult[21]>
#> 49: <ResampleResult[21]>
#> 50: <ResampleResult[21]>
#> 51: <ResampleResult[21]>
#> 52: <ResampleResult[21]>
#> 53: <ResampleResult[21]>
#> 54: <ResampleResult[21]>
#> 55: <ResampleResult[21]>
#> 56: <ResampleResult[21]>
#> 57: <ResampleResult[21]>
#> 58: <ResampleResult[21]>
#> 59: <ResampleResult[21]>
#> 60: <ResampleResult[21]>
#> 61: <ResampleResult[21]>
#> 62: <ResampleResult[21]>
#> 63: <ResampleResult[21]>
#> 64: <ResampleResult[21]>
#> 65: <ResampleResult[21]>
#> 66: <ResampleResult[21]>
#> 67: <ResampleResult[21]>
#> 68: <ResampleResult[21]>
#> 69: <ResampleResult[21]>
#> 70: <ResampleResult[21]>
#> 71: <ResampleResult[21]>
#> 72: <ResampleResult[21]>
#> 73: <ResampleResult[21]>
#> 74: <ResampleResult[21]>
#> 75: <ResampleResult[21]>
#> 76: <ResampleResult[21]>
#> 77: <ResampleResult[21]>
#>          resample_result

# subset the task and fit the final model
task$select(instance$result_feature_set)
learner$train(task)
# }