# mlr3fselect Package website: [release](https://mlr3fselect.mlr-org.com/) \| [dev](https://mlr3fselect.mlr-org.com/dev/) *mlr3fselect* is the feature selection package of the [mlr3](https://mlr-org.com/) ecosystem. It selects the optimal feature set for any mlr3 [learner](https://github.com/mlr-org/mlr3learners). The package works with several optimization algorithms e.g. Random Search, Recursive Feature Elimination, and Genetic Search. Moreover, it can automatically optimize learners and estimate the performance of optimized feature sets with [nested resampling](https://mlr3book.mlr-org.com/chapters/chapter6/feature_selection.html#sec-autofselect). The package is built on the optimization framework [bbotk](https://github.com/mlr-org/bbotk). ## Resources There are several section about feature selection in the [mlr3book](https://mlr3book.mlr-org.com). - Getting started with [wrapper feature selection](https://mlr3book.mlr-org.com/chapters/chapter6/feature_selection.html#sec-fs-wrapper). - Do a [sequential forward selection](https://mlr3book.mlr-org.com/chapters/chapter6/feature_selection.html#sec-fs-wrapper-example) Palmer Penguins data set. - Optimize [multiple performance measures](https://mlr3book.mlr-org.com/chapters/chapter6/feature_selection.html#sec-multicrit-featsel). - Estimate Model Performance with [nested resampling](https://mlr3book.mlr-org.com/chapters/chapter6/feature_selection.html#sec-autofselect). The [gallery](https://mlr-org.com/gallery.html) features a collection of case studies and demos about optimization. - Perform wrapper-based [Ensemble Feature Selection](https://mlr-org.com/gallery/technical/2025-01-12-efs/). - Utilize the built-in feature importance of models with [Recursive Feature Elimination](https://mlr-org.com/gallery/optimization/2023-02-07-recursive-feature-elimination/). - Run a feature selection with [Shadow Variable Search](https://mlr-org.com/gallery/optimization/2023-02-01-shadow-variable-search/). The [cheatsheet](https://cheatsheets.mlr-org.com/mlr3fselect.pdf) summarizes the most important functions of mlr3fselect. ## Installation Install the last release from CRAN: ``` r install.packages("mlr3fselect") ``` Install the development version from GitHub: ``` r # install.packages("pak") pak::pak("mlr-org/mlr3fselect") ``` ## Example We run a feature selection for a support vector machine on the [Spam](https://mlr3.mlr-org.com/reference/mlr_tasks_spam.html) data set. ``` r library("mlr3verse") tsk("spam") ``` ``` R ## ## ── (4601x58): HP Spam Detection ────────────────────────────────────────────────────── ## • Target: type ## • Target classes: spam (positive class, 39%), nonspam (61%) ## • Properties: twoclass ## • Features (57): ## • dbl (57): address, addresses, all, business, capitalAve, capitalLong, capitalTotal, charDollar, ## charExclamation, charHash, charRoundbracket, charSemicolon, charSquarebracket, conference, ## credit, cs, data, direct, edu, email, font, free, george, hp, hpl, internet, lab, labs, mail, ## make, meeting, money, num000, num1999, num3d, num415, num650, num85, num857, order, original, ## our, over, parts, people, pm, project, re, receive, remove, report, table, technology, telnet, ## will, you, your ``` We construct an instance with the [`fsi()`](https://mlr3fselect.mlr-org.com/reference/fsi.md) function. The instance describes the optimization problem. ``` r instance = fsi( task = tsk("spam"), learner = lrn("classif.svm", type = "C-classification"), resampling = rsmp("cv", folds = 3), measures = msr("classif.ce"), terminator = trm("evals", n_evals = 20) ) instance ``` ``` R ## ## ── ──────────────────────────────────────────────────────────────── ## • State: Not optimized ## • Objective: (classif.svm_on_spam) ## • Terminator: ``` We select a simple random search as the optimization algorithm. ``` r fselector = fs("random_search", batch_size = 5) fselector ``` ``` R ## ## ── : Random Search ───────────────────────────────────────────────────── ## • Parameters: batch_size=5 ## • Properties: single-crit and multi-crit ## • Packages: mlr3fselect ``` To start the feature selection, we simply pass the instance to the fselector. ``` r fselector$optimize(instance) ``` The fselector writes the best hyperparameter configuration to the instance. ``` r instance$result_feature_set ``` ``` R ## [1] "address" "addresses" "all" "business" ## [5] "capitalAve" "capitalLong" "capitalTotal" "charDollar" ## [9] "charExclamation" "charHash" "charRoundbracket" "charSemicolon" ## [13] "charSquarebracket" "conference" "credit" "cs" ## [17] "data" "direct" "edu" "email" ## [21] "font" "free" "george" "hp" ## [25] "internet" "lab" "labs" "mail" ## [29] "make" "meeting" "money" "num000" ## [33] "num1999" "num3d" "num415" "num650" ## [37] "num85" "num857" "order" "our" ## [41] "parts" "people" "pm" "project" ## [45] "re" "receive" "remove" "report" ## [49] "table" "technology" "telnet" "will" ## [53] "you" "your" ``` And the corresponding measured performance. ``` r instance$result_y ``` ``` R ## classif.ce ## 0.07042005 ``` The archive contains all evaluated hyperparameter configurations. ``` r as.data.table(instance$archive) ``` ``` R ## address addresses all business capitalAve capitalLong capitalTotal charDollar ## 1: TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE ## 2: TRUE TRUE TRUE FALSE FALSE TRUE TRUE TRUE ## 3: TRUE TRUE FALSE FALSE TRUE TRUE TRUE TRUE ## 4: TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE ## 5: FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE ## --- ## 16: FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE ## 17: FALSE FALSE FALSE TRUE TRUE TRUE FALSE FALSE ## 18: FALSE FALSE TRUE TRUE FALSE FALSE FALSE TRUE ## 19: TRUE TRUE TRUE TRUE FALSE TRUE TRUE TRUE ## 20: TRUE FALSE TRUE FALSE FALSE TRUE FALSE TRUE ## 58 variables not shown: [charExclamation, charHash, charRoundbracket, charSemicolon, charSquarebracket, conference, credit, cs, data, direct, ...] ``` We fit a final model with the optimized feature set to make predictions on new data. ``` r task = tsk("spam") learner = lrn("classif.svm", type = "C-classification") task$select(instance$result_feature_set) learner$train(task) ``` ## Citation If you use **mlr3fselect** in your work, please cite the package: > Becker M, Schratz P, Lang M, Bischl B, Zobolas J (2025). mlr3fselect: > Feature Selection for ‘mlr3’. R package version 1.4.0, > The **ensemble feature selection** components (hEFS) are described in the following study: > Zobolas, J., George, AM., López, A., Fischer, S., Becker, M., & > Aittokallio, T. Prognostic biomarker discovery in pancreatic cancer > through hybrid ensemble feature selection and multi-omics data. > BioData Mining (2026). # Package index ## Feature Selection Instances - [`FSelectInstanceAsyncMultiCrit`](https://mlr3fselect.mlr-org.com/reference/FSelectInstanceAsyncMultiCrit.md) : Multi-Criteria Feature Selection with Rush - [`FSelectInstanceAsyncSingleCrit`](https://mlr3fselect.mlr-org.com/reference/FSelectInstanceAsyncSingleCrit.md) : Single Criterion Feature Selection with Rush - [`FSelectInstanceBatchMultiCrit`](https://mlr3fselect.mlr-org.com/reference/FSelectInstanceBatchMultiCrit.md) : Class for Multi Criteria Feature Selection - [`FSelectInstanceBatchSingleCrit`](https://mlr3fselect.mlr-org.com/reference/FSelectInstanceBatchSingleCrit.md) : Class for Single Criterion Feature Selection - [`fsi()`](https://mlr3fselect.mlr-org.com/reference/fsi.md) : Syntactic Sugar for Feature Selection Instance Construction - [`fsi_async()`](https://mlr3fselect.mlr-org.com/reference/fsi_async.md) : Syntactic Sugar for Asynchronous Feature Selection Instance Construction - [`ObjectiveFSelect`](https://mlr3fselect.mlr-org.com/reference/ObjectiveFSelect.md) : Class for Feature Selection Objective - [`ObjectiveFSelectAsync`](https://mlr3fselect.mlr-org.com/reference/ObjectiveFSelectAsync.md) : Class for Feature Selection Objective - [`ObjectiveFSelectBatch`](https://mlr3fselect.mlr-org.com/reference/ObjectiveFSelectBatch.md) : Class for Feature Selection Objective - [`ArchiveAsyncFSelect`](https://mlr3fselect.mlr-org.com/reference/ArchiveAsyncFSelect.md) : Rush Data Storage - [`ArchiveAsyncFSelectFrozen`](https://mlr3fselect.mlr-org.com/reference/ArchiveAsyncFSelectFrozen.md) : Frozen Rush Data Storage - [`ArchiveBatchFSelect`](https://mlr3fselect.mlr-org.com/reference/ArchiveBatchFSelect.md) : Class for Logging Evaluated Feature Sets - [`fselect()`](https://mlr3fselect.mlr-org.com/reference/fselect.md) : Function for Feature Selection ## FSelectors - [`FSelector`](https://mlr3fselect.mlr-org.com/reference/FSelector.md) : FSelector - [`FSelectorAsync`](https://mlr3fselect.mlr-org.com/reference/FSelectorAsync.md) : Class for Asynchronous Feature Selection Algorithms - [`FSelectorBatch`](https://mlr3fselect.mlr-org.com/reference/FSelectorBatch.md) : Class for Batch Feature Selection Algorithms - [`mlr_fselectors_async_design_points`](https://mlr3fselect.mlr-org.com/reference/mlr_fselectors_async_design_points.md) [`FSelectorAsyncDesignPoints`](https://mlr3fselect.mlr-org.com/reference/mlr_fselectors_async_design_points.md) : Feature Selection with Asynchronous Design Points - [`mlr_fselectors_async_exhaustive_search`](https://mlr3fselect.mlr-org.com/reference/mlr_fselectors_async_exhaustive_search.md) [`FSelectorAsyncExhaustiveSearch`](https://mlr3fselect.mlr-org.com/reference/mlr_fselectors_async_exhaustive_search.md) : Feature Selection with Asynchronous Exhaustive Search - [`mlr_fselectors_async_random_search`](https://mlr3fselect.mlr-org.com/reference/mlr_fselectors_async_random_search.md) [`FSelectorAsyncRandomSearch`](https://mlr3fselect.mlr-org.com/reference/mlr_fselectors_async_random_search.md) : Feature Selection with Asynchronous Random Search - [`mlr_fselectors_design_points`](https://mlr3fselect.mlr-org.com/reference/mlr_fselectors_design_points.md) [`FSelectorBatchDesignPoints`](https://mlr3fselect.mlr-org.com/reference/mlr_fselectors_design_points.md) : Feature Selection with Design Points - [`mlr_fselectors_exhaustive_search`](https://mlr3fselect.mlr-org.com/reference/mlr_fselectors_exhaustive_search.md) [`FSelectorBatchExhaustiveSearch`](https://mlr3fselect.mlr-org.com/reference/mlr_fselectors_exhaustive_search.md) : Feature Selection with Exhaustive Search - [`mlr_fselectors_genetic_search`](https://mlr3fselect.mlr-org.com/reference/mlr_fselectors_genetic_search.md) [`FSelectorBatchGeneticSearch`](https://mlr3fselect.mlr-org.com/reference/mlr_fselectors_genetic_search.md) : Feature Selection with Genetic Search - [`mlr_fselectors_random_search`](https://mlr3fselect.mlr-org.com/reference/mlr_fselectors_random_search.md) [`FSelectorBatchRandomSearch`](https://mlr3fselect.mlr-org.com/reference/mlr_fselectors_random_search.md) : Feature Selection with Random Search - [`mlr_fselectors_rfe`](https://mlr3fselect.mlr-org.com/reference/mlr_fselectors_rfe.md) [`FSelectorBatchRFE`](https://mlr3fselect.mlr-org.com/reference/mlr_fselectors_rfe.md) : Feature Selection with Recursive Feature Elimination - [`mlr_fselectors_rfecv`](https://mlr3fselect.mlr-org.com/reference/mlr_fselectors_rfecv.md) [`FSelectorBatchRFECV`](https://mlr3fselect.mlr-org.com/reference/mlr_fselectors_rfecv.md) : Feature Selection with Recursive Feature Elimination with Cross Validation - [`mlr_fselectors_sequential`](https://mlr3fselect.mlr-org.com/reference/mlr_fselectors_sequential.md) [`FSelectorBatchSequential`](https://mlr3fselect.mlr-org.com/reference/mlr_fselectors_sequential.md) : Feature Selection with Sequential Search - [`mlr_fselectors_shadow_variable_search`](https://mlr3fselect.mlr-org.com/reference/mlr_fselectors_shadow_variable_search.md) [`FSelectorBatchShadowVariableSearch`](https://mlr3fselect.mlr-org.com/reference/mlr_fselectors_shadow_variable_search.md) : Feature Selection with Shadow Variable Search - [`mlr_fselectors`](https://mlr3fselect.mlr-org.com/reference/mlr_fselectors.md) : Dictionary of FSelectors - [`fs()`](https://mlr3fselect.mlr-org.com/reference/fs.md) [`fss()`](https://mlr3fselect.mlr-org.com/reference/fs.md) : Syntactic Sugar for Feature Selection Objects Construction - [`AutoFSelector`](https://mlr3fselect.mlr-org.com/reference/AutoFSelector.md) : Class for Automatic Feature Selection - [`auto_fselector()`](https://mlr3fselect.mlr-org.com/reference/auto_fselector.md) : Function for Automatic Feature Selection ## Nested Resampling - [`fselect_nested()`](https://mlr3fselect.mlr-org.com/reference/fselect_nested.md) : Function for Nested Resampling - [`extract_inner_fselect_archives()`](https://mlr3fselect.mlr-org.com/reference/extract_inner_fselect_archives.md) : Extract Inner Feature Selection Archives - [`extract_inner_fselect_results()`](https://mlr3fselect.mlr-org.com/reference/extract_inner_fselect_results.md) : Extract Inner Feature Selection Results ## Ensemble Feature Selection - [`ensemble_fselect()`](https://mlr3fselect.mlr-org.com/reference/ensemble_fselect.md) : Wrapper-based Ensemble Feature Selection - [`embedded_ensemble_fselect()`](https://mlr3fselect.mlr-org.com/reference/embedded_ensemble_fselect.md) : Embedded Ensemble Feature Selection - [`ensemble_fs_result`](https://mlr3fselect.mlr-org.com/reference/ensemble_fs_result.md) [`EnsembleFSResult`](https://mlr3fselect.mlr-org.com/reference/ensemble_fs_result.md) : Ensemble Feature Selection Result ## Callbacks - [`CallbackAsyncFSelect`](https://mlr3fselect.mlr-org.com/reference/CallbackAsyncFSelect.md) : Asynchronous Feature Selection Callback - [`CallbackBatchFSelect`](https://mlr3fselect.mlr-org.com/reference/CallbackBatchFSelect.md) : Create Feature Selection Callback - [`ContextAsyncFSelect`](https://mlr3fselect.mlr-org.com/reference/ContextAsyncFSelect.md) : Asynchronous Feature Selection Context - [`ContextBatchFSelect`](https://mlr3fselect.mlr-org.com/reference/ContextBatchFSelect.md) : Evaluation Context - [`callback_async_fselect()`](https://mlr3fselect.mlr-org.com/reference/callback_async_fselect.md) : Create Asynchronous Feature Selection Callback - [`callback_batch_fselect()`](https://mlr3fselect.mlr-org.com/reference/callback_batch_fselect.md) : Create Feature Selection Callback - [`mlr3fselect.async_freeze_archive`](https://mlr3fselect.mlr-org.com/reference/mlr3fselect.async_freeze_archive.md) : Freeze Archive Callback - [`mlr3fselect.backup`](https://mlr3fselect.mlr-org.com/reference/mlr3fselect.backup.md) : Backup Benchmark Result Callback - [`mlr3fselect.internal_tuning`](https://mlr3fselect.mlr-org.com/reference/mlr3fselect.internal_tuning.md) : Internal Tuning Callback - [`mlr3fselect.one_se_rule`](https://mlr3fselect.mlr-org.com/reference/mlr3fselect.one_se_rule.md) : One Standard Error Rule Callback - [`mlr3fselect.svm_rfe`](https://mlr3fselect.mlr-org.com/reference/mlr3fselect.svm_rfe.md) : SVM-RFE Callback - [`assert_async_fselect_callback()`](https://mlr3fselect.mlr-org.com/reference/assert_async_fselect_callback.md) [`assert_async_fselect_callbacks()`](https://mlr3fselect.mlr-org.com/reference/assert_async_fselect_callback.md) : Assertions for Callbacks ## Reexports - [`reexports`](https://mlr3fselect.mlr-org.com/reference/reexports.md) [`mlr_terminators`](https://mlr3fselect.mlr-org.com/reference/reexports.md) [`trm`](https://mlr3fselect.mlr-org.com/reference/reexports.md) [`trms`](https://mlr3fselect.mlr-org.com/reference/reexports.md) [`mlr_callbacks`](https://mlr3fselect.mlr-org.com/reference/reexports.md) [`clbk`](https://mlr3fselect.mlr-org.com/reference/reexports.md) [`clbks`](https://mlr3fselect.mlr-org.com/reference/reexports.md) : Objects exported from other packages ## Miscellaneous - [`faggregate()`](https://mlr3fselect.mlr-org.com/reference/faggregate.md) : Fast Aggregation of ResampleResults and BenchmarkResults - [`assert_fselectors()`](https://mlr3fselect.mlr-org.com/reference/mlr3fselect_assertions.md) [`assert_fselector_async()`](https://mlr3fselect.mlr-org.com/reference/mlr3fselect_assertions.md) [`assert_fselector_batch()`](https://mlr3fselect.mlr-org.com/reference/mlr3fselect_assertions.md) [`assert_fselect_instance()`](https://mlr3fselect.mlr-org.com/reference/mlr3fselect_assertions.md) [`assert_fselect_instance_async()`](https://mlr3fselect.mlr-org.com/reference/mlr3fselect_assertions.md) [`assert_fselect_instance_batch()`](https://mlr3fselect.mlr-org.com/reference/mlr3fselect_assertions.md) : Assertion for mlr3fselect objects ## Package - [`mlr3fselect`](https://mlr3fselect.mlr-org.com/reference/mlr3fselect-package.md) [`mlr3fselect-package`](https://mlr3fselect.mlr-org.com/reference/mlr3fselect-package.md) : mlr3fselect: Feature Selection for 'mlr3'