Container around a data.table::data.table() which stores all evaluated feature sets and performance scores.

Data structure

The table ($data) has the following columns:

  • One column for each feature of the task ($search_space).

  • One column for each performance measure ($codomain).

  • runtime_learners (numeric(1))
    Sum of training and predict times logged in learners per mlr3::ResampleResult / evaluation. This does not include potential overhead time.

  • timestamp (POSIXct)
    Time stamp when the evaluation was logged into the archive.

  • batch_nr (integer(1))
    Feature sets are evaluated in batches. Each batch has a unique batch number.

  • uhash (character(1))
    Connects each feature set to the resampling experiment stored in the mlr3::BenchmarkResult.

Each row corresponds to a single evaluation of a feature set.

The archive stores additionally a mlr3::BenchmarkResult ($benchmark_result) that records the resampling experiments. Each experiment corresponds to to a single evaluation of a feature set. The table ($data) and the benchmark result ($benchmark_result) are linked by the uhash column. If the results are viewed with as.data.table(), both are joined automatically.

Analysis

For analyzing the feature selection results, it is recommended to pass the archive to as.data.table(). The returned data table is joined with the benchmark result which adds the mlr3::ResampleResult for each feature set.

The archive provides various getters (e.g. $learners()) to ease the access. All getters extract by position (i) or unique hash (uhash). For a complete list of all getters see the methods section.

The benchmark result ($benchmark_result) allows to score the feature sets again on a different measure. Alternatively, measures can be supplied to as.data.table().

S3 Methods

  • as.data.table.ArchiveFSelect(x, unnest = NULL, exclude_columns = "uhash", measures = NULL)
    Returns a tabular view of all evaluated feature sets.
    ArchiveFSelect -> data.table::data.table()

    • x (ArchiveFSelect)

    • unnest (character())
      Transforms list columns to separate columns. Set to NULL if no column should be unnested.

    • exclude_columns (character())
      Exclude columns from table. Set to NULL if no column should be excluded.

    • measures (list of mlr3::Measure)
      Score feature sets on additional measures.

Super class

bbotk::Archive -> ArchiveFSelect

Public fields

benchmark_result

(mlr3::BenchmarkResult)
Stores benchmark result.

Methods

Public methods

Inherited methods

Method learner()

Retrieve mlr3::Learner of the i-th evaluation, by position or by unique hash uhash. i and uhash are mutually exclusive. Learner does not contain a model. Use $learners() to get learners with models.

Usage

ArchiveFSelect$learner(i = NULL, uhash = NULL)

Arguments

i

(integer(1))
The iteration value to filter for.

uhash

(logical(1))
The uhash value to filter for.


Method learners()

Retrieve list of trained mlr3::Learner objects of the i-th evaluation, by position or by unique hash uhash. i and uhash are mutually exclusive.

Usage

ArchiveFSelect$learners(i = NULL, uhash = NULL)

Arguments

i

(integer(1))
The iteration value to filter for.

uhash

(logical(1))
The uhash value to filter for.


Method predictions()

Retrieve list of mlr3::Prediction objects of the i-th evaluation, by position or by unique hash uhash. i and uhash are mutually exclusive.

Usage

ArchiveFSelect$predictions(i = NULL, uhash = NULL)

Arguments

i

(integer(1))
The iteration value to filter for.

uhash

(logical(1))
The uhash value to filter for.


Method resample_result()

Retrieve mlr3::ResampleResult of the i-th evaluation, by position or by unique hash uhash. i and uhash are mutually exclusive.

Usage

ArchiveFSelect$resample_result(i = NULL, uhash = NULL)

Arguments

i

(integer(1))
The iteration value to filter for.

uhash

(logical(1))
The uhash value to filter for.


Method print()

Printer.

Usage

ArchiveFSelect$print()

Arguments

...

(ignored).


Method clone()

The objects of this class are cloneable with this method.

Usage

ArchiveFSelect$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.