vp_suite.vpsuite

class VPSuite(device='cuda')

Bases: object

This class is the main workbench of vp-suite, in the sense that upon instantiating this class you’ve got access to all the functionalities of vp-suite: Loading and using datasets, loading or instantiating new models and training or testing them on the loaded datasets.

device

A string specifying which device to work on (‘cuda’ for GPU usage or ‘cpu’ for CPU).

Type

str

datasets

The loaded datasets.

Type

List[VPDatasetWrapper]

models

The loaded/created models.

Type

List[VPModel]

__init__(device='cuda')

Instantiates the VPSuite, setting device and clearing loaded models and datasets.

Parameters

device (str) – A string identifying whether to use GPU or CPU.

clear_datasets()

Empties the list of loaded datasets.

clear_models()

Empties the list of loaded models.

create_model(model_id, action_conditional=False, **model_kwargs)

Creates the model specified by given string ID and appends it to VPSuite’s list of loaded models.

Parameters

  • model_id (str) – The string ID corresponding to your desired model.

  • action_conditional (bool) – If the model supports actions, this variable determines whether the model will actually use provided actions for prediction.

  • **model_kwargs (Any) – Optional additional model configuration options.

download_dataset(dataset_id)

Downloads the dataset specified by given dataset ID.

Parameters

dataset_id (str) – The string ID mapping of the desired dataset.

hyperopt(optuna_config, n_trials=30, dataset_idx=- 1, model_idx=- 1, **run_kwargs)

Executes a hyperparameter optimization process using the bayesian optimization framework optuna. A pre-defined number of training runs (“trials”) are executed sequentially on the same model and datasets under varying (hyperparameter) configurations. After the optimization, the best configuration is printed to the terminal.

Parameters

  • optuna_config (dict) – Optuna run configuration, specifies the search space (parameters and their search ranges) of the optimization.

  • n_trials (int) – The number of training runs executed within the optimization.

  • dataset_idx (int) – The list index of the dataset that should be used for training.

  • model_idx (int) –

  • **run_kwargs (Any) – Optional specified run configuration parameters (will override the defaults). Apply to all runs within the optimizaion.

list_available_datasets()

Prints a list of all available datasets and their corresponding string IDs.

list_available_models()

Prints a list of all available models and their corresponding string IDs.

load_dataset(dataset_id, split='train', **dataset_kwargs)

Creates the dataset specified by given dataset id and appends it to VPSuite’s list of loaded datasets.

Parameters

  • dataset_id (str) – The string ID mapping of the desired dataset.

  • split (str) – This string specifies whether to load the dataset in training or testing mode.

  • **dataset_kwargs (Any) – Optional additional dataset configuration options.

load_model(model_dir, ckpt_name='best_model.pth')
Loads the model saved in the specified checkpoint file or the specified directory

and appends it to VPSuite’s list of loaded models.

Parameters

  • model_dir (str) – Relative path to the directory containing the saved model.

  • ckpt_name (str) – File name of the saved model.

reset_rng(seed)

Resets the RNG for all datasets and rng.

Parameters

seed (int) – The random seed for the rng’s.

test(brief_test=False, **run_kwargs)

Tests all loaded models and datasets, one dataset after the other. For each dataset test, the whole test set is iterated through and each data point is fed into each model to obtain performance metrics. After iteration, visualizations are created of the predictions of all models on randomly selected frame sequences, and performance metrics of each model on each dataset are logged in a comparable way.

Parameters

  • brief_test (bool) – If specified, only a brief sample check is done instead of fully iterating over the test set.

  • **run_kwargs (Any) – Optional testing configuration.

property test_sets

A list of all loaded test sets (last one in list is last loaded test set).

Type

Returns

train(trial=None, dataset_idx=- 1, model_idx=- 1, **run_kwargs)

Executes a training run on a single model and a single dataset. After preparation, the main training loops run training epochs until the max number of epochs or the time limit is reached.

During each epoch: 1. the whole dataset is iterated through in (model-specific) training iterations that employ forward and backward passes. 2. Each model is validated on the whole validation set and saved if it improved its performance on that set. 3. Every few epochs, prediction visualizations are created and saved to the disk. 4. Current model performance is logged.

Parameters

  • trial (Any) – If calling this function within a hyperparameter optimization run, this object cantains the necessary parameters. Otherwise, it’s None.

  • dataset_idx (int) – The list index of the dataset that should be used for training.

  • model_idx (int) –

  • **run_kwargs (Any) – Optional specified run configuration parameters (will override the defaults).

Returns: The best obtained validation loss (the corresponding model is saved as ‘best_model.pt’).

property training_sets

A list of all loaded training sets (last one in list is last loaded training set).

Type

Returns