High-Level API

Help

The AutoTranformers library automatically transforms data into machine learning models (AutoTransformer). This library supports many different domains (e.g. text, documents), as well as tasks (e.g. information extraction, classification). Additionally, each AutoTransformer can predict many tasks in parallel if required.

More details can be found at https://autotransformers.ai

autotransformers.help(domain=None)
The help function can be used to get more information about

  • Domains (e.g. text, docs)

  • Addons (e.g. active learning)

But also domain details such as

  • List of models (e.g. Bert, Roberta etc.)

  • Supported tasks (e.g. information extraction, single label classification etc.)

  • Implemented task heads (e.g. OMI)

  • Hyperparameter (e.g. learning rate)

  • Modules (e.g. WanDB tracking, early stopping etc.)

Parameters

domain (Optional) – If help for a given domain should be shown, provide this parameter (e.g. “text”)

AutoTransformer

The at module provides the high-level (HL) API to automatically transform arbitrary and unstructured data into machine learning models. Also addons such as active learning can be executed using this module.

<img src=”/_static/img/architecture/at.config_store@rt.png” />

autotransformers.at.init_logging(level=20)

Initialize internal logging of AutoTransformers with the given log level.

Parameters

level (int) – Log level as specified in the logging package.

autotransformers.at.init_deepspeed(*, optional=False, dist_backend='nccl')
Initialize the current process to run with DeepSpeed, if DeepSpeed is initialized and the script is executed

with deepspeed myscript.py.

Parameters

  • optional (bool) – If False, the function raises an exception if DeepSpeed is not installed or the current script is not executed with deepspeed myscript.py. If True, no exception is thrown and only a warning is shown. This is useful to provide a script for both, pytorch or DeepSpeed execution.

  • dist_backend (Optional[str]) – Torch distributed backend, e.g., nccl, mpi, gloo. Note that nccl is not supported for Windows.

Returns

True if DeepSpeed was initialized successfully, False otherwise. Note that this is only returned if optional=True.

class autotransformers.at.AutoTransformer(sys_config=None)

Creates an AutoTransformer that can be used either to load an already existing AutoTransformer from the disk or to automatically convert a given dataset into a model by calling the AutoTransformer.train function.

Parameters

sys_config (Union[dict, list, str, None]) – Dict, list with key values or a path (str) to your sys config which could contain details how to distribute etc.

class ATPrediction(input: Any, output: Any)

AutoTransformers inference returns

input: Any

Alias for field number 0

output: Any

Alias for field number 1

finish()

In case the current at should be freed, call this function.

set_config(key, value)

Set a single config value based on a given key where each hierarchy is separated with ‘/’. Returns the at instance again such that set_config can be called several times i.e. at.set_config(…).set_config(…) etc. To later on show all your configs simply call summary(). autotransformers.at.AutoTransformer

Note that this function must be called before you init.

summary(**kwargs)

This function (optionally) plots and returns all hyperparameters of your initialized AT

init(*, dataset_loader, model_name_or_path=None, path, custom_modules=None)

This function initializes a new AutoTransformer from scratch. Once we set the model_name or path, its no more possible to change those things!

Parameters

  • dataset_loader (Any) – The input dataset loader. This argument is required to do proper HPO.

  • model_name_or_path (Optional[str]) – The name or the path of the base model to that should be used. Note that task-heads are ignored.

  • path (Optional[str]) – Path to store data during training. Must be given if modules that persist data are used, such as Checkpointing or EarlyStopping.

  • custom_modules (Optional[list[Any]]) – Additional modules that can be injected to the AT in order to get callbacks during training. The training can be stopped, the training progress can be traced etc.

Returns

A newly initialized AutoTransformers instance

train(**kwargs)

Given a dataset loader, this method automatically creates an AutoTransformer model. If the path is a training checkpoint, the weights and optimizer state is loaded and training is continued. If the AutoTransformer was stored with AutoTransformers.save, no optimizer states are available and therefore, only the model weights are loaded and the training is started from epoch 0.

Parameters

  • dataset_loader – The DatasetLoader to train the model with.

  • eval_split – Fraction of the training set to split off in order to evaluate the model on unseen data after each epoch of training. This parameter is ignored if the dataset_loader supplies an evaluation set. Also if the eval_split is None and no evaluation set is given, evaluation is performed on the test set. This is not recommended and should only be used for debugging purposes. Sometimes this is also necessary to compare results against some papers as they do not use a validation set.

Returns

Test metrics.

evaluate(**kwargs)

Evaluate a trained AutoTransformer on a test dataset and report metrics.

Parameters

test_ds – The dataset that is used to test the performance of your model.

Returns

Dictionary of the AutoTransformer’s metrics. Only metrics that are enabled at testing time are calculated.

sample(**kwargs)

For unlabeled data, this function returns the acquisition_size samples that should be labeled first in order to gain the largest performance boost with minimal labeling effort.

Parameters

  • dataset – The unlabeled dataset pool where we search for the acquisition_size best samples to label next

  • method – The method that should be used for sampleng. Default: uncertainty_breaking_ties_sampling. Supports uncertainty_breaking_ties_sampling, uncertainty_entropy_sampling and random

  • acquisition_size – How many samples should be returned.

  • batch_size – The internal size of batches that is inferenced.

  • seed – For random sampling a seed can be provided

  • kwargs – Additional arguments which are directly forwarded to engine.__call__

Returns

(sampled_ds, rest_ds) where sampled_ds are acquisition_size samples and rest_ds are the remaining

samples that should not be labeled next.

save(**kwargs)

Saves the AutoTransformers to the given path that can be used for inference later on.

Note that the saved AutoTransformer can be used for inference, but optimizer checkpoints are not stored. Checkpoints are automatically created during training using the given path in __init__.

Parameters

  • path – Path where the current AutoTransformer instance should be stored.

  • makedirs – Whether the dirs should be created if they don’t exist.

load(path, *, optional=False, custom_modules=None)
This function loads an AutoTransformer from the disk stored.

Note that your AutoTransformer is automatically loaded from a training checkpoint if it exists. Otherwise the given path is loaded.

Parameters

  • path – The base path where the model should be loaded.

  • optional – If the model does not exist, return the randomly initialized AutoTransformer.

  • custom_modules (Optional[list[Any]]) – Additional modules that can be injected to the AT in order to get callbacks during training. The training can be stopped, the training progress can be traced etc.

Returns

The AutoTransformer as stored on the disk.