Progress tracking

AutoTransformers supports ClearML to log and visualize a model’s metrics during training. This example shows how you can track your model’s training progress using the ClearML Web Interface.

Before starting, create a ClearML account and set up the credentials on your machine. Refer to the ClearML tutorial for further instructions.

# Make sure ClearML is installed
%pip install ClearML

from autotransformers import AutoTransformer, DatasetLoader

As always, we use the googleplay dataset to test.

# The text snippets in this dataset are from "googleplay", a public dataset of app reviews on Google's Play Store.
dataset = {
    "meta": {
        "name": "example_singlelabel",
        "version": "1.0.0",
        "created_with": "wizard"
    },
    "config": [
        {
            "domain": "text",
            "type": "IText"
        },
        {
            "task_id": "task1",
            "classes": ["positive", "neutral", "negative"],
            "type": "TSingleClassification"
        }
    ],
    "train": [
        [
            {"value": "None of the notifications work. other people in the forums teport similar problems bht no fix. the app is nice but it isnt nearly as functional without notifications"},
            {"value": "negative"},
        ],
        [
            {"value": "It's great"},
            {"value": "positive"},
        ],
        [
            {"value": "Not allowing me to delete my account"},
            {"value": "negative"},
        ],
        [
            {"value": "So impressed that I bought premium on very first day"},
            {"value": "positive"},
        ],
    ],
    "test": [
        [
            {"value": "Can't set more than 7 tasks without paying an absurdly expensive weekly subscription"},
            {"value": "negative"},
        ]
    ],
}

In the configuration, we enable ClearML tracking and disable tracking on the console.

dl = DatasetLoader(dataset)

config = [
    ("engine/stop_condition/type", "MaxEpochs"),
    ("engine/stop_condition/value", 6),
    ("engine/modules/tracking.ClearML/enabled", True),
    ("engine/modules/tracking.ClearML/task", "MyExperiment"),
    ("engine/modules/tracking.Console/enabled", False),
]
at = AutoTransformer(config)

When starting a training run, a new ClearML page is automatically created, and logs are uploaded to it during training.

Once you start executing the cell below, a link to the tracking page will be displayed.

at.init(dataset_loader=dl, path=".models/example08")
at.train(dl)