Devcon Archive

Watch Your Models Learn: Real-Time AI/ML Experiments with Visdom

Duration: NaN:NaN:NaN

Speaker: Norbert Preining

Type: CLS - Talk (Open Tech Summit)

Expertise:

Event: Devcon

Date: Nov 2026

Training an AI model can take minutes, hours or days. Waiting until the end to inspect a log file is often too late: loss may already be diverging, gradients may be behaving unexpectedly, validation performance may have stopped improving, or a data pipeline may simply be producing the wrong results. Visdom provides an open-source way to see what is happening while an experiment is running. In this session, Norbert Preining will show how Python applications and machine-learning workflows can send live data to Visdom and turn a running experiment into an interactive dashboard. Starting with only a few lines of Python, we will look at streaming loss and accuracy curves, NumPy data, images and model outputs directly into the browser. The session will then move from individual plots to complete ML workflows: monitoring PyTorch and PyTorch Lightning training, collecting TensorFlow/Keras metrics, organizing experiments, recording hyperparameters and comparing runs. We will also look at how Visdom can be used to inspect images, embeddings and other intermediate results while a model is still training. A key idea behind Visdom is that visualization should remain separate from the computation itself. Your training framework computes the metrics; Visdom receives, stores and visualizes them. This keeps the integration small, makes Visdom usable with many different frameworks and allows developers to run the visualization server locally, on their own infrastructure or alongside remote training systems. The latest Visdom work also makes this approach useful for larger experiment workflows: experiments can carry parameters, metrics and tags, runs can be searched and compared, and live results can be explored through hyperparameter and experiment views. The talk will use short code examples and live visualizations to show how quickly a normal Python or ML training loop can become observable. Whether you are training a PyTorch model, working with TensorFlow/Keras, exploring NumPy data or building your own AI pipeline, the goal is the same: **don't wait for the experiment to finish to understand what it is doing.**