Lottery

by Javier Ideami

Title

Lottery

Artist

Javier Ideami

Medium

Digital Art - Artificial Intelligence Digital Art

Description

Lottery visualizes from a cenital view the loss landscapes of a Resnet18 (Mnist dataset) as the weights of the network are gradually being pruned (based on arxiv:1803.03635 by jonathan frankle, michael carbin).
Up to 80% pruning it can be observed in this specific network that the performance of the retrained networks with pruned weights can equal or exceed the original one when evaluating the test dataset.
Loss Landscape generated with real data: resnet18 / mnist, sgd-adam, bs=60, lr sched, eval mod, log scaled (orig loss nums) & vis-adapted.

This piece has been created through a multi day process composed of different stages that begin with the training of deep learning neural networks and proceeds through a number of phases until arriving to the final artwork.

When analyzing the loss landscape generated while increasing dropout, we see a noise layer taking over the landscape, a layer that is disruptive enough to help in preventing overfitting and the memorization of paths and routes across the landscape, and yet not disruptive enough to prevent convergence to a good minima (unless dropout is taken to extreme values).

Created by Javier Ideami.
For more about the Loss Landscape project visit https://losslandscape.com

Uploaded

July 28th, 2020

Statistics

Viewed 226 Times  -  Last Visitor from New York, NY on 04/23/2024 at 12:31 PM

Colors

Embed

Share

Sales Sheet

Tags