Multiplexed gradient descent: Fast online training of modern datasets on hardware neural networks without backpropagation

• DOI: 10.48550/arxiv.2303.03986
• Publication Date: 2023-01-01
• Authors: Adam N. McCaughan; Bakhrom Oripov; Natesh Ganesh; Sae Woo Nam; Andrew Dienstfrey; Sonia Buckley
• Keywords: MNIST database; Backpropagation; Hardware acceleration

We present multiplexed gradient descent (MGD), a framework designed to easily train analog or digital neural networks in hardware. MGD utilizes zero-order optimization techniques for online training of hardware networks. demonstrate its ability on modern machine learning datasets, including CIFAR-10 and Fashion-MNIST, compare performance backpropagation. Assuming realistic timescales parameters, our results indicate that these can network emerging platforms orders magnitude faster than the wall-clock time via backpropagation standard GPU, even presence imperfect weight updates device-to-device variations additionally describe how it be applied existing as part chip-in-the-loop training, integrated directly at level. Crucially, is highly flexible, process optimized compensate specific limitations such slow parameter-update speeds limited input bandwidth.