Scaling of Hardware-Compatible Perturbative Training Algorithms
With the rapid development of artificial intelligence (AI) technology, artificial neural networks (ANNs) have achieved significant success in multiple fields. However, traditional neural network training methods—especially the backpropagation algorithm—face numerous challenges in hardware implementation. Although the backpropagation algorithm is efficient in software, its hardware implementation requires reversible computational paths, substantial memory at each neuron, and the computation of activation function derivatives, which are difficult to achieve in hardware. Additionally, traditional complementary metal-oxide-semiconductor (CMOS) hardware consumes significant energy during the training and deployment of these algorithms, limiting their scalability and widespread adoption.
To address these issues, researchers have begun exploring brain-inspired hardware solutions, particularly analog neuromorphic hardware. This type of hardware can achieve similar computational capabilities at a lower energy cost, but effective training on analog hardware remains a challenge. Perturbative training methods have emerged as an alternative, estimating the gradient of the loss function by randomly perturbing network parameters, thereby avoiding the complexities of backpropagation in hardware. However, perturbative training methods are considered less scalable for large-scale problems because the time to estimate the gradient scales linearly with the number of network parameters.
The goal of this study is to explore a perturbative training framework called Multiplexed Gradient Descent (MGD) and validate its scalability and effectiveness in large-scale networks. MGD defines a set of time constants related to the perturbation process, enabling efficient gradient estimation in hardware and compatibility with existing optimization accelerators (e.g., momentum), thus providing a practical solution for future neuromorphic computing systems.
Source of the Paper
This paper was co-authored by B. G. Oripov, A. Dienstfrey, A. N. McCaughan, and S. M. Buckley, affiliated with the Department of Physics, University of Colorado Boulder, and the National Institute of Standards and Technology (NIST). The paper was published on April 17, 2025, in the journal APL Machine Learning, titled “Scaling of Hardware-Compatible Perturbative Training Algorithms,” as part of the special collection “Neuromorphic Technologies for Novel Hardware AI.” The DOI of the paper is 10.1063⁄5.0258271.
Research Process and Results
1. Research Process
a) Introduction and Extension of the MGD Framework
MGD is a hardware-friendly perturbative training framework that estimates the gradient of the loss function by randomly perturbing network parameters. Unlike traditional perturbative methods, MGD introduces three time constants corresponding to the time between weight updates, sample updates, and perturbation updates. By adjusting these time constants, MGD can implement various numerical gradient descent techniques, such as coordinate descent and simultaneous perturbation stochastic approximation (SPSA).
In this study, the authors extended the MGD framework to include weight perturbation and node perturbation, discussing the advantages and disadvantages of each approach. Weight perturbation directly perturbs each weight, while node perturbation perturbs the input to the activation function and computes weight updates through single-layer backpropagation.
b) Analysis of Gradient Estimation and Training Time
Through simulation experiments, the authors investigated the gradient estimation time and training time of MGD under different network sizes and task complexities. The experiments used a neural network architecture consisting of six convolutional layers and three fully connected layers, trained on the FashionMNIST dataset for classification tasks. The network size was varied by adjusting the depth (d) of each layer, with the number of parameters ranging from thousands to millions.
To measure the accuracy of gradient estimation, the authors generated a new gradient estimate in each iteration and compared it to the true gradient computed via backpropagation. The results showed that node perturbation outperformed weight perturbation in gradient estimation time, as node perturbation involves fewer independent perturbations.
c) Network Training and Optimization
The authors further examined the performance of MGD in training large-scale networks. The experimental results demonstrated that MGD could achieve the same test accuracy as backpropagation without gradient averaging. Additionally, the authors validated the compatibility of MGD with existing optimization algorithms, such as momentum and the Adam optimizer, and demonstrated their effectiveness within the MGD framework.
2. Key Results
a) Accuracy of Gradient Estimation
The experimental results showed that MGD’s gradient estimation could accurately approximate the true gradient after sufficient iterations. Node perturbation significantly outperformed weight perturbation in gradient estimation time, especially in large-scale networks. Specifically, the gradient estimation time for weight perturbation scaled linearly with the number of network parameters, while for node perturbation, it scaled with the square root of the number of parameters.
b) Scalability of Training Time
Although the gradient estimation time increased with network size, the training time did not follow the same linear scaling trend. The experiments showed that MGD could increase the training time by less than an order of magnitude while increasing the network size by three orders of magnitude. This indicates that MGD’s scalability in large-scale networks exceeds expectations.
c) Compatibility with Optimizers
The authors demonstrated the compatibility of MGD with momentum and the Adam optimizer. The experimental results showed that using the Adam optimizer could significantly reduce training time, further proving MGD’s potential in practical hardware.
Conclusions and Significance
This study demonstrates that MGD, as a hardware-compatible perturbative training method, can efficiently train large-scale networks and achieve accuracy comparable to backpropagation. MGD’s scalability challenges the conventional belief that perturbative methods are poorly scalable for large-scale problems, providing a practical solution for future neuromorphic computing systems.
Highlights of the Research
- Scalability Validation: MGD’s excellent scalability in large-scale networks breaks the limitations of traditional perturbative methods.
- Hardware Compatibility: MGD can be efficiently implemented in hardware and is compatible with existing optimization algorithms, offering broad application prospects.
- Comparison of Node and Weight Perturbation: Node perturbation outperforms weight perturbation in gradient estimation time, especially in large-scale networks.
Additional Valuable Information
The authors also explored the optimization potential of MGD on different hardware platforms. For example, for non-volatile memory with slow write speeds, increasing the gradient integration time can reduce the number of weight updates, thereby extending the hardware’s lifespan. Additionally, the flexibility of the MGD framework allows it to adapt to different hardware constraints and requirements.
This study provides an efficient and scalable solution for training neuromorphic hardware, offering significant scientific value and application potential.