A joint research team led by Xu Bo from the Institute of Automation and Mu-Ming Poo from the Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, have discovered that self-backpropagation, a form of mesoscale synaptic plasticity rule in natural neural networks, can elevate the accuracy and reduce the computational cost of spiking neural networks (SNNs) and artificial neural networks (ANNs).
Their findings were published in Science Advances on Oct. 20.
Previous studies proved that self-backpropagation (SBP) is caused by the rapid retrograde axonal transport of molecular signals. It is considered to be the key for efficient and flexible learning of biological neural networks.
The backpropagation (BP) algorithm in artificial neural networks uses a global strategy for optimization, which can achieve excellent performance but, at the same time, take too much computational cost during learning.
The researchers introduced biologically plausible SBP into a three-layer SNN. They found an elevated accuracy of network performance in three standard benchmark tasks, MNIST, NETtalk, and DvsGesture.
“The computational cost in terms of the product of mean epochs and algorithmic complexity per epoch was markedly reduced,” said Xu. Similar results were obtained by further applying SBP on Restricted Boltzmann Machine.
According to the study, SBP is a special mesoscale biological plasticity mechanism, indicating a similar important role of SBP in SNNs compared to its counterpart BP in ANNs.
This will attract attention in the field of machine learning because training SNNs with pure biologically plausible algorithms (e.g., spike-timing-dependent plasticity) is difficult, in which the information is spatio-temporal and carried by discontinuous spikes.
The study has paved a way towards biologically plausible effective learning on both SNNs and ANNs, with high accuracy and low computational cost for learning different tasks.
Brain-on-a-chip would need little training
More information:
Tielin Zhang et al, Self-backpropagation of synaptic modifications elevates the efficiency of spiking and artificial neural networks, Science Advances (2021). DOI: 10.1126/sciadv.abh0146
Provided by
Chinese Academy of Sciences
Citation:
Mesoscale neural plasticity helps in AI learning (2021, October 22)
retrieved 22 October 2021
from https://techxplore.com/news/2021-10-mesoscale-neural-plasticity-ai.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
part may be reproduced without the written permission. The content is provided for information purposes only.
