Evolution Sets the Stage for More Powerful Spiking Neural Networks

Indirect encoding of neural network components creates a pathway to complex applications for neural networks.

Image courtesy of Catherine Schuman
Left, an overview of the new evolutionary approach. Right, comparison of the new approach with another evolutionary approach to SNNs (EONS) showing the new approach reaches comparable performance to EONS, but does so more gradually, which can indicate better generalization ability of the resulting SNN.

The Science

Spiking neural networks (SNNs) are a type of artificial neural network that most closely replicates the structure of the human brain. This makes SNNs an important step on the road to developing artificial intelligence. Researchers recently advanced a key technique for training SNNs using an evolutionary approach. This approach involves recognizing and making use of the different strengths of individual elements of the SNN, called “encodings.” The researchers’ approach involves indirect encoding, which replicates encodings instead of creating an entirely new encoding for each individual. This is a much more efficient way to build large—and therefore smarter—SNNs.

The Impact

Indirect encoding helps scientists build significantly larger neural networks with more complex connections between neurons. These changes would help bring SNNs to increasingly complex applications. For example, improved SNNs would bring new power to the complex task of classifying large scientific images. By improving the performance of artificial intelligence systems, this research could enable scientists to find solutions to the challenges caused by climate change. It will also help scientists discover or improve clean energy technologies.

Summary

Spiking neural networks offer tremendous potential for the future of artificial intelligence. For one, they can be implemented efficiently on neuromorphic systems, which closely mimic biological brains. One of the challenges in building functioning SNNs is the training process. Standard training processes are difficult to apply to SNNs. In this work, scientists created an evolutionary approach for training SNNs implementing an indirect encoding of individuals. Specifically, the researchers evolved SNNs using Compositional Pattern Producing Networks (CPPNs), which can learn the connectivity patterns between neurons defined in a coordinate space. Using CPPNs rather than direct encodings to evolve parameters allows representations of small numbers of individual encodings (on the order of tens or hundreds of individual encodings) to potentially represent thousands to millions of SNN parameter values.

Contact

Daniel Elbrecht
University of Central Florida
[email protected]

Catherine Schuman
Oak Ridge National Laboratory
[email protected]

Funding

This research was funded by the Department of Energy Office of Science, Advanced Scientific Computing Research program.

Publications

Elbrecht, D. and Schuman, C., "Neuroevolution of Spiking Neural Networks Using Compositional Pattern Producing Networks." International Conference on Neuromorphic Systems, July 28-30, 2020. [DOI: 10.1145/3407197.3407198.]

Highlight Categories

Program: ASCR

Performer: University