Synthetic nervous system


Synthetic nervous system is a computational neuroscience model that may be developed with the Functional Subnetwork Approach to create biologically plausible models of circuits in a nervous system. The FSA enables the direct analytical tuning of dynamical networks that perform specific operations within the nervous system without the need for global optimization methods like genetic algorithms and reinforcement learning. The primary use case for a SNS is system control, where the system is most often a simulated biomechanical model or a physical robotic platform. An SNS is a form of a neural network much like artificial neural networks, convolutional neural networks, and recurrent neural networks. The building blocks for each of these neural networks is a series of nodes and connections denoted as neurons and synapses. More conventional artificial neural networks rely on training phases where they use large data sets to form correlations and thus "learn" to identify a given object or pattern. When done properly this training results in systems that can produce a desired result, sometimes with impressive accuracy. However, the systems themselves are typically "black boxes" meaning there is no readily distinguishable mapping between structure and function of the network. This makes it difficult to alter the function, without simply starting over, or extract biological meaning except in specialized cases. The SNS method differentiates itself by using details of both structure and function of biological nervous systems. The neurons and synapse connections are intentionally designed rather than iteratively changed as part of a learning algorithm.
As in many other computational neuroscience models, the details of a neural model are informed by experimental data wherever possible. Not every study can measure every parameter of the network under investigation, requiring the modeler to make assumptions regarding plausible parameter values. Rybak uses a sampling method where each node is composed of many neurons and each particular neuron's parameters are pulled from a probability distribution. Eliasmith uses what they call the Neural Engineering Framework in which the user specifies the functions of the network and the synaptic and neural properties are learned over time. SNS follows a similar approach via the Functional Subnetwork Approach. FSA allows parameters within the network to be designed analytically based on their intended function. As a result, it is possible to use this approach to directly assemble networks that perform basic functions, like addition or subtraction, as well as dynamical operations like differentiation and integration.

Background and history of synthetic nervous systems

Background

The details of the underlying control networks for many biological systems are not very well understood. However, recent advancements in neuroscience tools and techniques have clarified the cellular and biophysical mechanisms of these networks, and their operation during behavior in complex environments. Although there is a long-standing interest in biologically-inspired robots and robotic platforms, there is a recent interest in incorporating features of biomechanics and neural control, e.g., biomimicry. The SNS method uses data from neuroscience in control systems for neuromechanical simulations and robots. Designing both a robot's mechanics and controller to capture key aspects of a particular animal may lead to more flexible functionality while suggesting new hypotheses for how the animal's nervous system works.
Keeping neural models simple facilitates analysis, real time operation, and tuning. To this end, SNSs primarily model neurons as leaky integrators, which are reasonable approximations of sub-threshold passive membrane dynamics. The leaky integrator also models non-spiking interneurons which contribute to motor control in some invertebrates. If spiking needs to be incorporated into the model, nodes may be represented using the leaky integrate-and-fire models. In addition, other conductances like those of the Hodgkin-Huxley model can be incorporated into the model. A model may be initialized with simple components, and then details added to incorporate additional biological details. The modeler may then increase or decrease the level of biological detail depending upon the intended application. Keeping models simple in this way offers:
  • The ability to use dynamical systems analysis by way of balancing biological detail with analytical tractability.
  • Fast and computationally inexpensive network dynamic simulations to work effectively in a robotic controller. Thus, complex traditional models, like the cable equation or the full Hodgkin-Huxley action potential model, are typically avoided or simplified for the sake of computational efficiency.
  • Sparse function-dependent connectivity via the Functional Subnetwork instead of fully connected topologies, common in machine learning.
While the neuroscientific models are typically simplified for SNS, the method is flexible enough that more features can be incorporated. Consequently, the SNS method can accommodate demand driven complexity, only adding features specifically where they are needed. For example, persistent sodium channels can be added to just two neurons in a neural circuit to create a half- center oscillator pattern generator without changing the other neurons in the circuit. While these additions may increase computational cost, they grant the system the ability to perform a wider array of interesting behaviors.

History

The term "synthetic nervous system" has appeared in the literature since the year 2000 to describe several different computational frameworks for mimicking the functionality of biological nervous systems.
Cynthia Breazeal developed a social robot named "Kismet" while at MIT in the early 2000s. She used the term SNS to refer to her biologically-inspired hierarchical model of cognition, which included systems for low-level sensory feature extraction, attention, perception, motivation, behavior, and motor output. Using this framework, Kismet could respond to people by abstracting its sensory information into motivation for responsive behaviors and the corresponding motor output.
In 2005, Inman Harvey used the term in a review article on his field, Evolutionary Robotics. In his article, Harvey uses the term SNS to refer to the evolved neural controller for a simulated agent. He does not explicitly define the term SNS; instead, he uses the term to differentiate the evolved neural controller from one created via alternative approaches, e.g., multi-layer perceptron networks.
In 2008, Thomas R. Insel, the director of the National Institute of Mental Health, was quoted in an American Academy of Neurology interview calling for a "clear moon shot… a decade of new discovery basic research on brain anatomy". As part of that interview, Insel suggested building a "synthetic nervous system" as one such motivational moon shot to drive ongoing and future research. The technical details of what such a SNS would entail were not described.
An article published as part of the International Work-Conference on Artificial Neural Networks proposes a "synthetic nervous system" as an alternative to artificial neural networks based in machine learning. In particular, SNS should be able to include or learn new information without forgetting what it has already learned. However, the authors do not propose a computational neuroscience framework for constructing such networks. Instead, they propose a homeostatic network of the robot's "needs", in which the robot takes actions to satisfy its needs and return to homeostasis. Over time, the robot learns which actions to take in response to its needs.
A dissertation from Joseph Ayer's lab at Northeastern University uses a similar term in its title but never explicitly defines it. The topic of the dissertation is "RoboLobster, a biomimetic robot controlled by an electronic nervous system simulation". Other publications from Ayers use the term "electronic nervous system" to describe similar work. In each of these studies, Ayers uses a robot that is controlled by a network of simplified dynamical neural models whose structure mimic specific networks from the model organism. The choice of neural model reflects a balance between simulating the dynamics of the nervous system, which motivates mathematical complexity, while ensuring the simulation runs in real time, which motivates mathematical simplicity.
A 2017 research article from Alexander Hunt, Nicholas Szczecinski, and Roger Quinn use the term SNS and implicitly define it as "neural neuro-mechanical models…composed of non-spiking leaky integrator neuron models". Similar to work by Ayers et al., Hunt et al. apply the term SNS to refer to a simplified dynamical simulation of neurons and synapses used in the closed-loop control of robotic hardware. Subsequent articles by these authors present the Functional Subnetwork Approach for tuning SNS constructed from these and other simplified dynamical neural models , as well as further SNS models of the nervous system
Comparing the diversity of works that use the term SNS produces an implicit definition of SNS:
  • Their Network's structure and behavioral goals are grounded in biology
  • Their priority is to learn more about nervous system function, with the secondary goal of creating a more effective robot control system
  • They are typically posed as an alternative to more abstracted neural networks with simplified network structure
  • They are computational models of the nervous system meant for closed-loop control of the behavior of simulated or robotic agent within an environment.

    Comparison to other neural networks

SNSs share some features with machine learning networks like Artificial neural network, Convolutional neural network, and Recurrent neural network. All of these networks are composed of neurons and synapses inspired in some way by biological nervous systems. These components are used to build neural circuits with the express purpose of accomplishing a specific task. ANN simply refers to a collection of nodes connected such that they loosely model a biological brain. This is a rather broad definition and as a consequence there are many subcategories of ANN, two of which are CNN and RNN. CNNs are primarily used for image recognition and classification. Their layer-to-layer connections implement convolutional kernels across small areas of the image, which map the input to the system onto a collection of features. ANNs and CNNs are only loosely associated with SNS in that they share the same general building blocks of neurons and synapses, though the methods used to model each component varies between the networks. Of the three, RNNs are the most closely related to SNS. SNSs use the same leaky-integrator neuron models utilized in RNNs. This is advantageous as neurons inherently act as low pass filters, which is useful for robotic applications where such filtering is often applied to reduce noise for both sensing and control purposes. Both models also exhibit dynamic responses to inputs. While predicting the responses of a complicated network can be difficult, the dynamics of each node are relatively simple in that each is a system of first order differential equations. The key difference that distinguishes SNS from these neural networks are the synaptic connections and the general architecture of the neural circuit.
RNN structures generally present as large, highly connected or even all-to- all connected layers of neurons. Instead of these layers, SNS relies on functional subnetworks which are tuned to perform specific operations and then assembled into larger networks with explainable functions. These are significantly more tractable than a typical machine learning network. The tradeoff of SNS is that it typically takes more time to design and tune the network but it does not require a training phase involving large amounts of computing power and training data. The other key difference is that SNS synapses are conductance based rather than current based which makes the dynamics non-linear, unlike an RNN. This allows for the modelling of modulatory neural pathways since the synapses can alter the net membrane conductance of a postsynaptic neuron without injecting current. It also enables the functional subnetwork approach to encompass addition, subtraction, multiplication, division, differentiation, and integration operations using the same family of functions.