Electronic Synapses: One Step Closer to Creating Artificial Neural Networks

May 06, 2016 by Zabrel Holsman

Researchers at the Moscow Institute of Physics and Technology have come one step closer to creating an artificial neural network that mimics a human brain.

Researchers at the Moscow Institute of Physics and Technology have come one step closer to creating an artificial neural network that mimics a human brain.

Anyone who took electronics classes will be familiar with the circuit elements inductor, capacitor, and resistor. In 1971, a man named Leon Chua speculated that there should be a fourth class of elements for a fundamental passive circuit called a short memory resistor, or memristor.

The memristor concept showed invaluable properties pertaining to the augmentation of classical circuits. Until 2008, no one had given evidence of a physical model or pragmatic use of a memristor. Since then many research companies and industry entrepreneurs have been expending massive amounts of resources to realize the untapped potential of the memristor, as it has advantageous prospects for things such as solid state drives and ionic transport components.


Memristor layout. Image courtesy of NCBI.


As neuroscientists have taken on the task of replicating a human brain in an attempt to mimic human recognition, researchers from the Moscow Institute of Physics and Technology have been attempting to replicate the neural activity of a real brain by using analog components. In a recent paper published in the Journal of Nanoscale Research Letters, the MIPT team reinvented the memristor to get one step closer to creating artificial synapses.

The key component to creating a computerized brain is, of course, the memristor. In short, what a memristor does is regulate the flow of electrical current through a circuit and remember the charge value that passed through it. By their very nature, memristors are non-volatile: They can retain information with or without power.

An analogy is often made to a water pipe: Water flows through the pipe and expands in diameter, allowing water to pass through the pipe faster. When water flows back through the pipe, the diameter contracts and thus slows down the water flow; even when the water is turned off, the pipe retains the same structure.

Similarly in electrical standards, even when a memristor is turned off, it is capable of retaining the same resistance value. For example, if you cut the power of a computer with a memristor in it, the monitor would immediately be able to display the images and documents it displayed before the power was cut— as soon as power passed through it again.

It is remarkably difficult to replicate the mind of a human. However, we are currently capable of mimicking the brain of a cat. Neural network simulations such as Google’s DeepMind network have seen strange results, like placing the heads of dogs onto shapes by using image recognition software dubbed Deep Dream.


An image before (left) and after (right) being interpreted by Google DeepDream. Image generated via Deep Dream Generator. Original photo by Jay Ruzesky.


Unfortunately, the human brain is much faster and more complicated than its electrical counterparts. Biological brains can recognize images in a tenth of a second, so scientists want to imitate the brain, but the architecture of a brain is very different from that of a computer. In some models, neurons are connected and relay information via synapses, but each neuron can have over ten thousand connections with other neurons. Currently, it is postulated that changes in synapse conductivity are responsible for the processing of information, indicating that information essentially needs to pass through a giant chain of synapse changes to be "received".

What makes it incredibly hard to replicate the human brain is the fact that with classic systems, in order to replicate a synapse and neuron, one would need a massive amount of circuitry to mimic the billions of neurons and thousands of synapses. This would also consume a lot of power.


Image courtesy of the University of Buffalo.


Now what the researchers at MIPT did was recreate a memristor more suitable to mimic the brain. MIPT’s reinvention of the memristor made use of hafnium oxide (HfO2), a compound commonly used in CMOS fabrications. By applying voltage to the HfO2 layer, oxygen ions are driven from the lattice to electrodes. This leaves oxygen vacancies which allow for the transport of electrons. The resistance of the memristor is then defined by the number of oxygen vacancies in a channel, and can change over time by biasing the memristor.

The MIPT team is currently using their memristor-based electronic brain to simulate the functions of an organic brain such as memorizing and forgetting, noted as "long term potentiation" and "long term depression", respectively. This allows the electronic brain to gain plasticity in that it is capable of forgetting useless information in order to make room for new knowledge.

The combination of these elements is called spike-timing-dependent plasticity. In biology, this is the process that adjusts the strength of the connections between neurons in the brain. Connection strengths are adjusted based on the timing of neurons input and output action potentials, referred to as spikes. STDP has been proven to be the process of associated with learning. Thus, many researchers have been attempting to mimic the process using memristors.


A synapse spike. Image courtesy of MIPT.


Currently, no one has been able to devise a system that can integrate the memristor synapses into a full neural system, but configuring a memristor to become a synapse is the first step in emulating a human brain.

You can find the full published research paper here via the Journal of Nanoscale Research Letters.

  • W
    wattsknew May 09, 2016

    Replicating the human brain may not be a such great idea considering.Consider why we cannot “replicate” dodos (using old fashioned biology:breeding)- there are no more dodos left.

    Like. Reply
  • Natakel May 12, 2016

    One word:  Skynet . . .

    Like. Reply