Deep Learning

Keith Dillon
Fall 2019

Topic 2: Biological Inspiration

This topic:¶

Neurons in the brain
The artificial neuron model
Models of the brain

Reading:

en.wikiversity.org/wiki/Fundamentals_of_Neuroscience
Blumenfeld, "Neuroanatomy through clinical cases, 2e", 2010
Geron Chapter 10 (Artificial Neural Networks)

Outcomes¶

Point out key differences between biological and artificial neural networks
Introduction to key biological ideas seen in artificial neural networks

I. Neuroscience¶

The Nervous System¶

sensory signals in to brain
motor signals out from brain
- to muscles
- to organs & glands
intermediate processing at ganglia
- reflexes
- other simple tasks

Gray matter and White matter¶

Grey matter = cell bodies - think of as processing function step
White matter = axons - connections, tons of them

Diffusion weighted MRI¶

White matter in the brain is densely-packed with long-range connections between/among cortical regions and others

Biological Neurons¶

cell body containing the nucleus and most of the cell’s complex components,
many branching extensions called dendrites,
plus one very long extension called the axon. The axon’s length may be just a few times longer than the cell body, or up to tens of thousands of times longer.
Near its extremity the axon splits off into many branches called telodendria,
at the tip of these branches are minuscule structures called synaptic terminals (or simply synapses), which are connected to the dendrites (or directly to the cell body) of other neurons.
Biological neurons receive short electrical impulses called signals from other neurons via these synapses.
When a neuron receives a sufficient number of signals from other neurons within a few milliseconds, it fires its own signals.

Neuron schematic¶

A neuron is a cell with a special cell membrane

reshaped to produce interconnections
covered in ion channels which can be controlled to produce/carry action potentials
attach to other neurons with various kinds of synapses

Neuron model¶

ion channels
membrane potential
action potential ("spike")
synapses

Action potential¶

Info transfer via action potentials¶

Neuron doctrine¶

The brain is composed of neurons (and other cells) that are independent structurally, metabolically and functionally.
Information is transmitted from cell to cell across synapses

Synapses¶

There are a variety of different types of synapses which use different neurotransmitters.

The presynaptic neuro dumps neurotransmitter into the synaptic cleft (for most types of synapses)

The postsynaptic neuron has receptors to which the neurotransmitters (and other drugs present) may bind and cause change in potential.

Excitatory - increase membrane potential of post-synaptic neuron
Inhibitory - decrease membrane potential of post-synaptic neuron

Psychoactive drugs typically affect particular synapses at particular receptors, e.g. domaminergic (domamine), nicotinergic (affected by nicotene).

Hebbian Learning¶

"Neurons that fire together wire together"

Synapse between neurons is strenthened if they activate at same time

The Neuron - simplified mathematical model¶

Numbers representing input membrane potentials

Activation function represents threshold for spiking

Output number represents spike frequency

The Neuron as an algorithm¶

What is the mathematical function the artificial neuron computes?

Write code for a simple artificial neuron's behavior.

Networks of Neurons¶

Individual biological neurons seem to behave in a rather simple way,
But they are organized in a vast network of billions of neurons,
Each neuron typically connected to thousands of other neurons.
Highly complex computations can be performed by a vast network of fairly simple neurons
The architecture of biological neural networks "BNN"s is still the subject of active research
some parts of the brain have been mapped, and it seems that neurons are often organized in consecutive layers

Cross-section of cortical layers:

Neuron categories¶

Motor neurons - attach to and provide signals to drive muscles
Sensory neurons - return sensory signals to nervous system
Interneurons - connect neurons to other neurons for intermediate processing steps.

Example: reflex circuit - sensory signal activates muscle

Neuron types¶

Wide variety of differnt stereotyped neuron structures

Brodmann's cytoarchitectural areas¶

Regions of the brain can be differentiated based on cell types present

Also from neurology, we know different regions are important/necessary for different functions

Major brain regions¶

https://askabiologist.asu.edu/brain-regions

Vision pathways¶

Retinal cells (rods & cones) respond to light.
(retinal) Bipolar cells receive connections from one or more retinal cells
Ganglion cells receive connections from one or more bipolar cells (LGN = lateral geniculate nucleus of thalamus)
V1 (primary visual cotext) cells receive connections from LGN
"What" pathway - IT = inferiotemporal area
"Where" pathway - MT = medial temporal area