# Distributed Representations and Spike Timing

## Zoubin Ghahramani and Geoffrey Hinton

*Gatsby Computational Neuroscience Unit*

University College London

There is no general consensus about how the activities of spiking
neurons combine to represent multiple properties of an object, such as
its pose, deformation and illumination. We present a speculation about
how spiking neurons can represent and operate on full probability
distributions over the space of object properties. We start by
assuming that each neuron encodes a basis function over the space of
object properties. A neuron's activity (depolarization) represents a
multiplicative coefficient on its basis function. The weighted bases
combine additively across a population of neurons in the same group to
form an energy landscape (i.e. negative log probability density). In
this framework, since spikes cause a smooth depolarization in neurons
downstream (the EPSP), the timing of all-or-none spikes can be used to
represent real-valued coefficients.

This representation has several consequences: (1) Unlike disjunctive
codes (Anderson and van Essen, 1994), here neurons with very broad
spatio-temporal tuning curves (energy bases) can be combined to
represent very sharp spatio-temporal densities. (2) The Bayesian
operation of combining evidence from multiple sources with a prior
density is additive in the energy domain, and therefore trivial to
implement in these spiking neurons. (3) Spike timing can convey real
values quickly and accurately without requiring precise coincidence
detection, sub-threshold oscillations, or modifiable time delays
(Hopfield, 1995). (4) Lateral connections are required for ``explaining
away'' within a group: to correlate activities of nearby neurons so as
to produce a desired density. That is, when multiple neurons have
similar basis functions there are potentially many different ways of
representing the same density over object properties. But once one
neuron fires, its contribution to the energy landscape must be
subtracted from what nearby neurons have to represent. (5) Noise
models (uncertainty not inherent in the input) require the
introduction of nonlinear activity decay within a group of neurons.