Next: 5.4 Slow noise in Up: 5. Noise in Spiking Previous: 5.2 Statistics of spike

Subsections

# 5.3 Escape noise

There are various ways to introduce noise in formal spiking neuron models. In this section we focus on a noisy threshold' (also called escape or hazard model). In section 5.5 we will discuss noisy integration' (also called stochastic spike arrival or diffusion model). In both cases, we are interested in the effect of the noise on the distribution of interspike intervals.

In the escape model, we imagine that the neuron can fire even though the formal threshold has not been reached or may stay quiescent even though the formal threshold has been passed. To do this consistently, we introduce an escape rate' or firing intensity' which depends on the momentary state of the neuron.

## 5.3.1 Escape rate and hazard function

Given the input I and the firing time of the last spike, we can calculate the membrane potential of the Spike Response Model or the integrate-and-fire neuron from Eq. (5.1) or (5.2), respectively. In the deterministic model the next spike occurs when u reaches the threshold . In order to introduce some variability into the neuronal spike generator, we replace the strict threshold by a stochastic firing criterion. In the noisy threshold model, spikes can occur at any time with a probability density,

 = f (u - ) , (5.41)

that depends on the momentary distance between the (noiseless) membrane potential and the threshold; see Fig. 5.5. We can think of f as an escape rate similar to the one encountered in models of chemical reactions (van Kampen, 1992). In the mathematical theory of point processes, the quantity is called a stochastic intensity'. Since we use in the context of neuron models we will refer to it as a firing intensity.

Since u on the right-hand side of Eq. (5.41) is a function of time, the firing intensity is time-dependent as well. In view of Eqs. (5.1) and (5.2), we write

 (t|) = f[u(t|) - ] , (5.42)

where (t|) is the hazard introduced in Eq. (5.10). In other words, the escape rate f allows us to translate the membrane potential u(t|) into a hazard (t|).

Is Eq. (5.42) a sufficiently general noise model? We have seen in Chapter 2.2 that the concept of a pure voltage threshold is questionable. More generally, the spike trigger process could, for example, also depend on the slope = du/dt with which the threshold' is approached. In the noisy threshold model, we may therefore also consider an escape rate (or hazard) which depends not only on u but also on its derivative

 (t|) = f[u(t|),(t|)] . (5.43)

The choice of the escape function f in Eq. (5.42) or (5.43) is arbitrary. A reasonable condition is to require f 0 for u - . Below we discuss some simple examples of Eq. (5.42). We will return to Eq. (5.43) in Section 5.7.

Note that the hazard in Eq. (5.43) is implicitly time-dependent, via the membrane potential u(t|). In an even more general model, we could in addition include an explicit time dependence, e.g., to account for a reduced spiking probability immediately after the spike at . In the following examples we will stick to the hazard function as defined by Eq. (5.42).

### 5.3.1.1 Example: Hard and soft threshold

We have motivated the escape model by a noisy version of the threshold process. In order to explore the relation between noisy and deterministic threshold models, we consider an escape function f defined as

 f (u - ) = (5.44)

Thus, the neuron never fires if u < . On the other hand, if the mean escape time goes to zero, the neuron fires immediately when it crosses the threshold.

How can we soften' the sharp threshold? A simple choice for a soft threshold is an exponential dependence,

 f (u - ) =  exp[ (u - )] , (5.45)

where and are parameters. For , we return to the noiseless model of Eq. (5.44). Alternatively, we could introduce a piecewise linear escape rate,

 f (u - ) =  [u - ]+ = (5.46)

with slope for u > . For u > , the firing intensity is proportional to u - ; cf. Fig. 5.6. This corresponds to the intuitive idea that instantaneous firing rates increase with the membrane potential. Variants of the linear escape-rate model are commonly used to describe spike generation in, e.g., auditory nerve fibers (Miller and Mark, 1992; Siebert and Gray, 1963).

Finally, we can also use a sigmoidal escape rate (Wilson and Cowan, 1972; Abeles, 1982),

 f (u - ) = 1 + erf , (5.47)

with time constant and noise parameter . The error function is defined as

 erf(x) = exp(- y2) dy (5.48)

with erf(- x) = - erf(x). For u, the escape rate (5.47) saturates at a value f = independent of the noise parameter ; cf. Fig. 5.7B. For 0, we retrieve the step function f (u - ) =  (u - ).

### 5.3.1.2 Example: Motivating a sigmoidal escape rate

We want to motivate the sigmoidal escape rate by a model with stochastic threshold in discrete time tn = n . After each time step of length , a new value of the threshold is chosen from a Gaussian distribution of threshold values with mean ,

 P() = exp -  . (5.49)

The probability of firing at time step tn is equal to the probability that the momentary value of the threshold is below the membrane potential u(tn)

 Probspike at tn = Probu(tn) > = P() d ; (5.50)

cf. Fig. 5.7A. The firing probability divided by the step size can be interpreted as a firing intensity (Wilson and Cowan, 1972)

 f (u - ) =  exp -  dx = 1 + erf , (5.51)

which is the sigmoidal escape rate introduced in Eq. (5.47).

Instead of a model with stochastic threshold, we can also consider a model fixed threshold, but a membrane potential u(tn) + u(tn) with a stochastic component u. If u is chosen at each time step independently from a Gaussian distribution with variance and vanishing mean, we arrive again at formula (5.47) (Abeles, 1982; Geisler and Goldberg, 1966; Weiss, 1966).

The sigmoidal escape rate (5.51) has been motivated here for models in discrete time. There are two potential problems. First, if we keep fixed and take 0 we do not recover the deterministic threshold model. Thus the low-noise limit is problematic. Second, since the firing intensity diverges for 0, simulations will necessarily depend on the discretization . This is due to the fact that the bandwidth of the noise is limited by because a new value of or u is chosen at intervals . For 0, the bandwidth and hence the noise power diverge. Despite its problems, the sigmoidal escape rate is also used in neuronal models in continuous time and either motivated by a Gaussian distribution of threshold values (Wilson and Cowan, 1972) or else for fixed threshold by a Gaussian distribution of membrane potentials with band-limited noise (Abeles, 1982; Weiss, 1966). If we use Eq. (5.47) in continuous time, the time scale becomes a free parameter and should be taken proportional to the correlation time of the noise u in the membrane potential (i.e., proportional to the inverse of the noise bandwidth). If the correlation time is short, the model becomes closely related to continuous-time escape rate models (Weiss, 1966). A natural' correlation time of the membrane potential will be calculated in Section 5.5 in the context of stochastic spike arrival.

### 5.3.1.3 Example: Transition from continuous to discrete time

In the previous example, we have started from a model in discrete time and found that the limit of continuous time is not without problems. Here we want to start from a model in continuous time and discretize time as it is often done in simulations. In a straightforward discretization scheme, we calculate the probability of firing during a time step t of a neuron that has fired the last time at as (t'|) dt' (t|t. For u , the hazard (t|) = f[u(t|) - ] can take large values; see, e.g., Eq. (5.45). Thus t must be taken extremely short so as to guarantee (t|t < 1.

In order to arrive at an improved discretization scheme, we calculate the probability that a neuron does not fire in a time step t. Since the integration of Eq. (5.6) over a finite time t yields an exponential factor analogous to Eq. (5.7), we arrive at a firing probability

 Probspike in [t, t + t] | u(t|) 1 - exp - t f[u(t|) - ] . (5.52)

Even if f diverges for u, the probability remains bounded between zero and one. We see from Fig. 5.8A that an increase in the discretization t mainly shifts the firing curve to the left while the form remains roughly the same. An increase of the noise level makes the curve flatter; cf. Fig. 5.8B.

## 5.3.2 Interval distribution and mean firing rate

In this section, we combine the escape rate model with the concepts of renewal theory and calculate the input-dependent interval distribution PI(t|) for escape rate models.

We recall Eq. (5.9) and express the interval distribution in terms of the hazard ,

 PI(t|) = (t|) exp - (t'|) dt' . (5.53)

This expression can be compared to the interval distribution of the stationary Poisson process in Eq. (5.18). The main difference to the simple Poisson model is that the hazard (t|) depends on both the last firing time and the (potentially time-dependent) input. We know that immediately after firing a neuron is refractory and therefore not very likely to fire. Thus refractoriness strongly shapes the interval distribution of neurons; cf., e.g., (Berry and Meister, 1998). Escape models allow us to show the relation between the hazard (t|) and refractoriness.

To do so, we express by the escape rate. In order to keep the notation simple, we suppose that the escape rate f is a function of u only. We insert Eq. (5.42) into Eq. (5.53) and obtain

 PI(t|) = f[u(t|) - ] exp - f[u(t'|) - ] dt' . (5.54)

In order to make the role of refractoriness explicit, we consider the version SRM0 of the Spike Response Model. The membrane potential is

 u(t|) = (t - ) + h(t) (5.55)

with h(t) = (sI(t - s) ds; cf. Eq. (4.42). We insert Eq. (5.55) into (5.54) and find

 PI(t|) = f[(t - ) + h(t) - ] exp - f[(t' - ) + h(t') - ] ds' . (5.56)

Fig. 5.9 shows the interval distribution (5.56) for constant input current I0 as a function of s = t - . With the normalization (s)ds = 1, we have h0 = I0. Due to the refractory term , extremely short intervals are impossible and the maximum of the interval distribution occurs at some finite value of s. If I0 is increased, the maximum is shifted to the left. The interval distributions of Fig. 5.9A have qualitatively the same shape as those found for cortical neurons. The gain function = g(I0) of a noisy SRM neuron is shown in Fig. 5.9B.

### 5.3.2.1 Example: SRM0 with absolute refractoriness

We study the model SRM0 defined in Eq. (5.55) for absolute refractoriness

 (s) =  . (5.57)

The hazard is (t|) = f[h(t) - ] for t - > and (t|) = 0 for t - < since f 0 for u - . Hence, with r(t) = f[h(t) - ]

 PI(t|) = r(t)exp - r(t')dt' (t - - ) (5.58)

For stationary input h(t) = h0, we are led back to the Poisson process with absolute refractoriness; see Eq. (5.20). For 0, Eq. (5.58) is the interval distribution of an inhomogeneous Poisson process with rate r(t).

### 5.3.2.2 Example: Linear escape rates

In this example we show that interval distributions are particularly simple if a linear escape rate is adopted. We start with the non-leaky integrate-and-fire model. In the limit of , the membrane potential of an integrate-and-fire neuron is

 u(t|) = ur + I(t')dt' ; (5.59)

cf. Eq. (5.2). Let us set ur = 0 and consider a linear escape rate,

 (t|) =  [u(t|) - ]+ . (5.60)

For constant input current I0 and = 0 the hazard is (t|) = a0 (t - - ) with a0 =  I0/C and =  C/I0. The interval distribution for this hazard function has already been given in Eq. (5.14); see Fig. (5.2).

For a leaky integrate-and-fire neuron with constant input I0, the membrane potential is

 u(t|) = R I0 1 - e (5.61)

where we have assumed ur = 0. If we adopt Eq. (5.60) with = 0, then the hazard is

 (t - ) =  1 - e- (t-) , (5.62)

with =  R I0 and = . The interval distribution for this hazard function has been discussed in Section 5.2.3; cf. Eq. (5.17) and Fig. 5.2B. An absolute refractory time as in the hazard Eq. (5.16) could be the result of a positive threshold > 0.

### 5.3.2.3 Example: Periodic input

We study the model SRM0 as defined in Eq. (5.55) with periodic input I(t) = I0 + I1cos( t). This leads to an input potential h(t) = h0 + h1 cos( t + ) with bias h0 = I0 and a periodic component with a certain amplitude h1 and phase . We choose a refractory kernel with absolute and relative refractoriness defined as

 (s) = (5.63)

and adopt the exponential escape rate (5.45).

Suppose that a spike has occurred at = 0. The probability density that the next spike occurs at time t is given by PI(t|) and can be calculated from Eq. (5.53). The result is shown in Fig. 5.10. We note that the periodic component of the input is well represented in the response of the neuron. This example illustrates how neurons in the auditory system can transmit stimuli of frequencies higher than the mean firing rate of the neuron; see Chapter 12.5. We emphasize that the threshold in Fig. 5.10 is at =1. Without noise there would be no output spike. On the other hand, at very high noise levels, the modulation of the interval distribution would be much weaker. Thus a certain amount of noise is beneficial for signal transmission. The existence of a optimal noise level is a phenomenon called stochastic resonance and will be discussed below in Section 5.8.

Next: 5.4 Slow noise in Up: 5. Noise in Spiking Previous: 5.2 Statistics of spike
Gerstner and Kistler
Spiking Neuron Models. Single Neurons, Populations, Plasticity
Cambridge University Press, 2002