Tagged: Decoding

Brain-computer interfaces

The human brain is perhaps the most fascinating and complex signal processing machine in existence. It is capable of transducing a variety of environmental signals (the senses, including taste, touch, smell, sound, and sight) and extracting information from these disparate signal streams, ultimately fusing this information to enable behavior, cognition, and action. What is perhaps surprising is that the basic signal processing elements of the brain, i.e., neurons, transmit information at a relatively slow rate compared to transistors, switching about 106 times slower in fact. The brain has the advantage of having a tremendous number of neurons, all operating in parallel, and a highly distributed memory system of synapses (over 100 trillion in the cerebral cortex) and thus its signal processing capabilities may largely arise from its unique architecture. These facts have inspired a great deal of study of the brain from a signal processing perspective. Recently, scientists and engineers have focused on developing means in which to directly interface with the brain, essentially measuring neural signals and decoding them to augment and emulate behavior. This research area has been termed brain computer interfaces and is the topic of this issue of IEEE Signal Processing Magazine.

Mapping visual stimuli to perceptual decisions via sparse decoding of mesoscopic neural activity

In this talk I will describe our work investigating sparse decoding of neural activity, given a realistic mapping of the visual scene to neuronal spike trains generated by a model of primary visual cortex (V1). We use a linear decoder which imposes sparsity via an L1 norm. The decoder can be viewed as a decoding neuron (linear summation followed by a sigmoidal nonlinearity) in which there are relatively few non-zero synaptic weights. We find: (1) the best decoding performance is for a representation that is sparse in both space and time, (2) decoding of a temporal code results in better performance than a rate code and is also a better fit to the psychophysical data, (3) the number of neurons required for decoding increases monotonically as signal-to-noise in the stimulus decreases, with as little as 1% of the neurons required for decoding at the highest signal-to-noise levels, and (4) sparse decoding results in a more accurate decoding of the stimulus and is a better fit to psychophysical performance than a distributed decoding, for example one imposed by an L2 norm. We conclude that sparse coding is well-justified from a decoding perspective in that it results in a minimum number of neurons and maximum accuracy when sparse representations can be decoded from the neural dynamics.

Perceptual Decision Making Investigated via Sparse Decoding of a Spiking Neuron Model of V1

Recent empirical evidence supports the hypothesis that invariant visual object recognition might result from non-linear encoding of the visual input followed by linear decoding [1]. This hypothesis has received theoretical support through the development of neural network architectures which are based on a non-linear encoding of the input via recurrent network dynamics followed by a linear decoder [2], [3]. In this paper we consider such an architecture in which the visual input is non-linearly encoded by a biologically realistic spiking model of V1, and mapped to a perceptual decision via a sparse linear decoder. Novel is that we 1) utilize a large-scale conductance based spiking neuron model of V1 which has been well-characterized in terms of classical and extra-classical response properties, and 2) use the model to investigate decoding over a large population of neurons. We compare decoding performance of the model system to human performance by comparing neurometric and psychometric curves.

In a Blink of an Eye and a Switch of a Transistor: Cortically-coupled Computer Vision

Our society’s information technology advancements have resulted in the increasingly problematic issue of information overload-i.e., we have more access to information than we can possibly process. This is nowhere more apparent than in the volume of imagery and video that we can access on a daily basis-for the general public, availability of YouTube video and Google Images, or for the image analysis professional tasked with searching security video or satellite reconnaissance. Which images to look at and how to ensure we see the images that are of most interest to us, begs the question of whether there are smart ways to triage this volume of imagery. Over the past decade, computer vision research has focused on the issue of ranking and indexing imagery. However, computer vision is limited in its ability to identify interesting imagery, particularly as ¿interesting¿ might be defined by an individual. In this paper we describe our efforts in developing brain-computer interfaces (BCIs) which synergistically integrate computer vision and human vision so as to construct a system for image triage. Our approach exploits machine learning for real-time decoding of brain signals which are recorded noninvasively via electroencephalography (EEG). The signals we decode are specific for events related to imagery attracting a user’s attention. We describe two architectures we have developed for this type of cortically coupled computer vision and discuss potential applications and challenges for the future