Book Chapters

  • Nexus: A neural simulator for integrating top-down and bottom-up modeling

    January 18, 2017

    We have developed the NEXUS simulation environment as a tool for modeling large-scale neural systems. The software is written in C and runs under UNIX. A unique aspect of NEXUS is that it is particularly suited for simulating hybrid neural models (i.e. systems integrating different modeling paradigms and/or architectures.1) NEXUS is designed for large-scale simulations, and to facilitate model development, testing and analysis it incorporates several major features: network architectures based on topographic maps, programmable neural units, scalable and modular simulation, support for common learning paradigms including the generalized Hebb rule and backpropagation, and a user-friendly interface. These features make NEXUS a useful environment in which to study the “perceptual” properties of various network architectures.

  • Single-trial analysis of EEG during rapid visual discrimination: Enabling cortically-coupled computer vision

    January 17, 2017

    We describe our work using linear discrimination of multi-channel electroencephalography for single-trial detection of neural signatures of visual recognition events. We demonstrate the approach as a methodology for relating neural variability to response variability, describing studies for response accuracy and response latency during visual target detection. We then show how the approach can be utilized to construct a novel type of brain-computer interface, which we term cortically-coupled computer vision. In this application, a large database of images is triaged using the detected neural signatures. We show how ‘cortical triaging’ improves image search over a strictly behavioral response.

  • Bayesian networks for modeling cortical integration

    January 17, 2017

  • Single-trial analysis of EEG for enabling cognitive user interfaces

    January 17, 2017

  • Cortically-coupled computer vision

    January 13, 2017

    We have developed EEG-based BCI systems which couple human vision and computer vision for speeding the search of large images and image/video databases. We term these types of BCI systems “cortically-coupled computer vision” (C3Vision). C3Vision exploits (1) the ability of the human visual system to get the “gist” of a scene with brief (10’s–100’s of ms) and rapid serial (10 Hz) image presentations and (2) our ability to decode from the EEG whether, based on the gist, the scene is relevant, informative and/or grabs the user’s attention. In this chapter we describe two system architectures for C3Vision that we have developed. The systems are designed to leverage the relative advantages, in both speed and recognition capabilities, of human and computer, with brain signals serving as the medium of communication of the user’s intentions and cognitive state.

  • Signal processing and machine learning for single-trial analysis of simultaneously acquired EEG and fMRI

    January 13, 2017

    The simultaneous acquisition of electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) is a potentially powerful multimodal imaging technique for measuring the functional activity of the human brain. Given that EEG measures the electrical activity of neural populations while fMRI measures hemodynamics via a blood oxygenation-level-dependent (BOLD) signal related to neuronal activity, simultaneous EEG/fMRI (hereafter referred to as EEG/fMRI) offers a modality to investigate the relationship between these two phenomena within the context of noninvasive neuroimaging. Though fMRI is widely used to study cognitive and perceptual function, there is still substantial debate regarding the relation- ship between local neuronal activity and hemodynamic changes. Another rationale for EEG/fMRI is that, despite the fact that the individual modalities measure markedly different physiological phenomena, in terms of spatial and temporal resolution they are quite complementary. EEG offers millisecond temporal resolution; however, the spatial sampling density and ill-posed nature of the inverse model problem limit its spatial resolution. On the other hand, fMRI provides millimeter spatial resolution, but because of scanning rates and the low-pass nature of the BOLD response, the temporal resolution is limited. One approach that has been adopted to take advantage of this complementarity is to use fMRI activations to seed EEG source localization.

  • Construction of illusory surfaces by intermediate-level visual cortical networks

    December 28, 2016

    A model is proposed which directly links the perception of illusory contours to intermediate-level cortical processes for visual surface discrimination. An important assertion of the model is that illusory contours are reentered, via feedback, into surface discrimination processes with the result being the construction of illusory surfaces. The model is tested in a number of simulations which demonstrate surface completion, generation of illusory contours, and interactions with depth cues from stereopsis.

  • Cortical mechanisms for surface segmentation

    December 28, 2016

    Physiology has shown that the neural machinery of “early vision” is well suited for extracting edges and determining orientation of contours in the visual field. However, when looking at objects in a scene our perception is not dominated by edges and contours but rather by surfaces. Previous models have attributed surface segmentation to filling-in processes, typically based on diffusion. Though diffusion related mechanisms may be important for perceptual filling-in [4], it is unclear how such mechanisms would discriminate multiple, overlapping surfaces, as might result from occlusion or transparency. For the case of occlusion, surfaces exist on either side of a boundary and the problem is not to fill-in the surfaces but to determine which surface “owns” the boundary [1][3]. This problem of boundary “ownership” can also be considered a special case of the binding problem, with a surface being “bound” to a contour.