In this paper we use state-of-the-art multimodal neuroimaging to tease apart the spatio-temporal sequence of neural activity that “goes through a hitter’s mind” when they recognize a baseball pitch. Specifically we utilize electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) to investigate the neural networks activated for correct and incorrect pitch classifications. Our previous analysis has shown where in the trajectory of a pitch the hitter’s neural activity correctly discriminates a pitch type (e.g. fastball, curveball or slider). Here, we show that correct classifications correlate with a neural network including both visual and sub-cortical motor areas, likely demonstrating a link between visual identification and the required rapid motor response. Conversely, we find that not only is this activity lacking in incorrect classifications, but that it is instead replaced by prefrontal cortex activity, which has been shown to be responsible for more deliberative conflict resolution. Synthesizing these and other results, we hypothesize the potential uses of this technology in the form of a brain computer interface (BCI) to measure and enhance baseball player performance.