I'm in Finland where I've given a talk at the Brain and Mind Symposium, organized by the Doctoral Program of Brain and Mind of Aalto University and the University of Helsinki. I spoke about some new data we've been collecting at Penn that I'm excited about - more on that over the coming months. Overall a good trip so far, and I hope I can make it back before too long.
New grant funding from NIH
I'm happy to announce that we have just been awarded a five year research grant from the National Institute on Deafnesss and other Communication Disorders (NIDCD) to study some of the neural processes involved in listening effort. My talented co-investigators on the project are Kristin Van Engen and Mitch Sommers from the Psychology Department.
The sobering side of this news is that it remains a very tough funding climate, and there are many talented scientists with great ideas who are not being funded. We count ourselves very fortunate to have the opportunity to pursue this research over the next few years.
The official abstract for the grant follows. We'll be starting the project as soon as we can and will post updates here. Stay tuned!
Approximately 36 million Americans report having some degree of hearing impairment. Hearing loss is associated with social isolation, depression, cognitive decline, and economic cost due to reduced work productivity. Understanding ways to optimize communication in listeners with hearing impairment is therefore a critical challenge for speech perception researchers. A hallmark of recent research has been the development of the concept of listening effort, which emphasizes the importance of cognitive processing during speech perception: Listeners with hearing impairment can often understand spoken language, but with increased cognitive effort, taking resources away from other processes such as attention and memory. Unfortunately, the specific cognitive processes that play a role in effortful listening remain poorly understood. The goal of the current research is to provide a more specific account of the neural and cognitive systems involved in effortful listening, and investigate how these factors affect speech comprehension. The studies are designed around a framework of lexical competition, which refers to how listeners select a correct target word from among the possible words they may have heard (Was that word “cap” or “cat”?). Lexical competition is influenced by properties of single words (words that sound similar to many others, like “cat”, are more difficult to process), the acoustic signal (poorer acoustic clarity makes correct identification more difficult), and individual differences in cognitive processing (lower inhibitory ability makes incorrect targets more likely to be perceived). Neuroanatomically, these processes are supported by dissociable regions of temporal and frontal cortex, consistent with a large-scale cortical network that supports speech comprehension. Importantly, individual differences in both hearing impairment and cognitive ability interact with the type of speech being processed to determine the level of success a listener will have in understanding speech. The current research will involve collecting measures of hearing and cognition in all participants to investigate how individual differences in these measures impact speech perception. Converging evidence from behavioral studies, eyetracking, and functional magnetic resonance imaging (fMRI) will be used to explore the cognitive and neural basis of speech perception. Aim 1 evaluates the relationship between lexical competition and listening effort during speech perception. Aim 2 characterizes multiple cognitive processes involved in processing degraded speech. Aim 3 assesses how individual differences in hearing and cognition predict speech perception, relying on a framework of lexical competition to inform theoretical interpretation. These studies will show a relationship between lexical competition and the cognitive processes engaged when processing degraded speech, providing a theoretically-motivated framework to better explain the challenges faced by both normal-hearing and hearing-impaired listeners.
New paper: mapping speech comprehension with optical imaging (Hassanpour et al.)
Although fMRI is great for a lot of things, it also presents challenges, especially for auditory neuroscience. Echoplanar imaging is loud, and this acoustic noise can obscure stimuli or change the cognitive demand of a task (Peelle, 2014). In addition, patients with implanted medical devices can't be scanned.
My lab has been working with Joe Culver's optical radiology lab to develop a solution to these problems using high-density diffuse optical tomography (HD-DOT). Similar to fNIRS, HD-DOT uses light spectroscopy to image oxygenated and deoxygenated blood signals, related to the BOLD response in fMRI. HD-DOT also incorporates realistic light models to facilitate source reconstruction—this of huge importance for studies of cognitive function and facilitates combining results across subjects. A detailed description of our current large field-of-view HD-DOT system can be found in Eggebrecht et al. (2014).
Because HD-DOT is relatively new, an important first step in using it for speech studies was to verify that it is indeed able to capture responses to spoken sentences, both in terms of effect size and spatial location. Mahlega Hassanpour is a PhD student who enthusiastically took on this challenge. In our paper now out in NeuroImage (Hassanpour et al., 2015), Mahlega used a well-studied comparison of syntactic complexity looking at sentences containing subject-relative or object-relative center embedded clauses (taken from our previous fMRI study; Peelle et al 2010).
Consistent with previous fMRI work, we found a sensible increase from a low level acoustic control condition (1 channel vocoded speech) to subject-relative sentences to object-relative sentences. The results were seen at both the single subject level (with some expected noise) and the group level.
We are really glad to see nice responses to spoken sentences with HD-DOT and are already pursuing several other projects. More to come!
References:
Eggebrecht AT, Ferradal SL, Robichaux-Viehoever A, Hassanpour MS, Dehghani H, Snyder AZ, Hershey T, Culver JP (2014) Mapping distributed brain function and networks with diffuse optical tomography. Nature Photonics 8:448-454. doi:10.1038/nphoton.2014.107
Hassanpour MS, Eggebrecht AT, Culver JP, Peelle JE (2015) Mapping cortical responses to speech using high-density diffuse optical tomography. NeuroImage 117:319–326. doi:10.1016/j.neuroimage.2015.05.058 (PDF)
Peelle JE (2014) Methodological challenges and solutions in auditory functional magnetic resonance imaging. Frontiers in Neuroscience 8:253. doi:10.3389/fnins.2014.00253 (PDF)
Peelle JE, Troiani V, Wingfield A, Grossman M (2010) Neural processing during older adults' comprehension of spoken sentences: Age differences in resource allocation and connectivity. Cerebral Cortex 20:773-782. doi:10.1093/cercor/bhp142 (PDF)
NSF workshop on speech technology
I've just returned from a workshop at the National Science Foundation on "the role of speech science in developing robust processing applications". Participants included neuroscientists, speech scientists, psychologists, and engineers interested in speech production and perception. The goal was to foster interdisciplinary thinking about the future of speech technology, and the role NSF might play in supporting these directions. It was a very interesting workshop and I hope leads to future discussions!
2015 Lab Alumni
The 2014-2015 academic year was a busy one in the Peelle Lab: We had 3 seniors working on honors theses, and an AuD student working on a one year research project:
- Margaret Koeritzer completed her AuD capstone project on the contributions of hearing and cognitive ability to recognition memory for spoken sentences;
- Celia Kaufer finished her PNP honors project on the relationship between semantic ambiguity, acoustic detail, subjective clarity, and recognition memory for spoken words;
- Anna Mai completed her linguistics honors project on distributional text characteristics affect reading time and readers' beliefs about writers;
- Katie Vogel finished her PNP honors project looking at individual differences in rhythm reproduction.
In addition, research assistant Carol Iskiwitch, who was the lab's first paid employee ever, will be leaving soon as she prepares to head off to graduate school in the fall.
All of them have been valued lab members and we are sad to see them go. Good luck guys!
New paper: Prediction and constraint in audiovisual speech perception
A review paper on audiovisual speech perception from me and Mitch Sommers (2015) is now in press in Cortex (part of a forthcoming special issue on predictive processes in speech comprehension). In this review Mitch and I have tried to start unifying two separate lines of research. The first is that ongoing oscillations in auditory cortex affect perceptual sensitivity. There is continued interest in the role of cortical oscillations in speech perception, even for auditory-only speech, where there is evidence that cortical oscillations entrain to the ongoing speech signal (Giraud & Poeppel 2012; Peelle & Davis 2012). Aligning cortical oscillations to perceptual input can increase sensitivity (i.e., faster or more accurate at detecting near-threshold inputs). Entrainment is amplified by visual input, making multimodal integration in auditory cortex a viable mechanism for audiovisual processing (Schroeder et al., 2008).
Alongside this increased perceptual sensitivity comes the visual information that restricts the possible sounds (i.e., words). For example, when trying to make a "cat/cap" distinction, having the lips open gives a clear indication that "cap" is not correct. This perspective is described within the intersection density framework, which is a straightforward extension of unimodal lexical competition to audiovisual speech: speech perception is constrained to items that are compatible with both auditory and visual input.
We discuss these complementary types of integration in the context of schematic models of audiovisual speech processing. Although it seems like a basic point, from our perspective the available evidence suggests that multisensory processing influences perception at multiple levels (and in neuroantomically dissociable regions).
Finally, one very important aspect worth emphasizing: like all speech processing (Peelle 2012), the details of audiovisual speech processing are likely heavily influenced by the type of stimulus and task that we are doing. So, connected speech (sentences) may provide visual information that aids in processing that is simply unavailable in single words or phonemes. Similarly, phoneme studies (say, with a token of /da/) will not require the lexical competition and selection processes involved in word perception. This is not to say that any of these levels are more or less valid to study; however, we have to be cautious when trying to make generalizations, and sensitive to differences in visual information as a function of linguistic level (phoneme, word, sentence).
There are still many unresolved questions regarding the representations of visual-only speech, and audiovisual integration during speech processing. Hopefully the suggestions Mitch and I have made will be useful, and we look forward to having some more data in the coming years that speak to these issues.
References:
Giraud A-L, Poeppel D (2012) Cortical oscillations and speech processing: Emerging computational principles and operations. Nat Neurosci 15:511-517. doi:10.1038/nn.3063
Peelle JE (2012) The hemispheric lateralization of speech processing depends on what "speech" is: A hierarchical perspective. Frontiers in Human Neuroscience 6:309. doi:10.3389/fnhum.2012.00309 (PDF)
Peelle JE, Davis MH (2012) Neural oscillations carry speech rhythm through to comprehension. Frontiers in Psychology 3:320. doi:10.3389/fpsyg.2012.00320 (PDF)
Peelle JE, Sommers MS (2015) Prediction and constraint in audiovisual speech perception. Cortex. doi:10.1016/j.cortex.2015.03.006 (PDF)
Schroeder CE, Lakatos P, Kajikawa Y, Partan S, Puce A (2008) Neuronal oscillations and visual amplification of speech. Trends in Cognitive Sciences 12:106-113. doi:10.1016/j.tics.2008.01.002