New paper: mapping speech comprehension with optical imaging (Hassanpour et al.)

View fullsize

Although fMRI is great for a lot of things, it also presents challenges, especially for auditory neuroscience. Echoplanar imaging is loud, and this acoustic noise can obscure stimuli or change the cognitive demand of a task (Peelle, 2014). In addition, patients with implanted medical devices can't be scanned.

My lab has been working with Joe Culver's optical radiology lab to develop a solution to these problems using high-density diffuse optical tomography (HD-DOT). Similar to fNIRS, HD-DOT uses light spectroscopy to image oxygenated and deoxygenated blood signals, related to the BOLD response in fMRI. HD-DOT also incorporates realistic light models to facilitate source reconstruction—this of huge importance for studies of cognitive function and facilitates  combining results across subjects. A detailed description of our current large field-of-view HD-DOT system can be found in Eggebrecht et al. (2014).

Because HD-DOT is relatively new, an important first step in using it for speech studies was to verify that it is indeed able to capture responses to spoken sentences, both in terms of effect size and spatial location. Mahlega Hassanpour is a PhD student who enthusiastically took on this challenge. In our paper now out in NeuroImage (Hassanpour et al., 2015), Mahlega used a well-studied comparison of syntactic complexity looking at sentences containing subject-relative or object-relative center embedded clauses (taken from our previous fMRI study; Peelle et al 2010).

Consistent with previous fMRI work, we found a sensible increase from a low level acoustic control condition (1 channel vocoded speech) to subject-relative sentences to object-relative sentences. The results were seen at both the single subject level (with some expected noise) and the group level.

We are really glad to see nice responses to spoken sentences with HD-DOT and are already pursuing several other projects. More to come!


References:

Eggebrecht AT, Ferradal SL, Robichaux-Viehoever A, Hassanpour MS, Dehghani H, Snyder AZ, Hershey T, Culver JP (2014) Mapping distributed brain function and networks with diffuse optical tomography. Nature Photonics 8:448-454. doi:10.1038/nphoton.2014.107

Hassanpour MS, Eggebrecht AT, Culver JP, Peelle JE (2015) Mapping cortical responses to speech using high-density diffuse optical tomography. NeuroImage 117:319–326. doi:10.1016/j.neuroimage.2015.05.058 (PDF)

Peelle JE (2014) Methodological challenges and solutions in auditory functional magnetic resonance imaging. Frontiers in Neuroscience 8:253. doi:10.3389/fnins.2014.00253 (PDF)

Peelle JE, Troiani V, Wingfield A, Grossman M (2010) Neural processing during older adults' comprehension of spoken sentences: Age differences in resource allocation and connectivity. Cerebral Cortex 20:773-782. doi:10.1093/cercor/bhp142 (PDF)

NSF workshop on speech technology

I've just returned from a workshop at the National Science Foundation on "the role of speech science in developing robust processing applications". Participants included neuroscientists, speech scientists, psychologists, and engineers interested in speech production and perception. The goal was to foster interdisciplinary thinking about the future of speech technology, and the role NSF might play in supporting these directions. It was a very interesting workshop and I hope leads to future discussions!

2015 Lab Alumni

The 2014-2015 academic year was a busy one in the Peelle Lab: We had 3 seniors working on honors theses, and an AuD student working on a one year research project:

• Margaret Koeritzer completed her AuD capstone project on the contributions of hearing and cognitive ability to recognition memory for spoken sentences;
• Celia Kaufer finished her PNP honors project on the relationship between semantic ambiguity, acoustic detail, subjective clarity, and recognition memory for spoken words;
• Anna Mai completed her linguistics honors project on distributional text characteristics affect reading time and readers' beliefs about writers;
• Katie Vogel finished her PNP honors project looking at individual differences in rhythm reproduction.

In addition, research assistant Carol Iskiwitch, who was the lab's first paid employee ever, will be leaving soon as she prepares to head off to graduate school in the fall.

All of them have been valued lab members and we are sad to see them go. Good luck guys!

New paper: Prediction and constraint in audiovisual speech perception

A review paper on audiovisual speech perception from me and Mitch Sommers (2015) is now in press in Cortex (part of a forthcoming special issue on predictive processes in speech comprehension). In this review Mitch and I have tried to start unifying two separate lines of research. The first is that ongoing oscillations in auditory cortex affect perceptual sensitivity. There is continued interest in the role of cortical oscillations in speech perception, even for auditory-only speech, where there is evidence that cortical oscillations entrain to the ongoing speech signal (Giraud & Poeppel 2012; Peelle & Davis 2012). Aligning cortical oscillations to perceptual input can increase sensitivity (i.e., faster or more accurate at detecting near-threshold inputs). Entrainment is amplified by visual input, making multimodal integration in auditory cortex a viable mechanism for audiovisual processing (Schroeder et al., 2008).

Alongside this increased perceptual sensitivity comes the visual information that restricts the possible sounds (i.e., words). For example, when trying to make a "cat/cap" distinction, having the lips open gives a clear indication that "cap" is not correct. This perspective is described within the intersection density framework, which is a straightforward extension of unimodal lexical competition to audiovisual speech: speech perception is constrained to items that are compatible with both auditory and visual input.

Schematic models for audiovisual speech processing, proposing a single mechanism of multisensory integration (occurring early or late in the processing stream), or multiple complementary stages.

We discuss these complementary types of integration in the context of schematic models of audiovisual speech processing. Although it seems like a basic point, from our perspective the available evidence suggests that multisensory processing influences perception at multiple levels (and in neuroantomically dissociable regions).

Finally, one very important aspect worth emphasizing: like all speech processing (Peelle 2012), the details of audiovisual speech processing are likely heavily influenced by the type of stimulus and task that we are doing. So, connected speech (sentences) may provide visual information that aids in processing that is simply unavailable in single words or phonemes. Similarly, phoneme studies (say, with a token of /da/) will not require the lexical competition and selection processes involved in word perception. This is not to say that any of these levels are more or less valid to study; however, we have to be cautious when trying to make generalizations, and sensitive to differences in visual information as a function of linguistic level (phoneme, word, sentence).

There are still many unresolved questions regarding the representations of visual-only speech, and audiovisual integration during speech processing. Hopefully the suggestions Mitch and I have made will be useful, and we look forward to having some more data in the coming years that speak to these issues.

References:

Giraud A-L, Poeppel D (2012) Cortical oscillations and speech processing: Emerging computational principles and operations. Nat Neurosci 15:511-517. doi:10.1038/nn.3063

Peelle JE (2012) The hemispheric lateralization of speech processing depends on what "speech" is: A hierarchical perspective. Frontiers in Human Neuroscience 6:309. doi:10.3389/fnhum.2012.00309 (PDF)

Peelle JE, Davis MH (2012) Neural oscillations carry speech rhythm through to comprehension. Frontiers in Psychology 3:320. doi:10.3389/fpsyg.2012.00320 (PDF)

Peelle JE, Sommers MS (2015) Prediction and constraint in audiovisual speech perception. Cortex. doi:10.1016/j.cortex.2015.03.006 (PDF)

Schroeder CE, Lakatos P, Kajikawa Y, Partan S, Puce A (2008) Neuronal oscillations and visual amplification of speech. Trends in Cognitive Sciences 12:106-113. doi:10.1016/j.tics.2008.01.002

Talk at Macquarie University

View fullsize

I'm in Sydney, Australia where I've just given a talk at the Australian Hearing Hub at Macquarie University. It's my first visit to Macquarie and it's been great. The recently-established Hesring Hub combines researchers, clinicians, and industry partners in a single building (which includes a nice cafe on the ground floor...I'm jealous). This multi-pronged approach to hearing science is exemplary and a model for interdisciplinary collaboration. I look forward to good things happening here over the coming years!

One of the special treats on my visit was a visit to the state-of-the-art anechoic chamber in the basement. Now THAT is a proper sound booth.

New paper: A role for the angular gyrus in combinatorial semantics (Price et al.)

We know what a "leaf" is, and we know what "wet" means. But combining these concepts together into a "wet leaf" yields a new and possibly more specific idea. Similarly, a "brown leaf" is qualitatively different than any old leaf. Our ability to flexibly and dynamically combine concepts enables us to represent and communicate an enormous set of ideas from a relatively small number of constituents. The question of what neural systems might support conceptual combination has been a focus of research for Amy Price at Penn. Combinatorial semantics is an especially timely topic as there are ongoing debates about the anatomical systems most strongly involved in semantic memory more generally (angular gyrus? anterior temporal lobes? ventral visual regions?), as well as the nature of the information being represented (to what degree do concepts rely on sensorimotor cortices?).

View fullsize

In a new paper out this week in the Journal of Neuroscience (Price et al., 2015), Amy presents data from both fMRI and patients with neurodegenerative disease suggesting that the angular gyrus plays an important role in conceptual combination. Amy designed a clever task in which participants read word pairs that varied in how easily they could be combined into a single concept. For example, you could imagine that "turnip rock" is difficult to combine, whereas a "wet rock" is easier. Amy used all adjective-noun pairs, but still found a considerable amount of variability (for example a "plaid apple" combines less easily than a "plaid jacket"). This "ease of combination" was initially quantified using subject ratings, but Amy found that lexical co-occurrence statistics for these word pairs strongly correlate with their degree of combination, and thus co-occurrence measures were used in all analyses. 

These findings are in good agreement with previous work emphasizing an important role for the angular gyrus in semantic representation (Binder & Desai 2011; Bonner et al. 2013).

References:

Binder JR, Desai RH (2011) The neurobiology of semantic memory. Trends in Cognitive Sciences 15:527-536. doi:10.1016/j.tics.2011.10.001

Bonner MF, Peelle JE, Cook PA, Grossman M (2013) Heteromodal conceptual processing in the angular gyrus. NeuroImage 71:175–186. doi:10.1016/j.neuroimage.2013.01.006 (PDF)

Price AR, Bonner MF, Peelle JE, Grossman M (2015) Converging evidence for the neuroanatomic basis of combinatorial semantics in the angular gyrus. Journal of Neuroscience 35:3276–3284. http://www.jneurosci.org./content/35/7/3276.short (PDF)