I've just returned from a workshop at the National Science Foundation on "the role of speech science in developing robust processing applications". Participants included neuroscientists, speech scientists, psychologists, and engineers interested in speech production and perception. The goal was to foster interdisciplinary thinking about the future of speech technology, and the role NSF might play in supporting these directions. It was a very interesting workshop and I hope leads to future discussions!
2015 Lab Alumni
The 2014-2015 academic year was a busy one in the Peelle Lab: We had 3 seniors working on honors theses, and an AuD student working on a one year research project:
- Margaret Koeritzer completed her AuD capstone project on the contributions of hearing and cognitive ability to recognition memory for spoken sentences;
- Celia Kaufer finished her PNP honors project on the relationship between semantic ambiguity, acoustic detail, subjective clarity, and recognition memory for spoken words;
- Anna Mai completed her linguistics honors project on distributional text characteristics affect reading time and readers' beliefs about writers;
- Katie Vogel finished her PNP honors project looking at individual differences in rhythm reproduction.
In addition, research assistant Carol Iskiwitch, who was the lab's first paid employee ever, will be leaving soon as she prepares to head off to graduate school in the fall.
All of them have been valued lab members and we are sad to see them go. Good luck guys!
New paper: Prediction and constraint in audiovisual speech perception
A review paper on audiovisual speech perception from me and Mitch Sommers (2015) is now in press in Cortex (part of a forthcoming special issue on predictive processes in speech comprehension). In this review Mitch and I have tried to start unifying two separate lines of research. The first is that ongoing oscillations in auditory cortex affect perceptual sensitivity. There is continued interest in the role of cortical oscillations in speech perception, even for auditory-only speech, where there is evidence that cortical oscillations entrain to the ongoing speech signal (Giraud & Poeppel 2012; Peelle & Davis 2012). Aligning cortical oscillations to perceptual input can increase sensitivity (i.e., faster or more accurate at detecting near-threshold inputs). Entrainment is amplified by visual input, making multimodal integration in auditory cortex a viable mechanism for audiovisual processing (Schroeder et al., 2008).
Alongside this increased perceptual sensitivity comes the visual information that restricts the possible sounds (i.e., words). For example, when trying to make a "cat/cap" distinction, having the lips open gives a clear indication that "cap" is not correct. This perspective is described within the intersection density framework, which is a straightforward extension of unimodal lexical competition to audiovisual speech: speech perception is constrained to items that are compatible with both auditory and visual input.
Schematic models for audiovisual speech processing, proposing a single mechanism of multisensory integration (occurring early or late in the processing stream), or multiple complementary stages.
We discuss these complementary types of integration in the context of schematic models of audiovisual speech processing. Although it seems like a basic point, from our perspective the available evidence suggests that multisensory processing influences perception at multiple levels (and in neuroantomically dissociable regions).
Finally, one very important aspect worth emphasizing: like all speech processing (Peelle 2012), the details of audiovisual speech processing are likely heavily influenced by the type of stimulus and task that we are doing. So, connected speech (sentences) may provide visual information that aids in processing that is simply unavailable in single words or phonemes. Similarly, phoneme studies (say, with a token of /da/) will not require the lexical competition and selection processes involved in word perception. This is not to say that any of these levels are more or less valid to study; however, we have to be cautious when trying to make generalizations, and sensitive to differences in visual information as a function of linguistic level (phoneme, word, sentence).
There are still many unresolved questions regarding the representations of visual-only speech, and audiovisual integration during speech processing. Hopefully the suggestions Mitch and I have made will be useful, and we look forward to having some more data in the coming years that speak to these issues.
References:
Giraud A-L, Poeppel D (2012) Cortical oscillations and speech processing: Emerging computational principles and operations. Nat Neurosci 15:511-517. doi:10.1038/nn.3063
Peelle JE (2012) The hemispheric lateralization of speech processing depends on what "speech" is: A hierarchical perspective. Frontiers in Human Neuroscience 6:309. doi:10.3389/fnhum.2012.00309 (PDF)
Peelle JE, Davis MH (2012) Neural oscillations carry speech rhythm through to comprehension. Frontiers in Psychology 3:320. doi:10.3389/fpsyg.2012.00320 (PDF)
Peelle JE, Sommers MS (2015) Prediction and constraint in audiovisual speech perception. Cortex. doi:10.1016/j.cortex.2015.03.006 (PDF)
Schroeder CE, Lakatos P, Kajikawa Y, Partan S, Puce A (2008) Neuronal oscillations and visual amplification of speech. Trends in Cognitive Sciences 12:106-113. doi:10.1016/j.tics.2008.01.002
Talk at Macquarie University
I'm in Sydney, Australia where I've just given a talk at the Australian Hearing Hub at Macquarie University. It's my first visit to Macquarie and it's been great. The recently-established Hesring Hub combines researchers, clinicians, and industry partners in a single building (which includes a nice cafe on the ground floor...I'm jealous). This multi-pronged approach to hearing science is exemplary and a model for interdisciplinary collaboration. I look forward to good things happening here over the coming years!
One of the special treats on my visit was a visit to the state-of-the-art anechoic chamber in the basement. Now THAT is a proper sound booth.
New paper: A role for the angular gyrus in combinatorial semantics (Price et al.)
We know what a "leaf" is, and we know what "wet" means. But combining these concepts together into a "wet leaf" yields a new and possibly more specific idea. Similarly, a "brown leaf" is qualitatively different than any old leaf. Our ability to flexibly and dynamically combine concepts enables us to represent and communicate an enormous set of ideas from a relatively small number of constituents. The question of what neural systems might support conceptual combination has been a focus of research for Amy Price at Penn. Combinatorial semantics is an especially timely topic as there are ongoing debates about the anatomical systems most strongly involved in semantic memory more generally (angular gyrus? anterior temporal lobes? ventral visual regions?), as well as the nature of the information being represented (to what degree do concepts rely on sensorimotor cortices?).
In a new paper out this week in the Journal of Neuroscience (Price et al., 2015), Amy presents data from both fMRI and patients with neurodegenerative disease suggesting that the angular gyrus plays an important role in conceptual combination. Amy designed a clever task in which participants read word pairs that varied in how easily they could be combined into a single concept. For example, you could imagine that "turnip rock" is difficult to combine, whereas a "wet rock" is easier. Amy used all adjective-noun pairs, but still found a considerable amount of variability (for example a "plaid apple" combines less easily than a "plaid jacket"). This "ease of combination" was initially quantified using subject ratings, but Amy found that lexical co-occurrence statistics for these word pairs strongly correlate with their degree of combination, and thus co-occurrence measures were used in all analyses.
These findings are in good agreement with previous work emphasizing an important role for the angular gyrus in semantic representation (Binder & Desai 2011; Bonner et al. 2013).
References:
Binder JR, Desai RH (2011) The neurobiology of semantic memory. Trends in Cognitive Sciences 15:527-536. doi:10.1016/j.tics.2011.10.001
Bonner MF, Peelle JE, Cook PA, Grossman M (2013) Heteromodal conceptual processing in the angular gyrus. NeuroImage 71:175–186. doi:10.1016/j.neuroimage.2013.01.006 (PDF)
Price AR, Bonner MF, Peelle JE, Grossman M (2015) Converging evidence for the neuroanatomic basis of combinatorial semantics in the angular gyrus. Journal of Neuroscience 35:3276–3284. http://www.jneurosci.org./content/35/7/3276.short (PDF)
Talk at UCL
I gave a talk today at University College London, to the Speech Science Forum. UCL has a strong complement of speech, language, and cognitive scientists and it was a real pleasure to be here. There were a lot of interesting questions afterward, and several people helpfully pointed me towards some additional constraints or predictions that would be useful to consider.
As a side note, London will be the location of the 2016 annual meeting of the Society for the Neurobiology of Language..it will be a great opportunity to visit this amazing city.