Mechanisms of real-time speech interpretation in the human brain revealed

jg533 — Mon, 30 Sep 2019 19:00:00 +0000

In a study published today in the journal PNAS, researchers at the ̽��ֱ�� of Cambridge developed novel computational models of the meanings of words, and tested these directly against real-time brain activity in volunteers.

“Our ability to put words into context, depending on the other words around them, is an immediate process and it’s thanks to the best computer we’ve ever known: the brain in our head. It’s something we haven’t yet managed to fully replicate in computers because it is still so poorly understood,” said Lorraine Tyler, Director of the Centre for Speech, Language and the Brain at the ̽��ֱ�� of Cambridge, which ran the study.

Central to understanding speech are the processes involved in what is known as ‘semantic composition’ – in which the brain combines the meaning of words in a sentence as they are heard, so that they make sense in the context of what has already been said. This new study has revealed the detailed real-time processes going on inside the brain that make this possible.

By saying the phrase: “the elderly man ate the apple” and watching how the volunteers’ brains responded, the researchers could track the dynamic patterns of information flow between critical language regions in the brain.

As the word ‘eat’ is heard, it primes the brain to put constraints on how it interprets the next word in the sentence: ‘eat’ is likely to be something to do with food. ̽��ֱ��study shows how these constraints directly affect how the meaning of the next word in the sentence is understood, revealing the neural mechanisms underpinning this essential property of spoken language – our ability to combine sequences of words into meaningful expressions, millisecond by millisecond as the speech is heard.

“ ̽��ֱ��way our brain enables us to understand what someone is saying, as they’re saying it, is remarkable,” said Professor Tyler. “By looking at the real-time flow of information in the brain we’ve shown how word meanings are being rapidly interpreted and put into context.”

This research is funded by the European Research Council.

Reference
Lyu, B. et al; Neural dynamics of semantic composition. PNAS (2019). DOI: 10.1073/pnas.1903402116

Scientists have come a step closer to understanding how we’re able to understand spoken language so rapidly, and it involves a huge and complex set of computations in the brain.

̽��ֱ��way our brain enables us to understand what someone is saying, as they’re saying it, is remarkable. By looking at the real-time flow of information in the brain we’ve shown how word meanings are being rapidly interpreted and put into context.

Lorraine Tyler

̽��ֱ��text in this work is licensed under a Creative Commons Attribution 4.0 International License. Images, including our videos, are Copyright © ̽��ֱ�� of Cambridge and licensors/contributors as identified. All rights reserved. We make our image and video content available in a number of ways – as here, on our main website under its Terms and conditions, and on a range of channels including social media that permit your use and sharing of our content under their respective Terms.

Yes

̽��ֱ��communicative brain

lw355 — Tue, 29 Nov 2011 10:00:57 +0000

̽��ֱ��ability to communicate using language is fundamental to the distinctive and remarkable success of the modern human. It is this capacity that separates us most decisively from our primate cousins, despite all that we have in common across species as intelligent social primates.

A major challenge for the cognitive neurosciences is to understand this relationship: what is the neurobiological context in which human language and communication have emerged, and what are the special human properties that make language itself possible?

For the past 150 years, scientific thinking about this relationship has been dominated by the concept of a single, central language system built around the brain’s left hemisphere. Pioneering 19th-century neurologists Paul Broca and Carl Wernicke noticed that patients with left hemisphere brain damage had difficulties with language comprehension and language production. Two areas of the left frontal and temporal lobes, Broca’s area and Wernicke’s area, and the bundle of nerve fibres connecting them, were identified as critical for speaking and understanding language.

Recent research in our laboratories suggests major limitations to this classic approach to language and the brain. ̽��ֱ��Broca–Wernicke concept captures one important aspect of the neural language system – the key role of the left hemisphere network – but it obscures another, equally important one. This is the role of bi-hemispheric systems and processes, whereby both left and right hemispheres work together to provide the fundamental underpinnings for human communicative processes.

A more fruitful approach to human language and communication will require a dual neurobiological framework in which these capacities are supported by two intersecting but evolutionarily and functionally distinguishable subsystems. ̽��ֱ��historical failure to make this separation has, we suggest, severely undermined scientific attempts to understand language, both as a neurocognitive phenomenon in the modern human, and in terms of its evolutionary and neurobiological context.

Dual systems

A strong evolutionary continuity between humans and our primate relatives is provided by a distributed, bi-hemispheric set of capacities that support the dynamic interpretation of visual and auditory signals in the service of social communication. These capacities have been the object of intensive study in monkeys and apes, and there is good evidence that their basic architecture underpins related communicative functions in the human.

In the context of human language comprehension, the bi-hemispheric systems support the ability not only to identify the words a speaker is producing – typically by integrating auditory and visual cues in face-to-face interaction – but also to make sense of these word-meanings in the general context of the listener’s knowledge of the world and of the specific context of speaking.

Where we see divergence between humans and other primates is in the domain of grammatical (or syntactic) function. Primate communication systems are not remotely comparable to human language in their expressive capacities. Human language is much more than a set of signs that stand for things. It constitutes a powerful and flexible set of grammatical devices for organising the flow of linguistic information and its interpretation, allowing us to represent and combine abstract linguistic elements, where these elements convey not only meaning but also the subtle structural cues that indicate how these elements are linked together.

It is the fronto-temporal network of regions in the left hemisphere that mediates these core grammatical functions in humans. This is a network that differs neuroanatomically from those of the brains of other primates, showing substantial increases in size, complexity and connectivity.

Although it’s not yet understood just how these evolutionary changes in the left hemisphere provide the neural substrate on which grammatical functions depend, it is clear that they are essential. When the left hemisphere system is damaged, the parallel right hemisphere regions cannot take over these functions, even when damage is sustained early in childhood.

Critically, however, the left hemisphere system that has emerged in humans neither replaces nor displaces the bi-hemispheric system for social communication and action found in both humans and other primates. It interacts and combines with it to create a co-ordinated process of linguistically guided communication and social interaction.

Functional separability

̽��ֱ��most direct evidence for a dual system approach is the ability to separate these systems in the modern human. Using a combination of behavioural and neuroimaging techniques, we have been able to demonstrate this both in patients with left hemisphere brain damage and in unimpaired young adults.

In the research with patients (conducted with Dr Paul Wright in the Department of Experimental Psychology and Dr Emmanuel Stamatakis in the Division of Anaesthesia) we focus on the comprehension of spoken words and spoken sentences. In initial testing, patients perform classic measures of syntactic function, where they match different spoken sentences to sets of pictures. Shown three pictures – a woman pushing a girl, a girl pushing a woman and a woman teaching a girl – patients will correctly match the sentence ‘ ̽��ֱ��woman pushed the girl’ to the first picture but will incorrectly match the passive sentence ‘ ̽��ֱ��woman is being pushed by the girl’ to the same picture. ̽��ֱ��second sentence requires the use of syntactic cues to extract the right meaning – just using the order of words is not sufficient.

These behavioural tests of syntactic impairment are linked, in the same patients, to their performance in the neuroimaging laboratory, where they hear sentences that vary in their syntactic demands, and where the precise extent of the injury to their brains can be mapped out. When we put these different sources of information together, we see that damage to the left hemisphere system progressively impairs the syntactic aspects of language processing – the more damage, the worse the performance.

Critically, however, the amount of left hemisphere damage, and the extent to which it involves the key fronto-temporal circuit, does not affect the patients’ ability to identify the words being spoken or to understand the messages being communicated – so long as syntactic cues are not required to do so. These capacities are supported bi-hemispherically, and can remain relatively intact even in the face of massive left hemisphere damage.

In work carried out with Dr Mirjana Bozic, then based at the Medical Research Council (MRC) Cognition and Brain Sciences Unit in Cambridge, we have been able to delineate these systems in the undamaged brain, using functional neuroimaging to tease out the different processing regions that are engaged by speech inputs with different properties.

Listeners hear either words that are specifically linguistically complex (words like played, which have the grammatical inflection ‘ed’), or words that make more general demands on the language processing system (words like ramp, which have another word, ram, embedded in them). Using an analysis technique that identifies the separate dimensions of the brain’s response to these sets of words, we see that the linguistically complex words activate a response component that is restricted to the left fronto-temporal region. By contrast, words that are perceptually complex, due to increased competition between the whole word and the embedded word, activate a strongly bi-hemispheric set of regions, partially overlapping with the linguistic component. Even in the intact brain, therefore, we can see the dynamic allocation of processing resources across the two systems, as a function of their joint roles in the communicative process.

Implications

A dual systems account of the ‘communicative brain’ is likely to have important and illuminating consequences for the sciences of language and its disorders.

In the context of left hemisphere brain damage we can better appreciate – and build upon for rehabilitation – the substantial bi-hemispheric communicative capacities the patient may still possess. In first- and second-language acquisition, we can better understand the learning trajectories that lead to language proficiency in terms of the relative contributions of these two aspects of communicative function.

̽��ֱ��approach also provides a new perspective on the variation between languages, where different languages may load more or less heavily on the different computational resources made available by the two systems. Most importantly, it enables us to clarify and focus the core issues for a neurobiological account of language and communication, a scientific domain clouded by ideology and inconsistency.

What is it about the human brain that makes language possible? Two evolutionary systems working together, say neuroscientists Professor William Marslen-Wilson and Professor Lorraine Tyler.

William Marslen-Wilson and Lorraine Tyler

Functional neuroimaging of the human brain

This work is licensed under a Creative Commons Licence. If you use this content on your site please link back to this page.

Yes

̽��ֱ�� of Cambridge - William Marslen-Wilson

Mechanisms of real-time speech interpretation in the human brain revealed

̽��ֱ��communicative brain

Dual systems

Functional separability

Implications