̽��ֱ�� of Cambridge - Anders Hansen

Mathematical paradox demonstrates the limits of AI

sc604 — Thu, 17 Mar 2022 16:05:06 +0000

Like some people, AI systems often have a degree of confidence that far exceeds their actual abilities. And like an overconfident person, many AI systems don’t know when they’re making mistakes. Sometimes it’s even more difficult for an AI system to realise when it’s making a mistake than to produce a correct result.

Researchers from the ̽��ֱ�� of Cambridge and the ̽��ֱ�� of Oslo say that instability is the Achilles’ heel of modern AI and that a mathematical paradox shows AI’s limitations. Neural networks, the state-of-the-art tool in AI, roughly mimic the links between neurons in the brain. ̽��ֱ��researchers show that there are problems where stable and accurate neural networks exist, yet no algorithm can produce such a network. Only in specific cases can algorithms compute stable and accurate neural networks.

̽��ֱ��researchers propose a classification theory describing when neural networks can be trained to provide a trustworthy AI system under certain specific conditions. Their results are reported in the Proceedings of the National Academy of Sciences.

Deep learning, the leading AI technology for pattern recognition, has been the subject of numerous breathless headlines. Examples include diagnosing disease more accurately than physicians or preventing road accidents through autonomous driving. However, many deep learning systems are untrustworthy and easy to fool.

“Many AI systems are unstable, and it’s becoming a major liability, especially as they are increasingly used in high-risk areas such as disease diagnosis or autonomous vehicles,” said co-author Professor Anders Hansen from Cambridge’s Department of Applied Mathematics and Theoretical Physics. “If AI systems are used in areas where they can do real harm if they go wrong, trust in those systems has got to be the top priority.”

̽��ֱ��paradox identified by the researchers traces back to two 20th century mathematical giants: Alan Turing and Kurt Gödel. At the beginning of the 20th century, mathematicians attempted to justify mathematics as the ultimate consistent language of science. However, Turing and Gödel showed a paradox at the heart of mathematics: it is impossible to prove whether certain mathematical statements are true or false, and some computational problems cannot be tackled with algorithms. And, whenever a mathematical system is rich enough to describe the arithmetic we learn at school, it cannot prove its own consistency.

Decades later, the mathematician Steve Smale proposed a list of 18 unsolved mathematical problems for the 21st century. ̽��ֱ��18th problem concerned the limits of intelligence for both humans and machines.

“ ̽��ֱ��paradox first identified by Turing and Gödel has now been brought forward into the world of AI by Smale and others,” said co-author Dr Matthew Colbrook from the Department of Applied Mathematics and Theoretical Physics. “There are fundamental limits inherent in mathematics and, similarly, AI algorithms can’t exist for certain problems.”

̽��ֱ��researchers say that, because of this paradox, there are cases where good neural networks can exist, yet an inherently trustworthy one cannot be built. “No matter how accurate your data is, you can never get the perfect information to build the required neural network,” said co-author Dr Vegard Antun from the ̽��ֱ�� of Oslo.

̽��ֱ��impossibility of computing the good existing neural network is also true regardless of the amount of training data. No matter how much data an algorithm can access, it will not produce the desired network. “This is similar to Turing’s argument: there are computational problems that cannot be solved regardless of computing power and runtime,” said Hansen.

̽��ֱ��researchers say that not all AI is inherently flawed, but it’s only reliable in specific areas, using specific methods. “ ̽��ֱ��issue is with areas where you need a guarantee, because many AI systems are a black box,” said Colbrook. “It’s completely fine in some situations for an AI to make mistakes, but it needs to be honest about it. And that’s not what we’re seeing for many systems – there’s no way of knowing when they’re more confident or less confident about a decision.”

“Currently, AI systems can sometimes have a touch of guesswork to them,” said Hansen.“You try something, and if it doesn’t work, you add more stuff, hoping it works. At some point, you’ll get tired of not getting what you want, and you’ll try a different method. It’s important to understand the limitations of different approaches. We are at the stage where the practical successes of AI are far ahead of theory and understanding. A program on understanding the foundations of AI computing is needed to bridge this gap.”

“When 20th-century mathematicians identified different paradoxes, they didn’t stop studying mathematics. They just had to find new paths, because they understood the limitations,” said Colbrook. “For AI, it may be a case of changing paths or developing new ones to build systems that can solve problems in a trustworthy and transparent way, while understanding their limitations.”

̽��ֱ��next stage for the researchers is to combine approximation theory, numerical analysis and foundations of computations to determine which neural networks can be computed by algorithms, and which can be made stable and trustworthy. Just as the paradoxes on the limitations of mathematics and computers identified by Gödel and Turing led to rich foundation theories — describing both the limitations and the possibilities of mathematics and computations — perhaps a similar foundations theory may blossom in AI.

Matthew Colbrook is a Junior Research Fellow at Trinity College, Cambridge. Anders Hansen is a Fellow at Peterhouse, Cambridge. ̽��ֱ��research was supported in part by the Royal Society.

Reference:
Matthew J Colbrook, Vegard Antun, and Anders C Hansen. ‘ ̽��ֱ��difficulty of computing stable and accurate neural networks – On the barriers of deep learning and Smale’s 18th problem.’ Proceedings of the National Academy of Sciences (2022). DOI: 10.1073/pnas.2107151119

Humans are usually pretty good at recognising when they get things wrong, but artificial intelligence systems are not. According to a new study, AI generally suffers from inherent limitations due to a century-old mathematical paradox.

There are fundamental limits inherent in mathematics and, similarly, AI algorithms can’t exist for certain problems

Matthew Colbrook

Yuichiro Chino

Binary data wave

̽��ֱ��text in this work is licensed under a Creative Commons Attribution 4.0 International License. Images, including our videos, are Copyright © ̽��ֱ�� of Cambridge and licensors/contributors as identified. All rights reserved. We make our image and video content available in a number of ways – as here, on our main website under its Terms and conditions, and on a range of channels including social media that permit your use and sharing of our content under their respective Terms.

Yes

AI techniques in medical imaging may lead to incorrect diagnoses

sc604 — Tue, 12 May 2020 07:13:24 +0000

A team of researchers, led by the ̽��ֱ�� of Cambridge and Simon Fraser ̽��ֱ��, designed a series of tests for medical image reconstruction algorithms based on AI and deep learning, and found that these techniques result in myriad artefacts, or unwanted alterations in the data, among other major errors in the final images. ̽��ֱ��effects were typically not present in non-AI based imaging techniques.

̽��ֱ��phenomenon was widespread across different types of artificial neural networks, suggesting that the problem will not be easily remedied. ̽��ֱ��researchers caution that relying on AI-based image reconstruction techniques to make diagnoses and determine treatment could ultimately do harm to patients. Their results are reported in the Proceedings of the National Academy of Sciences.

"There’s been a lot of enthusiasm about AI in medical imaging, and it may well have the potential to revolutionise modern medicine: however, there are potential pitfalls that must not be ignored," said Dr Anders Hansen from Cambridge’s Department of Applied Mathematics and Theoretical Physics, who led the research with Dr Ben Adcock from Simon Fraser ̽��ֱ��. "We’ve found that AI techniques are highly unstable in medical imaging, so that small changes in the input may result in big changes in the output."

A typical MRI scan can take anywhere between 15 minutes and two hours, depending on the size of the area being scanned and the number of images being taken. ̽��ֱ��longer the patient spends inside the machine, the higher resolution the final image will be. However, limiting the amount of time patients spend inside the machine is desired, both to reduce the risk to individual patients and to increase the overall number of scans that can be performed.

Using AI techniques to improve the quality of images from MRI scans or other types of medical imaging is an attractive possibility for solving the problem of getting the highest quality image in the smallest amount of time: in theory, AI could take a low-resolution image and make it into a high-resolution version. AI algorithms ‘learn’ to reconstruct images based on training from previous data, and through this training procedure aim to optimise the quality of the reconstruction. This represents a radical change compared to classical reconstruction techniques that are solely based on mathematical theory without dependency on previous data. In particular, classical techniques do not learn.

Any AI algorithm needs two things to be reliable: accuracy and stability. An AI will usually classify an image of a cat as a cat, but tiny, almost invisible changes in the image might cause the algorithm to instead classify the cat as a truck or a table, for instance. In this example of image classification, the one thing that can go wrong is that the image is incorrectly classified. However, when it comes to image reconstruction, such as that used in medical imaging, there are several things that can go wrong. For example, details like a tumour may get lost or may falsely be added. Details can be obscured and unwanted artefacts may occur in the image.

"When it comes to critical decisions around human health, we can’t afford to have algorithms making mistakes," said Hansen. "We found that the tiniest corruption, such as may be caused by a patient moving, can give a very different result if you’re using AI and deep learning to reconstruct medical images – meaning that these algorithms lack the stability they need."

Hansen and his colleagues from Norway, Portugal, Canada and the UK designed a series of tests to find the flaws in AI-based medical imaging systems, including MRI, CT and NMR. They considered three crucial issues: instabilities associated with tiny perturbations, or movements; instabilities with respect to small structural changes, such as a brain image with or without a small tumour; and instabilities with respect to changes in the number of samples.

They found that certain tiny movements led to myriad artefacts in the final images, details were blurred or completely removed, and that the quality of image reconstruction would deteriorate with repeated subsampling. These errors were widespread across the different types of neural networks.

According to the researchers, the most worrying errors are the ones that radiologists might interpret as medical issues, as opposed to those that can easily be dismissed due to a technical error.

"We developed the test to verify our thesis that deep learning techniques would be universally unstable in medical imaging," said Hansen. " ̽��ֱ��reasoning for our prediction was that there is a limit to how good a reconstruction can be given restricted scan time. In some sense, modern AI techniques break this barrier, and as a result become unstable. We’ve shown mathematically that there is a price to pay for these instabilities, or to put it simply: there is still no such thing as a free lunch."

̽��ֱ��researchers are now focusing on providing the fundamental limits to what can be done with AI techniques. Only when these limits are known will we be able to understand which problems can be solved. "Trial and error-based research would never discover that the alchemists could not make gold: we are in a similar situation with modern AI," said Hansen. "These techniques will never discover their own limitations. Such limitations can only be shown mathematically."

Reference:
Vegard Antun et al. ‘On instabilities of deep learning in image reconstruction and the potential costs of AI.’ Proceedings of the National Academy of Sciences (2020). DOI: 10.1073/pnas.1907377117

Machine learning and AI are highly unstable in medical image reconstruction, and may lead to false positives and false negatives, a new study suggests.

To put it simply: there is still no such thing as a free lunch

Anders Hansen

Yes