̽��ֱ�� of Cambridge - automation

Robot ‘chef’ learns to recreate recipes from watching food videos

sc604 — Mon, 05 Jun 2023 01:00:00 +0000

̽��ֱ��researchers, from the ̽��ֱ�� of Cambridge, programmed their robotic chef with a ‘cookbook’ of eight simple salad recipes. After watching a video of a human demonstrating one of the recipes, the robot was able to identify which recipe was being prepared and make it.

In addition, the videos helped the robot incrementally add to its cookbook. At the end of the experiment, the robot came up with a ninth recipe on its own. Their results, reported in the journal IEEE Access, demonstrate how video content can be a valuable and rich source of data for automated food production, and could enable easier and cheaper deployment of robot chefs.

Robotic chefs have been featured in science fiction for decades, but in reality, cooking is a challenging problem for a robot. Several commercial companies have built prototype robot chefs, although none of these are currently commercially available, and they lag well behind their human counterparts in terms of skill.

Human cooks can learn new recipes through observation, whether that’s watching another person cook or watching a video on YouTube, but programming a robot to make a range of dishes is costly and time-consuming.

“We wanted to see whether we could train a robot chef to learn in the same incremental way that humans can – by identifying the ingredients and how they go together in the dish,” said Grzegorz Sochacki from Cambridge’s Department of Engineering, the paper’s first author.

Sochacki, a PhD candidate in Professor Fumiya Iida’s Bio-Inspired Robotics Laboratory, and his colleagues devised eight simple salad recipes and filmed themselves making them. They then used a publicly available neural network to train their robot chef. ̽��ֱ��neural network had already been programmed to identify a range of different objects, including the fruits and vegetables used in the eight salad recipes (broccoli, carrot, apple, banana and orange).

Using computer vision techniques, the robot analysed each frame of video and was able to identify the different objects and features, such as a knife and the ingredients, as well as the human demonstrator’s arms, hands and face. Both the recipes and the videos were converted to vectors and the robot performed mathematical operations on the vectors to determine the similarity between a demonstration and a vector.

By correctly identifying the ingredients and the actions of the human chef, the robot could determine which of the recipes was being prepared. ̽��ֱ��robot could infer that if the human demonstrator was holding a knife in one hand and a carrot in the other, the carrot would then get chopped up.

Of the 16 videos it watched, the robot recognised the correct recipe 93% of the time, even though it only detected 83% of the human chef’s actions. ̽��ֱ��robot was also able to detect that slight variations in a recipe, such as making a double portion or normal human error, were variations and not a new recipe. ̽��ֱ��robot also correctly recognised the demonstration of a new, ninth salad, added it to its cookbook and made it.

“It’s amazing how much nuance the robot was able to detect,” said Sochacki. “These recipes aren’t complex – they’re essentially chopped fruits and vegetables, but it was really effective at recognising, for example, that two chopped apples and two chopped carrots is the same recipe as three chopped apples and three chopped carrots.”

̽��ֱ��videos used to train the robot chef are not like the food videos made by some social media influencers, which are full of fast cuts and visual effects, and quickly move back and forth between the person preparing the food and the dish they’re preparing. For example, the robot would struggle to identify a carrot if the human demonstrator had their hand wrapped around it – for the robot to identify the carrot, the human demonstrator had to hold up the carrot so that the robot could see the whole vegetable.

“Our robot isn’t interested in the sorts of food videos that go viral on social media – they’re simply too hard to follow,” said Sochacki. “But as these robot chefs get better and faster at identifying ingredients in food videos, they might be able to use sites like YouTube to learn a whole range of recipes.”

̽��ֱ��research was supported in part by Beko plc and the Engineering and Physical Sciences Research Council (EPSRC), part of UK Research and Innovation (UKRI).

Reference:
Grzegorz Sochacki et al. ‘Recognition of Human Chef’s Intentions for Incremental Learning of Cookbook by Robotic Salad Chef.’ IEEE Access (2023). DOI: 10.1109/ACCESS.2023.3276234

Researchers have trained a robotic ‘chef’ to watch and learn from cooking videos, and recreate the dish itself.

We wanted to see whether we could train a robot chef to learn in the same incremental way that humans can – by identifying the ingredients and how they go together in the dish

Greg Sochacki

Robot ‘chef’ learns to recreate recipes from watching food videos

̽��ֱ��text in this work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. Images, including our videos, are Copyright © ̽��ֱ�� of Cambridge and licensors/contributors as identified. All rights reserved. We make our image and video content available in a number of ways – as here, on our main website under its Terms and conditions, and on a range of channels including social media that permit your use and sharing of our content under their respective Terms.

Yes

Taste of the future: robot chef learns to ‘taste as you go’

sc604 — Wed, 04 May 2022 04:00:00 +0000

Working in collaboration with domestic appliances manufacturer Beko, researchers from the ̽��ֱ�� of Cambridge trained their robot chef to assess the saltiness of a dish at different stages of the chewing process, imitating a similar process in humans.

Their results could be useful in the development of automated or semi-automated food preparation by helping robots to learn what tastes good and what doesn’t, making them better cooks.

When we chew our food, we notice a change in texture and taste. For example, biting into a fresh tomato at the height of summer will release juices, and as we chew, releasing both saliva and digestive enzymes, our perception of the tomato’s flavour will change.

̽��ֱ��robot chef, which has already been trained to make omelettes based on human taster’s feedback, tasted nine different variations of a simple dish of scrambled eggs and tomatoes at three different stages of the chewing process, and produced ‘taste maps’ of the different dishes.

̽��ֱ��researchers found that this ‘taste as you go’ approach significantly improved the robot’s ability to quickly and accurately assess the saltiness of the dish over other electronic tasting technologies, which only test a single homogenised sample. ̽��ֱ��results are reported in the journal Frontiers in Robotics & AI.

̽��ֱ��perception of taste is a complex process in humans that has evolved over millions of years: the appearance, smell, texture and temperature of food all affect how we perceive taste; the saliva produced during chewing helps carry chemical compounds in food to taste receptors mostly on the tongue; and the signals from taste receptors are passed to the brain. Once our brains are aware of the flavour, we decide whether we enjoy the food or not.

Taste is also highly individual: some people love spicy food, while others have a sweet tooth. A good cook, whether amateur or professional, relies on their sense of taste, and can balance the various flavours within a dish to make a well-rounded final product.

“Most home cooks will be familiar with the concept of tasting as you go – checking a dish throughout the cooking process to check whether the balance of flavours is right,” said Grzegorz Sochacki from Cambridge’s Department of Engineering, the paper’s first author. “If robots are to be used for certain aspects of food preparation, it’s important that they are able to ‘taste’ what they’re cooking.”

“When we taste, the process of chewing also provides continuous feedback to our brains,” said co-author Dr Arsen Abdulali, also from the Department of Engineering. “Current methods of electronic testing only take a single snapshot from a homogenised sample, so we wanted to replicate a more realistic process of chewing and tasting in a robotic system, which should result in a tastier end product.”

̽��ֱ��researchers are members of Cambridge’s Bio-Inspired Robotics Laboratory run by Professor Fumiya Iida of the Department of Engineering, which focuses on training robots to carry out the so-called last metre problems which humans find easy, but robots find difficult. Cooking is one of these tasks: earlier tests with their robot ‘chef’ have produced a passable omelette using feedback from human tasters.

“We needed something cheap, small and fast to add to our robot so it could do the tasting: it needed to be cheap enough to use in a kitchen, small enough for a robot, and fast enough to use while cooking,” said Sochacki.

To imitate the human process of chewing and tasting in their robot chef, the researchers attached a conductance probe, which acts as a salinity sensor, to a robot arm. They prepared scrambled eggs and tomatoes, varying the number of tomatoes and the amount of salt in each dish.

Using the probe, the robot ‘tasted’ the dishes in a grid-like fashion, returning a reading in just a few seconds.

To imitate the change in texture caused by chewing, the team then put the egg mixture in a blender and had the robot test the dish again. ̽��ֱ��different readings at different points of ‘chewing’ produced taste maps of each dish.

Their results showed a significant improvement in the ability of robots to assess saltiness over other electronic tasting methods, which are often time-consuming and only provide a single reading.

While their technique is a proof of concept, the researchers say that by imitating the human processes of chewing and tasting, robots will eventually be able to produce food that humans will enjoy and could be tweaked according to individual tastes.

“When a robot is learning how to cook, like any other cook, it needs indications of how well it did,” said Abdulali. “We want the robots to understand the concept of taste, which will make them better cooks. In our experiment, the robot can ‘see’ the difference in the food as it’s chewed, which improves its ability to taste.”

“Beko has a vision to bring robots to the home environment which are safe and easy to use,” said Dr Muhammad W. Chughtai, Senior Scientist at Beko plc. “We believe that the development of robotic chefs will play a major role in busy households and assisted living homes in the future. This result is a leap forward in robotic cooking, and by using machine and deep learning algorithms, mastication will help robot chefs adjust taste for different dishes and users.”

In future, the researchers are looking to improve the robot chef so it can taste different types of food and improve sensing capabilities so it can taste sweet or oily food, for example.

̽��ֱ��research was supported in part by Beko plc and the Engineering and Physical Sciences Research Council (EPSRC) Centre of Doctoral Training on Agri-Food Robotics (Agriforwards CDT). EPSRC is part of UK Research and Innovation (UKRI). Fumiya Iida is a Fellow of Corpus Christi College, Cambridge.

Reference:
Grzegorz Sochacki, Arsen Abdulali, and Fumiya Iida. ‘Mastication-Enhanced Taste-Based Classification of Multi-Ingredient Dishes for Robotic Cooking.’ Frontiers in Robotics & AI (2022). DOI: 10.3389/frobt.2022.886074

A robot ‘chef’ has been trained to taste food at different stages of the chewing process to assess whether it’s sufficiently seasoned.

If robots are to be used for certain aspects of food preparation, it’s important that they are able to ‘taste’ what they’re cooking

Grzegorz Sochacki

Taste of the future: robot chef learns to ‘taste as you go’

̽��ֱ��text in this work is licensed under a Creative Commons Attribution 4.0 International License. Images, including our videos, are Copyright © ̽��ֱ�� of Cambridge and licensors/contributors as identified. All rights reserved. We make our image and video content available in a number of ways – as here, on our main website under its Terms and conditions, and on a range of channels including social media that permit your use and sharing of our content under their respective Terms.

Yes

Machine learning to help develop self-healing robots that ‘feel pain’

Anonymous — Wed, 07 Aug 2019 09:11:57 +0000

̽��ֱ��goal of the €3 million Self-healing soft robot (SHERO) project, funded by the European Commission, is to create a next-generation robot made from self-healing materials (flexible plastics) that can detect damage, take the necessary steps to temporarily heal itself and then resume its work – all without the need for human interaction.

Led by the ̽��ֱ�� of Brussels (VUB), the research consortium includes the Department of Engineering ( ̽��ֱ�� of Cambridge), École Supérieure de Physique et de Chimie Industrielles de la ville de Paris (ESPCI), Swiss Federal Laboratories for Materials Science and Technology (Empa), and the Dutch Polymer manufacturer SupraPolix.

As part of the SHERO project, the Cambridge team, led by Dr Fumiya Iida from the Department of Engineering are looking at integrating self-healing materials into soft robotic arms.

Dr Thomas George Thuruthel, also from the Department of Engineering, said self-healing materials could have future applications in modular robotics, educational robotics and evolutionary robotics where a single robot can be 'recycled' to generate a fresh prototype.

“We will be using machine learning to work on the modelling and integration of these self-healing materials, to include self-healing actuators and sensors, damage detection, localisation and controlled healing,” he said. “ ̽��ֱ��adaptation of models after the loss of sensory data and during the healing process is another area we are looking to address. ̽��ֱ��end goal is to integrate the self-healing sensors and actuators into demonstration platforms in order to perform specific tasks.”

Professor Bram Vanderborght, from VUB, who is leading the project with scientists from the robotics research centre Brubotics and the polymer research lab FYSC, said: “We are obviously very pleased to be working on the next generation of robots. Over the past few years, we have already taken the first steps in creating self-healing materials for robots. With this research we want to continue and, above all, ensure that robots that are used in our working environment are safer, but also more sustainable. Due to the self-repair mechanism of this new kind of robot, complex, costly repairs may be a thing of the past.”

Researchers from the ̽��ֱ�� of Cambridge will use self-healing materials and machine learning to develop soft robotics as part of a new collaborative project.

Self-healing robots that ‘feel pain’

Bram Vanderborght

Robotic hand made of self-healing material that can heal at room temperature

Yes

One day of paid work a week is all we need to get mental health benefits of employment

fpjl2 — Wed, 19 Jun 2019 09:23:35 +0000

Latest research finds up to eight hours of paid work a week significantly boosts mental health and life satisfaction. However, researchers found little evidence that any more hours – including a full five-day week – provide further increases in wellbeing.

̽��ֱ��£2 billion vegetable and the agricultural future of the East

lw355 — Fri, 15 Mar 2019 11:00:01 +0000

From crop science to robotics, supply chains to economics, Cambridge ̽��ֱ�� researchers are working with farmers and industry to sustainably increase agricultural productivity and profitability.

Humans need not apply

lw355 — Thu, 05 Jul 2018 13:08:49 +0000

On googling ‘will a robot take my job?’ I find myself on a BBC webpage that invites me to discover the likelihood that my work will be automated in the next 20 years. I type in ‘editor’. “It’s quite unlikely, 8%” comes back. Quite reassuring – but, coming from a farming family, it’s a sobering moment when I type in ‘farmer’: “It’s fairly likely, 76%”.

̽��ֱ��results may well be out of date – such is the swiftness of change in labour market predictions – but the fact that the webpage even exists says something about the focus of many of today’s conversations around the future of work.

Many of the discussions are driven by stark numbers. According to a scenario suggested recently by consultancy McKinsey, 75–375 million workers (3–14% of the global workforce) will need to switch occupational categories by 2030, and all workers will need to adapt “as their occupations evolve alongside increasingly capable machines”.

Just recently, online retailer Shop Direct announced the closure of warehouses and a move to automation, putting nearly 2,000 jobs at risk. Automation – or ‘embodied’ artificial intelligence (AI) – is one aspect of the disruptive effects of technology on the labour market. ‘Disembodied AI’, like the algorithms running in our smartphones, is another.

Dr Stella Pachidi from Cambridge Judge Business School believes that some of the most fundamental changes in work are happening as a result of ‘algorithmication’ of jobs that are dependent on information rather than production – the so-called knowledge economy.

Algorithms are capable of learning from data to undertake tasks that previously needed human judgement, such as reading legal contracts, analysing medical scans and gathering market intelligence.

“In many cases, they can outperform humans,” says Pachidi. “Organisations are attracted to using algorithms because they want to make choices based on what they consider is ‘perfect information’, as well as to reduce costs and enhance productivity.”

But these enhancements are not without consequences, says Pachidi, who has recently started to look at the impact of AI on the legal profession.

“If routine cognitive tasks are taken over by AI, how do professions develop their future experts?” she asks. “Expertise and the authority it gives you is distributed in the workplace. One way of learning about a job is ‘legitimate peripheral participation’ – a novice stands next to experts and learns by observation. If this isn’t happening, then you need to find new ways to learn.”

Another issue is the extent to which the technology influences or even controls the workforce. For over two years, Pachidi was embedded in a telecommunications company. There she observed “small battles” playing out that could have vast consequences for the future of the company.

“ ̽��ֱ��way telecoms salespeople work is through personal and frequent contact with clients, using the benefit of experience to assess a situation and reach a decision. However, the company had started using a data analytics algorithm that defined when account managers should contact certain customers about which kinds of campaigns and what to offer them.”

̽��ֱ��algorithm – usually built by external designers – often becomes the curator of knowledge, she explains. “In cases like this, a myopic view begins to creep into working practices whereby workers learn through the ‘algorithm’s eyes’ and become dependent on its instructions. Alternative explorations – the so-called technology of foolishness where innovation comes out of experimentation and intuition – is effectively discouraged.”

Pachidi and colleagues have even observed the development of strategies to ‘game’ the algorithm. “Decisions made by algorithms can structure and control the work of employees. We are seeing cases where workers feed the algorithm with false data to reach their targets.”

It’s scenarios like these that many researchers in Cambridge and beyond are working to avoid by increasing the trustworthiness and transparency of AI technologies (see issue 35 of Research Horizons), so that organisations and individuals understand how AI decisions are made.

In the meantime, says Pachidi, in our race to reap the undoubted benefits of new technology, it’s important to avoid taking a laissez-faire approach to algorithmication: “We need to make sure we fully understand the dilemmas that this new world raises regarding expertise, occupational boundaries and control.”

While Pachidi sees changes ahead in the nature of work, economist Professor Hamish Low believes that the future of work will involve major transitions across the whole life course for everyone: “ ̽��ֱ��traditional trajectory of full-time education followed by full-time work followed by a pensioned retirement is a thing of the past.”

“Disruptive technologies, the rise of the ad hoc ‘gig economy’, living longer and the fragile economics of pension provision will mean a multistage employment life: one where retraining happens across the life course, and where multiple jobs and no job happen by choice at different stages.”

His research examines the role of risk and the welfare system in relation to work at these various life stages. “When we are talking about the future of work,” he says, “we should have in mind these new frameworks for what people’s lives will look like, and prepare new generations for a different perspective on employment.”

On the subject of future job loss, he believes the rhetoric is based on a fallacy: “It assumes that the number of jobs is fixed. If in 30 years, half of 100 jobs are being carried out by robots that doesn’t mean we are left with just 50 jobs for humans. ̽��ֱ��number of jobs will increase: we would expect there to be 150 jobs.”

Dr Ewan McGaughey, at Cambridge’s Centre for Business Research and King’s College London, agrees that “apocalyptic” views about the future of work are misguided. “It’s the laws that restrict the supply of capital to the job market, not the advent of new technologies that causes unemployment.”

His recently published research answers the question of whether automation, AI and robotics will mean a ‘jobless future’ by looking at the causes of unemployment. “History is clear that change can mean redundancies – after World War II, 42% of UK jobs were redundant, but social policy maintained full employment. Yes, technology can displace people. But social policies can tackle this through retraining and redeployment.”

He adds: “ ̽��ֱ��big problem won’t be unemployment it will be underemployment – people who want to work but can’t because they have zero-hours contracts. If there is going to be change to jobs as a result of AI and robotics then I’d like to see governments seizing the opportunity to improve policy to enforce good job security. We can ‘reprogramme’ the law to prepare for a fairer future of work and leisure.”

This might mean revitalising fiscal and monetary policies such as a universal social security and taxing the owners of robots.

McGaughey’s findings are a call to arms to leaders of organisations, governments and banks to pre-empt the coming changes with bold new policies that ensure full employment, fair incomes and a thriving economic democracy.

“ ̽��ֱ��promises of these new technologies are astounding. They deliver humankind the capacity to live in a way that nobody could have once imagined,” he adds. “Just as the industrial revolution brought people past subsistence agriculture, and the corporate revolution enabled mass production, a third revolution has been pronounced. But it will not only be one of technology. ̽��ֱ��next revolution will be social.”

Inset image: read more about our research on the topic of work in the ̽��ֱ��'s research magazine; download a pdf; view on Issuu.

Will automation, AI and robotics mean a jobless future, or will their productivity free us to innovate and explore? Is the impact of new technologies to be feared, or a chance to rethink the structure of our working lives and ensure a fairer future for all?

If routine cognitive tasks are taken over by AI, how do professions develop their future experts?

Stella Pachidi

̽��ֱ��District

Linking research to policy makers

Dr Koen Jonkers is at the Joint Research Centre – the European Commission’s science and knowledge service in Brussels – and also a policy fellow at Cambridge’s Centre for Science and Policy (CSaP).

Over the past few months, Jonkers has been discussing the future of work with academic experts in Cambridge as part of his research for a special JRC report aimed at providing evidence for the European Commission’s employment and social affairs policies.

“Among the megatrends that will affect the future of work – an ageing workforce, migration, globalisation, urbanisation, and so on – the impact of technology is one where we seem to be witnessing a step change in the relationship that many people have with their work,” says Jonkers, who is one of the scientists employed by the JRC to provide independent scientific advice and support to EU policy.

“Some people have said there will be a major shock in terms of joblessness. Others that it is part of a trend that is ongoing and that it will bring opportunity. We want to give an overview of all the viewpoints, to analyse how well societies are equipped to deal with change, to mitigate potential adverse consequences, and to come up with an idea of what is likely to happen.

“As well as reskilling and upskilling current workers, governments will be keen to look at anticipatory actions to prepare young people to have a different type of work life to that of their parents and grandparents, so that they will be used to a world where people and machines work together.”

̽��ֱ��mission of CSaP is to improve public policy – in the UK and Europe – through the more effective use of evidence and expertise. “Through the CSaP Fellowship, it’s been very refreshing to talk with people with very high levels of expertise in fields other to my own,” says Junkers. “In such a multifaceted areas as the future of work, it’s been important for me to have expert analysis of the themes that are playing out.”

Yes

All in a day’s work

lw355 — Tue, 12 Jun 2018 08:54:24 +0000

READ THE STORY HERE

Researchers at the ̽��ֱ�� of Cambridge are helping to understand the world of work – the good, the bad, the fair and the future.

Christopher Burns on Unsplash

Yes

Let’s get statted

tdk25 — Wed, 03 Jun 2015 14:13:27 +0000

“I keep saying that the sexy job in the next 10 years will be statisticians, and I’m not kidding,” Hal Varian, Chief Economist at Google famously observed in 2009. It seems a difficult assertion to take seriously, but six years on, there is little question that their skills are at a premium.

Indeed, we may need statisticians now more than at any time in our history. Even compared with a decade ago, we can now gather, produce and consume unimaginably large quantities of information. As Varian predicted, statisticians who can crunch these numbers are all the rage. A new discipline, ‘Data Science’, which fuses statistics and computational work, has emerged.

“People are awash in data,” reflects Zoubin Ghahramani, Professor of Information Engineering at Cambridge. “This is occurring across industry, it’s changing society as we become more digitally connected, and it’s true of the sciences as well, where fields like biology and astronomy generate vast amounts of data.”

Over the past few years, Richard Samworth, Professor of Statistics, has watched the datarati step out from the shadows. “It’s probably fair to say that statistics didn’t have the world’s best PR for quite a long time,” he says. “Since this explosion in the amount of data that we can collect and store, opportunities have arisen to answer questions we previously had no hope of being able to address. These demand an awful lot of new statistical techniques.”

‘Big data’ is most obviously relevant to the sciences, where large volumes of information are gathered to answer questions in fields such as genetics, astronomy and particle physics, but it also has more familiar applications. Transport authorities gather data from electronic ticketing systems like Oyster cards to understand more about passenger movements; supermarkets closely monitor customer transactions to react to shoppers’ predilections. As users of social media, many of us disclose data about ourselves that is as valuable to marketing as it is relevant to psychoanalytics. Increasingly, we are also ‘lifeloggers’, monitoring our own behaviour, health, diet and fitness, through smart technology.

This information, as Ghahramani points out, is no use on its own: “It fills hard drives, but to extract value from it, we need methods that learn patterns in the data and allow us to make predictions and intelligent decisions.” This is what statisticians, computer scientists and machine learning specialists bring to the party – they build algorithms, which are coded as computer software, to see patterns. At root, the datarati are interpreters.

Despite their ‘sexy’ new image, however, not enough data scientists exist to meet this rocketing demand. Could some aspects of the interpretation be automated using artificial intelligence instead, Ghahramani wondered? And so, in 2014 and with funding from Google, the first incarnation of ̽��ֱ��Automatic Statistician was launched online. Despite minimal publicity, 3,000 users uploaded datasets to it within a few months.

Once fed a dataset, the Automatic Statistician assesses it against various statistical models, interprets the data and – uniquely – translates this interpretation into a short report of readable English. It does this without human intervention, drawing on an open-ended ‘grammar’ of statistical models. It is also deliberately conservative, only basing its assessments on sound statistical methodology, and even critiquing its own approach.

Ghahramani and his team are now refining the system to cope with the messy, incomplete nature of real-world data, and also plan to develop its base of knowledge and to offer interactive reports. In the longer term, they hope that the Automatic Statistician will learn from its own work: “ ̽��ֱ��idea is that it will look at a new dataset and say, ‘Ah, I’ve seen this kind of thing before, so maybe I should check the model I used last time’,” he explains.

While automated systems rely on existing models, new algorithms are needed to extract useful information from evolving and expanding datasets. Here, the role of human statisticians is vital.

To characterise the problem, Samworth presents a then-and-now comparison. During the past century, a typical statistical problem might, for instance, have been to understand the relationship between the initial speed and stopping distance of cars based on a sample size of 50.

These days, however, we can record information on a huge number of variables at once – the weather, road surface, make of car, wind direction, and so on. Although the extra information has the potential to yield better models and reduce uncertainty, in many areas, the number of features measured is so high it may even exceed the number of observations. Identifying appropriate models in this context is a serious challenge, which requires the development of new algorithms.

To resolve this, statisticians rely on a principle called ‘sparsity’; the idea that only a few bits of the dataset are really important. ̽��ֱ��statistician identifies these needles in the haystack. Various algorithms have been developed to select the important variables, so that the initial sprawl of information starts to become manageable and patterns can be extracted.

Together with his colleague Dr Rajen Shah in the Department of Pure Mathematics and Mathematical Statistics, Samworth has developed a method for refining any such variable selection technique called ‘Complementary Pairs Stability Selection’. This applies the original method to random subsamples of the data instead of the whole, and does this over and over again. Eventually, the variables that appear on a high proportion of the subsamples emerge as those meriting further attention.

Scanning Google Scholar for citations of the paper in which this was proposed, Samworth finds that his algorithm has been used in numerous research projects. One looks at how to improve fundraising for disaster zones, another examines potential biomarkers for breast cancer survival, and a third identifies risk factors connected with childhood malnutrition.

How does he feel when he sees his work being applied so far and wide? “It’s funny,” he says. “My training is in mathematics and I still get a kick from proving a theorem, but it’s also rewarding to see people using your work. It’s often said that the good thing about being a statistician is that you get to play in everyone’s back yard. I suppose this demonstrates why that’s true.”

Inset image: left to right, Zoubin Ghahramani and Richard Samworth

With more information than ever at our fingertips, statisticians are vital to innumerable fields and industries. Welcome to the world of the datarati, where humans and machines team up to crunch the numbers.

It fills hard drives, but to extract value from it, we need methods that learn patterns in the data and allow us to make predictions and intelligent decisions

Zoubin Ghahramani

Automatic Statistician

̽��ֱ��text in this work is licensed under a Creative Commons Attribution 4.0 International License. For image use please see separate credits above.

Yes