̽��ֱ�� of Cambridge - visual processing

Robot trained to read braille at twice the speed of humans

sc604 — Mon, 29 Jan 2024 06:04:52 +0000

̽��ֱ��research team, from the ̽��ֱ�� of Cambridge, used machine learning algorithms to teach a robotic sensor to quickly slide over lines of braille text. ̽��ֱ��robot was able to read the braille at 315 words per minute at close to 90% accuracy.

Although the robot braille reader was not developed as an assistive technology, the researchers say the high sensitivity required to read braille makes it an ideal test in the development of robot hands or prosthetics with comparable sensitivity to human fingertips. ̽��ֱ��results are reported in the journal IEEE Robotics and Automation Letters.

Human fingertips are remarkably sensitive and help us gather information about the world around us. Our fingertips can detect tiny changes in the texture of a material or help us know how much force to use when grasping an object: for example, picking up an egg without breaking it or a bowling ball without dropping it.

Reproducing that level of sensitivity in a robotic hand, in an energy-efficient way, is a big engineering challenge. In Professor Fumiya Iida’s lab in Cambridge’s Department of Engineering, researchers are developing solutions to this and other skills that humans find easy, but robots find difficult.

“ ̽��ֱ��softness of human fingertips is one of the reasons we’re able to grip things with the right amount of pressure,” said Parth Potdar from Cambridge’s Department of Engineering and an undergraduate at Pembroke College, the paper’s first author. “For robotics, softness is a useful characteristic, but you also need lots of sensor information, and it’s tricky to have both at once, especially when dealing with flexible or deformable surfaces.”

Braille is an ideal test for a robot ‘fingertip’ as reading it requires high sensitivity, since the dots in each representative letter pattern are so close together. ̽��ֱ��researchers used an off-the-shelf sensor to develop a robotic braille reader that more accurately replicates human reading behaviour.

“There are existing robotic braille readers, but they only read one letter at a time, which is not how humans read,” said co-author David Hardman, also from the Department of Engineering. “Existing robotic braille readers work in a static way: they touch one letter pattern, read it, pull up from the surface, move over, lower onto the next letter pattern, and so on. We want something that’s more realistic and far more efficient.”

̽��ֱ��robotic sensor the researchers used has a camera in its ‘fingertip’, and reads by using a combination of the information from the camera and the sensors. “This is a hard problem for roboticists as there’s a lot of image processing that needs to be done to remove motion blur, which is time and energy-consuming,” said Potdar.

̽��ֱ��team developed machine learning algorithms so the robotic reader would be able to ‘deblur’ the images before the sensor attempted to recognise the letters. They trained the algorithm on a set of sharp images of braille with fake blur applied. After the algorithm had learned to deblur the letters, they used a computer vision model to detect and classify each character.

Once the algorithms were incorporated, the researchers tested their reader by sliding it quickly along rows of braille characters. ̽��ֱ��robotic braille reader could read at 315 words per minute at 87% accuracy, which is twice as fast and about as accurate as a human Braille reader.

“Considering that we used fake blur the train the algorithm, it was surprising how accurate it was at reading braille,” said Hardman. “We found a nice trade-off between speed and accuracy, which is also the case with human readers.”

“Braille reading speed is a great way to measure the dynamic performance of tactile sensing systems, so our findings could be applicable beyond braille, for applications like detecting surface textures or slippage in robotic manipulation,” said Potdar.

In future, the researchers are hoping to scale the technology to the size of a humanoid hand or skin. ̽��ֱ��research was supported in part by the Samsung Global Research Outreach Program.

Reference:
Parth Potdar et al. ‘High-Speed Tactile Braille Reading via Biomimetic Sliding Interactions.’ IEEE Robotics and Automation Letters (2024). DOI: 10.1109/LRA.2024.3356978

Researchers have developed a robotic sensor that incorporates artificial intelligence techniques to read braille at speeds roughly double that of most human readers.

Can robots read braille?

Parth Potdar

Robot braille reader

̽��ֱ��text in this work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. Images, including our videos, are Copyright © ̽��ֱ�� of Cambridge and licensors/contributors as identified. All rights reserved. We make our image and video content available in a number of ways – on our main website under its Terms and conditions, and on a range of channels including social media that permit your use and sharing of our content under their respective Terms.

Yes

Just made coffee while chatting to a friend? Time to thank your ‘visuomotor binding’ mechanism…

amb206 — Fri, 14 Mar 2014 15:00:00 +0000

We talk about being ‘on automatic’ when we’re describing carrying out a familiar series of actions without being aware of what we’re doing.

Now researchers have for the first time found evidence that a dedicated information highway or ‘visuomotor binding’ mechanism connects what we see with what we do. This mechanism helps us to coordinate our movements in order to carry out all kinds of tasks from dunking a biscuit in your coffee, while maintaining eye contact with someone else, to playing basketball on a crowded court.

̽��ֱ��UCL-led research (published yesterday in the journal Current Biology) was a collaboration between Dr Alexandra Reichenbach, of the UCL Institute of Cognitive Neuroscience, and Dr David Franklin, of the Computational and Biological Learning Lab at Cambridge’s Department of Engineering.

Their research suggests that a specialised mechanism for spatial self-awareness links visual cues with body motion. ̽��ֱ��finding could help us understand the feeling of disconnection reported by schizophrenia patients and could also explain why people with even the most advanced prosthetic limbs find it hard to coordinate their movements.

Standard visual processing relies on us being able to ignore distractions and pay attention to objects of interest while filtering out others. “ ̽��ֱ��study shows that our brains also have separate hard-wired systems to track our own bodies visually even when we are not paying attention to them,” explained Franklin. “This allows visual attention to focus on objects in the world around us rather than on our own movements.”

̽��ֱ��newly-discovered mechanism was identified when three experiments were carried out on 52 healthy adults. In all three experiments, participants used robotic interfaces to control cursors on two-dimensional displays, where cursor motion was directly linked to hand movement. They were asked to keep their eyes fixed on the centre of the screen, a requirement checked by eye tracking. “ ̽��ֱ��robotic virtual reality system allowed us to instantaneously manipulate visual feedback independently of the physical movement of the body,” said Franklin.

In the first experiment, participants controlled two separate cursors – equally close to the centre of the screen – with their right and left hands. Their goal was to guide each cursor to a corresponding target at the top of the screen. Occasionally the cursor or target on each side would jump left or right, requiring participants to take corrective action. Each jump was ‘cued’ by a flash on one side, but this was random and did not always correspond to the side about to change.

Not surprisingly, people reached faster to cursor jumps when their attention was drawn to the ‘correct’ side by the cue. However, reactions to jumps were fast regardless of cuing, suggesting that a separate mechanism independent of attention is responsible for tracking our movements.

“ ̽��ֱ��first experiment showed us that we react very quickly to changes relating to objects directly under our own control, even when we are not paying attention to them,” explained Reichenbach. “This provides strong evidence for a dedicated neural pathway linking motor control to visual information, independently of the standard visual systems that are dependent on attention.”

̽��ֱ��second experiment was similar to the first but introduced changes in brightness to demonstrate the attention effect on the visual perception system. In the third experiment, participants were asked to guide one cursor to its target in the presence of up to four dummy targets or cursors, acting as ‘distractors’ alongside the real ones. In this experiment, responses to cursor jumps were less affected by distractors than responses to target jumps. Reactions to cursor jumps remained strong with one or two distractors but decreased significantly in the presence of four.

“These results provide further evidence of a dedicated visuomotor binding mechanism that is less prone to distractions than standard visual processing,” said Reichenbach. “It looks like the specialised system has a higher tolerance for distractions but in the end it is effected by them. Exactly why we evolved a separate mechanism remains to be seen but the need to react rapidly to different visual clues about ourselves and the environment may have enough to necessitate a separate pathway.”

For more information about this story contact Alexandra Buxton, Office of Communications, ̽��ֱ�� of Cambridge, amb206@admin.cam.ac.uk, 01223 761673

Experiments have identified a dedicated information highway that combines visual cues with body motion. This mechanism triggers responses to cues before the conscious brain has become aware of them.

̽��ֱ��study shows that our brains also have separate hard-wired systems to track our own bodies visually even when we are not paying attention to them.

David Franklin

Jenny Downing

Dunking a cookie into a cup of coffee

This work is licensed under a Creative Commons Licence. If you use this content on your site please link back to this page.

Yes

Licence type:

Attribution