By placing a camera inside a robot, researchers can infer how the person is touching it and what the person’s intent is just by looking at the shadow images. | Credit: Cornell University

Cornell University researchers created a low-cost method for soft robots to detect a range of physical interactions, from pats to punches to hugs, without relying on touch at all. Instead, a USB camera located inside the robot captures the shadow movements of hand gestures on the robot’s skin and classifies them with machine-learning software.

The new ShadowSense technology is the latest project from Cornell’s Human-Robot Collaboration and Companionship Lab. The group published a paper on the research called “ShadowSense: Detecting Human Touch in a Social Robot Using Shadow Image Classification.”

“Touch is such an important mode of communication for most organisms, but it has been virtually absent from human-robot interaction,” said Guy Hoffman, associate professor at Cornell’s Sibley School of Mechanical and Aerospace Engineering and the paper’s senior author. “One of the reasons is that full-body touch used to require a massive number of sensors, and was therefore not practical to implement. This research offers a low-cost alternative.”

The technology originated as part of a collaboration to develop inflatable robots that could guide people to safety during emergency evacuations. Such a robot would need to be able to communicate with humans in extreme conditions and environments. Imagine a robot physically leading someone down a noisy, smoke-filled corridor by detecting the pressure of the person’s hand.

Rather than installing a large number of contact sensors – which would add weight and complex wiring to the robot, and would be difficult to embed in a deforming skin – the team took a counterintuitive approach. In order to gauge touch, they looked to sight.

“By placing a camera inside the robot, we can infer how the person is touching it and what the person’s intent is just by looking at the shadow images,” said Yuhan Hu, the paper’s lead author. “We think there is interesting potential there, because there are lots of social robots that are not able to detect touch gestures.”

Prototype robot

The prototype robot consists of a soft inflatable bladder of nylon skin stretched around a cylindrical skeleton, roughly four feet in height, that is mounted on a mobile base. Under the robot’s skin is a USB camera, which connects to a laptop. The researchers developed a neural network-based algorithm that uses previously recorded training data to distinguish between six touch gestures – touching with a palm, punching, touching with two hands, hugging, pointing and not touching at all – with an accuracy of 87.5 to 96%, depending on the lighting.

The robot can be programmed to respond to certain touches and gestures, such as rolling away or issuing a message through a loudspeaker. And the robot’s skin has the potential to be turned into an interactive screen.

By collecting enough data, a robot could be trained to recognize an even wider vocabulary of interactions, custom-tailored to fit the robot’s task, Hu said. The robot doesn’t even have to be a robot. ShadowSense technology can be incorporated into other materials, such as balloons, turning them into touch-sensitive devices.

“While the technology has certain limitations, for example requiring a line of sight from the camera to the robot’s skin, these constraints could actually spark a new approach to social robot design that would support a visual touch sensor like the one we proposed,” Hoffman said. “In the future, we would like to experiment with using optical devices such as lenses and mirrors to enable additional form factors.”

Increasing privacy

In addition to providing a simple solution to a complicated technical challenge, and making robots more user-friendly to boot, ShadowSense offers a comfort that is increasingly rare in these high-tech times: privacy.

“If the robot can only see you in the form of your shadow, it can detect what you’re doing without taking high fidelity images of your appearance,” Hu said. “That gives you a physical filter and protection, and provides psychological comfort.”

The ability to physically interact and understand a person’s movements and moods could ultimately be just as important to the person as it is to the robot.

“Touch interaction is a very important channel in terms of human-human interaction. It is an intimate modality of communication,” Hu said. “And that’s not easily replaceable.”

Editor’s Note: This article was republished from Cornell University.

The post How shadows can help robots understand human touch appeared first on The Robot Report.

Just when it seemed like robots couldn’t get any cooler, Cornell University researchers have created a soft robot muscle that can regulate its temperature through sweating.

This form of thermal management is a basic building block for enabling untethered, high-powered robots to operate for long periods of time without overheating, according to the Rob Shepherd, associate professor of mechanical and aerospace engineering, who led the project.

The team’s paper, “Autonomic Perspiration in 3D Printed Hydrogel Actuators,” published Jan. 29 in Science Robotics.

One of the hurdles for making enduring, adaptable and agile robots is managing the robots’ internal temperature, according to Shepherd, the paper’s senior author. If the high-torque density motors and exothermic engines that power a robot overheat, the robot will cease to operate.

This is a particular issue for soft robots, which are made of synthetic materials. While more flexible, they hold their heat, unlike metals, which dissipate heat quickly. An internal cooling technology, such as a fan, may not be much help because it would take up space inside the robot and add weight.

So Shepherd’s team took inspiration from the natural cooling system that exists in mammals: sweating.

“The ability to perspire is one of the most remarkable features of humans,” said co-lead author T.J. Wallin, M.S. ’16, Ph.D. ’18, a research scientist at Facebook Reality Labs. “Sweating takes advantage of evaporated water loss to rapidly dissipate heat and can cool below the ambient environmental temperature. … So as is often the case, biology provided an excellent guide for us as engineers.”

Shepherd’s team partnered with the lab of Emmanuel Giannelis, the Walter R. Read Professor of Engineering, to create the necessary nanopolymer materials for sweating via a 3D-printing technique called multi-material stereolithography, which uses light to cure resin into predesigned shapes.

“Our contribution is the making of mixtures of nanoparticles and polymeric materials that basically allow us to control the viscosity, or flow, of these fluids,” said Giannelis, also Cornell’s vice provost for research and vice president for technology transfer, intellectual property and research policy.

The researchers fabricated finger-like actuators composed of two hydrogel materials that can retain water and respond to temperature – in effect, “smart” sponges. The base layer, made of poly-N-isopropylacrylamide, reacts to temperatures above 30 C (86 F) by shrinking, which squeezes water up into a top layer of polyacrylamide that is perforated with micron-sized pores. These pores are sensitive to the same temperature range and automatically dilate to release the “sweat,” then close when the temperature drops below 30 C.

Credit: Cornell University

The evaporation of this water reduces the actuator’s surface temperature by 21 C within 30 seconds, a cooling process that is approximately three times more efficient than in humans, the researchers found. The actuators are able to cool off roughly six times faster when exposed to wind from a fan.

“The best part of this synthetic strategy is that the thermal regulatory performance is based in the material itself,” said Wallin. “We did not need to have sensors or other components to control the sweating rate. When the local temperature rose above the transition, the pores would simply open and close on their own.”

The team incorporated the actuator fingers into a robot hand that could grab and lift objects, and they realized that autonomous sweating not only cooled the hand, but lowered the temperature of the object as well. While the lubrication could make a robot hand slippery, Shepherd says that modifications to the hydrogel texture could compensate by improving the hand’s grip, much like wrinkles in skin.

One disadvantage of the technology is that it can hinder a robot’s mobility. There is also a need for the robots to replenish their water supply, which has led Shepherd to envision soft robots that will someday not only perspire like mammals, but drink like them, too.

The ability of a robot to secrete fluids could also lead to methods for absorbing nutrients, catalyzing reactions, removing contaminants and coating the robot’s surface with a protective layer, the researchers wrote.

“I think that the future of making these more biologically analogous materials and robots is going to rely on the material composition,” Shepherd said. “This brings up a point [about the importance of] multidisciplinary research in this area, where really no one group has all the answers.”

Editor’s Note: This article was republished from the Cornell Chronicle.

The post Soft robot sweats autonomously to prevent overheating appeared first on The Robot Report.