Project CETI is a nonprofit scientific and conservation initiative that aims to decode whale communications. | Source: Project CETI

Off the idyllic shores of Dominica, a country in the Caribbean, hundreds of sperm whales gather deep in the sea. While their communication sounds like a series of clicks and creaks to the human ear, these whales have unique, regional dialects and even accents. A multidisciplinary group of scientists, led by Project CETI, is using soft robotics, machine learning, biology, linguistics, natural language processing, and more to decode their communications. 

Founded in 2020, Project CETI, or the Cetacean Translation Initiative, is a nonprofit organization dedicated to listening to and translating the communication systems of sperm whales. The team is using specially created tags that latch onto whales and gather information for the team to decode. Getting these tags to stay on the whales, however, is no easy task. 

“One of our core philosophies is we could never break the skin. We can never draw blood. These are just our own, personal guidelines,” David Gruber, the founder and president of Project CETI, told The Robot Report. 

“[The tags] have four suction cups on them,” he said. “On one of the suction cups is a heart sensor, so you can get the heart rate of the whale. There’s also three microphones on the front of it, so you hear the whale that it’s on, and you can know the whales that’s around it and in front of it.

“So you’ll be able to know from three different microphones the location of the whales that are speaking around it,” explained Gruber. “There’s a depth sensor in there, so you can actually see when the whale was diving and so you can see the profiles of it going up and down. There’s a temperature sensor. There’s an IMU, and it’s like a gyroscope, so you can know the position of the whale.”

Learn from Agility Robotics, Amazon, Disney, Teradyne and many more.

Finding a humane way to tag whales

One of the core principles of Project CETI, according to Gruber, is to use technology to bring people closer to animals. 

“There was a quote by Stephen Hawking in a BBC article, in which he posited that the full development of AI and robotics would lead to the extinction of the human race,” Gruber said. “And we thought, ‘This is ridiculous, why would scientists develop something that would lead to our own extinction?’ And it really inspired us to counter this narrative and be like, ‘How can we make robots that are actually very gentle and increase empathy?’”

“In order to deploy those tags onto whales, what we needed was a form of gentle, stable, reversible adhesion,” Alyssa Hernandez, a functional morphologist, entomologist, and biomechanist on the CETI team, told The Robot Report. “So something that can be attached to the whale, where it would go on and remain on the whale for a long amount of time to collect the data, but still be able to release itself eventually, whether naturally by the movements of the whale, or by our own mechanism of sort of releasing the tag itself.”

This is what led the team to explore bio-inspired techniques of adhesion. In particular, the team settled on studying suction cups that are common in marine creatures. 

“Suction discs are pretty common in aquatic systems,” said Hernandez. “They show up in multiple groups of organisms, fish, cephalopods, and even aquatic insects. And there are variations often on each of these discs in terms of the morphology of these discs, and what elements these discs have.”

Hernandez was able to draw on her biology background to design suction-cup grippers that would work particularly well on sperm whales that are constantly moving through the water. This means the suction cup would have to withstand changing pressures and forces. They can stay on a whale’s uneven skin even when it’s moving. 

“In the early days, when we first started this project, the question was, ‘Would the soft robots even survive in the deep sea?’” said Gruber. 

An overview of Project CETI’s mission. | Source: Project CETI

How suction cup shape changes performance

“We often think of suction cups as round, singular material elements, and in biology, that’s not usually the case,” noted Hernandez. “Sometimes these suction disks are sort of elongated or slightly different shaped, and oftentimes they have this sealing rim that helps them keep the suction engaged on rough surfaces.”

Hernandez said the CETI team started off with a standard, circular suction cup. Initially, the researchers tried out multiple materials and combinations of stiff backings and soft rims. Drawing on her biology experience, Hernandez began to experiment with more elongated, ellipse shapes. 

“I often saw [elongated grippers] when I was in museums looking at biological specimens or in the literature, so I wanted to look at an ellipse-shaped cup,” Hernandez said. “So I ended up designing one that was a medium-sized ellipse, and then a thinner ellipse as well. Another general design that I saw was more of this teardrop shape, so smaller at one end and wider at the base.” 

Hernadez said the team also looked at peanut-shaped grippers. In trying these different shapes, she looked for one that would provide increased resistance over the more traditional circular suction cups. 

“We tested [the grippers] on different surfaces of different roughness and different compliance,” recalled Hernandez. “We ended up finding that compared to the standard circle, and variations of ellipses, this medium-sized ellipse performed better under shear conditions.” 

She said the teardrop-shaped gripper also performed well in lab testing. These shapes performed better because, unlike a circle, they don’t have a uniform stiffness throughout the cup, allowing them to bend with the whale as it moves. 

“Now, I’ve modified [the suction cups] a bit to fit our tag that we currently have,” Hernandez said. “So, I have some versions of those cups that are ready to be deployed on the tags.”

Project CETI uses drones to monitor sperm whale movements and to place the tags on the whales. | Source: Project CETI

Project CETI continues iterating

The Project CETI team is actively deploying its tags using a number of methods, including having biologists press them onto whales using long poles, a method called pole tagging, and using drones to press the tags onto the whales. 

Once they’re on the whale, they stay on for anywhere from a few hours to a few days. Once they fall off, the CETI team has a mechanism that allows them to track the tags down and pull all of the gathered data off of them. CETI isn’t interested in making tags that can stay on the whales long-term, because sperm whales can travel long distances in just a few days, and it could hinder their ability to track the tags down once they fall off. 

The CETI team said it plans to continue iterating on the suction grippers and trying new ways to gently get crucial data from sperm whales. It’s even looking into tags that would be able to slightly crawl to different positions on the whale to gather information about what the whale is eating, Gruber said. The team is also interested in exploring tags that could recharge themselves. 

“We’re always continuing to make things more and more gentle, more and more innovative,” said Gruber. “And putting that theme forward of how can we be almost invisible in this project.”

The post Project CETI develops robotics to make sperm whale tagging more humane appeared first on The Robot Report.

While humanoid robots have burst into mainstream attention in the past year, and more and more companies have released their own models, many operate similarly. The typical humanoid uses arms and grippers to handle objects, and their rigid legs provide a mode of transportation. Researchers at the Toyota Research Institute, or TRI, said they want to take humanoids a step further with the Punyo robot. 

Punyo isn’t a traditional humanoid robot in that it doesn’t yet have legs. So far, TRI‘s team is working with just the torso of a robot and developing manipulation skills. 

“Our mission is to help people with everyday tasks in our homes and elsewhere,” said Alex Alspach, one of TRI’s tech leads for whole-body manipulation, in a video (see above). “Many of these manipulation tasks require more than just our hands and fingers.” 

When humans have to carry a large object, we don’t just use our arms to carry it, he explained. We might lean the object against our chest to lighten the load on our arms and use our backs to push through doors to reach our destination.

Manipulation that uses the whole body is tricky for humanoids, where balance is a major issue. However, the researchers at TRI designed its robot to do just that. 

“Punyo does things differently. Taking advantage of its whole body, it can carry more than it could simply by pressing with outstretched hands,” added Andrew Beaulieu, one of TRI’s tech leads for whole-body manipulation. “Softness, tactile sensing, and the ability to make a lot of contact advantageously allow better object manipulation.” 

TRI said that the word “punyo” is a Japanese word that elicits the image of a cute yet resilient robot. TRI’s stated goal was to create a robot that is soft, interactive, affordable, safe, durable, and capable.

Learn from Agility Robotics, Amazon, Disney, Teradyne and many more.

Robot includes soft limbs with internal sensors

Punyo’s hands, arms, and chest are covered with compliant materials and tactile sensors that allow it to feel contact. The soft materials allow the robot’s body to conform with the objects it’s manipulating.

Underneath it are two “hard” robot arms, a torso frame, and a waist actuator. TRI says it aimed to combine the precision of a traditional robot with the compliance, impact resistance, and sensing simplicity of soft robotic systems. 

The entirety of Punyo’s arms are covered in air-filled bladders or bubbles. These bubbles connect via a tube to a pressure sensor. This sensor can feel forces applied to the outer surfaces of the bubble.

Each bubble can also be individually pressurized to a desired stiffness, and add around 5 cm of compliance to the surface of the robot’s arms. 

Instead of traditional grippers, Punyo has “paws” made up of a single high-friction latex bubble with a camera inside. The team printed the inside of these bubbles with a dot pattern. The camera watches for deformities in this pattern to estimate forces. 

Left: Under Punyo’s sleeves are bubbles, air tubes, and pressure sensors that add compliance and tactile sensing to the arms. Right: Closeup of a pair of arm bubbles. | Source: Toyota Research Institute

Punyo learns to use full-body manipulation

Punyo learned contact-rich policies using two methods: diffusion policy and example-guided reinforcement learning. TRI announced its diffusion policy method last year. With this method, the robot uses human demonstrations to learn robust sensorimotor policies for hard-to-model tasks.

Example-guided reinforcement learning is a method that requires tasks to be modeled in simulation and with a small set of demonstrations to guide the robot’s exploration. TRI said it uses this kind of learning to achieve robust manipulation policies for tasks it can model in simulation. 

When the robot can see demonstrations of these tasks it can more efficiently learn them. It also gives TRI team more room to influence the style of motion the robot uses to achieve the task.

The team uses adversarial motion priors (AMP), which are traditionally used for stylizing computer-animated characters, to incorporate human motion imitation into its reinforcement pipeline. 

Reinforcement learning does require the team to model tasks in simulation for training. To do this, TRI used a model-based planner for demonstrations instead of teleoperation. It called this process “plan-guided reinforcement learning.”

TRI claimed that using a planner makes longer-horizon tasks that are difficult to teleoperate possible. The team can also automatically generate any number of demonstrations, reducing its pipeline’s dependence on human input. This moves TRI closer to scaling up the number of tasks tha tPunyo can handle. 

The post Punyo is a soft robot from TRI designed for whole-body manipulation appeared first on The Robot Report.

Researchers at the University of Southern California (USC) have developed a design for a sensor inspired by the folding patterns of origami that uses 3D electrodes to track deformation in robots.

The project was led by Hangbo Zhao, who holds dual appointments as an assistant professor in the Department of Aerospace and Mechanical Engineering and the Alfred E. Mann Department of Biomedical Engineering. Zhao wanted to find a new way to measure stretchability.

Typically, stretchability and recovery, which are crucial metrics for predicting and controlling the motion of a robot, are measured using cameras. This process, however, doesn’t work well outside of a lab, as when robots are out in the world, operating in space, or within the human body, they can’t be surrounded by multiple cameras.

Additionally, soft robots that stretch and deform are typically made of a soft material like rubber. While these materials are good at stretching, they can also undergo irreversible changes in the material properties through repeated use.

Instead of using cameras and soft materials, Zhao and his team leveraged their previous work in the designs and manufacturing of small-scale 3D sculptures that apply principles of origami. These methods allowed them to create a sensor that can measure a strain range up to three times higher than a typical sensor.

To do this, the USC team built a 3D structure of electrodes that converts stretch and release to a process of folding and unfolding. This process allows the shape of the robot to change without transforming the substance of the material itself.

As these electrodes unfold, they capture the strength of the electrical field. The team then developed a model that converts this electrical field reading into a measurement of deformation. This method allows the sensors to be used repeatedly and to give precise readings even when measuring large and dynamic deformations of soft bodies.

This approach is best suited for responding to large deformations that existing sensors aren’t capable of identifying accurately. This is because, through folding, engineers can achieve large jumps in dimensions without causing a change in material.

“We integrate the 3D origami-inspired electrodes with a soft, stretchable substrate through covalent bonding,” Zhao said. “This unique combination allows us to measure a very large deformation, as much as 200 percent strain, with an ultra-low hysteresis of around 1.2 percent. There’s also a very fast response, within 22 milliseconds.”

These sensors can be attached to soft bodies in motion, which includes anything from mechanical tendons found in prosthetic legs to human internal organs.

The high-performing design of these sensors means they are capable of rapidly measuring high deformation with maximum precision. The sensors also have a sensing area of just a few square millimeters, which allowed the team to measure deformation locally. The sensors can also detect strain from different directions.

While these sensors were designed for controlling soft robotics, they can also be suited for innovations in biomedicine.

“We can apply these sensors as wearable or implantable biomedical devices for healthcare monitoring,” Zhao said. “For example, tracking the movement and flexibility of our skin or our joints. There’s also high demand for developing implantable sensors that can continuously monitor the functional status of internal organs that undergo cyclic expansion and contraction.”

The USC team’s paper, “High-Stretchability and Low-Hysteresis Strain Sensors Using Origami-Inspired 3D Mesostructures,” was published in the journal Science Advances.

The post USC develops origami-inspired sensors for soft robotics appeared first on The Robot Report.

Soft Robotics grippers can move items that might be damaged by classic mechanical grippers. | Credit: Soft Robotics

Robots are finally getting a grip. 

Developers have been striving to close the gap on robotic gripping for the past several years, pursuing applications for multibillion-dollar industries. Securely gripping and transferring fast-moving items on conveyor belts holds vast promise for businesses. 

Soft Robotics, a Bedford, Mass. startup, is harnessing NVIDIA Isaac Sim to help close the sim-to-real gap for a handful of robotic gripping applications. One area is perfecting gripping for pick and placement of foods for packaging. 

Food packaging and processing companies are using the startup’s mGripAI system which combines soft grasping with 3D Vision and AI to grasp delicate foods such as proteins, produce, and bakery items without damage.

“We’re selling the hands, the eyes and the brains of the picking solution,” said David Weatherwax, senior director of software engineering at Soft Robotics. 

Unlike other industries that have adopted robotics, the $8 trillion food market has been slow to develop robots to handle variable items in unstructured environments, says Soft Robotics. 

The company, founded in 2013, recently landed $26 million in Series C funding from Tyson Ventures, Marel and Johnsonville Ventures.

Companies such as Tyson Foods and Johnsonville are betting on the adoption of robotic automation to help improve safety and increase production in their facilities. Both companies rely on Soft Robotics technologies. 

Soft Robotics is a member of the NVIDIA Inception program, which provides companies with GPU support and AI platform guidance. 

Getting a grip with synthetic data

Soft Robotics develops unique models for every one of its gripping applications, each requiring specific data sets. And picking from piles of wet, slippery chicken and other foods can be a tricky challenge. 

Utilizing Omniverse and Isaac Sim, the company can create 3D renderings of chicken parts with different backgrounds, like on conveyor belts or in bins and with different lighting scenarios. 

The company taps into Isaac Replicator to develop synthetic data, generating hundreds of thousands of images per model and distributing that among an array of instances in the cloud. Isaac Replicator is a set of tools, APIs, and workflows for generating synthetic data using Isaac Sim.

It also runs pose estimation models to help its gripping system see the angle of the item to pick. 

NVIDIA A100 GPUs on site enable Soft Robotics to run split-second inference with the unique models for each application in these food-processing facilities. Meanwhile, simulation and training in Isaac Sim offer access to NVIDIA A100s for scaling up workloads.

“Our current setup is fully synthetic, which allows us to rapidly deploy new applications. We’re all in on Omniverse and Isaac Sim, and that’s been working great for us,” said Weatherwax. 

Solving issues with occlusion, lighting 

A big challenge at Soft Robotics is solving issues with occlusion for an understanding of how different pieces of chicken stack up and overlap one another when dumped into a pile. “How those form can be pretty complex,” Weatherwax said.

Glares on wet chicken can potentially throw off detection models. “A key thing for us is the lighting, so the NVIDIA RTX-driven ray tracing is really important,” he said. 

The glares on wet chicken is a classic lighting and vision problem that requires a new approach for training machine learning vision models. | Credit: Soft Robotics

But where it really gets interesting is modeling it all in 3D and figuring out in a split second which item is the least obstructed in a pile and most accessible for a robot gripper to pick and place. 

Building synthetic data sets with physics-based accuracy, Omniverse enables Soft Robotics to create such environments. “One of the big challenges we have is how all these amorphous objects form into a pile,” Weatherwax said. 

Boosting production line pick accuracy

Production lines in food processing plants can move fast. But robots deployed with application-specific models promise to handle as many as 100 picks per minute. 

Still a work in progress, success in such tasks hinges on accurate representations of piles of items, supported by training data sets that consider every possible way items can fall into a pile. 

The objective is to provide the robot with the best available pick from a complex and dynamic environment. If food items fall off the conveyor belt or otherwise become damaged then it is considered waste, which directly impacts yield.

Driving production gains 

Meat-packing companies rely on lines of people for processing chicken, but like so many other industries they have faced employee shortages. Some that are building new plants for food processing can’t even attract enough workers at launch, said Weatherwax. 

“They are having a lot of staffing challenges, so there’s a push to automate,” he said.

The Omniverse-driven work for food processing companies has delivered a more than 10X increase in its simulation capacity, accelerating deployment times for AI picking systems from months to days. 

And that’s enabling Soft Robotics customers to get a grip on more than just deploying automated chicken-picking lines — it’s ensuring that they are covered for an employment challenge that has hit many industries, especially those with increased injury and health risks. 

“Handling raw chicken is a job better suited for a robot,” he said.

The post Soft Robotics mGripAI uses simulation to train in NVIDIA Isaac Sim appeared first on The Robot Report.

This week, we have a special video edition of The Robot Report podcast. This is the video feed from our recent interview with Tatum Robotics founder and CEO, Samantha Johnson. The video features American Sign Language (ASL) translation so that hearing-impaired individuals can also enjoy the content.

Tatum Robotics is building a robotic device shaped like a human hand and arm, that can mimic a human translator for deafblind individuals. Currently, deafblind individuals communicate by touching the hand of their translator. The human translator uses finger spelling and ASL signs to communicate.

Tatum Robotics is building a robotic analog to the human hand, designed to replicate the interaction between a translator and a deafblind user. Ultimately, Tatum Robotics wants to open up the world of ebooks for consumption by deafblind individuals. This will be followed by remote communication (i.e. over the web) between both hearing individuals and deafblind individuals, or even between two deafblind individuals.

As Samantha Johnson discusses in the video, until now, deafblind individuals are often isolated and bored for long periods of time, with no ability to communicate without a translator.

An early prototype of the Tatum Robotics communication robot for deafblind individuals. | Credit: Tatum Robotics

We want to thank the ASL translators on this project: Tymber Marsh and Sean Havas for their amazing translation skills. Tatum Robotics is currently recruiting additional ASL signers to contribute their unique ASL techniques to the robot design. If you are interested, contact Tatum Robotics directly for how you can contribute.

The post Video podcast episode featuring interview with Tatum Robotics founder appeared first on The Robot Report.

Soft Robotics’ mGrip soft gripper can handle even delicate foods, like cupcakes, without smashing them. | Source: Soft Robotics

Soft Robotics brought in $26 million in the first closing of its Series C funding round. This brings the robotic picking company’s total funding to $86 million, according to Crunchbase. 

Soft Robotics plans to use the latest round of funding to expand commercial deployments of mGripAI, its robotic picking product. mGripAI is an IP69K-rated automation package that uses 3D vision and artificial intelligence (AI) to allow industrial arms to perform automated bulk picking in food processes. 

mGripAI, originally brought to market in 2021, can perform over 90 picks per minute. The system includes perception modules that capture high-resolution, 3D images. These images are sent to an intelligence module, which translates them into action for the robotic arm and gripper. The mGrip soft gripper works in unison with the intelligence module to pick the product. 

The mGripAI system is able to track objects in real time for maximum pick accuracy. The system is also capable of grasp optimization, intelligent robot motion control and embedded object understanding. 

“We’re delighted that some of the world’s leaders in the food production and automation markets have decided to join existing investors in supporting SRI’s continuing growth journey,” Jeff Beck, CEO of Soft Robotics, said. “SRI’s technologies are increasingly crucial to enabling and scaling efficient and safe production of several food categories. This round of growth capital strengthens SRI’s ability to rapidly develop, deploy and support those technologies.”

Tyson Ventures, the venture capital arm of Tyson Foods and an existing Soft Robotics customer, led the funding round. Marel and Johnsonville, another Soft Robotics customer, also joined the funding round as new investors. The round also included participation from the company’s existing investors. 

“At Tyson, we are continually exploring new areas in automation that can enhance safety and increase the productivity of our team members,” Rahul Ray, Senior Director of Tyson Ventures, said. “Soft Robotics’ revolutionary robotic technology, computer vision and AI platform have the potential to transform the food industry and will play a key role in any company’s automation journey.”

While the company has primarily focused on automated bulk food processes, in May, Soft Robotics announced that it will be expanding the commercial focus of its mGripAI, a soft gripping solution for automating bulk food picking processes. The company plans to make the product available for order fulfillment, sortation, decanting and kitting. 

The post Soft Robotics picks up $26M appeared first on The Robot Report.

Harvard’s tentacle-like gripper wrapping around a succulent. | Source: Harvard Microrobotics Lab/Harvard SEAS

Researchers at the Harvard John A. Paulson School of Engineering and Applied Sciences (SEAS) have created a tentacle-like gripper that can grasp irregularly shaped or soft objects without damaging them. 

The gripper is made up of many thin, soft tentacles that rely on inflation to wrap themselves around an object without any sensing, planning or feedback control. Individually, each tentacle is too weak to pick up many objects, but with many working together the gripper can gently lift heavy and oddly shaped objects.

Each tentacle is made up of a foot-long hollow, rubber rubes. The tubes are made with thicker plastic on one side, so that was the tube is pressurized it curls like a pigtail. As the tube curls, it wraps and entangles itself around an object. Each added tentacle increases the strength of this hold. The gripper releases the object by simply depressurizing the tentacles. 

When designing the gripper, the research team took inspiration from nature. The gripper’s tentacles act similarly to how a jellyfish stuns its prey. 

To test how effective the gripper was, the research team used simulation and experiments where the gripper was tasked with handling a range of objects, including different houseplants and toys. 

The team hopes that the gripper can be used to grasp fragile objects, like soft fruits and vegetables in agricultural production and distribution and delicate tissue in medical settings, as well as irregularly shaped objects, like glassware, in warehouses. The gripper could replace traditional grippers that rely on embedded sensors, complex feedback loops and advanced machine-learning algorithms to work. 

The team’s research was published in the Proceedings of the National Academy of Sciences (PNAS). It was co-authored by Clark Teeple, Nicholas Charles, Yeonsu Jung, Daniel Baum and James C. Weaver, and supported by the Office of Naval Research, the National Science Foundation, the Simons Foundation and the Henri Seydoux fund. 

The post Harvard researchers create soft, tentacle-like robot gripper appeared first on The Robot Report.

MIT researchers have created a 3D printed material with embedded sensors that can sense how its moving. | Source: MIT/CSAIL

Researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have developed programmable materials that can sense their own movements. The team created lattice materials with networks of air-filled channels, which allows researchers to measure changes in air pressure within the channels when the material is being moved or bent. 

The lattice structure created by the team is a kind of architected material, meaning when you change the geometry of the features in the material, its mechanical properties, like stiffness or toughness, are altered. For a lattice, the denser the network of cells making up the structure, the stiffer it is. 

It’s difficult to integrate sensors into these materials because of the sparse, complex shapes that make them up. Putting sensors outside the structure, however, doesn’t provide enough information to get a complete picture of how the material is deforming or moving. 

CSAIL’s team used digital light processing 3D printing to incorporate the air-filled channels into the struts that form the lattice structure of the team’s material. The researchers drew the structure out of a pool of resin and hardened it into a precise shape using projected light. In this method, an image is projected onto the wet resin, and areas struct by the light are cured. Researchers used pressurized air, a vacuum and intricate cleaning to remove any excess resin before it was cured.

When the resulting structure is moved or squeezed, the channels formed by the 3D printing are deformed, causing the volume of air inside to change. The team used an off-the-shelf pressure sensor to measure these changes in pressure and get feedback on how the material is deforming. 

The CSAIL team then built off of their results by building sensors into a class of materials developed for motorized soft robotics called handed shearing auxetics (HSAs). HSAs can be twisted or stretched, making them good for soft robotic actuators. Like architected materials, HSAs are difficult to embed sensors into because of their complex structure. 

The team ran the sensorized HSA material through a series of movements for over 18 hours, and used the sensor data they gathered to train a neural network to accurately predict the robot’s motion. 

In the future, the team hopes its technology could be used to create soft, flexible robots with embedded sensors. These robots could understand their own posture and movements. The CSAIL team also sees potential for their technology to be used to create wearable devices that provide feedback on how the user is moving or interacting with their environment. 

The team recently published the results of their study in Science Advances. Daniela Rus, the Andrew and Erna Viterbi Professor of Electrical Engineering and Computer Science and director of CSAIL, was the lead author on the paper. Co-authors included Lillian Chin, a graduate student at MIT CSAIL, Ryan Truby, former CSAIL postdoc and now assistant professor at Northwestern University, and Annan Zhang, CSAIL graduate student. 

The post MIT CSAIL creates materials that can sense the way they move appeared first on The Robot Report.

The GelSight fin ray gripper was able to feel the pattern on Mason jars. | Source: CSAIL

A research team at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) has developed a robotic gripper with Fin Ray fingers that are able to feel the objects it manipulates. 

The Perceptual Science Group at CSAIL, led by professor Edward Adelson and Sandra Liu, a mechanical engineering PhD student, created touch sensors for their gripper, allowing it to feel with the same or more sensitivity as human skin. 

The team’s gripper is made of two Fin Ray fingers. The fingers act similar to a fish’s tail, which will bend towards an applied force rather than away, and are 3D printed from a flexible plastic material. Typical Fin Ray grippers have cross-struts that run through the interior, but the CSAIL team decided to hollow out the interior to make room for their sensory components. 

The inside of the gripper is illuminated by LEDs. On one end of the hollowed-out gripper sits a camera mounted to a semi-rigid backing. The camera faces a layer of pads made of a silicone gel called GelSight. The layer of pads is glued to a thin sheet of acrylic material, which is attached to the opposite end of the inner cavity. 

The gripper is designed to fold seamlessly around the objects it grips. The camera determines how the silicone and acrylic sheets deform as it touches an object. From these observations, the camera, with computational algorithms, can figure out the general shape of the object, how rough its surface is, its orientation in space and the force being applied by, and imparted to, each finger. 

Using this method, the gripper was able to handle a variety of objects, including a mini-screwdriving, a plastic strawberry, an acrylic paint tube, a Ball Mason jar and a wine glass. 

While holding these objects, the gripper was able to detect fine details on their surfaces. For example, on the plastic strawberry, the gripper could identify individual seeds on its surface. The fingers could also feel the lettering on the Mason jar, something that vision-based robotics struggle with because of the way glass objects refract light. 

Additionally, the gripper could squeeze a paint tube without breaking the container and spilling its contents, and pick up and put down a wine glass. The gripper could sense when the base of the glass touched the tabletop, resulting in proper placement seven out of 10 times. 

The team hopes to improve on the sensor by making the fingers stronger. By removing the cross-struts, the team also removed much of the structural integrity, meaning the fingers have a tendency to twist while gripping things. The CSAIL team also want to create a three fingered gripper that could pick up fruits and vegetables and evaluate their ripeness. 

The team’s work was presented at the 2022 IEEE 5th International Conference on soft robotics. 

The post MIT CSAIL develops robotic gripper that can feel what it grabs appeared first on The Robot Report.

A simulated ball covered in tiny robots that can distort its shape. | Source: University of Bath

A research team from the Universities of Bath and Birmingham are hoping to reinvent the way we design robots. 

Typically, robots, like robotic arms, are controlled by a single central controller. The research team, however, is hoping to create robots that are made from many individual units that act individually but cooperatively to determine the machine’s movement. 

The researchers are doing this using active matter, collections of a large number of active agents that resist the way that ordinary soft materials move. Typical soft materials will shrink into a sphere, like water beading into droplets, to create the smallest surface area possible. 

Active matter could be used in many ways. For example, a layer of nanorobots wrapped around a rubber ball could distort the shape of the ball by working in unison. 

“Active matter makes us look at the familiar rules of nature – rules like the fact that surface tension has to be positive – in a new light,” Dr. Jack Binysh, the first author on the study, said. “Seeing what happens if we break these rules, and how we can harness the results, is an exciting place to be doing research.”

During the study, the researchers simulated a 3D soft solid with a surface that experiences active stresses. The stresses expand the surface of the material, pulling the solid underneath with it. This can cause the entire shape to change. This change could even be tailored by altering the elastic properties of the material. 

“This study is an important proof of concept and has many useful implications,” Dr. Anton Souslov, another author on the paper, said. “For instance, future technology could produce soft robots that are far squishier and better at picking up and manipulating delicate materials.”

Eventually, the scientists hope to develop machines with arms made of flexible material with tiny robots embedded in the surface. The technology could also be used to coat the surface of nanoparticles in a responsive, active material. This would allow them to customize the size and shape of drug delivery capsules. 

For now, the researchers have already started their next phase of work: applying the general principles they’ve learned to designing specific robots. The results of their study were published in Science Advances. 

The post Researchers aim to create soft, shape-changing robots appeared first on The Robot Report.

These nano drones use soft actuators to fly through the air. | Source: MIT

Researchers at MIT have created a tiny robot, weighing less than 1 gram, that can zip around with insect-like agility and resilience. The research team isn’t the first to take on the challenge of making tiny flying robots. After all, they do have a number of real-world applications such as pollinating a field of crops or searching for survivors in disaster areas.

Typically, these kinds of robots are made with rigid actuators built from piezoelectric ceramic materials. The rigid materials help the robot to fly, but they also make it fragile. These types of robots are unlikely to survive the number of collisions it would face in the real world.

Soft actuators can be much more resilient than rigid ones, but they present a different problem for tiny robots. They require higher voltages than rigid actuators, and more voltage means bigger power electronics that the robot isn’t capable of lifting.

Kevin Chen, an assistant professor in the department of electrical engineering and computer science, and senior author of the study, worked with his team to create a new fabrication technique for soft actuators. The produced actuators operate with 75% lower voltage than current versions and can carry 80% more payload.

The team’s rectangular robot has four sets of wings, each driven by an actuator, that beat nearly 500 times per second. The actuators work like artificial muscles, and are made up of layers of elastomer placed between two very thin electrodes and rolled into a squishy cylinder.

The wings flap by applying voltage to the actuator, which makes the electrodes squeeze the elastomer.

One defect after another

One challenge the team faced was creating more surface area for the robot. The actuator requires less voltage with more surface area, so the team tried to build as many thin layers of elastomer and electrode as possible. However, the thinner the elastomer layers are, the more unstable the robot is.

The team was able to create layers that were only 10 micrometers in thickness by rethinking the fabrication process. For example, during the spin coating process, where an elastomer is poured onto a flat surface and rapidly rotated to make thinner layers of film, the team had to contend with tiny air bubbles that the spinning process created.

“In this process, air comes back into the elastomer and creates a lot of microscopic air bubbles. The diameter of these air bubbles is barely 1 micrometer, so previously we just sort of ignored them,” Chen said. “But when you get thinner and thinner layers, the effect of the air bubbles becomes stronger and stronger. That is traditionally why people haven’t been able to make these very thin layers.”

To get around this, the team performed the vacuuming process after spin coating, before the elastomer had a chance to dry, and then baking the actuator to dry it. By solving this problem, the team was able to increase the power output of the actuator by more than 300% as well as increase the lifespan of the actuator.

Baking the elastomer layers also helped to reduce the curing time of the actuator.

The artificial muscles improve the robot’s payload and allow it to achieve better hovering performance. | Photo Credit: Kevin Chen

“The first time I asked my student to make a multilayer actuator, once he got to 12 layers, he had to wait two days for it to cure. That is totally not sustainable, especially if you want to scale up to more layers,” Chen said.

The team also ran into issues with the thin electrodes in the actuator. The electrodes are made up of carbon nanotubes that are about 1/50,000 the diameter of a human hair. The more nanotubes the actuator has the higher the actuator’s power output is.

The nanotubes, however, have sharp ends that could pierce the elastomer, so the team had to find the optimal number of nanotubes.

“Two years ago, we created the most power-dense actuator and it could barely fly. We started to wonder, can soft robots ever compete with rigid robots? We observed one defect after another, so we kept working and we solved one fabrication problem after another, and now the soft actuator’s performance is catching up,” Chen said. “They are even a little bit better than the state-of-the-art rigid ones. And there are still a number of fabrication processes in material science that we don’t understand. So, I am very excited to continue to reduce actuation voltage.”

The resulting robot could hover for 20 seconds. According to Chen, this is the longest ever recorded by a sub-gram robot. It also had a longer lifespan than other actuators, lasting more than two million cycles.

The team plans to continue testing fabrication techniques in a clean room where it won’t have to contend with dust in the air when creating the actuator layers. Chen hopes to make the layers as thin as 1 micrometer.

The post Soft actuators help MIT power nano drones appeared first on The Robot Report.

The unique bi-stable design of the actuator allows it to snap back and forth quickly with temperature changes. | Source: North Carolina State University

A research team at North Carolina State University (NCSU) has developed a quicker thermal actuator for soft robotic devices. Actuators create motion by converting energy into work.

“Using thermal actuation is not new for soft robots, but the biggest challenge for soft thermal actuators was that they were relatively slow – and we’ve made them fast,” Yong Zhu, corresponding author of the paper and the Andrew A. Adams Distinguished Professor of mechanical and aerospace engineering at NCSU, said.

The key to the team’s actuator’s quick movements is in its bi-stable design, and the shape it prefers depends on the temperature.

The research team put silver nanowires between two layers of different materials on top of each other. The two materials will heat and expand at different temperatures, as that happens the structure will bend.

At a certain point, once the structure reaches a critical temperature, it will snap into place. Once it snaps into place the structure is stable again. The same process occurs as the structure cools. The critical temperatures, however, will be different, with the heating critical temperature being higher.

“Think of a snap hair clip. It’s stable until you apply a certain amount of energy (by bending it over), and then it snaps into a different shape – which is also stable,” Shuang Wu, first author of the paper and Ph.D. student at NCSU, said.

The researchers created two prototypes to test their method. The first snaps shut or open, mimicking the movement of a venus flytrap. The second’s snapping movement helps it to crawl, moving at more than one body length per second.

The actuator has potential applications in a variety of industries, from biomedical applications to manufacturing.

Moving forward, the team hopes to automate the process by creating sensor and control mechanisms.

“We’re also interested in exploring other possible materials, so that we could fine-tune the thermal and mechanical properties,” Zhu said. “This could allow us to tailor both actuator speed and force.”

Editor’s Note: You can read the team’s entire research here.

The post NCSU researchers make faster thermal actuator appeared first on The Robot Report.

A hexapod robot with fin-ray inspired grippers that have altered crossbeams.

Many soft robotic grippers are inspired by the way a fish’s fin bends. They involve triangular components that bend inward, around an object to grip it. These components have transverse beams that sit at a 90° angle, which help the gripper to bend around an object.

These traditional grippers require a lot of force to pick up an object, and aren’t always capable of handling delicate objects without crushing them. Researchers from five different universities, led by Poramante Manoonpong from Syddansk University in Odense, are taking their inspiration from the way insects walk and grip things to develop a different way to make robotic grippers.

Instead of transverse beams sitting at a 90° angle, the international research team in Odense put the beams at 10° or 30° angles, similar to insect attachment pads. By changing the angle of these beams, researchers found that their grippers were able to bend more around objects and pick them up using 20% less force.

“We know that many animals are able to [grip or attach themselves] to surfaces using compliant pads or feet,” said Manoonpong. “We could see that the angle of insects’ attachment pads was different. Previous work has already investigated changing the angle of the gripper, but none have conducted detailed investigation on the effect of different crossbeam angles inside its frame.”

Using less force to pick up an object allows the gripper to handle more delicate items, like food items, without crushing them. When the researchers tested grippers with beams at 90° or 120°, it took much more force for the gripper to pick up an item.

The new grippers also improved the way that robots walk. When faced with uneven surfaces, like rocks, or round ones, like tubes to walk across, the robots were able to navigate the situation without complex control. The grippers resemble flippers, which can bend easily around the surface the robot walks across.

Inspired by nature

Manoonpong’s team of researchers were inspired by insect attachment pads, but they’re not the only researchers looking to nature to make advancements in robotics.

In December 2020, researchers at the University of Georgia College of Engineering developed a robotic gripper inspired by beanpoles. These soft grippers require only one pneumatic control, which simplified the operation. The gripper also had a fiber optic sensor in the robot’s elastic spine that can sense the item it’s picking up and any external disturbances.

In November 2020, researchers at UNSW Sydney created a gripper inspired by an elephant’s trunk. This soft fabric gripper was designed to handle heavy objects as well as fragile ones.

The Odense team tested their gripper model using only soft grippers. Their next challenge is to make one strong enough to handle harsher environments.

The Odense team’s entire research can be found here.

The post These robotic grippers are inspired by insects appeared first on The Robot Report.

The smart hand is soft and elastic, weighs about half a pound, and costs a fraction of comparable prosthetics.

For the more than 5 million people in the world who have undergone an upper-limb amputation, prosthetics have come a long way. Beyond traditional mannequin-like appendages, there is a growing number of commercial neuroprosthetics — highly articulated bionic limbs, engineered to sense a user’s residual muscle signals and robotically mimic their intended motions.

But this high-tech dexterity comes at a price. Neuroprosthetics can cost tens of thousands of dollars and are built around metal skeletons, with electrical motors that can be heavy and rigid.

Now engineers at MIT and Shanghai Jiao Tong University have designed a soft, lightweight, and potentially low-cost neuroprosthetic hand. Amputees who tested the artificial limb performed daily activities, such as zipping a suitcase, pouring a carton of juice, and petting a cat, just as well as — and in some cases better than — those with more rigid neuroprosthetics.

The researchers found the prosthetic, designed with a system for tactile feedback, restored some primitive sensation in a volunteer’s residual limb. The new design is also surprisingly durable, quickly recovering after being struck with a hammer or run over with a car.

The smart hand is soft and elastic, and weighs about half a pound. Its components total around $500 — a fraction of the weight and material cost associated with more rigid smart limbs.

“This is not a product yet, but the performance is already similar or superior to existing neuroprosthetics, which we’re excited about,” said Xuanhe Zhao, professor of mechanical engineering and of civil and environmental engineering at MIT. “There’s huge potential to make this soft prosthetic very low cost, for low-income families who have suffered from amputation.”

Zhao and his colleagues have published their work today in Nature Biomedical Engineering. Co-authors include MIT postdoc Shaoting Lin, along with Guoying Gu, Xiangyang Zhu, and collaborators at Shanghai Jiao Tong University in China.

Big Hero hand

The team’s pliable new design bears an uncanny resemblance to a certain inflatable robot in the animated film “Big Hero 6.” Like the squishy android, the team’s artificial hand is made from soft, stretchy material — in this case, the commercial elastomer EcoFlex. The prosthetic comprises five balloon-like fingers, each embedded with segments of fiber, similar to articulated bones in actual fingers. The bendy digits are connected to a 3-D-printed “palm,” shaped like a human hand.

Related: Watch a soft robotic hand play Mario Bros.

Rather than controlling each finger using mounted electrical motors, as most neuroprosthetics do, the researchers used a simple pneumatic system to precisely inflate fingers and bend them in specific positions. This system, including a small pump and valves, can be worn at the waist, significantly reducing the prosthetic’s weight.

Lin developed a computer model to relate a finger’s desired position to the corresponding pressure a pump would have to apply to achieve that position. Using this model, the team developed a controller that directs the pneumatic system to inflate the fingers, in positions that mimic five common grasps, including pinching two and three fingers together, making a balled-up fist, and cupping the palm.

The pneumatic system receives signals from EMG sensors — electromyography sensors that measure electrical signals generated by motor neurons to control muscles. The sensors are fitted at the prosthetic’s opening, where it attaches to a user’s limb. In this arrangement, the sensors can pick up signals from a residual limb, such as when an amputee imagines making a fist.

The team then used an existing algorithm that “decodes” muscle signals and relates them to common grasp types. They used this algorithm to program the controller for their pneumatic system. When an amputee imagines, for instance, holding a wine glass, the sensors pick up the residual muscle signals, which the controller then translates into corresponding pressures. The pump then applies those pressures to inflate each finger and produce the amputee’s intended grasp.

Going a step further in their design, the researchers looked to enable tactile feedback — a feature that is not incorporated in most commercial neuroprosthetics. To do this, they stitched to each fingertip a pressure sensor, which when touched or squeezed produces an electrical signal proportional to the sensed pressure. Each sensor is wired to a specific location on an amputee’s residual limb, so the user can “feel” when the prosthetic’s thumb is pressed, for example, versus the forefinger.

Good grip

To test the inflatable hand, the researchers enlisted two volunteers, each with upper-limb amputations. Once outfitted with the neuroprosthetic, the volunteers learned to use it by repeatedly contracting the muscles in their arm while imagining making five common grasps.

After completing this 15-minute training, the volunteers were asked to perform a number of standardized tests to demonstrate manual strength and dexterity. These tasks included stacking checkers, turning pages, writing with a pen, lifting heavy balls, and picking up fragile objects like strawberries and bread. They repeated the same tests using a more rigid, commercially available bionic hand and found that the inflatable prosthetic was as good, or even better, at most tasks, compared to its rigid counterpart.

One volunteer was also able to intuitively use the soft prosthetic in daily activities, for instance to eat food like crackers, cake, and apples, and to handle objects and tools, such as laptops, bottles, hammers, and pliers. This volunteer could also safely manipulate the squishy prosthetic, for instance to shake someone’s hand, touch a flower, and pet a cat.

In a particularly exciting exercise, the researchers blindfolded the volunteer and found he could discern which prosthetic finger they poked and brushed. He was also able to “feel” bottles of different sizes that were placed in the prosthetic hand, and lifted them in response. The team sees these experiments as a promising sign that amputees can regain a form of sensation and real-time control with the inflatable hand.

The team has filed a patent on the design, through MIT, and is working to improve its sensing and range of motion.

“We now have four grasp types. There can be more,” Zhao said. “This design can be improved, with better decoding technology, higher-density myoelectric arrays, and a more compact pump that could be worn on the wrist. We also want to customize the design for mass production, so we can translate soft robotic technology to benefit society.”

Editor’s Note: This article was republished from MIT News.

The post Inflatable robotic hand gives amputees real-time tactile control appeared first on The Robot Report.

A team of researchers from the University of Maryland has 3D printed a soft robotic hand that is agile enough to play Nintendo’s Super Mario Bros. – and win!

The feat demonstrates a promising innovation in the field of soft robotics, which centers on creating new types of flexible, inflatable robots that are powered using water or air rather than electricity. The inherent safety and adaptability of soft robots has sparked interest in their use for applications like prosthetics and biomedical devices. Unfortunately, controlling the fluids that make these soft robots bend and move has been especially difficult – until now.

The key breakthrough by the team, led by University of Maryland assistant professor of mechanical engineering Ryan D. Sochol, was the ability to 3D print fully assembled soft robots with integrated fluidic circuits in a single step.

“Previously, each finger of a soft robotic hand would typically need its own control line, which can limit portability and usefulness,” explains co-first author Joshua Hubbard, who performed the research during his time as an undergraduate researcher in Sochol’s Bioinspired Advanced Manufacturing (BAM) Laboratory at UMD. “But by 3D printing the soft robotic hand with our integrated fluidic transistors, it can play Nintendo based on just one pressure input.”

As a demonstration, the team designed an integrated fluidic circuit that allowed the hand to operate in response to the strength of a single control pressure. For example, applying a low pressure caused only the first finger to press the Nintendo controller to make Mario walk, while a high pressure led to Mario jumping. Guided by a set program that autonomously switched between off, low, medium, and high pressures, the robotic hand was able to press the buttons on the controller to successfully complete the first level of Super Mario Bros. in less than 90 seconds.

“Recently, several groups have tried to harness fluidic circuits to enhance the autonomy of soft robots,” said recent Ph.D. graduate and co-first author of the study Ruben Acevedo, “but the methods for building and integrating those fluidic circuits with the robots can take days to weeks, with a high degree of manual labor and technical skill.”

To overcome these barriers, the team turned to “PolyJet 3D Printing,” which is like using a color printer, but with many layers of multi-material ‘inks’ stacked on top of one another in 3D.

“Within the span of one day and with minor labor, researchers can now go from pressing start on a 3D printer to having complete soft robots – including all of the soft actuators, fluidic circuit elements, and body features – ready to use,” said study co-author Kristen Edwards.

The choice to validate their strategy by beating the first level of Super Mario Bros. in real time was motivated by science just as much as it was by fun. Because the video game’s timing and level make-up are established, and just a single mistake can lead to an immediate game over, playing Mario provided a new means for evaluating soft robot performance that is uniquely challenging in a manner not typically tackled in the field.

In addition to the Nintendo-playing robotic hand, Sochol’s team also reported terrapin turtle-inspired soft robots in their paper. The terrapin happens to be UMD’s official mascot, and all of the team’s soft robots were printed at UMD’s Terrapin Works 3D Printing Hub.

Another important benefit of the team’s strategy is that it’s open source, with the paper open access for anyone to read as well as a link in the supplementary materials to a GitHub with all of the electronic design files from their work.

“We are freely sharing all of our design files so that anyone can readily download, modify on demand, and 3D print – whether with their own printer or through a printing service like us – all of the soft robots and fluidic circuit elements from our work,” said Sochol. “It is our hope that this open-source 3D printing strategy will broaden accessibility, dissemination, reproducibility, and adoption of soft robots with integrated fluidic circuits and, in turn, accelerate advancement in the field.”

At present, the team is exploring the use of their technique for biomedical applications including rehabilitation devices, surgical tools, and customizable prosthetics. As Sochol is a faculty affiliate of the Fischell Department of Bioengineering as well as a member of both the Maryland Robotics Center and the Robert E. Fischell Institute for Biomedical Devices, the team has an exceptional environment to continue advancing their strategy to address pressing challenges in biomedical fields.

Editor’s Note: This article was republished from the University of Maryland.

The post Watch a soft robotic hand play Super Mario Bros. appeared first on The Robot Report.