They move the hand of the robot onto the part of an object they want it to grip, like the rim of a bowl or the handle of a mug, and record the locations of the fingertips.īecause the NDF has learned so much about 3D geometry and how to reconstruct shapes, it can infer the structure of a new shape, which enables the system to transfer the demonstrations to new objects in arbitrary poses, Du explains. The researchers use this trained NDF model to teach a robot a new skill with only a few physical examples. Instead, our approach automatically discovers these parts from the shape reconstruction,” Du says.
“If you wanted to do this with another approach, you’d have to hand-label all the parts. For instance, it learns that the handles of mugs are similar, even if some mugs are taller or wider than others, or have smaller or longer handles. “This equivariance is what allows us to much more effectively handle cases where the object you observe is in some arbitrary orientation,” Simeonov says.Īs the NDF learns to reconstruct shapes of similar objects, it also learns to associate related parts of those objects. With this property, if the model is shown an image of an upright mug, and then shown an image of the same mug on its side, it understands that the second mug is the same object, just rotated. The team designed the NDF with a property known as equivariance. While the network was trained in simulation on a large dataset of synthetic 3D shapes, it can be directly applied to objects in the real world. The data points can be obtained from a depth camera that provides information on the distance between the object and a viewpoint.
The model computes the geometric representation for a specific item using a 3D point cloud, which is a set of data points or coordinates in three dimensions. To overcome this challenge, the researchers created a new type of neural network model, a Neural Descriptor Field (NDF), that learns the 3D geometry of a class of items.
This is one reason it is so hard for machine-learning systems to generalize to new object orientations. The research will be presented at the International Conference on Robotics and Automation.Ī robot may be trained to pick up a specific item, but if that object is lying on its side (perhaps it fell over), the robot sees this as a completely new scenario. Newton Career Development Professor of Cognitive Science and Computation in the Department of Brain and Cognitive Sciences and a member of the Computer Science and Artificial Intelligence Laboratory (CSAIL) Alberto Rodriguez, the Class of 1957 Associate Professor in the Department of Mechanical Engineering and senior authors Pulkit Agrawal, a professor in CSAIL, and Vincent Sitzmann, an incoming assistant professor in EECS. Simeonov wrote the paper with co-lead author Yilun Du, an EECS graduate student Andrea Tagliasacchi, a staff research scientist at Google Brain Joshua B. The concept of generalization by construction is a fascinating capability because this problem is typically so much harder,” says Anthony Simeonov, a graduate student in electrical engineering and computer science (EECS) and co-lead author of the paper. “Our major contribution is the general ability to much more efficiently provide new skills to robots that need to operate in more unstructured environments where there could be a lot of variability. In simulations and using a real robotic arm, the researchers show that their system can effectively manipulate never-before-seen mugs, bowls, and bottles, arranged in random poses, using only 10 demonstrations to teach the robot. With just a few demonstrations, the system uses what the neural network has learned about 3D geometry to grasp new objects that are similar to those in the demos. The technique uses a neural network specifically designed to reconstruct the shapes of 3D objects. Within 10 to 15 minutes, the robot would be ready to perform a new pick-and-place task. This machine-learning method enables a robot to pick up and place never-before-seen objects that are in random poses it has never encountered. Reprogramming that robot involves hand-labeling thousands of images that show it how to grasp these new mugs, then training the system all over again.īut a new technique developed by MIT researchers would require only a handful of human demonstrations to reprogram the robot. Everything is humming along, until the warehouse processes a change and the robot must now grasp taller, narrower mugs that are stored upside down. With e-commerce orders pouring in, a warehouse robot picks mugs off a shelf and places them into boxes for shipping.
0 kommentar(er)
