Just as so many humans learn to cook by watching YouTube videos, so are robots. Scientists at the University of Maryland  and NICTA, Australia, are developing ways to make cooking robots by having them watch YouTube videos.

The main idea behind making cooking robots is simplifying normally complicated actions. Humans have the ability to manipulate their tools, such as using a knife for countless different kitchen tasks, whereas robots, so far, have a limited range of movement. The scientists want to make it so that all of the common actions used in the kitchen, such as stirring, pouring, twisting, and so on, can be done using claws or pincers, or even with the Versaball, or universal graspers.

Then comes the trouble of how to teach the robot to cook in the first place. The initial thought was to program the robot directly, and another was to use motion capture gloves or hand trackers to record the needed motions. The decision ultimately came down to finding a way to have a robot teach itself – through YouTube

Scientists at the University of Maryland are working on ways for the cooking robots to learn how to study human actions through these videos, and then turn them into commands that the machines are able to duplicate. The difficult part with this is that YouTube videos are unpredictable, unlike videos specifically made for machines. Between backgrounds, lightning, and other variables, there are a lot of specifics for scientists to account for.

By breaking down the movements seen in the video through Convolutional Neural Network (CNN) based recognition modules, robots can simultaneously work out what kind of grip the chef is using to do the action, and at the same time learn how the hand moves and what it’s doing. The scientists are aiming to classify different cooking scenarios and then gradually break them down further and more directly so that the robot knows, each time, exactly what to do. It will take a while, but those involved plan to turn each movement into numbers and data, which makes a learning robot a more plausible scenario.

Photo Credit: Jeremy Brooks