Teaching Robots with Minecraft

Humans can walk into an environment with unlimited possibilities and make intuitive choices when they have a goal in mind. They can ignore the noise, but robots, not so much. So, researchers from Brown University decided to develop an algorithm that would allow robots to learn how to approach these real-world spaces, using Minecraft as a study aid.

The researchers highlighted the problem in their paper:

“Robots operating in unstructured, stochastic environments such as a factory floor or a kitchen face a difficult planning problem due to the large state space and the very large set of possible tasks…”

They explain that robots lack the intuition to ignore possible actionable objects when given these expansive environments. The array of choices becomes too large for a robot to handle. Stefanie Tellex, assistant professor of computer science, explained in a press release:

“It’s a really tough problem. We want robots that have capabilities to do all kinds of different things, but then the space of possible actions becomes enormous. We don’t want to limit the robot’s capabilities, so we have to find ways to shrink the search space.”

So, why Minecraft?

Tellex explains that the sandbox computer game “is a really good model of a lot of these robot problems. There’s a huge space of possible actions somebody playing this game can do, and it’s really cheap and easy to collect a ton of training data. It’s much harder to do that in the real world.”

The researchers worked to outfit the robot with the proper algorithms to plan and assess which actions and object would help it reach a particular goal. Tellex was happy to report that the robot made significant progress in this arena.

“It’s able to learn that if you’re standing next to a trench and you’re trying to walk across, you can place blocks in the trench. Otherwise don’t place blocks. If you’re trying to mine some gold under some blocks, destroy the blocks. Otherwise don’t destroy blocks.”

The researchers then moved the robot into a real-world space, giving it the task of baking brownies. It got so good at anticipating certain priors that when a carton of eggs appeared in the workspace, it knew to hand the cook a whisk. Clever bot.

Looking to the future of machine learning is Microsoft Director of Search Stefan Weitz; he explains that its success will ride on teaching artificial intelligence to identify patterns. There’s a major difference between a search engine that can critically analyze your search queries rather than simply scouring the web’s index of results.