OpenAI Taught a Neural Network How to Play Minecraft

OpenAI has detailed its efforts to teach a neural network how to play Minecraft.

The organization says(Opens in a new window) it used a “massive unlabeled video dataset of human Minecraft play,” along with “a small amount of labeled contractor data,” with a technique called Video PreTraining (VPT) as it trained the neural network how to play the popular block-based title.

But the process wasn’t as simple as making a computer watch a bunch of Minecraft videos on YouTube. OpenAI says it first trained an Inverse Dynamics Model (IDM) with 2,000 hours of footage that showed what keys a player was pressing when a certain action was performed.

That IDM was then used to assist with training the VPT Foundation Model, as depicted here:

“Trained on 70,000 hours of IDM-labeled online video,” OpenAI says, “our behavioral cloning model (the ‘VPT foundation model’) accomplishes tasks in Minecraft that are nearly impossible to achieve with reinforcement learning from scratch. It learns to chop down trees to collect logs, craft those logs into planks, and then craft those planks into a crafting table; this sequence takes a human proficient in Minecraft approximately 50 seconds or 1,000 consecutive game actions.”

OpenAI didn’t stop there. Its model also learned how to perform other actions—including “swimming, hunting animals for food, and eating that food”—as well as a technique called “pillar jumping” that involves “repeatedly jumping and placing a block underneath yourself.”

The organization says it’s “open sourcing our contractor data, Minecraft environment, model code, and model weights” so others can explore the possibilities afforded by VPT. It also published a paper(Opens in a new window) with additional information about its findings from this experiment.