Project Malmo

Malmo is an AI experimentation platform built on top of Minecraft, and designed to support fundamental research in AI.

[Project Malmo @Project_Malmo]

Join me in attending the next episode of Research Forum, a series from Microsoft Research that explores bold ideas, new advances, and important discussions with the global research community in the era of general AI. Register: https://t.co/yVzC7NsUFY

[Project Malmo @Project_Malmo]

MineLand: Simulating Large-Scale Multi-Agent Interactions with Limited Multimodal Senses and Physical Needs https://t.co/m05lkmQFzI

[Project Malmo @Project_Malmo]

Large Language Models as Minecraft Agents https://t.co/QfctiA2LBK

[Project Malmo @Project_Malmo]

Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents https://t.co/qlTTvo0uCd

[AK @_akhaliq]

JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models

paper page: https://t.co/jo9SkrD7re

Achieving human-like planning and control with multimodal observations in an open world is a key milestone for more functional generalist agents. Existing approaches can handle certain long-horizon tasks in an open world. However, they still struggle when the number of open-world tasks could potentially be infinite and lack the capability to progressively enhance task completion as game time progresses. We introduce JARVIS-1, an open-world agent that can perceive multimodal input (visual observations and human instructions), generate sophisticated plans, and perform embodied control, all within the popular yet challenging open-world Minecraft universe. Specifically, we develop JARVIS-1 on top of pre-trained multimodal language models, which map visual observations and textual instructions to plans. The plans will be ultimately dispatched to the goal-conditioned controllers. We outfit JARVIS-1 with a multimodal memory, which facilitates planning using both pre-trained knowledge and its actual game survival experiences. In our experiments, JARVIS-1 exhibits nearly perfect performances across over 200 varying tasks from the Minecraft Universe Benchmark, ranging from entry to intermediate levels. JARVIS-1 has achieved a completion rate of 12.5% in the long-horizon diamond pickaxe task. This represents a significant increase up to 5 times compared to previous records. Furthermore, we show that JARVIS-1 is able to self-improve following a life-long learning paradigm thanks to multimodal memory, sparking a more general intelligence and improved autonomy.

[Project Malmo @Project_Malmo]

The ToMCAT Dataset, a rich, multimodal dataset consisting of data from 40 teams of three humans conducting simulated urban search-and-rescue (SAR) missions in a Minecraft based testbed, collected for the ToMCAT project. https://t.co/akwqjkrH7s

[Julia Kiseleva @julia_kiseleva]

The #greenlands platform is now live! 🤖🧑‍🤝‍🧑 With the ability to test #AI agents with real humans and receive invaluable feedback, this could be a game-changer for evaluation methods! Don't miss out, check it out now! #artificialintelligence #gaming #evaluation #Minecraft

[MsftTechNews @MsftTechNews]

Students can learn Azure via server creation, dive into #AI using Minecraft & GPT3 OpenAI or learn with Minecraft using Microsoft Research Project Malmo. @wongcyrus shares how to set up a free private Minecraft server in Azure via #EDU yearly credit. https://t.co/6vxpxfU6eI

[Embedded House @embedded_house]

Nvidia AI plays Minecraft, wins machine&learning conference award: NeurIPS 2022 honors MineDojo for playing Minecraft when instructed by written prompts. https://t.co/SWW5SwyPCF #technews #ev #tesla

[iglu-contest @ NeurIPS @IgluContest]

📢📢We are happy to announce the winners IGLU NLP track at @NeurIPSConf:
🥇First place – Craftsmanfly team: zubingou, wxl1999, guo_qingyan, jason_zhang
🥈Second place – try1try team
🥉Third place – felipe_b team: Felipe Bivort Haiek
https://t.co/LvMFD4U7Bz

[Julia Kiseleva @julia_kiseleva]

Happy to share our job post for *in person* summer internships!
https://t.co/5rnuxjgrZR

[iglu-contest @ NeurIPS @IgluContest]

📢It's never too late!
There is still one month left to the IGLU 2022
@NeurIPSConf
NLP task deadline.
Get your hands dirty by playing with our baseline.
"What to ask as Clarifying questions?"
🎁 $9,000 cash prize + Co-authorship
#NeurIPS2022 #NLP #AI
https://t.co/Nmzzr3AKZX

[Sam Devlin @smdvln]

Heading to London (for the first time in over 2 years🤯) to talk @ic_arl @ICComputing about Human-AI collaboration for improved player experience today at 5pm (BST).

Free registration for in-person or remote is still open at: https://t.co/LdW0FTL7Xk

[ICARL @ic_arl]

You can register now for our next seminar with
@smdvln from @MSFTResearch who will be talking about human-AI team-work collaboration!🤾🤖
Sign up here https://t.co/0tCDV9iclc
📍@imperialcollege Huxley 308
@ICComputing #AI

[MineRL Project @minerl_official]

There’s still time to apply! ✍️

[SingularityNET @SingularityNET]

In this recently published article we explore why an AGI-ish Minecraft agent is valuable, what problems it uniquely solves, and why it is valuable for solving complex real-world problems. 👇

🔗 https://t.co/4C7tG2UImR

#SNETCommunity #Minecraft #AI 🧵1/2 ⬇️

[Julian Togelius @togelius]

Sam Devlin @smdvln taking about designing games for AI to play at #gameaischool2022

[AI and Games School 2026 @GameAISchool]

Designing Games for AI to Play at #GameAISchool2022 with @smdvln from @MSFTResearch

[Generative Design in Minecraft Competition (GDMC) @GenDesignMC]

Congratulations to Mike's Angels, the winners of the 2022 AI Settlement Generation Challenge in @Minecraft.

This is a thread about their entry and the GDCM competition.

Feel free to share. #pcg #Minecraft #GameAI
@UniLeidenNews @mike_preuss

[MineRL Project @minerl_official]

Missing the MineRL Diamond 💎 competition? Don’t worry: it is now the intro track for the BASALT competition… with a twist. You can develop reinforcement and imitation learning agents to get a diamond *shovel*, then move to the BASALT tasks! Start here: https://t.co/3GqaWOtKsO