Google DeepMind unveils Genie 4: a world model with audio and real-time steering

On Tuesday, Google DeepMind released Genie 4, its newest world model, as a research preview. According to the company, it produces the most photorealistic and interactive worlds of any Genie model so far. Users can type prompts mid-simulation to change the scene. The model also generates real-time audio to match the world, including music, ambient noise, and interactive dialogue.

A crowded and consolidating field

Genie 4 arrives a couple of months after AI video startup Runway released its second world model, GWM Worlds 2. That model is also playable, responds dynamically to user input, and generates audio. Meanwhile, the industry is starting to consolidate. Fei-Fei Li's World Labs is being acquired by AMD, and Amazon is in talks to buy Decart. Rivian has acquired Third Dimension to bring simulation for self-driving testing in-house.

What has changed since Genie 3

Genie 3 launched as a research preview a little over a year ago. Beyond greater realism and audio, Genie 4 supports freeform edits in real time: users can change the weather or camera angle, or add new people and objects. It can keep a scene consistent for up to two minutes, twice as long as its predecessor. It also offers better physics and more human-like characters.

Genie 4 is also more tightly integrated with Gemini. The language model acts as a director, planning steps and checking frames while Genie renders the visuals. In a research experiment tied to Gemini Robotics, a Gemini 4 Argon model broke tasks into steps that Genie 4 simulated in real time. This approach would let robot scenarios be tested in simulation before they play out in the real world.

"What we'll increasingly see in the future are these hybrid world models, where you have a combination of Gemini writing code and text, and Genie ingesting that in real time and simulating reality at different levels of abstraction," Google research director Tejas Kulkarni said during a briefing.

From games to storyboards

Like other world models, Genie 4 is pitched at gaming, interactive learning, and robotics. One speculative learning scenario: stepping into a painting of a historical scene and talking with its subjects about the event it depicts.

Trusted testers, the people who get first access to Google's cutting-edge products, are pointing to another use: prototyping films. Google DeepMind engineer Diego Rivas said the real-time aspect matters most where staying in a creative flow is valuable, such as ideation and storyboarding.

Google already offers Veo, a model that generates high-definition video with synchronized audio and is gaining popularity among filmmakers. Rivas said users could eventually turn such prototypes into finished creative assets with Gemini Omni, a multimodal model. Those capabilities, however, are still some way off.

Limitations and availability

Rivas stressed that Genie 4 is still at an early stage and that the final product experience remains unclear. The team also acknowledged gaps. Details and objects drift the longer a simulation runs. The team has also not yet found a way to guarantee accurate audio or legible in-world text.

Genie 4 is available now to trusted testers. In the coming weeks, it will roll out to Google AI Ultra subscribers in Project Genie, Google's tool for generating interactive worlds from text or image prompts. Early access partners of the Gemini API and Google AI Studio will also get the model.

Source: TechCrunch