Revolutionary open-source interactive world model for real-time video generation and controllable game world simulation
Matrix-Game 2.0, developed by Skywork AI, is an open-source, real-time, streaming interactive world model for generating minute-length videos at 25 FPS using few-step auto-regressive diffusion. It supports keyboard and mouse action injection, enabling frame-level control over generated video sequences.
Another perspective presents Matrix-Game 2.0 as a lightweight, browser-ready AI world model aimed at rapid experimentation in reinforcement learning, multi-agent systems, and interactive AI environments. It delivers interactive worlds at 25 FPS directly in the browser.
Matrix-Game refers to an advanced interactive world foundation model designed for controllable game world generation, specifically demonstrated with Minecraft environments. It uses an image-to-world generation paradigm, starting from a single reference image and generating a continuous interactive game world with temporal coherence over long video durations.
Generate interactive videos at 25 FPS with real-time streaming synthesis and precise frame-level action injection for responsive gaming experiences.
Support for keyboard and mouse action injection with fine-grained control over character actions and camera movements in virtual worlds.
Fully open-source with model weights and evaluation benchmarks available for research and development in interactive game world generation.
Outperforms benchmarks like Oasis in image quality, temporal consistency, input accuracy, and object coherence across multiple metrics.
Trained on over 2,700 hours of unlabeled gameplay and 1,000+ hours of finely annotated clips with 17 billion parameters for detailed control.
Maintains high visual quality, smooth temporal transitions, and faithfully models physical rules of game worlds with minimal artifacts.
Visual fidelity and clarity
Input response precision
Cursor control precision
Ensure you have a compatible GPU (RTX 4090 recommended) and sufficient VRAM. Matrix-Game runs smoothly on high-end hardware with optimized performance for real-time generation.
Clone the repository from GitHub and follow the installation guide. You can also access pre-trained models directly through HuggingFace for quick testing.
Use keyboard and mouse inputs to control character actions and camera movements. Matrix-Game supports real-time action injection with precise frame-level control for immersive gameplay experiences.
Explore applications in reinforcement learning, multi-agent systems, robotics research, and game development. The model extends beyond Minecraft to support various interactive environments.