The system uses a streaming frame-level autoregressive approach with sliding window attention and learnable attention sink embeddings to manage long contexts while maintaining performance on-device.

Demos showcased MIDI steering where sustained notes or chords guide an ensemble generation, plus a gesture-based MIDI input demo illustrating local MRT2 control.

Control modalities include text, audio, MIDI, and note/drums signals, with multi-signal conditioning updated at 25 Hz per frame for responsive reactions across inputs.

Technical design features a SpectroStream codec, frame-aligned conditioning, causal sliding window attention, and a C++ inference engine powered by Apple MLX, with a 2.4B base model requiring newer Macs and a 230M variant running on any Apple Silicon MacBook.

Compared with MRT1, MRT2 delivers roughly 15x lower latency, enabling frame-by-frame generation at about 40 ms and integrates directly into DAWs and other music software.

Google released Magenta RealTime 2 (MRT2), an open-weights on-device live music synthesis model that runs locally on Apple Silicon Macs without internet or cloud access, with two model sizes (mrt2_base at 2.4B params and mrt2_small at 230M), the latter capable of real-time performance on M1 and newer Macs.

Four accompanying applications ship with MRT2: Jam for standalone use with MIDI control, Collider for two-dimensional prompt mixing, MRT2 Plugin as an Audio Unit for DAWs like Logic and Ableton, and Creative Coding Extensions for Max/MSP, PureData, and SuperCollider.

MRT2 supports MIDI steering, text-to-synth prompts, audio cloning from short snippets, prompt mixing, and gesture control via LFOs, MIDI controllers, or camera input for expressive live performance.

Inference-time masking and CFG scales provide flexible creative control, supporting partially unconditional generation and robustness to missing or noisy inputs.

Auto-Strum modes and a compact 4-token vocabulary govern onsets, offs, and continuations, with drums controlled via training-time cues and a dedicated inference-time drum-off switch.

MRT2 is documented with an open-weights model on HuggingFace, an open-source Python library (magenta-rt), and a macOS Apple Silicon bundle, all built around a native C++ engine for efficient streaming.

The release marks Magenta’s push into production-ready, on-device music AI collaboration, announced on June 4, 2026, alongside free instrument apps and DAW plugins.