A MIDI loop goes in. A responsive musical partner grows around it.
Neuraloop is a work in progress: an instrument concept for turning a played MIDI loop into complementary musical parts, while keeping the performer in control.
Work in progress
The interaction and split hardware architecture are being designed. Model integration and the playable hardware prototype are still ahead; real-time co-performance is the goal, not a completed capability.
The problem
Most generative AI systems are designed around text or audio. Real-time MIDI co-performance requires a different architecture: controllable symbolic generation, low latency, role-aware orchestration, and tight interaction between performer, instrument, and model.
Design & implementation
Neuraloop explores how symbolic music generation can behave like an instrument rather than a content generator. The project is centered on live MIDI interaction, controllable transformation, and role-aware musical generation.
The system is designed around a split architecture. An embedded ESP32-S3 orchestrator manages MIDI routing, controls, and system state in real time, while a separate compute node handles symbolic note generation. This keeps timing-critical behavior close to the hardware and pushes heavier inference away from the performance layer.
On the model side, Neuraloop is moving toward a role-aware generation approach instead of a generic LLM setup. The training direction combines groove-oriented datasets for rhythmic parts, broader melodic datasets for harmonic material, and multitrack datasets to learn arrangement structure and inter-part relationships.
Design goals
- Real-time MIDI co-composition centered on performer control
- Split architecture between embedded orchestration and remote inference
- Role-aware multitrack generation for drums, bass, synth, and pad
- Focus on low latency, interpretability, and live playability
- Training strategy built from complementary symbolic music datasets
Why I built it
Neuraloop explores the gap between model capability and real-world musical interaction. The project is less about generating music in the abstract and more about making generation controllable, interpretable, and playable inside a live creative workflow.
