Beat-Aligned Spectrogram-to-Sequence Generation of Rhythm-Game Charts

Jayeon Yi, Sungho Lee, Kyogu Lee

jayeonyi@umich.edu

Github | Extended Abstract

.

In the heart of "rhythm games" - games where players must perform actions in sync with a piece of music - are "charts", the directives to be given to players. We newly formulate chart generation as a sequence generation task and train a Transformer using a large dataset. We also introduce tempo-informed preprocessing and training procedures, some of which are suggested to be integral for a successful training. Our model is found to outperform the baselines on a large dataset, and is also found to benefit from pretraining and finetuning.

.

Inference Examples

Yu-Peng Chen @HOYO-MiX - Duel in the Mist (target difficulty: 3.61, result: 4.29)

katagiri - Angel's Salad (target difficulty: 2.19, result: 2.29)

Aiobahn - I'm here (feat. rionos) (target difficulty: 3.68, result 3.47)

PSYQUI - Stepper (three charts placed side-by-side)

(target difficulty: 1.86, 2.54, 3.30; result 1.72, 2.42, 3.38)