Developed an experimental pipeline that transforms AI-generated music into unique 2D platformer levels in Unity. Leveraging GPT-2, TensorFlow, CUDA, and PyTorch, we converted text prompts into AI-generated songs, then into MIDI files, which Unity used to generate custom levels. Collaborating with a team of four, we divided tasks based on our strengths, addressing challenges early to ensure efficient and timely project completion. 

We used Omnizart, a Python library for automatic music transcription, to extract musical notes as piano notes for our project. For transforming MIDI files into data usable for level generation, we integrated the pretty-midi library. This allowed us to extract each note’s pitch, start time, finish time, and BPM, saving this information into a text file for Unity to generate unique levels. By using pretty-midi’s estimate_tempo() method, we synchronized the game tempo with the music’s rhythm, enabling dynamic, rhythm-responsive gameplay.