Artificial intelligence (Ai) is an especially disruptive technology, impacting a growing number of domains in ways both beneficial and detrimental. It is even showing surprising impacts in the Arts, provoking questions fundamental to philosophy, law, and engineering, not to mention practices in the Arts themselves. MUSAiC is an interdisciplinary research venture confronting questions and challenges at the frontier of the AI disruption of music.
HIGHLIGHTS
2024
Paper accepted for presentation: Elin Kanhov, “Entanglements with Deepfake: AI Voice Models and their Diffractive Potential”, Intersectional Materialisms: Diversity in Creative Industries, Methods and Practices, Aug 26-28 2024, Maynooth, Ireland..
The First International Conference in AI Music Studies: Prospects, Challenges and Methodologies of Studying AI Music in the Humanities and Social Sciences is set to occur Dec. 10-12 in Stockholm.
Apr 5 2024, Sturm will speak at the kick-off event of the newly funded project, “Spirits in Complexity – Making kin with experimental music systems“.
Paper accepted for presentation: Anna-Kaisa Kaila (co-authors Elin Kanhov & Bob L. T. Sturm), “Ethnographic Considerations and Critical Reflections on the Impacts of AI on Traditional Irish Music”, British Forum for Ethnomusicology & International Council for Traditional Music Ireland Joint-Annual Conference, Cork, Ireland University College Cork, 4-7 April 2024.
On March 2, 2024 Sturm gave a Rayson Huang Lecture at The University of Hong Kong on March 2 as part of the international symposium “AI, Music, and Creativity: At a Crossroads”
Kanhov and Sturm, “’Why make music with AI?’ Potentials in ethnographic studies of AI music service users”, will be presented at the International Council for Traditions of Music and Dance Nordic-Baltic Network Conference in Helsinki, October 8-10th 2024.
Sturm will give an invited seminar on February 12 at the Georgia Tech Center for Music Technology. The talk is titled, “Generative AI helps one express things for which they may not have expressions (yet) — but perhaps at the cost of one’s time and voice”
New paper released on SocArXiv: B. L. T. Sturm, K. Déguernel, R. S. Huang, A. Holzapfel, O. Bown, N. Collins, J. Sterne, L. Cros Vila, L. Casini, D. Alberto Cabrera, E. A. Drott, and O. Ben-Tal, “MusAIcology: AI Music and the Need for a New Kind of Music Studies”, SocArXiv, 2024.
Two papers accepted to EvoMUSART 2024: Casini, Jonason and Sturm, “Investigating the viability of Masked Language Modeling for symbolic music generation in abc-notation” (nominated for best paper); and Dalmazzo, Déguernel and Sturm, “The Chordinator: Modeling Music Harmony By Implementing Transformer Networks and Token Strategies”.
.
2023
MUSAiC members Kaila and Sturm appear in article on AI and music in Opus Magasin (no. 121, Dec 2023).
MUSAiC appears at KTH TEDx, presented by Luca Casini, Laura Cros Vila and David Dalmazzo.
Sturm gives an invited seminar at the Royal Northern College of Music, UK: “Visions of a New Discipline called ‘AI Music Studies'”.
Four “late breaking demos” presented at ISMIR 2023:
- Dalmazzo, Déguernel and Sturm, “The Chordinator: Chord Progression Modeling and Generation using Transformers”.
- Jonason, Casini and Sturm, “Retrieval Augmented Generation of Symbolic Music with LLMs”.
- Casini, Jonason and Sturm, “Generating folk-like music in abc-notation with Masked Language Models”.
- Cros Vila and Sturm, “Cosine Contours: A Case Study with Melodies from Irish Traditional Dance Music”.
Paper posted to arXiv: Jonason, et al. “DDSP-based Neural Waveform Synthesis of Polyphonic Guitar Performance from String-wise MIDI Input“.
Three of Sturm’s “Music of the Crowdworkers” programmed for the SEMS Sounding Board 2023 exhibition.
Amerotti, Benford, Sturm, Vear, “A Live Performance Rule System Informed by Irish Traditional Dance Music“, in Proc. Computer Music Multidisciplinary Research 2023.
Sturm and Flexer, “A Review of Validity and its Relationship to Music Information Research“, in Proc. ISMIR 2023.
Two papers published at AI Music Creativity 2023: Sturm, “The Ai Music Generation Challenge 2022: Summary and Results“; and Cros Vila and Sturm, “Statistical evaluation of abc-formatted music at the levels of items and corpora“. Also presented was Alt-AIMC paper Kaila, Holzapfel and Sturm, “Are we solving the wrong problems – and doing harm in the process?“. The Fourth Annual Machine Folk Music School was also presented.
“Simple Love (ChatGPT + Sturm&Sturm)” receives Honorary Mention at Human + Machine Song Contest.
Déguernel and Sturm, “Bias in Favour or Against Computational Creativity: A Survey and Reflection on the Importance of Socio-cultural Context in its Evaluation” published at ICCC 2023.
Sturm appears on Swedish National Television to discuss AI and music.
Falk, Sturm and Ahlbäck, “Automatic Legato Transcription Based on Onset Detection”, in Proc. Sound and Music Computing 2023.
Chapter appears in M. Clancy’s new book, Artificial Intelligence and Music Ecosystem, Routledge, 2023. The chapter is, Huang, Holzapfel and Sturm, “Global Ethics – From Philosophy to Practice A Culturally Informed Ethics of Music AI in Asia”
Article publication: Huang, Holzapfel, Sturm and Kaila, “Beyond Diverse Datasets: Responsible MIR, Interdisciplinarity, and the Fractured Worlds of Music“, Trans. of the International Society for Music Information Retrieval, Vol. 6, No. 1, pp. 43–59, 2023. This is an invited extension of our 2022 ISMIR paper.
Sturm gives a keynote lecture at the course, “Musical Creativity with AI“, April 26-27, in Barcelona.
Results of The Ai Music Generation Challenge 2022 are announced. There were several submissions in three sub-challenges: generating tunes, judging tunes, and titling tunes.
Publication of Sturm and Flexer, “Validity in Music Information Research Experiments”, arXiv:2301.01578, 2023. This article is a “tutorial” about validity of conclusions drawn from experiments, and how it appears in music informatics research. A version appears at ISMIR 2023.
Special issue of Computer Music Journal co-edited by MUSAiC members: Musical Interactivity in Human–AI and AI–AI Partnerships
2022
Presentation of Jonason and Sturm, “Audio Latent Space Cartography“, Late Breaking Demo at ISMIR 2022. Nicolas has created a very interesting way of visualizing latent spaces of audio synthesis models. Plenty of creative applications to come!
The MUSAiC Festival 2022 Nov. 22-24 was a big success, attracting many people to a variety of concerts and talks, as well the concluding workshop teaching the application of machine learning to audio-visual synthesis.
Déguernel, Sturm and Aguilar, “Investigating the relationship between liking and belief in AI authorship in the context of Irish traditional music“, published at the CREAI 2022 Workshop on Artificial Intelligence and Creativity,
Nov 28–Dec 02 2022 in Udine, Italy.
Sturm and Holzapfel organized a round table on Music and AI at the WASP-HS community meeting, Oct 13 2022 and contributed to the final report.
Sturm, “Generative AI helps one express things for which they may not have expressions (yet)”, Proc. Generative AI and HCI, CHI 2022 Workshop.
.
Paper, music piece, and workshop all accepted to 2022 AI Music Creativity Conference. Sturm “The Ai Music Generation Challenge 2021: Summary and Results“. Sturm, “The shadow still lay where he had been standing (a growing opera for YouTube)”. The 3rd Machine Folk Music Workshop.
Holzapfel and Sturm, “Interwoven Listening with the Music Listening Machine”, presented at the 2022 joint meeting of the American Musicological Society (AMS), Society for Ethnomusicology (SEM), and Society for Music Theory (SMT).
Song by MUSAiC team Maskinspelmanslaget made it to the final round of the 2022 AI Song Contest. It placed 14 out of 46.
Jonason & Sturm, “Neural Music Instrument Cloning from Few Samples” published at 2022 Digital Audio Effects conference. Video of talk here. See the accompanying webpage of demos.
Results of The Ai Music Generation Challenge 2021 are announced! Former visiting researcher Luca Casini has won with the transformer system he developed at KTH. (See next news item.)
Casini & Sturm “Tradformer: A Transformer Model of Traditional Music” presented at the special track AI, the Arts and Creativity at IJCAI 2022.
MUSAiC PhD Laura Cros Vila will give a talk on “Musical Instrument Recognition using the Scattering Transform” at the Kymatio’22: Deep learning meets wavelet theory workshop, held on May 19-20 at LS2N, École Centrale de Nantes, France.
Sturm presents a talk at the Dagstuhl workshop: Deep Learning and Knowledge Integration for Music Audio Analysis, Feb 20-25. Full proceedings here. The talk was about responsible engineering in music informatics.
Over a year in the making, the “AI and Musical Creativity” special issue of the Transactions of ISMIR is out! Nine fantastic articles contribute technical and philosophical discussions to a fast-developing field. Sturm (MUSAiC PI) is lead guest editor.
The Ai Music Generation Challenge 2022 announced. There are three sub-challenges: generation of plausible Irish reels; artificial judge; and tune titling.
Launch of AHRC-funded research network Datasounds, datasets and datasense: Unboxing the hidden layers between musical data, knowledge and creativity. Sturm (MUSAiC PI) is a part of this network, and will host a meeting of project partners in November 2022. More information. Final summary here.
2021
Journal publication: Ben-Tal, Harris and Sturm, “How Music AI Is Useful: Engagements with Composers, Performers and Audiences“, Leonardo (2021) 54 (5): 510–516.
Journal publication: Sturm, B. & Maruri-Aguilar, H. (2021) “The Ai Music Generation Challenge 2020: Double Jigs in the Style of O’Neill’s 1001””, Journal of Creative Music Systems. 5(1). doi: https://doi.org/10.5920/jcms.950
Sturm’s research featured in E&T article, Can AI be music to our ears? Nov. 11 2021.
Sturm’s composition, “Three Tunes from the AI Frontiers”, performed at ISMIR 2021 and the 2021 NeurIPS workshop Machine Learning for Creativity and Design.
Sturm invited to present his music Ai research at the 2021 Summit on Machine Intelligence co-organized by Central Conservatory of Music in Beijing, China (October 22-24).
Huang, Sturm and Holzapfel, “De-centering the West: East Asian Philosophies and the Ethics of Applying Artificial Intelligence to Music” published at ISMIR 2021, and awarded Best Special Call Paper.
Sturm delivers invited talks to Traditions in Transition “Transcultural Technologies for Creative Expression“, and STS Italia 2021 “Effect of dataveillance on artistic and cultural production: Exploiting user data to shape user preferences and create new content”. Video for STS Italia.
Sturm invited to deliver keynote to Audio Mostly 2021. The talk highlights the MUSAiC project, its outcomes so far, and what the future holds for traditional music.
The Ai Music Generation Challenge 2021 has begun! Watch this video to get an idea of the kind of music this challenge entails.
Two papers presented at the AI Music Creativity 2021 conference: Huang and Sturm, “Reframing ‘Aura’: Authenticity in the Application of Ai to Irish Traditional Music”; and Sturm, “An Artificial Critic of Irish Double Jigs”. See also: “58,105 Irish-style Double Jigs“.
A MUSAiC team entered a song – “Evigt Förlorad (Forever Lost)” – into the 2021 AI Song Contest. Out of 38 entries, our song placed third with the jury, and fifth overall. It was also mentioned in a NY Time article, Robots Can Make Music, but Can They Sing?
2020
The Ai Music Generation Challenge 2020 was a success! Tunes generated by seven Ai systems were evaluated by four Irish traditional music experts, and two winners were picked and performed.
Poetic research blog launched: Tunes from the Ai Frontiers. This is a personal exploration of machine-generated folk music, and how it fits with my practice of Irish traditional music.
The 2020 Joint Conference on AI Music Creativity – the kickoff event of MUSAiC – brought together two overlapping research forums for a week-long virtual conference involving over 200 participants around the world. Here are technical and logistical details about how I ran the conference online. The keynotes and selected papers from this conference were invited for expansion and published in a 2022 special issue of the Journal of Creative Music Systems.
“Music from EDSAC” (circa 1960) by D. G. Champernowne, rediscovered and performed. This is a string quartet composed from material generated by a computer in the UK.
MUSAiC is a project that has received funding from the European Research Council under the European Union’s Horizon 2020 research and innovation programme (Grant agreement No. 864189).
MUSAiC 