Machine Learning for Creativity and Design

NeurIPS 2020 Workshop

December 12ᵗʰ, remote

Image credit: Sofia Crespo's "On the seabed gazing up" (2020).


Generative machine learning and machine creativity have continued to grow and attract a wider audience to machine learning. Generative models enable new types of media creation across images, music, and text - including recent advances such as StyleGAN2, Jukebox and GPT-3. This one-day workshop broadly explores issues in the applications of machine learning to creativity and design. We will look at algorithms for generation and creation of new media, engaging researchers building the next generation of generative models (GANs, RL, etc). We investigate the social and cultural impact of these new models, engaging researchers from HCI/UX communities and those using machine learning to develop new creative tools. In addition to covering the technical advances, we also address the ethical concerns ranging from the use of biased datasets to replicating artistic work. Finally, we’ll hear from some of the artists and musicians who are adopting machine learning including deep learning and reinforcement learning as part of their own artistic process. We aim to balance the technical issues and challenges of applying the latest generative models to creativity and design with philosophical and cultural issues that surround this area of research.

The goal of this workshop is to bring together researchers interested in advancing art and music generation to present new work, foster collaborations and build networks with the aim of solving the most pressing problems in the field.

Invited Speakers

Sofia Crespo, Artist

Stephanie Dinkins, Artist and Professor at Stony Brook University

Scott Eaton, Artist

Jesse Engel, Magenta, Google Brain

Angela Fan, Facebook

Aaron Hertzmann, Adobe Research

Janelle Shane, AI Weirdness


All times are EST (UTC -5).

You will find links to all the sessions on our NeurIPS workshop website (registered participants only).

Time Event
8:15 AM Introduction and Art Gallery Overiew Luba Elliott
8:30 AM Poster Session 1
Posters 1-32 apart from 29
9:30 AM Farewell to Fart: Working with generated text in the age of huge neural nets
Janelle Shane (AI Weirdness)
9:56 AM Artist+AI: Figures&Form
Scott Eaton
10:21 AM Artificial biodiversity
Sofia Crespo
10:40 AM Art Showcase 1
11:00 AM Panel Discussion 1
Janelle Shane, Sofia Crespo, Scott Eaton and Angela Fan chaired by Adam Roberts and Luba Elliott
11:30 AM Social 1
12:15 PM magenta: Empowering Creative Agency with Machine Learning
Jesse Engel
12:45 PM Art Showcase 2
1:20 PM Computation and the Human Visual Perception of Art
Aaron Hertzmann (Adobe Research)
1:47 PM LIGHT: Language in Games with Humans and Text
Angela Fan (Facebook)
2:09 PM Not the Only One
Stephanie Dinkins (Stony Brook University)
2:35 PM Art Showcase 1
2:50 PM Audio-reactive Latent Interpolations with StyleGAN
Hans Brouwer
3:00 PM Towards realistic MIDI instrument synthesizers
Rodrigo Castellon
3:10 PM Creative Sketch Generation
Songwei Ge
3:20 PM Agence: an interactive film exploring multi-agent systems and human agency
Dante Camarena
3:30 PM Art Showcase 2
3:45 PM Poster Session 2
Posters 1-29
4:45 PM Panel Discussion 2
Jesse Engel, Aaron Hertzmann and Stephanie Dinkins chaired by Holly Grimm and Tom White
5:15 PM Social 2
6:00 PM End

Accepted papers

  1. Audio-reactive Latent Interpolations with StyleGAN
    • Hans Brouwer
  2. Creative Sketch Generation
    • Songwei Ge, Vedanuj Goswami, C. Lawrence Zitnick, Devi Parikh
  3. Agence: an interactive film exploring multi-agent systems and human agency
    • Dante Camarena, Pietro Gagliano, Alexander Bakogeorge, Nicholas Counter, Anuj Patel, Casey Ryan Blustein, Erin Ray, David Oppenheim, Laura Mingail, Kory W. Matthewson, Pablo S. Castro
  4. GANterpretations
    • Pablo Samuel Castro
  5. Neural Style Transfer for Casual Creation
    • Simon Colton
  6. Network Bending Neural Vocoders
    • Louis McCallum, Matthew Yee-King
  7. Colorization Transformer
    • Manoj Kumar, Dirk Weissenborn, Nal Kalchbrenner
  8. White-box Audio VST Effect Programming
    • Christopher Mitcheltree, Hideki Koike
  9. TräumerAI: Dreaming Music with StyleGAN
    • Dasaem Jeong, Seungheon Doh, Taegyun Kwon
  10. A Note on Data Biases in Generative Models
    • Patrick Esser, Robin Rombach, Björn Ommer
  11. Randomized Overdrive Neural Networks
    • Christian J. Steinmetz, Joshua D. Reiss
  12. Latent Space Oddity: Exploring Latent Spaces to Design Guitar Timbres
    • Jason Taylor
  13. Resolution Dependant GAN Interpolation for Controllable Image Synthesis Between Domains
    • Justin N. M. Pinkney, Doron Adler
  14. Image Generation With Neural Cellular Automatas
    • Mingxiang Chen, Zhecheng Wang
  15. Behaviour Aesthetics of Reinforcement Learning in a Robotic Art Installation
    • Sofian Audry, Rosalie Dumont Gagné, Hugo Scurto
  16. LiveGAN
    • Cameron Smith
  17. Mask-Guided Discovery of Semantic Manifolds in Generative Models
    • Mengyu Yang, David Rokeby, Xavier Snelgrove
  18. A Speech-Based Music Composition Tool with Transformer
    • Jason d’Eon, Sri Harsha Dumpala, Daniel Oore, Sageev Oore
  19. Transformer-GAN: Symbolic music generation using a learned loss
    • Aashiq Muhamed, Liang Li, Xingjian Shi, Suri Yaddanapudi, Wayne Chi, Dylan Jackson, Rahul Suresh, Zachary Lipton, Alexander J. Smola
  20. Latent Compass
    • Sarah Schwettmann, Hendrik Strobelt, Mauro Martino
  21. Choreo-Graph: Learning Latent Graph Representations of the Dancing Body
    • Mariel Pettee, Santiago Miret, Somdeb Majumdar, Marcel Nassar
  22. Painting from Music using Neural Visual StyleTransfer
    • Ingrid Odlen, Prateek Verma, Constantin Basica, Pamela Davis Kivelson
  23. Horses With Blue Jeans - Creating New Worlds by Rewriting a GAN
    • David Bau, Steven Liu, Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba
  24. Spatial Assembly:Generative Architecture With Reinforcement Learning, Self Play and Tree Search
    • Panagiotis Tigas, Tyson Hosmer
  25. Text to Dialog: Using Semantic Similarity to Extend Narrative Immersion in Virtual Worlds
    • Sherol Chen, Ben Pietrzak, Kory W. Matthewson, Dan Cary, Anna Kipnis, Erin Hoffman-John
  26. Towards realistic MIDI instrument synthesizers
    • Rodrigo Castellon, Chris Donahue, Percy Liang
  27. Musical Diary - AI Application for Music Making and Journaling
    • Suyash Joshi, Devin Lane, Stephen Haney
  28. Copyspace: Where to Write on Images
    • Jessica M. Lundin, Michael Sollami, Brian Lonsdorf, Alan Ross, Owen Schoppe, David Woodward, Sönke Rohde
  29. Weird AI Yankovic: Generating Parody Lyrics
    • Mark O. Riedl
  30. Generating Novel Glyph without Human Data by Learning to Communicate
    • Seung-won Park
  31. Diptychs of human and machine perceptions
    • Vivien Cabannes, Thomas Kerdreux, Louis Thiry
  32. A Framework and Dataset for Abstract Art Generation via CalligraphyGAN
    • Jinggang Zhuo, Ling Fan, Harry Jiannan Wang


If you have any questions, please contact us at

Workshop website:

Previous years:

The art submissions from previous years can be viewed here

How to attend

You should purchase a ticket on This will allow you to access our website on NeurIPS with links to the livestream, poster session and socials.


Luba Elliott, AI Curator

Sander Dieleman, DeepMind

Adam Roberts, Magenta, Google Brain

Tom White, Victoria University of Wellington

Holly Grimm, Artist and freelance creative technologist

Mattie Tesfaldet, McGill University / MILA

Samaneh Azadi, UC Berkeley

Daphne Ippolito, University of Pennsylvania / Google Brain