Machine Aesthetics: Transcoding Space Time

The use of generative AI models increasingly involves the reliance on a few black box pretrained and centralized models where design intent is conveyed through language prompts. However, language is an imperfect medium for conveying design and artistic intent especially in the visual arts and architecture. 
Underpinning modern AI is a multitude of techniques borrowed from data science, digital signal processing, information theory, computer graphics …. The machinery of Machine Learning and information theory provides a unified framework for representing, analyzing and synthesizing data, regardless of whether these data map to images, sound, 3d models or something else.

In this course we will explore the creative potential of Machine Learning and Signal Analysis techniques when we treat the environment and the things we create as signals. This year we will focus on the transcoding between temporal and spatial forms, with emphasis on the relationship between audio and three-dimensional geometry.

We will visit certain audio and video analysis techniques including the Fourier transform, various types of spectrogram and ML tools, that will help us extract structure from temporal artifacts. Students will have an opportunity to familiarize with certain concepts behind signal analysis that offer us alternative ways to think about and connect the structure of temporal and spatial objects.

In addition, new will look at several techniques for synthesizing three dimensional objects that are suitable for these types of workflows.

Students will work in small groups to design and produce a three-dimensional object/space/interactive installation or experience, virtual or that represents a temporal object (a piece of music, video, performance, time series data…). The emphasis is on how the structure of one object (geometry, curvature, rhythm …) maps onto another.

The class is structured as a small studio with a series of workshops introducing the tools necessary. No coding knowledge will be required but coding material and technical discussion will be offered to students that are interested and have prior knowledge.

When possible, all material in class / workshops will be given in three different versions, Unity3d (with emphasis on interactivity / real time), Grasshopper (with emphasis on geometric operations), Scripting (for advanced students that want more customization). Students are free to choose the technological framework that matches their project and technical knowledge level.