Enactive Design: Creative Applications through Concurrent Human-Machine Interaction

Enactive Design is an advanced research seminar on human-computer interaction. We will explore the role of real-time, bidirectional communication between human and digital agents in a design context, and leverage the potentials of this interactive relationship to establish new creative domains.

Digital interfaces provide computational frameworks for creative exploration in disciplines such as architecture, design and art. However, in many instances such as traditional computer-aided design software, integrated development environments, numerically-controlled fabrication machines, or certain kinds of robots and artificial intelligence systems, the machine is subservient to the orders of its human counterpart. While this model might be a convenient human-machine relationship for production-oriented scenarios, in the case of design environments, a higher degree of machine agency could be desired, as it may generate new models of ideation, creative exploration and design through human-computer collaboration.

Our investigation is inspired by the concept of Enactivism, a philosophy which argues that cognition arises from the interactions of an agent with its context. Rather than in an abstract or intangible way, knowledge and learning in agents arise from purposeful, situated and embodied interaction with their context. Translated to design environments, what would it mean to create with tools that have a certain degree of agency of their own? How would that inform and expand our creativity? What kind of opportunities may arise from designing as a conversation rather than an imposition? Can design be conceived as the human curation of the suggestions of an artificial intelligence? How can the power and precision of computation be amplified by the decision-making capacity of humans-on-the-loop? Are these new forms of collaborative art?

We will address these questions, and many others, through the design of concurrent human-machine interactive platforms, with a particular focus on the computational aspects of the system. Participants will learn techniques such as applied artificial intelligence, network communication, asynchronous programming and RESTful APIs, and additional topics such as physical sensing, interactive fabrication and robot control will be discussed. The course will rely heavily on the use of state of the art, off-the-shelf generative AI models, including large language models, diffusion models, etc. (i.e. no foundational machine learning topics will be covered). Students are expected to use these tools to build prototype applications as demonstrators of their design questions.

The course will be conducted through a series of lectures, readings, discussions and hands-on workshops. Exercises will experiment with real-time communication between human and digital agents, leading to a semester-long personal/group final project and a conference-grade research paper. This is an advanced research seminar: a high degree of maturity, independence and initiative are expected.

This course is the last installment of a three-part course series on Computational Design preceded by SCI- 6338: Introduction to Computational Design (Fall), and SCI-6483: Procedural Fields, Functional Design of Discrete Hyperdimensional Spaces (Spring), taught by the same instructor.

The first day of classes, Tuesday, September 3rd, is held as a MONDAY schedule at the GSD. As this course meets on Monday, the first meeting of this course will be on Tuesday, September 3rd. It will meet regularly thereafter.