Project 1:

Do you have a technology-dependent creative project in your head that you want to build? Do you look at a particular kind of work by other interactive designers/artists and think "I wish I knew how to make that!" In this project, we're dropping you in at the deep end. We want you to propose a creative technology project, and we're going to try to help you build it. Or, more accurately, we're going to help you identify and kludge together a small critical piece of it. The question of scope – figuring out what that "small critical piece" is – is fundamental here, and will underlie many of our discussions as your projects progress.

To "kludge" is to use an ill-assorted collection of parts to make something. It typically has negative connotations, suggesting something that has been quickly and shoddily put together. But kludging can be a valuable part of an iterative, exploratory, creative design process as we move back and forth between thinking and making, concept and prototype. This is especially true when working with (and designing) interactive technologies, where we are concerned with dynamic experiences and behaviors.

Tips for a productive project

  • Don't spend time trying to think of a completely novel idea before you start. In this project it's totally OK – we encourage it, even – to be motivated by the desire to replicate interactive and/or technological characteristics you have identified in others' work.
  • The project might turn out not the way you envisioned it.
  • Although we hope to learn some practical things, we are equally interested in the process of thinking through making.
  • Physical computing might be too much of a challenge in this short intro project, unless you already have components or can get them asap.


  • Upload diagrams and images of your building work / process.
  • Show the piece that you built in context of the system diagram
  • Discuss how the scope of your prototype changed as you worked through your ideas (include illustrations).

>>> Weekly Schedule

Week 1: Sept. 13 - 17


Warm-up assignment:

  • - Choose a technological artifact – a consumer product, an interactive artwork, a piece of software, an installation, anything – which exhibits some characteristics of a 'creative tech' interest of yours. (Choose more than one if you are torn between multiple interests!) Find an image of your artifact and drop it in the Slack channel.
  • - Consider: how was this artifact created? What are the parts and the components? How can they be categorized, grouped? How do they connect to each other in a system? What are some of the tools and resources used to create those parts and bring them together? Be prepared to share your thoughts when we next meet.

Homework briefing and discussion, including diagramming examples [link].

GitHub Workshop Zoom Recording


Make a system diagram of your chosen artifact. (You can continue with what you already have, or choose something new.) Your diagram should communicate the types of connection and configuration of key components of your artifact. But (!) we want you to go above and beyond this strict engineering definition of a system diagram. As designers we are interestested in tangibility, materiality, experience, aesthetic, emotion, cultural context, social connections... What are other qualities you could foreground and integrate into the diagram?

Consider: If you were to build one specific technological part of this artifact, what would that be? This question is an entirely subjective one, and refers to your own aspirations as a maker. We strongly encourage you not to get too caught up in conceptual interests at this point. Is there an interaction, behavior, or function you would like to emulate, or a technological effect you are yearning to achieve? What tools do you think you would use? What skills would you need?

Prepare to come to the next class with the homework on 2-3 slides which we can add to a shared deck at the start of the meeting.

Week 2: Sept. 20 - 24


Quick 'round the room' update from everyone: share your system diagrams and give us an update of your thinking of how you might build one part of your system (~3 minutes each)

Discussion about defining scope.

Determine thematic tech support groups. Configure remaining class time.


Get started EARLY and start to kludge your project together.

Use the Slack channel to share resources and ask questions to each other and faculty.

Prepare to come to the next class ready to dive in and demo what you've made and quickly and clearly communicate any challenges.

Week 3: Sept. 27 - Oct. 1


First two hours of class: problem-solving lab, working together or in groups.

Last hour: Everyone quickly demos their project for the group.

Zoom Recording


Finish and polish your documentation.

Project 2:
Simulation & Design

Simulations are generated by models that mimic alternate conditions. Before the digital era, simulations were mostly associated with non-design fields, such as engineering, epidemiology, astronomy, meteorology, psychology, and economics. These models were typically used to predict or infer performance or events in possible, or future, or otherwise unknowable states.

When software-enabled simulation eventually emerged, it evolved rapidly in non-research domains such as gaming and visual effects for cinema, with echoes in the art world from the very beginning. From there, crossovers continued into architecture and countless media applications. Today, sophisticated simulations are ubiquitous: in VR, AR, games, movies, GUIs (not to mention the simulations of disasters - like pandemics, fires, and hurricanes - we consume daily through the media.)

AR, VR, gaming, and animation all employ tools that could be classified as simulations of one type or another, and in this workshop will be experimenting with these fundamentals using a game engine to apply behaviors and conditions into a digital environment.

Alongside this technical exposure, however, we will consider - in our projects and conversation - the influence of tools for simulation on design and art. Unlike simulation, representation has long been the realm of the designer: illustrating the world for the purposes of communication or as a form of instrumentalized image-making for developing texts, media, objects, and places to be produced.

Is this significant? Is it merely a distinction without a difference?

How can the designer/artist use simulation critically, productively, originally? How does one shift their attention from questions of form to questions of behavior and performance (or are they the same thing)? When using tools developed for specific industries (namely, gaming and visual effects) does one intervene to default qualities of the expected use? Does the capability of creating digital models capable of self-generating and simulating outcomes alter one's perceptions as a maker in general?


Climatological, social, and technological changes are occurring increasingly at a global, rather than regional, scale. While addressing issues of this complexity has always required the collaboration of multiple forms of scientific expertise, inexpensive super-computation and accessible software allows designers to engage with topics long considered strictly the purview of these scientific researchers and engineers.

For this project, you will create a vivarium: a micro-world where a defined set of properties and behaviors are placed and developed. As a starting point, you will be assigned one of the following verbs, each inspired by a current global crisis, nearing a tipping point: a condition where a controlling system is overwhelmed, sometimes beyond potential return or repair.

  • Flooding (rising sea levels) - Zoey Wang
  • Melting (dissolving glaciers) - Eric Schubert, Fanxuan Zhu
  • Blowing (powerful hurricanes) - Zeyu Wang
  • Burning (stronger wildfires) - Mario Santanilla
  • Migrating (mass movements of people and wildlife) - Yiran Mao
  • Spreading (pandemics, epidemics) - Guowei Lyu
  • Colliding (space junk) - Hongming Li, Annie Wang
  • Drying (drought) - Miaoqiong
  • Accreting (garbage/pollution) - Cha Gao, Qq Yuwen
  • Clouding (air/water pollution) - Noah Curtis
  • Densifying (urban growth) - Shiyi Chen
  • Growing (super blooms) - Alan Amaya, Yue Xi


  • 1. In this world-building exercise, choices about visual representation are as important as simulated behavior. Make sure you have sufficiently developed the visual language of your project to move it far away from Unity’s default look and feel. (Unless, of course, your use of the default is intentional.) Consider: what elements in your world are figurative, what’s literal? What are you referencing through these choices? There is also a strategic, practical reason for doing this; if your project looks default people likely won’t be interested in it.
  • 2. Use screen recordings of your running simulation as 'raw footage'. Create an edited sequence of short video clips/GIFs (as well as images) to document various features, moments, events. How do you tell a story about your world through this clip sequence? What details do you crop in on? How do you use captions in combination with your collection of clips? Consider this documentation requirement a fundamental part of the design brief.

>>> Weekly Schedule

Week 4

Friday, Oct 8th:

Introduction, project briefing

Unity intro workshop with John Brumley: Zoom Recording, In-class file

Saturday, Oct 9th:

Unity workshop with John (continued from Friday): Zoom recording, In-class file

Discussion of project ideas and directions


Build on workshop exercises and make some initial experiments. Re-read brief and review examples. Consider: What are the elements of my simulation? How do I represent them? How are they interacting?

Week 5

Tuesday, Oct 12th:

Individual meetings with John and Ben (sign-up details to follow)

Friday, Oct 15th:

In-progress project reviews and group technical help session

Zoom recording

In-class files:


Develop your projects!

Week 6

Tuesday, Oct 19th:

Individual meetings with John and Ben (sign-up details to follow)

In-class file, keijiro's shadernoise package

Friday, Oct 22nd:

Project reviews and documentation briefing

Zoom recording


Project documentation assignment

Acknowledgement: This project brief and examples slides were developed with Tim Durfee

Week 7: Oct. 25 - 29 — NO CLASS

Project 3:
Synthetic relationships

Communication lies at the core of all media. From speech to the written word, from radio to social networks - the core function of media has been to enable and facilitate communication. In this project we're going to take on this radical approach to communication, and try to get to the root of mediated exchange. We will examine how data is collected, how it is transformed into information, how it can be encoded, transmitted, and decoded. In the process we will encounter the need to build contextualized systems of meaning: systems of representation that maintain internal coherence, but might not make a lot of sense when seen from the outside.

Working with a partner you will construct two electronic beings capable of percieving some aspects of the environment around them. Use whatever sensors are necessary to create their "worldview." Then you will need to connect them into a network that would enable them to communicate their findings, and respond to the communications of their counterpart.

Things to consider in this project: Are both creatures going to be in the same environment or in different ones? Will their "worldviews" (the sensory data they receive) be the same, opposite, or unrelated? Will they be stationary or mobile - and if mobile: self-propelled, or moving with the help of another being. How will their sensory perceptions be processed internally - and is there difference in the way one processes data from the other? How will the result of this process be encoded for transmission? Will the transmission be continuous or intermittent? Will they be able to transmit and receive information at the same time or will they take turns? Is their communication going to be preceptible by the outsiders?

Group assignments:

  • Guowei Lyu, Fanxuan Zhu
  • Hongming Li, Zhiyan Wang
  • Qianyue Yuwen, Zheng Wang
  • Shiyi Chen, Miaoqiong
  • Alan Amaya, Yue
  • Yining Gao, Mario Santanilla
  • Zeyu Wang, Eric Schubert
  • Yiran Mao, Noah Curtis


Write down your responses to the questions posted in the brief, and how your project addresses them. Create a diagram outlining the communication schema and depicting all your decisions. Create a video explaning the inner workings of your system and showing it in action. Write about the process of making from iteration to iteration. You can reference this document to see examples of how the tecniques of collage, illustration, photography and sketching can be used to explain your vision for the possibilites and implications of your project.

>>> Weekly Schedule

Week 8: Nov. 1 - 5


Overview of electronic components: sensors, actuators, microcontrollers. Analog/digital read-write operations. Storing and retreiving data from AdafruitIO using ESP32 and P5js.

ThinkerCad links: Potentiometer and LED, Two and three pin sensors/Multiple values over serial

Zoom recording


Schedule some time with your partner to talk about the possible directions of the project. Review and make decisions about the environment(s) your creatures will be exploring, the information they will be collecting and how the information will be stored and exchanged. Use diagramming tools to help you understand your ideas in details. Review the Adafruit IO Basics Tutorial: Analog Output to learn how to read data from the Adafruit IO cloud with an ESP32-like device and represent it with a simple output.

Week 9: Nov. 8 - 12

Wednesday, Nov 10, 3pm - 5pm

Process check-in and tech troubleshooting with Maxim.

Friday, Nov 12, 1pm - 4pm

Group meetings with Ben and Maxim.


Build out your projects!

Week 10: Nov. 15 - 19

Wednesday, Nov 17, 4pm - 6pm

Process check-in and tech troubleshooting with Maxim - in the Windtunnel building, open lab.

Friday, Nov 19, 1pm - 4pm

Workshop: finish and present your projects.

Code example from class

Zoom recording


Compile and upload the documentation.

Week 11: Nov. 22 - 26 — NO CLASS

Project 4:
Computational aesthetic

If you already have experience making drawings with computers, you know that in that process you draw a circle, then a rectangle, a line, some triangles until you compose the image you want. That process is very similar to writing a letter or a book by hand - it is a set of instructions that do one task after another.

Shaders are also a set of instructions, but the instructions are executed all at once for every single pixel on the screen. That means the code you write has to behave differently depending on the position of the pixel on the screen. Like a type press, your program will work as a function that receives a position and returns a color, and when it's compiled it will run extraordinarily fast.

- The Book of Shaders


>>> Weekly Schedule

Week 12: Nov. 29 - Dec. 3


Workshop: Introduction to fragment shaders. Tools and methods for programing shaders.


Create an object in Unity or any other environment of your choosing and color it using shaders in a way that “makes sense.“ Makes sense” can be interpreted broadly. Some options:

  • Make a lava lamp where the inside blobs are driven by a shader.
  • Make some bubbling soup or a cauldron.
  • Make washing machine where the window part is a shader.

Think about something that can't be created easily without a shader. Think about a shader's strengths and weaknesses. This homework is most of all meant to be a prompt to encourage exploration, which is why its left to broad interpretation.

Week 13: Dec. 6 - 10

Wednesday, Dec 8, 4pm

Office hours with Char - on Zoom.



Finilize your documentation for the semester and post it online.

Week 14: Dec. 13 - 17 — NO CLASS