While data has been visualized and sonified over and over, it is mostly the final, static output of that data that is being used to find a matching visual or audio cue in order to provide the audience with alternative ways to understand that data set. In my presentation, I aim to look at the use of image and sound in relation to data, as it is being processed and generated, rather than focusing a definite, immobile representation of what that data has to tell us. Developed as a Master’s Thesis at the NYU Game Center, Social Contact is a digital simulation of political theory, and explores the implications of a developer’s subjectivity in the process of programming -that is, generating and representing data.
Using both research from the field of software studies as well as research from the field of agent- based modeling, Social Contact is as much about the process than the output. This process of developing a simulation of general, universal, loosely defined ideas (e.g. simulating democracy, culture, language), still implies that these theories, turned into computable data through the process of writing code, must be interpreted and understood by both machine and audience.
Sound and visual design in interactive software have largely been devoted to the conveying of information to the user by choosing the cue that would represent the action/situation best to the eyes of the player, this presentation will also focus on how Social Contact has incorporated visual and sound design into the very process of development, generating those sounds and visuals at the same time as the data. How can the same process of close-reading of source code be applied to sound and visual design in interactive experiences? What does a thorough examination of how sound is generated say about the sound designer?
Using Processing’s 2D geometric primitives for the visual component and a modular synthesis Java library for the soundscape, the complete software does not contain any external assets, and thus relies on both the subjective composition of these minimalist building blocks by the developer, and the emergent characteristics embedded in the simulation. This immediate relation directly affects the ever-changing audio/visual representation of the simulation.
This presentation will then aim to take a closer look at that tension between the subjective writing of the designer and the objective output of the compiled code, as unexpected results emerge through the simulation, and allow the audience to interact with a self-contained piece of software.