Rehearsal (Visuo-Spatial Scratch-Pad) and Rehearsal (Articulatory Loop) are a companion pair of installations, they explore the constituent parts of our sensory memory – the visual and the linguistic – using computational techniques to attempt to understand, remember, and recall the world. As with our own system of memory, their processes are volatile and susceptible to environmental influence.

Rehearsal (Visuo-Spatial Scratch-Pad)

Rehearsal (Visuo-Spatial Scratch-Pad) views the scene before it, attempting to trace the details it sees, this feeds back into its vision, recursively affecting what it depicts. As with our visuo-spatial memory the system can only hold a certain amount of information in its ‘buffer’

The traced depiction descends into a frantic, fragmented depiction of the details of the scene, as the limited buffer cannot keep up with the added interpreted information it loses important parts and creates new ones providing an ever shifting representation of the forms in front of it.

The project uses computer vision to capture and analyse the slice of the world it is presented with, tracing its understanding of this, key features and outlines, via projection mapping back onto the space as it views it. This, in turn, interferes with its sight adding artefacts and unexpected variations which compound through the system’s feedback loop into increasing noise, an inability for the system to ‘focus’.

Rehearsal (Articulatory Loop)

In Rehearsal (Articulatory Loop) the system is seeded with this passage from ‘Funes the Memorious’ by Jorge Luis Borges:

“I suspect, however, that he was not very capable of thought. To think is to forget differences, generalize, make abstractions. In the teeming world of Funes, there were only details, almost immediate in their presence.”

It then attempts to recreate this via a repeated process of speech → text → speech, misinterpreting and recreating the passage along the way. The passage of the sound through the air offers opportunities for the surroundings and viewer to affect the transmission and understanding.

An example of the transformed, misinterpreted output:

Screen Shot 2014-05-26 at 10.23.26

An earlier version of the project can be seen here.

For more information about the project, see the process pages.