previous

3. Research results

Capturing Unstable Media presents a complimentary approach to the widespread material- and object-focused, rather static approach in preservation of contemporary art. For unstable media art, it is difficult to define the notion of 'original state' of an art object. Documenting the context of electronic art activities is important, as well as a perspective of process over product. Unstable media art activities often take place in dynamic, networked environments and are the result of an artistic research and development process. Such aRt&D processes can have very diverse outcomes, ranging from tools and installations to presentations and symposia. Each of those needs to be valued in the context in which they are produced, and, if necessary, needs to be captured. 

In short, the following characteristics of electronic art need to be taken into account when trying to capture unstable media: (1) electronic art activities are process-based; (2) context is very important; (3) the activities are heterogeneous in materials and practices; (4) projects are usually created through interdisciplinary or multidisciplinary collaboration; (5) user interaction is an essential activity in the dissemination phase of many projects; (6) it is important to speak about electronic art activities, and not just artworks; (7) preserving and reconstructing objects becomes less relevant than in contemporary art preservation.

A. Analysis of case studies

In order to distinguish the essential aspects of electronic art activities, and to define aspects that need special attention in this research, various projects from V2_'s archive were investigated [19], including two major case studies co-developed at V2_Lab: whisper [20] by Thecla Schiphorst and Susan Kozel, a project related to wearable technologies, performance and the body; and DataCloud 2.0 [21] and previous and later DataClouds, projects that involve web-based 2D or 3D visualizations of complex information structures.

First, essential components of activities and their terminologies were defined for all projects. As many concepts as possible were defined, focusing not so much on the reconstruction and display of a finalized artwork, but rather on several manifestations in a process, all possible components of these manifestations and the interplay of these components. Next, the available documentation for each case was studied, from the broadest perspective (including their social environment, development stages and research context) to the lowest meaningful level of detail (such as technical environment and network specifications). For whisper and DataCloud 2.0, the whole range of data and documentation, generated as a by-product of the research process, stored on project sites and in multi-user file versioning systems (CVS), was analyzed in depth [22]. For this research summary, we focus on an overview of the whisper project.

whisper is a research trajectory on "wearable architectures", dealing with small wearable devices and intelligent garments that are linked to a network. These computing devices gather physical data and signals generated by the body, and respond to these. During the DEAF03 festival [23] organized by V2_, whisper took place as a performance in an installation space. Participants entered the space wearing data suits and could interact with each other and with the devices, networked to a central database server. The server translates these behaviors in an aesthetic, shared audiovisual experience. The whole initial concept of the project was written down in a project proposal [24].

whisper consists of a number of intertwined research and test activities, some of which took place at the V2_Lab, producing data and resulting in several manifestations, like presentations and performances. During the first artist-in-residence period at the V2_Lab (June-July 2002), the research focused mainly on hardware and system development and design issues. The project was also presented at Anarchives, Connection-machines [25], a conference organized by V2_. The second residency (January-February 2003) emphasized the practical implementation of the participatory public installation and on wireless communication between the different modules of the whisper system. Next, whisper was performed in a public installation at the DEAF03 exhibition and at the E-Culture Fair.

B. Projects, occurrences and components

Electronic art projects, like whisper , consist of several research phases and manifestations (occurrences), each of which is documented by different types and genres of documentation. On a more detailed level, these occurred activities and products can be subdivided into smaller functional components, such as installation objects, network setups, software and hardware components in a specific configuration. In the case of whisper , several components were researched and developed at different co-production phases at the V2_Lab and were covered by different types and genres of documentation. See figure 2 for an overview of occurrences and components of whisper .

Figure 2. Capturing scheme of whisper

The whisper team members focused on different components during their first residency. Some designed the aesthetics of the input and output sensors, documented by photographs and sketches, while others focused on the technical design of these devices and their wireless Bluetooth communication, as outlined in a wiring diagram. A mathematician developed special application software, the so-called particle system, which generated the visual output projected on the floor of the space where the performances took place. The source code for this software is stored on an internal CVS server at V2_.

Between both residency periods, several activities took place outside V2_, including workshops on experience modeling. These were not part of V2_'s collaboration and are therefore not covered in this research project - although the research findings of Capturing Unstable Media include recommendations on interoperability between complementary archives (see below).

During the second artist in residence period, a team of software and hardware designers worked on the communication between hardware and software components, like the storage device of the system ("data collector" in Figure 3) and the different servers ("Main Brain" server, pool visuals servers and sound server). They focused on a serial protocol (the 1-Wire protocol) that enables communications between the data collector server and the JavaStamp microcontroller to which the sensors have been connected. The results of this research trajectory are summarized in a technical research report.

Figure 3. Deployment diagram with a schematic overview of the different modules of the whisper systems design and its dependencies.

For capturing on a detailed level, it is important to be able to describe how an occurrence (in the case of whisper: its public installation) has functioned. How did the different components work together? As demonstrated in Figure 3, the breath and pulse sensors (input devices) for the participatory installation are integrated with other components, like the garments participants wear (installation objects) and are part of the wearable configuration. This configuration (called "wearable system" in the above diagram) comprises a central processing unit, CPU, (JavaStamp microprocessor), an input/output device (Bluetooth module), and an output device (LEDs sewed on the garments). So, one could identify systems design and configuration as special technical components of electronic art projects.

For the actual implementation of the installation and for the public performance during DEAF03, the physical properties of the installation space were mapped in a plan and in sketches. The required wireless network was set up, together with the sensors, documented by installation instructions. Finally, lighting, visuals and audio were implemented, also documented in installation instructions. Photographs and a digital video report served as documentation for the resulting public performance during DEAF03 and the corresponding interaction between participants.

C. Problematic aspects of unstable media

A series of problematic aspects for the capturing of unstable media projects were encountered. User interaction is an important characteristic of many projects, but is difficult to describe and capture in a generic and objective manner. Furthermore, an attempt has been made to describe the very diverse modes of distributed authorship of a project and to describe, in a basic way, the dependencies of a work on hardware and software components. In Capturing Unstable Media, a series of proposals were formulated for dealing with these problematic aspects; proposed solutions include documentation strategies and an elaborated conceptual model, featuring metadata sets for some of these problematic aspects. These proposals will be described below.

D. Documentation strategies and recommendations

For Capturing Unstable Media, institutions need to decide on the level of detail in documenting a project (how extensive should the documentation be?) and on the point of interest in a specific project (which stages, versions or components need to be documented?). During the research it also became clear that there is much confusion on types and genres of documentation. There is a lack of standard terminology and a need for agreement on definitions.

An important aspect of Capturing Unstable Media is the act of collecting documentation that is most appropriate to the entities of the object of research that need to be captured. Institutions need to make a selection in the documentation, depending on the relative importance of the object or activity and to the level of detail in which it will be described; furthermore, documentation can be selected on the basis of its quality, variety and standardized readability [26].

A few general guidelines, resulting from the research on the case studies for Capturing Unstable Media, can already be formulated [27]. Some categories of documentation will not be relevant for certain types of institutions; for example, organizations without a media lab or workshop will not have access to, or generate, documentation related to research processes - which does not mean that this type of documentation is not important.

aRt&D process

Electronic art projects are often researched and developed through interdisciplinary or multidisciplinary collaboration [28] and generate documentation on concept, research objectives, design issues, the flow of technical experimentation and innovation, the technologies used and the results of the research process. This category of documentation also includes information on the operation, interrelationships and interdependencies of the components of a system, as well as its software, hardware and network configurations. Most often this documentation will be textual or visual in nature and include genres such as source code of applications, research reports and schemes and diagrams of interdependent components. Depending on the perspective and goals of the institution that is involved in this process, the selected documentation will rather emphasize finalized, implemented versions of the documented object, or may, on the contrary, emphasize the evolution of the research process itself.

Implementation

Electronic art projects are implemented once or several times in a physical (installation), digital (application) or mixed environment. Documentation for implementation will include practical information and instructions on the technical and/or physical environment in which the electronic art manifests itself, as well as user-oriented instructions. This category of documentation is mainly textual or visual, and may include genres of a directive nature, such as installation instructions, technical riders, schemes plans and documentation on ambient sound or lighting conditions.

User interaction

In this research project, a formal, restricted metadata model for describing user interaction was designed, including parameters on number of users, temporal and place dimensions, sensory mode (visual, auditive, tactile etc.) and on the level of intensity of the interaction (examples are navigation, participatory interaction, intercommunication between users) [29]. However, the specific, subjective characteristics and quality of a user's interaction with an electronic art piece cannot be captured by metadata alone; specific documentation of a user's experience is needed here. Available documentation may be textual or visual use case scenarios. However, for a good understanding of user interaction, additional documentation often needs to be actively created, such as audiovisual reports or registrations of someone interacting with a piece. Interviews with users may also prove very useful; in general, recordings and registrations of user testing activities are rare, but interesting documentation materials.

Interdisciplinary collaboration, distributed authorship

Individual contributors' involvement in a project can be outlined in a formal model; the Capturing Unstable Media Conceptual Model offers a proposal for dealing with this issue (see further). However, it is also important to gather additional documentation on contributors and their specific roles or functions in the project as related to the specific aspects of each activity. This includes documentation on copyright and biographical information about artists, main contributors and technical staff, mostly of textual nature.

The documentation that is obtained along these guidelines needs to be appropriately mapped to the relevant entities as defined in the Capturing Unstable Media Conceptual Model (see further). Digitalized or digitally-born materials need to be stored according to well-defined procedures, respecting the basic principles of maintenance of a digital media archive [30].

E. Capturing Unstable Media Conceptual Model

Apart from gathering documentation about a researched object, describing the object and its documentation in an appropriate formal model is an important step in the process of capturing. A formal model, such as an archival metadata structure, makes large numbers of research objects accessible and makes it possible to compare and study them on an equal level. Each organization will inevitably develop its own modeling system according to its own activities and goals; the Capturing Unstable Media Conceptual Model (CMCM) was designed as a source of inspiration for providing clear concepts and terminologies that can be used in such an individualized model.

The CMCM describes electronic art activities in a generic way, by using a structure of typical concepts. The concepts were derived from the study of projects from V2_Archive and the analysis of the case studies (see above). The idea was to create an abstract reference model for outlining the different (levels of) concepts through which activities in the field of electronic media art can be captured. Such activities may be as varied as long-term international research projects and specific, short-lived artworks, R&D activities and pieces of hardware, workshops and user interfaces.

The CMCM is an ontology with a multi-hierarchical and object-oriented structure of interrelated concepts or classes (described below). The choice for an ontology, rather than another abstract information structure such as a topic map, was made because of the following considerations:

• The capturing model needs to be a standardized instrument that can be used by a wide variety of actors and institutions; a choice was made for opening the possibility of integration with Semantic Web technologies [31]. For ontologies, various markup standards exists, such as the older DAML + OIL or the newer OWL; these are open, W3C-supported markup standards.
• The capturing model needs to accommodate a fine meshwork of interrelations and dependencies between concepts or classes; for this purpose, an ontology is especially suitable.
• The model also needs to be an instrument for enabling archival interoperability between institutions (see further).

The ontology is not intended as the fundament of a database structure; it can exist independently from the varied metadata and database systems that are used at various electronic media art institutions throughout the world. Rather, the CMCM may function as an independent reference framework, useable as a basis for interoperability, where each institution can map its individual concepts in its database structure to the corresponding concepts in the model. Not every concept in the CMCM needs to be included in an individual institution's data model; the choice of concepts depends on the institution's profile. Some institutions might focus on the description of technical details within R&D projects or artists' works, while others may emphasize the description of larger contextualization of projects, activities and actors.

The CMCM offers a series of basic concepts that are of interest to the activities in the field of electronic media art, along with suggestions on how these concepts could interrelate with each other. An interactive overview of all concepts can be consulted in an HTML export of the CMCM ontology [32].

Figure 4. Schematic overview of the main concept levels used in the CMCM [33]:

	Projects are long-term activities that are subject to change (e.g. a research project which runs over the course of two years).
	Occurrences can be activities or products with a distinct, short time span and an autonomous character.
	These occurrences may contain or be related to components: small, non-autonomous entities, often of a technical nature, such as software or hardware.
	Users interact with occurrences or components through various interaction models.
	Projects, occurrences and components can each have their own authors or makers.
	Each entity (project, occurrence, component) can be described through one or more pieces of documentation.

More in detail, subtypes of these concepts include [34]:

• Occurrences
  • OccurredProduct
    • PublicInstallation, Application, Performance
  • OccurredActivity
    • Meeting, Presentation, Performance, R&DPeriod
• Components
  • ComponentPhysical
    • Installation, Publication, Hardware, Network
  • ComponentDigital
    • Software, Network
  • ComponentConceptual
    • Design
  • ComponentContent
    • DataCollection, Audio, MovingImage, StillImage...
  • ComponentTechnical
    • Configuration, SystemsDesign

Apart from this, the model focuses on several problematic aspects in the area of capturing unstable media and suggests metadata solutions: terminology for describing electronic art (through a special thesaurus [35]), genres and types of documentation [36], describing distributed authorship [37], hardware and software dependencies [38] and user interaction [39].

The CMCM includes an extended system for integrating and interrelating relevant documentation of each activity in the field. It also deals with the complex issue of interdisciplinary, distributed authorship in the field of unstable media art, by defining the roles of all involved actors as related to the specific aspects of each activities. This creates a horizontal and interdisciplinary meshwork of involvement, rather than a rigid list of credits with inherent hierarchy.

The research project also includes a proposal to describe hardware and software dependencies in a more generic way. It is often necessary to describe how a well-defined group of components work together. In the above description of whisper , the configuration and design of the different modules of its system were outlined (see Figure 3.). Both SystemsDesign and Configuration are entities in the CMCM that should be used for expressing meaningful clusters of technical components of an occurrence.

next