Bodies in code interfaces with digital media

Yet, if Krueger's theory and work resonate with the second-generation paradigm of mixed reality, this is due less to his original aim (provoking the flat-footed humanism of the art worldi) than to the singular convergence of the "artificial" and the "natural" in the electronics technologies ubiquitous in our world today. This is a reality not lost on Krueger, who has recently noted: "Since I was arguing for convenience, naturalness, and obviousness, my concepts were well-positioned for technological advances as they unfolded. Since 99% of applications are 2D and 99% of 3D applications are driven by 2D interfaces, there has been very little immediate interest in HMD immersion systems in the general office environment," not to mention the world beyond. As if echoing (or ventriloquizing) the proponents of the mixed reality paradigm, Krueger speaks of a "progression towards external realities' driven by the technical development of low-cost projectors and organic LED displays that can be ubiquitously embedded in the environment; rather than withdrawing from the physical domain, today's digital technologies are literally virtualizing the physical.

Krueger's calculated defense of the artificial notwithstanding, the operative principle of his research and artistic experimentation could easily be summarized in the form of a practical maxim: make use of what nature has given us!"In our physical reality," Krueger observes, "We use our bodies to interact with objects. We move our bodies or turn our heads to see better. We see other people and they can see us. We have acquired a consistent set of expectations through a lifetime of experience"(148). Making use of this evolutionarily realized heritage will allow us to adapt to our ever more rapidly changing technical environment because such heritage provides a stable background against which to assimilate new interactional spaces and share new affordances: “Any system that observes these conventions will be instantly understood by everyone on the planet” (148).

BODIES IN CODE

27

More than a simple desire to buck the technicist trend, Krueger's work is informed by his conviction that the best way for us to bring technologies to bear on human existence—to shepherd our ongoing technogenesis as it undergoes its most accelerated phase to date—is to channel them through our evolutionarily acquired embodiment. This explains his lifelong commitment to a humanist model in which human interaction (encompassing interaction with the self, with others, and with machines) prevails over any notion of technicist instrumentalization. Krueger's environments function practically, following the characterization of Howard Rheingold, as "laboratories for finding out how humans might harmonize with" technical environments whose sway is (and has always been) inescapable.iii,3

The benefits of such an artistic (and scientific) program are twofold: On one hand, human embodiment serves to "naturalize" technical modifications of the world (and, potentially, of the body); on the other hand, these modifications provide an important source for decoupling or deterritorialization by which the body's habitual intercourse with the world gets disturbed and (potentially) expanded. Embodiment accommodates and self-reorganizes in the face of the ever expanding scope of technics in our world today.

In a series of works culminating in the ground-breaking video projection system Videoplace, Krueger has sought to materialize his conviction that "the focus of interface research should be on human nature, not on the transient computer" (Krueger, “An Easy Entry," 147). This conviction has led him to construct environments that eschew the logic emanating from computer code in favor of emergent logics rooted in social conventions. Never an end in itself in Krueger's work, the computer is always a vehicle for exploring and expanding embodied (human) interaction with the world and with other human beings. "In the ultimate interface," he stresses, "input should come from our voices and bodies and output should be directed at all our senses. Since we will also interact with each other through computers, the ultimate interface should also be judged by how well it helps us to relate to each other” (147).

This principled emphasis on human embodiment as mediator between computer and world represents something new in the history of our technogenesis: in its role as primary access to a (now) highly technologized lifeworld, embodiment serves to couple body and world, as well as to actualize the potential of digital (virtual reality) technologies to modify the lifeworld (and, thereby, to infiltrate that primary enactive, worldconstitutive coupling). Embodied enaction is, quite literally, the agent through which technics has an impact on life and the lifeworld.

BODIES IN CODE

28

If Krueger's entire career seems dedicated to exposing nothing more or less than this primacy of embodied enaction, the latter—as he points out—ushers in an important, and enabling, margin of indetermination: "The logical consequence of this thought process [yielding the conception of the ultimate interface] was the concept of an artificial reality in which the laws of cause and effect were designed to facilitate the functions that interested the user" (148). It would hardly be an exaggeration to claim that this margin of indetermination—one directly tied to technics—comprises the operative principle of Krueger's embodied aesthetics of new media. The new technical environments afford nothing less than an opportunity to suspend habitual causal patterns and, subsequently, to forge new patterns through the medium of embodiment—that is, by tapping into the flexibility (or potentiality) that characterizes humans as fundamentally embodied creatures.

That this principle comprises the explicit focus of Krueger's first, predigital responsive environment amply attests to its centrality in his aesthetics. Glowflow (1969) is a light-sound environment—a room with four horizontal light tubes running along its walls—that deploys visual and auditory means to disturb the visitor's habitual mode of perceiving space (see Figure 1.1). The room is entirely dark except for the multicolored phosphorescent light tubes which form the only visual point of reference for the visitor. Movement within the darkened space inevitably causes the visitor to step on floor sensors that release into the tubes light and sound elements from one of four enclosed light columns (one per tube). Outside of this element of interactivity, the pigments running through the tubes remain arbitrarily determined.

What proved compelling about the environment and most captivated Krueger's aesthetic interest was the way that viewers inevitably, one might even want to say “naturally," attempted to make sense of the environment by coupling intentional gestures (e.g., speaking or moving) with the (mostly arbitrary) patterns of light in the tubes. As Krueger recounts in Artificial Reality 2,

If a tube started glowing after [the visitors] spoke, they would assume that their speech had turned it on. If a sound occurred after they hit the wall, they would assume that striking the wall would elicit more sounds. Often, they would persist in the behavior long after the result should have convinced them that their hypotheses were incorrect.4

BODIES IN CODE

29

This emergent and unexpected phenomenon alerted Krueger to the possibility and value of exploiting the margin of indetermination afforded by experience with new technologies and media environments.i As Söke Dinkla has observed,

The behavior of the visitor comprised efforts to figure out the rule system of Glowflow. The process of rational clarification of cause and effect proceeded precisely opposite to its normal pattern: an effect would dictate its cause and not the reverse [Einer Wirkung wurde eine Ursache zugeschrieben und nicht umgekehrt]. Such exploratory behavior is typical when it is a question of accommodating oneself to foreign surroundings whose rules are unknown. (67)

In one way or another, all of Krueger's environments exploit this technique of disturbing habitual cause and effect couplings to domesticate novel experiential domains through the primary medium of embodiment.

With his subsequent (and properly digital) environments, however, Krueger added a crucial element to this technique of domestication: the calculated feedback of new sensory experiences into embodied enaction for the express purpose of expanding the latter's agency (together with the scope of its coupling with the environment).' What made this possible and served to distinguish Krueger's environments from the majority of art environments from the 1960s was Krueger's insistence on clarifying for the visitor precisely how his or her actions called forth reactions from the environment. Because this insistence is due, in part at least, to the analytical precision afforded by the computer as a material support for the interactive environment, it yields a specific, technical (but not technicist) conception of interactivity: “dealings [einen Umgang] with the computer in which the receiver/user is able clearly to correlate her actions with the reactions of the system" (Dinkla, 70). Such correlation, affirms artist Simon Penny, comprises “the first and fundamental law of the aesthetics of interactive installation" (Simon Penny, email to the author, January 3, 2006).

BODIES IN CODE

30

On this conception, interactivity can be distinguished from prevailing notions of the period and from more contemporary technicist visions. It is at once more codified than the radically open-ended environments of process art and happenings and yet more open and flexible than the preprogrammed response repertoires that one finds in socalled interactive cinema and hypertext. What is crucial in Krueger's conception of interactivity, as in his humanist conception of artificial reality more generally, is the privilege accorded embodied (human) agents. Differences notwithstanding, the point of all of his environments is to facilitate new kinds of world-construction and intersubjective communication. If these environments do still serve as interfaces to the computer, they do not do so, as do all technicist conceptions of the "Human-Computer Interface" (including the mouse and the graphicaluser interface), by instrumentalizing—and thus reducing—embodied enaction; here, rather, it is the technology that remains instrumentalized and human action that gets privileged.

From 1970 to 1984 (when he ceased development of his interactive platform, Videoplace), Krueger devoted himself to the task of producing an environment that would realize the potential of the computer to expand human capacities for embodied enaction and communication. Not surprisingly, this realization—accomplished in Videoplace—proceeded through several discrete stages, each of which marks an important advance in the transfer of agency from the computer to embodiment in open-ended feedback with itself.

Metaplay (1970) combined closed-circuit (video) technology with the computer to explore the conditions for the correlation by the visitor of her actions and the system's reactions. This environment established a real-time communication circuit between participants in a gallery, who encounter a projected two-dimensional image of their bodies, and the artist who, located in a separate space, employs a data-tablet and pen to draw directly on the projection screen, thereby provoking responses from and stimulating further interaction with the visitors (see Figure 1.2). As Andy Cameron has recently pointed out, Metaplay offered Krueger an opportunity to “explore the kinds of interactions which he had been unable to explore in Glowflow--an interaction which is explicit and which is capable of engaging people within a new kind of aesthetic experience."5

BODIES IN CODE

31

Rather than the amorphous and nondirected action-reaction couplings that spontaneously (and most often erroneously) emerged in Glowflow, Metaplay clarifies the correlation of (visitor) action and (system) reaction precisely by isolating it and embedding it within a tightly coupled, clearly delineated circuit. To achieve this isolation, the environment sacrifices the autonomy of response. As Cameron notes, because the technology to create a truly interactive computer-supported responsive environment was unavailable (at least to Krueger), he was forced “to simulate interactive technology, using people behind a screen [the artist, in the case of Metaplay] to provide the system responses. In this odd and oddly productive reversal of the classic Turing test, the “participants would think they were interacting with an interactive environment, but in fact they would be interacting with concealed human beings" (Cameron, 18).

BODIES IN CODE

32

What the environment gains from this sacrifice is the creativity of indirection, here instantiated in the margin of indeterminacy emerging in the space between Krueger's doodlings and the visitor's responses. Because the action-response circuit involves something beyond what can be contained in or determined by any technological interface— something unpredictable from the system's perspective—it privileges creativity over technicity. As Cameron suggests, this achievement-generating a working prototype of a new kind of communication with the paltriest of resources—is Krueger's most impressive: "By prototyping the experience, rather than the technology, ... Krueger was able to explore the aesthetic space of an interactive installation before the technology existed" (Cameron, 18).

When the source of indirection is subsequently integrated into the technical circuit, as it is in Psychic Space (1971), it simply cannot betoken any kind of break with Krueger's thoroughgoing privileging of the aesthetic over the technical. Rather, what is at stake here is a (partial) restoration of autonomy to the responsive environment, a restoration accomplished through the incorporation of feedback—and with it, of the element of unpredictability—as an internal aspect of the installation's solicitation and process-based generation of experience (see Figure 1.3). In this sense, Psychic Space marks a monumental accomplishment: the extrinsic source of creative indirection—Krueger's doodlings—is replaced by an intrinsic and ineliminable source—the embodied viewer's self-movement.

BODIES IN CODE

33

The operative technical element of this intrinsic feedback circuit is the use of motion sensors (in the place of the artist's external intervention); by placing the visitor into real-time interaction with data capturing the movement of her reactions within the environment, Psychic Space transforms embodied movement into the source of creative indetermination and, consequently, into the aesthetic focus of the environment. Dinkla emphasizes the (relative) novelty of this configuration:

[Psychic Space] confronts the visitor ... with the graphic, projected image of a lozenge that follows her movement. Just as Krueger moved the pen across the data tablet in Metaplay, now the visitor can couple herself to the graphic symbol through her embodied movement. The fact that the visitor engages in communication by way of her body stands in stark contrast to the typical interchange with the computer from this time .... (Dinkla, 73, emphasis added)

Although the visitor is initially invited to focus on the lozenge's function as a symbol of her self, the lozenge soon becomes enclosed by a small square that, triggered by the visitor's effort to move beyond this confine, then becomes a maze within which she is beckoned to wander freely. The aesthetic interest of the environment is thus shifted from the feedback circuit to the interactive possibilities that it facilitates.

In stark contrast to typical computer games, these possibilities offered by Psychic Space are not guided by any clearly defined goal and do not culminate in the successful accomplishment of a task. Rather, the environment is designed in such a way as to hinder the viewer's progress toward any such goal; for example, the computer will suddenly readjust the configuration of the maze at the moment when the visitor is nearing its exit, or it will shift the output generated by the visitor's movement so that it correlates with horizontal rather than vertical movement of the graphic symbol. In sum, the computer here functions to perturb the natural tendency of the primarily motile visitor to pursue a prescripted goal and thereby to foreground the flexibility and generativity of unreflective, learned social rules against the rigidity of the explicitly coded “if-then," rules governing the operation of computers (as well as the predominant visions of human-computer interface at the time).

As Dinkla puts it, Psychic Space “automatizes" socially coded, embodied behavior by prototyping its deployment as a source of creative feedback for adaptation to constantly changing situations (74). In this way, the environment continually solicits the visitor to affirm her agency, which is to say, the fact that the remapping and extension of her movement into this new space is the environment's driving purpose.

BODIES IN CODE

34

Videoplace, Krueger's interactive platform, marks a further and, in some sense, ultimate, stage in the restoration of autonomy to the responsive environment, understood as encompassing—and indeed as the interactional domain structurally coupled to —the embodied visitor. Like Psychic Space, Videoplace works by capturing an image of the visitor's movement, only in this case the image presents the outline of the visitor's body processed (and distorted in various ways) by the computer. Several different programs facilitate viewer interaction with the schematic traces of her bodily movement; for example, they allow her to fill in the space within the lines with colored images of body parts or to interact with temporally divergent and continually reverberating captures of her movement (see Figure 1.4).

More than anything else, Videoplace is differentiated from Krueger's earlier responsive environments by the technical and aesthetic accomplishment of complete action-response synchronicity. Here, the computer does not react to the visitor with a discrete response that in turn triggers an action on the part of the visitor (and so on); rather, because the computer's activity—some programmed modification of the data capturing the visitor's movement—coincides absolutely with that movement it becomes (or is experienced as) an indissociable part of its agency. This accomplishment of synchronicity shifts the aesthetic interest of the environment from the clarification of the visitor-system correlation (Krueger's guiding concern up to this point) to the newfound capacity for expanded agency:

BODIES IN CODE

35

On account of the synchronicity [of the movement of the video image] with the movement of the [visitor's] body, it is no longer a question of distinguishing between the activity of the system and the activity of the visitor. The computer system's role as interaction partner fades into the background, and it now makes itself available as an instrument [Instrumentarium] for the visitor to use. (Dinkla, 82)

The accomplishment of synchronicity is accompanied by a displacement of the artist as programmer and a wholesale inversion of the problematic of simulation. With the shift away from the visitor-system correlation to the action of a now unified single enactive “system-agent," the function of programming undergoes profound transformation from a closelooped technical element to an open-ended social capability. As Dinkla argues, the environments supported by Videoplace are designed “to learn from the visitors." Krueger's goal, she continues, was “to delegate his role as programmer to the public” (78).

With this shift in aesthetic emphasis comes an equally profound shift in technical emphasis that is definitive for the entire trajectory of media art that I am attempting to sketch here. Rather than calling on the user to adapt to the computer and to learn its language (as is the case with most HCI systems), here the computer is called upon to “learn to understand the natural means of communication" employed by embodied humans: in this case, gestures (Dinkla, 78). It should be clear by now that such "understanding" is the technical and aesthetic precondition for the synchronicity of system and visitor action. Without it, the visitor-system correlation could never move beyond the stage of clarifying exploration; it could never be directed to the world of action and could never become creative in the proper sense.

Thus, Videoplace ultimately achieves nothing short of a wholesale humanistic dissolution of the infamous—and for many, highly problematic—Turing test (together with Turing's abstract conception of a general-purpose computing machine). Far from maintaining an emphasis on the behavioral indiscernibility of human and computer and on the general problem of simulation, Videoplace seeks to engineer a human-computer cooperation so seamless that functional synchronicity becomes possible in practice. Put another way, Videoplace replaces the observation of behavioral output central to the Turing paradigm of simulation with a functional criterion: the operational "congruence of actions of the user and reactions of the system" (Dinkla, 83). Its purpose is neither epistemological nor ontological—to fool a human observer— but purely functional: to integrate the computer as seamlessly as possible into the motor activity of embodied human agents.

BODIES IN CODE

36