Balancing Art and Complexity:

Joseph Nechvatal's Computer Virus Project

 

by

Stphane Sikora

 

Introduction

Since his first robotic-assisted paintings in 1986, the artist Joseph Nechvatal has always questioned the relationship between reality and virtuality. By working in-between these two spaces, Nechvatal has shown their complex interaction. This reciprocity is what Nechvatal sees as typical of viractualism, an art theory term he developed in 1999. This term viractualism (and viractuality) emerged out of his doctoral research into the philosophy of art and new technology concerning immersive virtual reality at Roy Ascott's Center for Advanced Inquiry in the Interactive Arts (CAiiA), at the University of Wales College, Newport, UK. There he developed this viractual concept, which strives to identify and create an interface between the biological and the technological. Viractualism is central to his work as an artist. [Nechvatal, 2011]

The basis of the viractual conception is that virtual producing computer technology has become a noteworthy means for making and understanding contemporary art. This brings art to a place where one finds the emerging of the computed (the virtual) with the uncomputed corporeal (the actual). This amalgamate is what he calls the viractual. Digitization is a key metaphor for viractuality in the sense that it is the elementary translating procedure today. For Nechvatal, the viractual recognizes and uses the power of digitization while being culturally aware of the values of monumentality and permanency - qualities that can be found in some compelling analog art. [Nechvatal 2011]

The Computer Virus Project's initial goal was to produce physical paintings by using algorithms implementing viral processes. It is based on a simulation tool which allows Nechvatal to virtually introduce artificial organisms into a digitized reproduction of an earlier work of his, and let them transform and destroy that original image. During these attacks , novel still images can be extracted and painted on canvas, which is a way to realize them; i.e. to bring back the virtual into the real.

After an historical presentation of the Computer Virus Project, this paper will describe the simulation model in detail, and how we attempted to reconcile art with random exploration and complex processes.

 

1 Origin of the project

Following his first series of innovative paintings that were created using a digital-robotic painting machine in 1986, Nechvatal sought to create paintings around the concept of the computer virus. He developed this idea by considering an image as a host for the viruses: active agents whose role it is to manipulate and degrade the information contained in the image. The negative connotations of the HIV virus as a vector of disease is reflected in the principle of degradation of the image. But here, the virus is also the basis of a creative process, producing newness in reference to the major influence of the virus on evolution in biological systems.

 

1.1 Computer Virus 1.0

Nechvatals work with Jean-Philippe Massonie of the Laboratory MIS at the Franche-Comte University in 1991-2 allowed him to develop the first implementation of the Computer Virus Project. The computer language used for this was Basic Hypercard. Figure 1 shows an acrylic painting on canvas where the host image has been "attacked" by the Hypercard virus. The resulting blue colored marks are a product of the viral algorithm. Also the first version of this code was used as a design element for the composition of a painting. (Fig. 2)

 

 

Figure 1 : viral attaque: the conquest Of the horrible 1993. A painting done with the first viral algorithm developed in 1992.

 

 

Figure 2: Virus cOde, 1993, 430 x 220 cm.

 

 

1.2 Computer Virus 2.0

Computer Virus 2.0 began in 2001 in collaboration with the author, Stphane Sikora. The project reached a new form by becoming a real-time artificial life simulation. Artificial life is a field that studies artificial objects that exhibit properties of life [Langton, 1989] [Adami, 1998]. Here, viruses are modeled as autonomous agents inhabiting an image (the host) and try to survive by consuming or 'eating' the colors contained in the image. From this version, it became possible for Nechvatal to make digital video projections of the process being computed that he used in art installations [Couchot, 2007]. Unlike a recorded video, these projections are live and constantly renewed, where each attack is presented as a unique event.

Figure 3 shows stages of such an attack. Three colonies of viruses are initially injected into the picture. The first consumes red and blue, and leaves a green trace. The second uses green, and leaves a purple trace. The third eats the colors red and green, leaving a blue trace. When different colonies meet, the combination of their actions consumes all resources, leaving it predominantly black.

 

Figure 3: Stages of a viral attack on an image by three colonies of viruses able to consume colors.

 

1.3 Evolutions

Since 2001, Computer Virus 2.0 has varied over many exhibition situations. One of the major areas of development was to strengthen the immersive aspect, enriching the experience of the audience. Experiments were made in the area of the visualization of the viruses, the shape of the virus their size and color - but also in how the environment is represented. For example, the possibility to display only a portion of the image was introduced, allowing to show more details, like when looking into a microscope.

 

a                                                b

Figure 4:(a) Viral infected still diptych painting from 2003 Orgiastic abattOir : flawless ignudiO, computer-robotic assisted acrylic diptych on canvas, 224 x 16 cm. (b) Real time viral projection at show After Virus, Galerie RLBQ, Marseille, France.

 

In 2010 a combination of paintings and animations on small screens was produced for the exhibition in Paris called Art rtinal revisit: histoire de loeil at Galerie Richard.

 

Figure 5: Partial installation view of Art rtinal revisit: histoire de loeil at Galerie Richard 2010.

 

1.3.1 Sound development

Computer Virus development consisted next with the addition of real-time audio production. This proved useful in enhancing the immersive nature of the installation Viral Counter-Attack that was mounted at Espace Landowski in Paris in 2004. To best suit the viral image, sound is synthesized in real time from the activity of the virus. The algorithm used is a form of granular synthesis [Bowcott, 1989] applied to audio files, in which parameters are modulated according to statistics extracted from the simulation: viruss reproduction rate and resource consumption. These statistics are calculated for different regions in the image, thereby producing spatialized sound. For example, if the viruses are more active on one side of the screen, the sound will be stronger on that side. This mechanism allows the installation to draw the viewer's eye toward regions of the image where important events are taking place.

Some attacks have been captured and recorded - and were worked on by Andrew Deutsch and Matthew Underwood at the Institute for Electronic Art at Alfred University in New York for the creation of the initial movement of Nechvatals Viral Symphony. [Nechvatal, 2006]

In 2009, the composer Rhys Chatham contributed to the Computer Virus Project by producing a soundtrack from one of his compositions for 400 guitars entitled Crimson Grail. This collaborative work is entitled Viral Venture and was publicly first shown in 2011 on a large screen at the Beatrice Theater of the School of Visual Arts in New York City. The superposition of the hundreds of instruments produced rhythms and harmonies that matched particularly well with the vibrational movements of the virus, as if each virus was associated with a guitar string.

 

Figure 6: CD cover of Viral Symphony, 2009.

 

 

1.3.2 Interactivity

In a series of exhibition installations entitled Viral Counter-Attack (2004) the Computer Virus Project was presented for the first time in a multi-user, interactive version. Here the audience was invited to influence the path of the viruss movement in real-time as they passed their hands over a sensitive surface. (Fig. 7)

 

Figure 7:Viral Counter-Attack interactive device that allowed the public to change the movement of the viruses by attracting them to specific areas of the image.

 

1.4 Reversing the process

In 2010, emphasis shifted somewhat from destruction to creation by inverting the viral activity so that while the image was being destroyed, another image was being constructed simultaneously. This was achieved by using two hosts concurrently. Instead of a series of single attacks, continuous animations were created. Still images were then captured from this creative-destructive process, for example Figure 8.

Figure 8: sOuth pOle, 2011, computer-robotic assisted acrylic on canvas, 50 x 50 cm, Galerie Richard, New York.

 

This creative-destructive process was also used in the creation of what is called the Penelope project. This project revisits Nechvatal's early body of drawings from the 1980s that used a dense network of lines that concealed and revealed suggestive representational material. Penelope project picks up on this palimpsest technique by continuously revealing underlaying drawings as the top drawings are eaten away. This generative animation is paired with some actual drawings on paper for exhibition.

 

 

Figure 9: Penelope Project, 2010 still image from animation.

 

As of 2011, the Computer Virus Project is still in development. Particularly emergent is the enriched capacity of the viruses to act with new behavioral instructions, always offering more complexity in the behavior exhibited by the virus.

 

2 Simulation model

The Computer Virus Project is based on a multi-agent simulation [Ferber, 1995], where the viruses (the agents) inhabit within an image. Viral activity is simulated as a continuous loop of perception and action in interaction with the environment [Meyer, 1997]. Each virus is autonomous because it extracts information from its local environment to decide what actions to take. Its goal is to survive, and to do this, it must consume the colors of the pixels of the image.

2.1 Environment

The environment - named host - is the world in which the viruses live. It is modeled as two dimensional grid of square cells that correspond to the pixels of the image-host. As is in the cellular automata work of John von Neumann and in the Game of Life by John Horton Conway [Gardner, 1960], each cell contains resources, and may host one ore more viruses. For each cell, neighborhood is defined as the cell itself and the 8 adjacent cells (Fig. 9). Viruses have only access to data contained in the neighboring cells. They can move from cell to cell - with no exception to this rule. If one of them steps out on one side of the image, it will end up on the other side (Fig. 9b). This type of environment is commonly referred to as toroidal. As defined in the field of complex systems, it has no center and no boundaries, thus avoiding edge effects [Weisburger, 1989].

 

Figure 10: (a) Neighborhood of a cell corresponds to the eight adjacent cells (b). The world is toroidal: there is no center. Cells located on opposite borders of the environment are considered adjacent.

 

The host is built from an image encoded in RGB color space (red, green, blue) by associating a resource to each color channel. Each cell corresponds to a pixel of the image and contains quantities of resources R, G, B according to the color of its associated pixel.

During the attack, viruses change the quantities of resources distributed in the host, resulting in a change in the pixel color of the image.

 

2.2 Behavior of the virus

A virus is able to collect data from its surrounding environment using sensors and is also capable of achieving actions through its actuators (Fig. 9). It can move to a nearby cell, or change the amount of resources on the cell it occupies. The perceptual abilities of the viruses are rather limited, yet this capacity is sufficient for them to orient themselves and decide what kind of action to take.

 

Figure 11: Situated agent cybernetic loop: at each simulation step, viruses perceive local data conditions and decide what actions to take.

 

2.2.1Internal energy

The only goal of a virus is to survive. To survive a virus must maintain a minimal level of vital energy, called E. E reduces itself by a constant amount E- at every step of the simulation. When E reaches 0, the virus is considered dead, thus it is eliminated from the simulation. Therefore, in order to stay alive, the virus must gain energy. In Computer Virus Project, the only way for a virus to do this is to alter the color of the host: the more it changes its color, the more energy it gains. Therefore, energy gained (E+) is proportional to the amount (D) of resources that are exchanged with the environment.

With (Rt, Gt, Bt) and (Rt+1, Gt+1, Bt+1) resource values at respectively t and t+1 simulation steps.

Finally, energy E is computed for each simulation step t+1, as shown in the formula below, where E+t+1 is the amount of energy gained by a virus at simulation step t+1, E- the energy lost by the virus:

Thus, viral survival requires appropriate actions and the ability to find places where resources can be exploited. This energy need will have a major role in structuring the behavior of the virus, as it will favor the emergence of efficient viruses, and the removal of inefficient ones. Also E- directly impacts the difficulty to survive: high values drastically reduce the time for viruses to find resources before dying.

2.2.2 Behavioral Program

Each virus is controlled by its behavioral program. It defines its actions according to its internal state and its local environment, making the link between sensors and actuators. Figure 10 shows the tree representation of such a program. Each node corresponds to a program instruction. Some nodes have sub-nodes, which correspond to sub-programs. At every step in the simulation process, each virus interprets its behavioral program and performs corresponding actions. The first node of the tree (the root) is interpreted first, and sub-nodes are then interpreted recursively.

 

Figure 12: Example of behavioral program.

 

Figure 10's program is interpreted as follows: if vital energy of the virus is high, then it divides; otherwise it consumes red resources and goes to areas where red resources are available. This program will perform well on a picture containing red color, and poorly on pictures where red color is absent.

This example shows two categories of instructions: some instructions, such as 'Seq' and 'If Energy High' give the tree its structure.

       "Seq" is used to interpret a sequence of instructions. Each sub node is interpreted one after the other.

       Conditional instructions such as "if High Energy" evaluate a condition, interpreting the 1st subtree it the condition is met, and the second one, if not. In Fig. 10, if the viral energy E of the virus is above a threshold (60% of the maximum energy) then the first sub program will be executed.

 

Other instructions located on the leaves of the tree correspond to actions performed by the virus.

       Some action instructions alter the amount of resources in the environment. These include "EatR", "EatG" and "EatB" - which are used to consume resources (respectively R, G and B). The instruction "Dark" causes the consumption of three resources simultaneously. Fig. 13f shows the results obtained with the "invert" instruction. In this latter case, the attack will never end, as the statement "invert" produces resources.

       By default, viruses move randomly, but some instructions can attract the virus in some directions. "FollowR", "FollowG", "FollowB" for example, guides the virus to the closest cell that contains the highest amount of one of the three resources. The command "FollowEdge makes the virus follow the lines in the host.

 

       Finally, the duplication instruction produces a copy of the virus. This copy inherits the genetic program of the initial virus (the parent). The energy of the parent virus is divided equally between the two viruses (Fig. 11). Only one duplication can be performed by a virus at each simulation step, even when multiple occurrences of duplication instruction were encountered during the program's interpretation.

 

These instructions altering pixels color were the first one introduced, many other were added later.

 

 

 

 

 

Figure 13: Different examples of filters obtained by different combinational instructions in behavioral programs of viruses, applied on host (a).

 

 

Figure 14: The energy of the parent virus is divided equally between the two viruses during duplication.

 

 

2.3 Evolving programs

We have seen that the survival of a virus depends on the amount of energy dissipated at every step of the simulation, but also (and especially) its ability to extract energy from its environment. If a program is sufficiently adapted, the virus will have the ability to remain active as long as resources are sufficient. In the example given in Fig. 10, the virus will reproduce by creating a copy of itself if it has enough energy. Otherwise, it will seek to absorb energy by reducing the amount of the resource R where it is located, while moving to the pixel neighbor who has the most of this R resource. The behavioral program is suitable for images containing much red.

So far I have described the basics of the simulation model implemented in the Computer Virus Project.

At this step, it is possible to write behavioral programs by hand and observe the resulting attack on various images/hosts. But designing such programs is a laborious and tricky task, as it involves creating a program for each virus, and may be reconsidered for each image/host. A better way to fully explore the possibilities of the simulation model is to generate these programs automatically.

Genetic programming [Koza, 1992] is an optimization method based on genetic algorithms [Goldberg, 1989] for automatically writing functions or programs by means of evolutionary processes starting from a population of random programs. To avoid the necessity of having to write behavioral programs, and for more diversity within the population of viruses, I therefore used this automatic writing scheme to bring out behavioral programs tailored to the survival of the virus.

Among the viruses produced at random, some will disappear quickly, while others will manage to stay alive and to reproduce. To explore other behaviors (in addition to those obtained the random generation of the initial population) a mutation operator is applied during duplication. It involves replacing a node chosen randomly by a new subroutine. (Fig. 12)

 

Figure 15: Mutation operation.

 

This mechanism allows the emergence of new behaviors during an attack, on the different regions of the image. Among adapted viruses, some are able to execute duplications, which will have the effect of producing new viruses that have a high probability of being also adapted to their environment, and sometime superior to its parent, thanks to genetic operators. As a consequence, the number of these viruses may grow exponentially, resulting in a very large population. Therefore, a limit on the number of viruses had to be introduced to avoid slow downs in real time simulations.

 

3 Conclusion: balancing randomness

 

Above I have outlined the software architecture governing the simulation bases of Joseph Nechvatals Computer Virus Project. This software permits the exploration of complex dynamics while adhering to Nechvatals specific aesthetic demands. It creates a balance between art and complexity: on one hand it uses the constraints of order, and on the other hand, it leaves a part of the process to take place at random.

 

The project's aesthetic function is achieved by writing a dynamic mechanism for automatically evolving the code of the Computer Virus Project through the establishment of an artificial selection force, typical of genetic algorithms. This inter-twinning activity is itself evident of Nechvatal's theory of the viractual.

 

 

 

 

 

4 References

 

[Adami, 1998]

C. Adami. Introduction to Artificial Life, Springer Verlag, 1998

 

[Bowcott, 1989]

P. Bowcott, Cellular automation as a means of high level compositional control of granular synthesis, Proceedings of International computer music conference. 1989, San Francisco. ICMA.

 

[Couchot, 2007]

Edmond Couchot, Des Images, du temps et des machines, dit Actes Sud, 2007, pp. 263264

 

[Ferber, 1995]

J. Ferber. Les Systmes Multi-Agents, Inter Edition, Paris, 1995.

 

[Gardner, 1970]

M. Gardner. The Fantastic Combinations of John Conways Game of Life, in Scientific American Vol. 223:4, 1970, pp. 120-123.

 

[Goldberg, 1989]

D.E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning.

Addislon-wesley, 1989.

 

[Koza, 1992]

J.R. Koza, Genetic Programming: On the Programming of computers by Means of Natural Selection. MIT Press, 1992.

 

[Langton, 1989]

C. Langton. Artificial Life : Proceedings of the 1st Workshop on the Synthesis ant the Simulation of Living Systems 1987, C.Langton Ed., Addison-Wesley, 1989

 

[Meyer, 1997]

J.-A. Meyer. From natural to artificial life: Biomimetic mechanisms in animat designs. Robotics and Autonomous Systems, 22:3 -21, 1997.

 

[Moulon, 2004]

Dominique Moulon, L'art numrique: spectateur-acteur et vie artificielle, Les images numriques 47-48, 2004, pp. 124125

 

[Nechvatal, 2006]

Joseph Nechvatal, Viral Symphony, Institute of Electronic Arts, 2006. http://www.archive.org/details/ViralSymphony

 

[Nechvatal, 2011]

Joseph Nechvatal, Immersion Into Noise, Open Humanities Press, University of Michigan, 2011. p. 244

 

[Popper, 2007]

Frank Popper, From Technological to Virtual Art, MIT Press, pp.120123

 

[Weisbuch, 1989]

G. Weisbuch. Introduction la dynamique des systmes complexes InterEditions/CNRS, 1989

 

 

 

 

 

 

RETURN