Balancing
Art and Complexity:
Joseph
Nechvatal's Computer Virus Project
by
Stphane
Sikora
Since his
first robotic-assisted paintings in 1986, the artist Joseph Nechvatal has
always questioned the relationship between reality and virtuality. By working
in-between these two spaces, Nechvatal has shown their complex interaction.
This reciprocity is what Nechvatal sees as typical of viractualism, an art
theory term he developed in 1999. This term viractualism (and viractuality) emerged
out of his doctoral research into the philosophy of art and new technology
concerning immersive virtual reality at Roy Ascott's Center for Advanced
Inquiry in the Interactive Arts (CAiiA), at the University of Wales College,
Newport, UK. There he developed this viractual concept,
which strives to identify and create an interface between the biological and
the technological. Viractualism is central to his work as an artist.
[Nechvatal, 2011]
The basis of the viractual conception is that
virtual producing computer technology has become a noteworthy means for making
and understanding contemporary art. This brings art to a place where one finds
the emerging of the computed (the virtual) with the uncomputed corporeal (the
actual). This amalgamate is what he calls the viractual. Digitization is a key metaphor for viractuality in the
sense that it is the elementary translating procedure today. For Nechvatal, the
viractual recognizes and uses the power of digitization while being culturally
aware of the values of monumentality and permanency - qualities that can be
found in some compelling analog art. [Nechvatal 2011]
The Computer
Virus Project's initial goal was to produce physical paintings by using
algorithms implementing viral processes. It is based on a simulation tool which
allows Nechvatal to virtually introduce artificial organisms into a digitized
reproduction of an earlier work of his, and let them transform and destroy that
original image. During these attacks , novel still images can be extracted
and painted on canvas, which is a way to realize them; i.e. to bring back the
virtual into the real.
After an
historical presentation of the Computer Virus Project, this paper
will describe the simulation model in detail, and how we attempted to reconcile
art with random exploration and complex processes.
Following his first
series of innovative paintings that were created using a digital-robotic
painting machine in 1986, Nechvatal sought to create paintings around the
concept of the computer virus. He developed this idea by considering an image
as a host for the viruses: active agents whose role it is to manipulate and
degrade the information contained in the image. The negative connotations of
the HIV virus as a vector of disease is reflected in the principle of
degradation of the image. But here, the virus is also the basis of a creative
process, producing newness in reference to the major influence of the virus on
evolution in biological systems.
Nechvatals work with
Jean-Philippe Massonie of the Laboratory MIS at the Franche-Comte University in
1991-2 allowed him to develop the first implementation of the Computer Virus
Project.
The computer language used for this was Basic Hypercard. Figure 1 shows an
acrylic painting on canvas where the host image has been "attacked"
by the Hypercard virus. The resulting blue colored marks are a product of the
viral algorithm. Also the first version of this code was used as a design
element for the composition of a painting. (Fig. 2)
|
Figure 1 : viral
attaque: the conquest Of the horrible 1993. A painting
done with the first viral algorithm developed in 1992.
|
Figure 2: Virus cOde, 1993, 430 x 220 cm.
Computer Virus 2.0 began in 2001 in
collaboration with the author, Stphane Sikora. The project reached a new form
by becoming a real-time artificial life simulation. Artificial life is a field
that studies artificial objects that exhibit properties of life [Langton, 1989]
[Adami, 1998]. Here, viruses are modeled as autonomous agents inhabiting an
image (the host) and try to survive by consuming or 'eating' the colors
contained in the image. From this version, it became possible for Nechvatal to
make digital video projections of the process being computed that he used in
art installations [Couchot, 2007]. Unlike a recorded video, these projections
are live and constantly renewed, where each attack is presented as a unique
event.
Figure 3 shows stages of
such an attack. Three colonies of viruses are initially injected into the
picture. The first consumes red and blue, and leaves a green trace. The second
uses green, and leaves a purple trace. The third eats the colors red and green,
leaving a blue trace. When different colonies meet, the combination of their
actions consumes all resources, leaving it predominantly black.
|
Figure 3: Stages of a viral attack on
an image by three colonies of viruses able to consume colors.
Since 2001, Computer
Virus 2.0
has varied over many exhibition situations. One of the major areas of
development was to strengthen the immersive aspect, enriching the experience of
the audience. Experiments were made in the area of the visualization of the
viruses, the shape of the virus their size and color - but also in how the
environment is represented. For example, the possibility to display only a
portion of the image was introduced, allowing to show more details, like when
looking into a microscope.
a
b
Figure 4:(a) Viral infected still
diptych painting from 2003 Orgiastic abattOir : flawless ignudiO, computer-robotic
assisted acrylic diptych on canvas, 224 x 16 cm. (b) Real time viral projection
at show After Virus, Galerie RLBQ, Marseille, France.
In 2010 a combination of
paintings and animations on small screens was produced for the exhibition in
Paris called Art rtinal revisit: histoire de loeil at Galerie Richard.
|
Figure 5: Partial installation
view of Art rtinal revisit: histoire de loeil at Galerie Richard
2010.
Computer Virus development consisted
next with the addition of real-time audio production. This proved useful in
enhancing the immersive nature of the installation Viral Counter-Attack that was mounted at
Espace Landowski in Paris in 2004. To best suit the viral image, sound is
synthesized in real time from the activity of the virus. The algorithm used is
a form of granular synthesis [Bowcott, 1989] applied to audio files, in which
parameters are modulated according to statistics extracted from the simulation:
viruss reproduction rate and resource consumption. These statistics are
calculated for different regions in the image, thereby producing spatialized
sound. For example, if the viruses are more active on one side of the screen,
the sound will be stronger on that side. This mechanism allows the installation
to draw the viewer's eye toward regions of the image where important events are
taking place.
Some
attacks have been captured and recorded - and were worked on by Andrew Deutsch
and Matthew Underwood at the Institute for Electronic Art at Alfred University
in New York for the creation of the initial movement of Nechvatals Viral
Symphony.
[Nechvatal, 2006]
In 2009, the composer
Rhys Chatham contributed to the Computer Virus Project by producing a
soundtrack from one of his compositions for 400 guitars entitled Crimson
Grail.
This collaborative work is entitled Viral Venture and was publicly first
shown in 2011 on a large screen at the Beatrice Theater of the School of Visual
Arts in New York City. The superposition of the hundreds of instruments
produced rhythms and harmonies that matched particularly well with the
vibrational movements of the virus, as if each virus was associated with a
guitar string.
|
Figure 6: CD cover of Viral Symphony, 2009.
In a series of
exhibition installations entitled Viral Counter-Attack (2004) the Computer
Virus Project
was presented for the first time in a multi-user, interactive version. Here the
audience was invited to influence the path of the viruss movement in real-time
as they passed their hands over a sensitive surface. (Fig. 7)
|
Figure
7:Viral Counter-Attack interactive device that allowed the public to change the
movement of the viruses by attracting them to specific areas of the image.
In
2010, emphasis shifted somewhat from destruction to creation by inverting the
viral activity so that while the image was being destroyed, another image was
being constructed simultaneously. This was achieved by using two hosts
concurrently. Instead of a series of single attacks, continuous animations were
created. Still images were then captured from this creative-destructive
process, for example Figure 8.
|
Figure 8: sOuth pOle, 2011, computer-robotic
assisted acrylic on canvas, 50 x 50 cm, Galerie Richard, New York.
This
creative-destructive process was also used in the creation of what is called
the Penelope project.
This project revisits Nechvatal's early body of drawings from the 1980s that
used a dense network of lines that concealed and revealed suggestive
representational material. Penelope project picks up on this palimpsest technique by
continuously revealing underlaying drawings as the top drawings are eaten away.
This generative animation is paired with some actual drawings on paper for
exhibition.
|
Figure 9: Penelope
Project,
2010 still image from animation.
As of 2011, the Computer
Virus Project
is still in development. Particularly emergent is the enriched capacity of the
viruses to act with new behavioral instructions, always offering more
complexity in the behavior exhibited by the virus.
The
Computer Virus Project is based on a multi-agent simulation [Ferber, 1995], where
the viruses (the agents) inhabit within an image. Viral activity is simulated
as a continuous loop of perception and action in interaction with the
environment [Meyer, 1997]. Each virus is autonomous because it extracts
information from its local environment to decide what actions to take. Its goal
is to survive, and to do this, it must consume the colors of the pixels of the
image.
The environment - named
host - is the world in which the viruses live. It is modeled as two dimensional
grid of square cells that correspond to the pixels of the image-host. As is in
the cellular automata work of John von Neumann and in the Game of Life by John Horton Conway
[Gardner, 1960], each cell contains resources, and may host one ore more
viruses. For each cell, neighborhood is defined as the cell itself and the 8
adjacent cells (Fig. 9). Viruses have only access to data contained in the
neighboring cells. They can move from cell to cell - with no exception to this
rule. If one of them steps out on one side of the image, it will end up on the
other side (Fig. 9b). This type of environment is commonly referred to as
toroidal. As defined in the field of complex systems, it has no center and no
boundaries, thus avoiding edge effects [Weisburger, 1989].
|
Figure
10: (a) Neighborhood of a cell corresponds to the eight adjacent cells (b). The
world is toroidal: there is no center. Cells located on opposite borders of the
environment are considered adjacent.
The
host is built from an image encoded in RGB color space (red, green, blue) by
associating a resource to each color channel. Each cell corresponds to a pixel
of the image and contains quantities of resources R, G, B according to the
color of its associated pixel.
During the attack,
viruses change the quantities of resources distributed in the host, resulting
in a change in the pixel color of the image.
A
virus is able to collect data from its surrounding environment using sensors
and is also capable of achieving actions through its actuators (Fig. 9). It can
move to a nearby cell, or change the amount of resources on the cell it
occupies. The perceptual abilities of the
viruses are rather limited, yet this capacity is sufficient for them to orient
themselves and decide what kind of action to take.
|
Figure 11: Situated agent cybernetic loop: at each
simulation step, viruses perceive local data conditions and decide what actions
to take.
The
only goal of a virus is to survive. To survive a virus must maintain a minimal
level of vital energy, called E. E
reduces itself by a constant amount E- at
every step of the simulation. When E reaches 0, the virus is
considered dead, thus it is eliminated from the simulation. Therefore, in order
to stay alive, the virus must gain energy. In Computer Virus Project, the
only way for a virus to do this is to alter the color of the host: the more it
changes its color, the more energy it gains. Therefore, energy gained (E+)
is proportional to the amount (D) of resources that are exchanged with the environment.
With (Rt, Gt,
Bt) and (Rt+1, Gt+1, Bt+1) resource
values at respectively t and t+1 simulation steps.
Finally,
energy E is computed for each simulation step t+1, as shown in the formula
below, where E+t+1 is the
amount of energy gained by a virus at simulation step t+1, E- the
energy lost by the virus:
Thus, viral survival
requires appropriate actions and the ability to find places where resources can
be exploited. This energy need will have a major
role in structuring the behavior of the virus, as it will favor the emergence
of efficient viruses, and the removal of inefficient ones. Also E-
directly impacts the difficulty to survive: high values drastically reduce the
time for viruses to find resources before dying.
Each virus is controlled
by its behavioral program. It defines its actions according to its internal
state and its local environment, making the link between sensors and actuators.
Figure 10 shows the tree representation of such a program. Each node corresponds
to a program instruction. Some nodes have sub-nodes, which correspond to
sub-programs. At every step in the simulation process, each virus interprets
its behavioral program and performs corresponding actions. The first node of
the tree (the root) is interpreted first, and sub-nodes are then interpreted
recursively.
|
Figure 12: Example of behavioral program.
Figure
10's program is interpreted as follows: if vital energy of the virus is high,
then it divides; otherwise it consumes red resources and goes to areas where
red resources are available. This program will perform well on a picture
containing red color, and poorly on pictures where red color is absent.
This example shows two
categories of instructions: some instructions, such as 'Seq' and 'If Energy
High' give the tree its structure.
"Seq"
is used to interpret a sequence of instructions. Each sub node is interpreted
one after the other.
Conditional
instructions such as "if High Energy" evaluate a condition,
interpreting the 1st subtree it the condition is met, and the second one, if
not. In Fig. 10, if the viral energy E of the virus is above a threshold (60%
of the maximum energy) then the first sub program will be executed.
Other
instructions located on the leaves of the tree correspond to actions performed
by the virus.
Some action instructions
alter the amount of resources in the environment. These include
"EatR", "EatG" and "EatB" - which are used to
consume resources (respectively R, G and B). The instruction "Dark"
causes the consumption of three resources simultaneously. Fig. 13f shows the
results obtained with the "invert" instruction. In this latter case,
the attack will never end, as the statement "invert" produces
resources.
By
default, viruses move randomly, but some instructions can attract the virus in
some directions. "FollowR", "FollowG", "FollowB"
for example, guides the virus to the closest cell that contains the highest
amount of one of the three resources. The command "FollowEdge makes the
virus follow the lines in the host.
Finally,
the duplication instruction produces a copy of the virus. This copy inherits
the genetic program of the initial virus (the parent). The energy of the parent
virus is divided equally between the two viruses (Fig. 11). Only one
duplication can be performed by a virus at each simulation step, even when
multiple occurrences of duplication instruction were encountered during the
program's interpretation.
These instructions
altering pixels color were the first one introduced, many other were added
later.
|
|
Figure 13: Different examples of filters obtained
by different combinational instructions in behavioral programs of viruses,
applied on host (a).
|
Figure 14: The energy of
the parent virus is divided equally between the two viruses during duplication.
We
have seen that the survival of a virus depends on the amount of energy
dissipated at every step of the simulation, but also (and especially) its
ability to extract energy from its environment. If a program is sufficiently
adapted, the virus will have the ability to remain active as long as resources
are sufficient. In the example given in Fig. 10, the virus will reproduce by
creating a copy of itself if it has enough energy. Otherwise, it will seek to
absorb energy by reducing the amount of the resource R where it is located,
while moving to the pixel neighbor who has the most of this R resource. The
behavioral program is suitable for images containing much red.
So far I have described
the basics of the simulation model implemented in the Computer Virus Project.
At
this step, it is possible to write behavioral programs by hand and observe
the resulting attack on various images/hosts. But designing such programs is a
laborious and tricky task, as it involves creating a program for each virus,
and may be reconsidered for each image/host. A better way to fully explore the
possibilities of the simulation model is to generate these programs
automatically.
Genetic
programming [Koza, 1992] is an optimization method based on genetic algorithms
[Goldberg, 1989] for automatically writing functions or programs by means of
evolutionary processes starting from a population of random programs. To avoid
the necessity of having to write behavioral programs, and for more diversity within
the population of viruses, I therefore used this automatic writing scheme to
bring out behavioral programs tailored to the survival of the virus.
Among
the viruses produced at random, some will disappear quickly, while others will
manage to stay alive and to reproduce. To explore other behaviors (in addition
to those obtained the random generation of the initial population) a mutation
operator is applied during duplication. It involves replacing a node chosen
randomly by a new subroutine. (Fig. 12)
|
Figure 15: Mutation operation.
This mechanism allows
the emergence of new behaviors during an attack, on the different regions of
the image. Among adapted viruses, some are able to execute duplications, which
will have the effect of producing new viruses that have a high probability of
being also adapted to their environment, and sometime superior to its parent,
thanks to genetic operators. As a consequence, the number of these viruses may
grow exponentially, resulting in a very large population. Therefore, a limit on
the number of viruses had to be introduced to avoid slow downs in real time
simulations.
Above I have outlined
the software architecture governing the simulation bases of Joseph Nechvatals Computer
Virus Project.
This software permits the exploration of complex dynamics while adhering to
Nechvatals specific aesthetic demands. It creates a balance
between art and complexity: on one hand it uses the constraints of order, and
on the other hand, it leaves a part of the process to take place at random.
The
project's aesthetic function is achieved by writing a dynamic mechanism for
automatically evolving the code of the Computer Virus Project through the
establishment of an artificial selection force, typical of genetic algorithms.
This inter-twinning activity is itself evident of Nechvatal's theory of the
viractual.
[Adami, 1998] |
C. Adami. Introduction to Artificial Life, Springer Verlag,
1998 |
[Bowcott, 1989] |
P. Bowcott, Cellular automation as a means
of high level compositional control of granular synthesis, Proceedings of
International computer music conference. 1989, San Francisco. ICMA. |
[Couchot, 2007] |
Edmond Couchot, Des Images, du temps
et des machines,
dit Actes Sud, 2007, pp. 263264 |
[Ferber, 1995] |
J. Ferber. Les Systmes Multi-Agents, Inter
Edition, Paris, 1995. |
[Gardner, 1970] |
M. Gardner. The Fantastic Combinations of
John Conways Game of Life, in Scientific American Vol. 223:4, 1970, pp. 120-123. |
[Goldberg, 1989] |
D.E. Goldberg, Genetic Algorithms in
Search, Optimization and Machine Learning. Addislon-wesley, 1989. |
[Koza, 1992] |
J.R. Koza, Genetic Programming: On the
Programming of computers by Means of Natural Selection. MIT Press, 1992. |
[Langton, 1989] |
C. Langton. Artificial Life : Proceedings
of the 1st Workshop on the Synthesis ant the Simulation of Living Systems 1987, C.Langton Ed.,
Addison-Wesley, 1989 |
[Meyer, 1997] |
J.-A. Meyer. From natural to artificial
life: Biomimetic mechanisms in animat designs. Robotics and
Autonomous Systems, 22:3 -21, 1997. |
[Moulon, 2004] |
Dominique Moulon, L'art numrique:
spectateur-acteur et vie artificielle, Les images numriques 47-48, 2004, pp.
124125 |
[Nechvatal, 2006] |
Joseph Nechvatal, Viral Symphony, Institute of
Electronic Arts, 2006. http://www.archive.org/details/ViralSymphony |
[Nechvatal, 2011] |
Joseph Nechvatal, Immersion Into Noise, Open Humanities Press, University of Michigan, 2011. p.
244 |
[Popper, 2007] |
Frank Popper, From Technological to
Virtual Art,
MIT Press, pp.120123 |
[Weisbuch, 1989] |
G. Weisbuch. Introduction la dynamique
des systmes complexes InterEditions/CNRS, 1989 |