"SIDEBURNSPACE: A Corpus-Centric View of Inter/Artiface Design and Creation"
Much has been made of the leaving-behind of the body by cyberspace voyagers. The ability to fly, perform feats normally physically impossible, and generally to leave behind the "meat" and indulge the so-called "pure intellect" have all been mentioned in various writings about virtual experience. Yet, paradoxically, all these activities wind up affecting the meat in very direct ways.
Our paper presents a view of the human body as the ultimate receptor of cyberspatial activities and examines how this viewpoint shapes the creation of principles for creating what we term "Artifaces" - computer interfaces designed specifically to convey artistic meaning above and beyond a Cartesian world of tessellated polygons.
We start with the assertions that the mind/body duality does not exist and that Art is an action of experiencing information transfer from the outside world, through the mind, to the body. We discuss our view that the mind is the part of the body which interfaces to the world and that the body is at the top of the sensory data abstraction hierarchy; we consider the notion of the body as self-modifying software code. We examine the idea of a "Science of Art" i.e. can one discuss Art in meaningful information-theoretic terms and draw useful conclusions about interface design which lead towards a theoretical foundation for the creation of integrated-media art works? We consider the concept of ambiguity as a form of data compression and reach the conclusion that a massive rebirth of Poesie is needed to aid researchers and designers of virtual environments cope with non-linear representational situations. We end by discussing some guidelines for designing cyberspatial environments based on the ideas developed in this paper.
1.0 Introduction
Our goal in this paper is to set out for discussion our ideas about a theory of cyberspatial aesthetics. This theory is very directly based on current scientific views about the human mind and body and, in turn, "feeds back", having direct application in the world of Science. We shall show how these ideas developed, their relationship to science, and some design rules resulting from their application.
2.0 Intersection; Simultanaeity
Designing a virtual environment, whether for data visualization tasks or solely for Art's sake has more to do with classic notions about painting or sculpture than it does with simply ordering collections of data along lines or paths or programming direct one-to-one mappings of data to colour, position, et al. As trite as it may sound, we are presently in an age where the worlds of Art and Science are once again merging after being apart for several hundred years. The artist can no longer talk to his scientist/programmer to get the job done any more than he can talk to his painter and have a picture painted. To achieve the best art or science, the artist must be the scientist/programmer and the scientist/programmer must be the artist; each must have enough mastery of the techniques of the other so that the line between vision and visualization can be seamless.
There are two great reunifications going on presently in the world of intellectual thought. The first is the reunification of body and mind. It is beginning to look as if what has long been regarded as one of Western Thought's primary dualisms may actually be a single entity after all. The second, reconnecting what was divided several hundred years ago, is the reunification of Art and Science. Once again, skillful practitioners in each area are starting to walk and talk like one and the same. There are some important relationships between these two "new" unities. These relationships are the subject of this paper.
3.0 The Matter of Mind; Biologic Foundations of Art Theory
At least since Plato wrote his philosophical treatise "The Republic" , humans have wondered about what is real and what is imagined, about the difference between what they perceive to be outside of them and what seems solely internal. Sometimes it seems that certain things/events can be both - anyone who has ever been in Love has experienced some feelings of simultanaeity with their lover - the feeling that he/she is me; feeling it is not clear where oneself ends and where the other person begins. Bound up with all of this is the ancient Greek idea of the duality of mind and body, the idea that somehow, the two can be considered, treated, and trained independently even though they are related to each other and influence each other. It seems fairly clear what the body is - we each have one and are generally aware of where it starts and where it ends; its limitations and dimensions. As for the mind, its location is less clear. To be sure, we know that the brain is the organ of the body where conscious thought resides and where a great deal of real-time regulatory activity is computed such as the firing of the heart's vagus nerve to trigger heart beats, etc. - but we still aren't really sure where the mind fits in relation to the brain. In traditional thought, there is a clear demarcation between mind and body based on the ancient Greek traditions.
We take issue with this idea. While the brain is undoubtedly the organ in which conscious thought occurs, we must acknowledge the fact that modern day research is discovering that mind and body are very intimately connected through a wide variety of co-modifying feedback paths. A full survey of this research is beyond the scope of this paper but to provide some partial evidence in favour of our assertion that mind and body interpenetrate each other to a large degree, we can consider the research being done into the mechanisms behind the functioning of acupuncture therapy.
There seems little doubt that acupuncture and its related therapies work and can be very useful therapeutic tools. During the last few decades, researchers have begun to establish a theoretical basis for explaining why applying pressure to certain areas of the body causes relief of pain, moderation of blood pressure, and other desirable effects. In his book "Scientific Aspects of Acupuncture", Felix Mann writes:
"Acupuncture is based on the fact that stimulating the skin has an effect on the internal organs and on other parts of the body, a relatively simple reflex whose therapeutic application is largely ignored in the West. Various experiments demonstrate the existence of this cutaneo-visceral reflex."
However, the more one studies this "relatively simple reflex", the more one learns about the interconnection of feedback paths between the mind, brain, and nerve pathways. It is possible to have stimulation in one area affect another region physically far from the stimulation. It is also possible to have "referring" of sensation from one area to another, sometimes along the same nerve path but sometimes resulting from a complex interaction between stimulatory nerve signals and responses mediated by the brain's ganglia. This process is ultimately complex enough that Mr. Mann writes, near the end of his book: "I think quite a substantial part of the success of acupuncture lies somewhere between the mental and the physical aspects: if there is only the mental treatment it does not help; if there is only physical treatment it likewise fails...It is this peculiar combination of the physical stimulus of a needle prick, together with psychological factors, which seems to produce the results one sees in acupuncture. This is of course, a largely new concept in medicine, a concept which should be considered when designing research projects. A rose is a physical object, but a beautiful rose is a nonscientific concept as it is a combination of something physical and non-physical."
This last is an extremely important statement and we shall return to it later in our discussion. For now it is enough to note that stimulating the body directly affects the brain, which affects the mind which, in turn, also affects the brain (causing other neurons to fire) which affects the body again, coming completely around in our circle of cause and effect.
Given these interactions it becomes less clear just where one's brain/mind ends and the body begins. Is the spinal cord part of the brain? What about nerve endings connected to the acupuncture points? Is there non-conscious information processing within our bodies decentralized from the brain that ultimately communicates with it? Perhaps this occurs in various cells or organs of our bodies or maybe in all of them? If this is the case, where can the mind be said to reside? In recent years it has become clear that human consciousness is only a small part of the information processing that goes on within a human body. Consciousness has been likened to a debug trace in a computer - something that is invoked only to solve a problem solvable by no other means and, when invoked, something which proceeds very slowly and labouriously relative to the normal speed of the computer's software. While human consciousness may be created by the firing of the brain's neurons, it is far from clear that this is the sole location of the human mind.
We can thus postulate a situation like that depicted in Figure 1 where mind and body are intertwined to some degree with only a relatively small portion of the information processing bandwidth reserved for the processes of conscious thought.
3.1 Data Flow; Rivers of Experience Carving the Beds that Hold Them Upright
In computer terms, human beings have what would be described as a "data flow architecture". We process and benefit from a great deal of information which is abstracted by our senses on a moment-to-moment basis but we store very little of that information for later use. Most of our data processing is used for real-time decision making and is jettisoned a few moments later to make way for new incoming information.
In a similar manner to the diagram in Figure 1, we can establish a hierarchy of information processing such as that depicted in Figure 2 where the senses gather information from the world outside of us, preprocess it to abstract meaningful information, communicate at least some of this information to the brain which in turn communicates it to the mind which, in turn, communicates back to the brain and possibly directly back to other cells/organs in the body. This is what we will refer to as the "Data Flow of Experience". This view opposes previous hierarchies which, typically, have information flowing from the world, through the senses, to the mind. We hypothesize that this is much too simplistic a view. The body/mind is the ultimate receptor of this heavily abstracted information and this has some interesting implications.
Perhaps the body, being the ultimate repository of the individual's genetic code, is the ultimate long term recipient of whatever information processing is performed by the mind and, at least in this sense, should be regarded as being at the top of the data abstraction hierarchy. There is little doubt the mind can alter the body in the sense of aiding or delaying the onset of disease. Perhaps the mind can alter the coding. Can we reengineer ourselves to some limited degree? Is the mind/body an extremely complex system of self-modifying software/firmware/hardware? Despite the discrediting of Lamarckian biologic theories, more research is needed into the question of self-modification. Our DNA does change as we go through life but it is not clear what the significance of these changes are.
4.0 Art, Science, Duality, Myth: The Science of Art; The Art of Science
Stipulating our diagram of the Data Flow of Experience and its biologic foundations, what are some of the implications for someone encountering a work of Art? The philosopher John Dewey wrote a book entitled "Art as Experience" . His views on this topic are relevant here, although we are inclined to frame the conclusions within a biological/computational world-view: Art is not a thing, it is an information processing event, like all of what we term "experience". Humans are information processing survival machines, evolved to serve the needs of our genes and their chaos-based, firmware-scripted, recipes. It is never possible to directly perceive what is outside of us; all our senses are abstractions constructed out of incoming data flows. Previously, in a book on Art and Technology, we wrote a description of such abstraction that seems relevant: "Patterns recognized at low levels are linked and correlated to other patterns to make still more abstract structures until finally, at a conscious level, we see a bouquet of daisies or hear twelfth-century choral music".
Figure 3 depicts this process in relation to art "objects": our senses abstract information about the art "objects" that are outside us and transfer this information into our minds which interpret it based on our context of prior knowledge.
Given that one can think of the experience of Art as information processing, one can posit the possibility of a "Science of Art" based on the physiologic/psychologic rules which govern this information processing event. At least in principle it seems possible to formulate guidelines, algorithms, and heuristics to aid the creator of the artwork to attain his/her desired results.
If one thinks of Art as an experience of information transfer or communication, it seems evident that the structural devices long discussed in traditional art theories are the source of the information. For example, what distinguishes music from noise is the order and structure that is imposed on it. From the choice of timbres to the choices of rhythm to the volumes and articulations of the individual notes - all of this imposition of structure is what defines the piece and gives it its musicality. This is most clear when listening to compositions involving so-called "found sounds" such as machinery, nature sounds, or speech sampled from other contexts. These works can still be very musical and their musicality lies in the structuring of the found sounds rather than the sounds themselves which are, per se, not musical.
5.0 Ambiguous Data; Greater Meaning
As an example of some useful implications stemming from an information-processing-based theory of Art, we can put forward the concept of "sensual bandwidth". Each of our senses can perceive limited frequency ranges in a number of different dimensions. For example, our ability to distinguish separate images is lost if the frequency of presentation of these images exceeds 20 or 30 Hz. For frequencies above this limit, the time-constant of our eyesight and retinal abstraction precludes conscious recognition of these separate images and we perceive only a flow of animated movement. Any artist working with animated images needs to be aware of this fact in order to properly present his/her materials.
Similarly with the sense of hearing, psychoacoustics plays an important role in the ability of people to localize sounds in a perceived 3D space. The volume of the sound arriving at each ear, the phase difference between the sound at each ear over a range of frequencies, plus the physical structure of the ear itself (i.e. its "head-related transfer function") determine one's ability of perceive location of sound sources. Any integrated-media artist working with ideas of sound localization must come to know and understand some of these basic principles in order to work effectively - much as previous generations needed to master the understanding of perspective and theories of colour mixing.
Continuing in this direction, when considering the idea of art as information transfer, one is tempted to experiment with the application of concepts and theorems from the telecommunications world much as previous generations of artists worked with ideas about colour theory, perception, and geometry.
Of particular interest in this situation is that what in the scientific world would be regarded as a mistake or a problem can, in the artistic world of human-to-human communication, actually be an asset. For example, traditional means of encoding information have relied on creating coding schemes which were bereft of ambiguity. A particular code within a data modulation scheme represented one thing and one thing only. Relatively recently, telecom researchers have realized that one can effect great economies of data compression by indicating a context in which the data is to be received. In some ways, this recognition parallels the increasing recognition of the importance of context in the world of post-modern art theory. For post-modern theorists and critics, the context in which one experiences a work of art, be it writing, painting, dance, music, or theatre, is of great importance. It is the context of one's prior knowledge and experience plus the setting in which the work is presented that strongly influences what one feels. If we choose symbols and contexts properly, it is possible to achieve extensive amounts of data compression by making each symbol mean several different things simultaneously depending on in which context they are interpreted.
In the artistic world, a very good example of this compression is poetry. A skilled poet is a master of ambiguity. It serves as a crucial device for packing meanings together in a much denser form than ordinary prose would allow. In Art, the utility of ambiguity and simultanaeity of meanings is undeniable. Part of the pleasure of appreciation of a work is interpreting the relationships one discovers, whether consciously or unconsciously. Viewed from an information theoretic standpoint, the use of ambiguity in Art is a form of high-ratio data compression which allows a great deal of meaning to be conveyed in a single phrase, image, shape, or posture. Based on this, we can appreciate ambiguity as a crucially important theoretical construct without which it would be very difficult to communicate those things that need to be communicated. Ambiguity is not about being vague but rather about saying a lot of things otherwise very difficult to express. This concept has some interesting implications for designers of virtual environments - artistic and non-artistic alike.
6.0 Explicit Postures; Implicit Rendering: Sub-Texts in Data Space
There is currently an urgent need for a solid theoretical basis surrounding the creation of art works involving what we have come to term "integrated-media" . Integrated-media, as opposed to "multi-media", involves the bringing together of several disparate media which, taken together, have an effect greater than the mere sum of their parts. Integrated-media is not a new thing - opera and some ballets can be considered to be early forms of it. What is new, however, is the ease with which this sort of thing can now be accomplished. This difference can be likened to that between using a flute to play multiple notes simultaneously - something which is possible after a great deal of practice within certain note ranges - and playing chords using a piano. In the case of the flute, it is not the normal idiom of the instrument whereas for the piano, it is something it was designed to do and which it excels at. "Instruments" which were specifically designed with the task of media integration in mind are just now coming into existence. One of them, invented by us, the GAMS (Gesture And Media System) allows a user to coordinate electronic video, music, and lighting using a 3D tracking device which maps the gestures of one's hand or the positions of one's body to control and synchronize these multiple media. As this genre of instruments are perfected in the coming years, we shall perhaps see a great increase in expressive capacity of performers, and great increases in communications ability between those using similar systems for more mundane activities such as collaborative work, negotiation and conflict resolution, etc.
An equally urgent need exists for such a solid theoretical foundation in the area of data visualization. The need for data visualization has always existed in the world of science but has recently assumed new importance in light of advances in virtual reality technology that open many new possibilities. In fact, there are so many possibilities open that it is often very difficult to tell which are worth pursuing for a given problem.
It seems as if integrated-media art and data visualization are basically two views of the same object. If Art and Science are once again very intimately related, this should hardly be surprising. However, there are some corollaries that are not entirely obvious. Our art-theoretic notions of "Art as information processing" and "Ambiguity as data compression" have importance vis a vis scientific visualization problems and, particularly, design of virtual scientific environments. Designers of virtual visualization environments often encounter difficulty in finding the most useful ways to represent data so that humans examining the data can recognize information of interest. It is our contention that much of this problem comes from the use of Cartesian coordinates with their concomitant orthogonal dimensions of measurement. While Cartesian space is very useful for some applications, it is not an appropriate tool for other non-linear data representation tasks, much the same way that written language is not an appropriate tool for representing a symphony.
What seems more appropriate than geometry is poetry. Consider a poetry of data spaces; to be able to represent multiple meanings and relationships by the interrelations and positions of a few abstracted symbols chosen so that their contexts (i.e. their relationships amongst themselves) provide as much information as their Cartesian locations. This seems more efficacious. This sort of approach is much more suited to the "new science" approach of considering the actions of entire systems in chaos-theoretic terms rather than isolating individual parts with the hope of thereby understanding the functioning of the whole. It is difficult enough to understand even comparatively simple machines such as internal combustion engines by solely studying the actions of their individual parts. It seems close to impossible to understand the activities of complex machines such as human bodies without developing extremely poetic, meaningful, ways of abstracting and representing the functionality of the whole system with its multiplicity of feedback paths. Perhaps the best creators of tomorrow's cyberspatial environments will be a reborn and rearmed force of Data Poets, impassioned by working with hyper-dimensional, virtual, art objects.
7.0 Artifaces: Design, Art, Madness, Structure; A Uno Mismo
Is our theoretical discussion simply that or have there been any practical results? Over the past decade we (and those with whom we have collaborated) have worked with art and electronic media in a variety of ways. Projects have ranged from "technological theatre" to "walk-in databases" to "hyper-dimensional art objects". Looking back, we can see a development of a style of work which, at this point, is distinct enough that we have invented a name for it. We call these types of works "Artifaces" and they differ from normal portrayals of data in a number of ways.
In general terms, artifaces are virtual, integrated-media, environments where users can explore virtual art/science "objects". We have developed a number of guiding principles of artiface design which are directly related to our previously expressed theoretic ideas about Art, information processing, and the human mind/body. The techniques developed for these artifaces that have shaped the development of the aesthetic ideas just mentioned. In their turn, these ideas have influenced practice, providing indications and directions in the creation of these artistic projects. Summarized, these principles are given below:
a) One's resultant whole must always be greater than the sum of the parts. The resultant affect of all of the media taken together must feed back and amplify/modify each of them, engendering a great deal of structure (i.e. information content) based on the relationships between the different modes of expression.
b) One must pay careful attention to the particular expressive qualities of the individual media. Understanding the human mind/body's perceptive limits is essential to constructing well-rendered works.
c) Choose your symbolic language wisely. With mixed media available, many choices are available. One must choose symbols and symbolic grammars that complement what needs to be expressed.
d) Architectural elegance is an important part. A sort of minimalist "Occam's Razor" of expression is generally preferred over romantic excess in order to manage the complexity of what is generated.
e) Structure good, randomness bad. Since we are concerned with information transfer, structure is to be preferred (in most cases) to random, aleatoric, events such as those occurring in some multi-media art pieces from the 1960's and 70's. Unless the meaning being imparted is a meaning of randomness, aleatory activity simply diminishes the communications bandwidth - like noise on a telephone line.
f) Become one with your data. Immersive experience is qualitatively different from simply viewing something on a screen. As with Dr. Einstein and his famous thought experiments, a great deal of insight can be gained by placing your body and mind so that they figuratively and literally interpenetrate the data sets being presented. Most successful artifaces give users the feeling of sharing their own private 3D physical space with fields of 3D data flux.
A properly constructed artiface provides a very intense, dense, experience - usually of many things simultaneously overlaid and related to each other as if one were receiving a highly compressed data file and decompressing it in real time. It is powerful, gripping, emotional, and insight provoking in a completely non-linear, non-Cartesian way. One can travel from Rome to Venice and not pass through Florence; one can experience an epiphany.
We now return again to Felix Mann's statement, quoted previously: "A rose is a physical object, but a beautiful rose is a nonscientific concept as it is a combination of something physical and non-physical." Is a beautiful rose any more nonscientific than a neurally-based theory of acupuncture? Both are based on interactions between body and mind, mind and body. Both have real measurable effects on the physical world we inhabit. What about our theory of cyberspatial aesthetics? According to it, there is no reason why a beautiful rose should not be an eminently scientific phenomenon. A corollary is that the creation of artifaces should be an important contribution toward the evolution of science as well as art. Perhaps not everything can be studied in situ, perhaps to obtain maximum insight into phenomena, one needs a Poet's bandwidth.
8.0 Conclusion; Beginning
We have put forward reasoned views, based on about ten years of serious practical experience working and thinking within the world of interactive electronic art, science, and human interface design. Some of these views have a philosophical basis and, for the sake of clarity, we have explained them at some length in this context. However, the reader should not consider this paper as a discussion of scientific, logical-positivist, "fact" but rather as a provocation - a manifesto; an invitation to think about things in a somewhat different way and see where these different thoughts may lead. Experiments are the next step after this - what concerned us here was still imagination, possibilities, hypotheses, and conjecture. These are the first steps in any successful scientific or artistic venture and we are certainly at the beginning, not at the end, of our thinking about matters such as these. |
