Art and the Semiotics of Images: Three Questions About Visual Meaning

This little survey of graphic signification will draw on painting, photography, and digital graphics, there being no sharp line distinguishing the latter two and all three appearing via reproductions, on the Web. To be sure, some (the "post photography" folks like William J. Mitchell) have argued that the case is quite altered with digitally manipulated images which give up the claim or even appearance of representing some part of the material world, and J. David Bolter and Richard A. Grusin argue for a line of development in Western graphic culture toward pure presentation (unmediated by a representer artist) which culminates in contemporary Net graphics. There is some point to this--digital artists take their images where they find them, http://muse.jhu.edu/journals/con figurations/v004/4.3bolter.html whether in a box of old photographs, scans of objects sitting on top of the scanner, stock photos, their browsers' caches--and we may imagine the gaze of digital taking/making as directed not through a viewfinder or past an easel, but at a monitor screen. But just as we imagine ourselves in scenes of seeing (though at one remove from the photographer's or painter's seeing) we can continue to do so at two removes, perhaps more.

To see how much energy and interest can be generated from splitting of placard and image, consider the "Statuary" series by Jacqueline Hayden on www.zonezero.com; the first one of 10 is here in the margin. These pictures are presented one by one in a highlighted oval (museum lighting) against a rich dark maroon field; each comes with a little placard button that when pressed opens a window, as here, with the placard. (The picture also can be enlarged.) The placard text in each case seems utterly unaware of the modification Hayden has made to the antique torso and thus enacts the obliviousness of the Western fine arts tradition to the look of bodies past the age of fifty. The images are rather small platinum prints done with great care and fine finish, and the exhibition is not a joke or mockery of age by youth or of museum culture by the realities of the aging body or preposterous vanity of those past their physical prime. These tensions are evoked but not resolved (since images don't say anything); rather the gaze they call forth is a compassionate one seeking and finding a certain kind of beauty.

But that is getting ahead of the story, which begins with the standard arrangement whereby text may discreetly assist us in getting the image to float in the right directions.

To begin with the simple determining function of text, compare the following two images from an exhibit catalog from which superimposed words have been removed so that you can experience their "float" without words; you can then add the words by clicking the "Add Text" button. This first is an abundant display of supermarket prepared food, and one could imagine several possible lines of intention (they are all Kraft food products, they run heavily to cheese and preserves, they are a riot of color, shape and detail that severely challenges computer resolution, they are unbounded in all directions) but (you've clicked it already, haven't you?) the words (enlarged for legibility) anchor the display to a very conventional dismissal of "American processed food."

In this second graphic, the words"Post Human" seem to point to some kind of future world or tendency; it echoes the other "posts" --certainly poststructualism is post humanist--but what part of the "post human" world are we contemplating, and with what attitude? The image is also a bit hard to make out because of the angles; the woman may be partially submerged (but upside down?) and the light is no help either. Is this some kind of cryosleep in zero gravity? There are a lot of things that might be called post human . . . .

There are better clues available than the words on the image: this graphic, like the preceding one, comes from an exhibition catalog for a show sponsored by the Deste Foundation for Contemporary Art in Athens, USA (Ohio?) in 1990. Called "Artificial Nature," the catalog pursues the phrase post human through many pictures of the artificial replacing, altering, and glossing over traditional human limits. It even provides another view of the striped lady, who apparently is lying in a few inches of water at the bottom of a whirlpool bath. Clearly the text does not close down interpretation here, or even give it much assistance.

In these first, rather simple cases, one has the impression that the image came first, and the words were added to interpret what was already there. When we speak of illustration, however, we are usually thinking of adding an image to an already existing text, and this relation too would seem to anchor the image. At times, however, the image seems to interpret the text quite broadly or even undermine it. Consider for example the following work from Wired magazine.

Each issue of Wired includes a 4 page (2 double) spread before the Contents page which cites a line or two from a featured article later in the magazine and functions as a teaser (or highly graphic "abstract") for the article. The sentence to be quoted and graphicked is usually long enough to support the two stage setup (double page one followed by double page two), as for example additive or contrastive pairings, or cause and effect.

The "Data" set of pages is built on lines from an article about a Seattle company that recovers old email, even deleted email. The lines seem rewritten over themselves. The line in "Data 1," "Backups containing millions of email messages are the digital equivalent of formaldehyde," offers a simile which is the basis of the green liquid look with its bit of magnified mosquito or crane fly in it.

Turning the page, the color changes to fiery red and hotter yellow, to a lake of fire or furnace with old disks, a key, some more cranefly wing, numbers and labels. The text says explicates the simile: "a medium where nothing decays." The fire could be taken as what puts companies in the hot seat, but it can also attract traditional connotations of Hell, the place where nothing is forgotten or forgiven. For me, seeing a sort of doll's face or mask in the fire invites this human association with the digital eternally unforgotten. This I should add carries the significance of the graphic far into a spiritual dimension that has little to do with the content of the article, which mostly turns on CYA for corporations. If the reader turns to the indicated page and begins to read the article, she likely will be disappointed by the absence of metaphysical grandeur. Which is to say that the artist takes the lines out of context and composes a visual meditation upon them; the graphic, however, is still an illustration of what the words propose.

Usually Wired's graphic serves the bit of quoted text; the next example is unusual in its relation to the quoted words. Gary Wolf's featured article in June 1999 Wired profiles Sir John Templeton and his investments in religion, specifically in showing that good religion is good business. The two double-page spread is built on lines from one of Templeton's operatives and is neither explicitly endorsed nor derided in the text of the piece. In context, it both celebrates the triumph of world capitalism and outlines the next area for it to annex, namely the realm of moral values.

On the first two pages, the two spray cleanser containers on the right margin seem to express the result of the end of the struggle for markets. Photographed in hard focus and bright light against dead black with nothing but the text to support them, they illustrate what Kress and van Leeuwen call "hyperreal" modality, which in this case links to sensual pleasure focussing on the consumer object typical of food and drink adverts (p. 169). (see also John Berger, Ways of Seeing , pp. 140-141) When we match these pages with their text declaring the settling of the fundamental battle over the free market, it is hard to avoid the effect of severe understatement (or underrepresentation) amounting to ironic deflation ("the late consumer capitalist market economy as epitomized by the choice of cleansers now dominates the scene"--with Bruce Springsteen's "57 channels and nothin' on" in the background). In the second pair of pages, the two packs of cigarettes (on sale in Japan, I hear) fill the position of the cleansers and would appear to represent the not yet realized victory of the market in the sphere of morals. (And here they bear their own texts ("Peace" and "Hope") which push even beyond "Fantastic" and "Fabulous" as Orwellian perversions of the words.) The graphics thus mock the words from Templeton's agent by reducing the grand phrases to their practical consequences in daily life: "capitalist marketing of morality would offer us immoral commodity choices packaged with positive words"--how backward can people be to withhold assent! In this display from Wired , graphics comes as close as it can to making a counter statement.

This degree of distance between graphic and text is typical of political cartoons and demonstration placards. The graphic style, however, is usually not realistic but exaggerated and parodic, which is to say of lower (less realistic) modality (as if: "this is the way the world would be if these views were real"--conditional if not irrealis, one might say, not indicative). John Heartfield's "The Butter is Gone" (1935) is a famous exemplar. The text is a quotation from a speech of Hermann Göring's, "Bronze has always made a nation strong; butter and fat at best make a people plump." And so, the butter being gone, the family is dining on metal. Although the graphic is a montage of photos, the swastika wallpaper and general absurdity of the actions lower the modality.

Victor Burgin developed exactly the opposite relation of text and graphic in his political work of the 1970s: here the image is "appropriated" from an advert and the text written on it is social critique or theory. One quite well-known one ("Possession") was done at the time of an exhibit of contemporary artists in Newcastle. The Arts Council asked for some publicity posters, and Burgin responded with "Possession" 200 copies of which were pasted up on the streets of Newcastle. Burgin intended for the diametrical opposition of text and image to catch the gaze and trigger thought. Follow-up research indicated that not many passersby remembered what the posters said, much less what they implied. For a few more years, Burgin continued to exhibit large photographs with substantial text (often over 100 words) at odds in various ways with the image. The effect is sometimes a rather professorial and preachy enumeration of the "contradictions" of late capitalist consumer society, but at other times it is more suggestive, enigmatic, or tensely ironic, as when he quotes Foucault's description of the Panopticon in a picture of a Berlin peep show with circular stage.

In "Life Demands a Little Give and Take" text and image are in the opposite relation to "Possession," namely, the text is from the commercial advert and the image is from the street. I am not sure how readily the image would make sense with no context, but in a collection of pictures that deal with the contradiction between manipulative, obfuscating culture (ideology) and real material conditions, it is not hard to see this picture as an exposure of the racist overtones of pale=beautiful. That is, we have ordinary people waiting for a bus on a nondescript street corner in modern Britain, among whom the camera's gaze falls on a woman who is distinctly not pale and who does not qualify as one of the targeted audience of the fashion magazine spiel.

Burgin was certainly not alone making text+image displays in the 1970s; much conceptualist art would fall under this rubric. Keith Arnatt, for example, exhibited a similar display, this time with a philosophic theme. Tony Godfrey, who cites this work, says, "It is uncertain whether the photograph acted as a critique of the philosophy or was merely the pretext for quoting it" ( Conceptual Art, p, 172), which is a way of saying he is not sure whether the image illustrates or undermines the text. He finds the text unmemorable and finally unnecessary, saying that it serves "ultimately only to underline what is implicit." In a sense you can always say that, even when you don't say what is implicit, but for me it does highlight certain themes inherent in the situation ("the contradictoriness of all self-authenticating gestures," "uncertainty (a la Austin) of what accusation is being denied," "the making of such a photograph declaring oneself to be a real artist--is it real art?" "is the art more real with the accompanying text?" The photo might in itself cast the viewer into its reflexive abyss, but the text certainly does help. This is art that makes you think.

Not all Conceptualist artists played the big discourses of politics and philosophy off against images; some, like Karen Knorr played bits of self description or art cliches off against exquisitely photographed interiors to engage the viewer in reflection. Berger's and Richon's own contributions to the collection are even more oblique in the relation of text to image, as if the textual material is slipping out of alignment with the visual. The texts certainly do not dominate over these images, and this may partly have to do with the extreme degree of deliberation and high degree of technical finish the images exhibit. Without the texts, however, I am not sure we would have much of a clue as to what context to place the images in (they do appear in sets in these publications.)

But, so the argument goes, are we not being too hard on ourselves? Art is not a quarter a minute peep show, and anyway, aren't they supposed to know to pull down the blinds (and put some up!)? If they don't do it, and wander around at night naked with the lights on, are they not exhibiting their nakedness? Are they not perhaps even posing for us? Thus we come to Michael Fried's rich and celebrated discussions of absorption: the idea gained some currency in France from the time of Diderot that the viewer's pleasure in a painting with human figures, its ability to wholely engross the viewer, depends on the assumption that the subjects are not posing for us; rather, they are absorbed in whatever they are about, and this absorption is a condition for our absorption in viewing them. Since painters have always used models, this absorption effect is essentially an illusion, the conjuring of which posed a continuing problem for painters which they dealt with in various ways.

Among the most inventive were Gustav Courbet, Fried argues, and Édouard Manet, whose most famous paintings are radical and modern precisely because they overturn the whole absorption game; Olympia and his famous picnicker are naked and do not merely vaguely look toward the viewer (is this "demand" or "offer"?); rather, they stare back in a fashion usually felt to be challenging or socially amused. Suddenly the old, stable arrangement of viewer and object is up for renegotiation, and has never been settled since. Indeed, a great deal of art photography and commentary in the last thirty years has worked over and upon this theme.

One way for a photographer to work these themes is to revisit the key paintings, especially those of Manet, and remake them (so we overlap the topic of intertextuality here). Here I will look a some photographs by Jeff Wall, who seems to have set himself the goal of becoming the portrayer of modern life in late twentieth century Vancouver that Manet was in mid-nineteenth century Paris.

Jeff Wall has been working mainly in large transparencies (which indicate a strong liking for the translumination that we are now used to in Net display of graphics). Stereo (here shown as installed/exhibited and also closer up) alludes doubtless to many of the nude ladies of the great oil tradition, and strikingly by way of contrast to Manet's Olympia, who as noted breaks out of the tradition to engage the viewer. Here we have the new absorption of the Walkman which disengages the young man from any sense of being viewed. The couch too contrasts with Olympia's--it is a $50 Salvation Army special with hairoil stains, tattered piping and a nice, prominent stain. He doesn't need anybody or anything as long as he has his head space.

Walkman-induced neoabsorption also caught the eye of Jan Saudek, who gave Velasquez's Rokeby Venus a similar makeover. The Velasquez original fits nicely into this theme of absorption and gaze. In it, Venus turns her back toward us and appears to be entirely absorbed in her own image in the mirror held by Cupid. But wait, if we can see her image, then she cannot: she sees our image, and so, more indirectly and discreetly than Olympia, she gazes back. (Click on the thumbnails--the filiation between the pictures is not evident otherwise.)

Clearly Saudek's take on the irruption of Walkmans into modern life is similar to Wall's: once again, a gaze that existed in the original is absorbed by the black hole of the "personal listening device." The idea of these images spoke so much to Saudek that he did a second "Walkman" version with a classical Narcissus image. Note here the very close attention to replicating the inner and outer fabrics and the position of the feet, which is just different enough to make it clear the whole assembly was photographed anew.

Another of Jeff Wall's depictions of modern life (in particular, modern life in the Pacific Northwest) turns on a remaking of Manet's famous "Dejeuner sur la Herbe" (which keeps peeping throught the moving "reader" slit in the online version of this paper) as the very large transparency "The Storyteller" (229 x 437 cm). Here too we have gatherings in public park spaces, though the setting on the landscaped banks of a freeway overpass is a far cry from the Paris "herb" and the temperature is cooler, judging by the clothes of the figures and their little fire. Clearly, it is Vancouver (Wall taught Art History at Simon Fraser University for many years). The principle point of contact with Manet's Dejeuner is the group of three, most particularly the posture of the man, elbow on knee. Manet's grouping is directly lifted from Marcantonio Raimondi's The Judgment of Paris (--see Fried, é Manet's Modernism , p. 56). But the relations are strikingly different: the three members of the group are engaged in the woman's story, and no one, naked or otherwise, has any awareness of or interest in us. I find this a salutary treatment for those who might yearn to go to Paris and live in the Impressionist period.

(We should perhaps note that Manet's "Dejeuner" is a very strange painting--very hard to make narrative sense of (why is she sitting there nekid, the men clothed, and no one paying the slightest attention, except us?). At least one reader, namely the Barbie parodist Dean Brown, has visually shown another story painted over in the picture as we have it today.

3.2 including the shooter

The set of engagements (and non-engagements) is further enriched when the photographer includes himself or herself in the scene. I am not thinking primarily of Cindy Sherman, who includes herself as the main subject, but of photographers who depict themselves depicting. Such acts require mirrors and break the conventional twining of viewer's and photographer's eyes. That is, the viewer cannot be the "implicit photographer" when she sees the photographer represented behind the camera (assuming it is the camera that took the picture, shooting into a mirror). If she sees the photographer viewing through the taking lens, where is she viewing from? The classical precedents for such pictures are the grand canvases of Velasquez ( Los Meninas may it rest in peace) and Courbet ( The Painter's Studio ), but as paintings, the "viewer as painter" is less compelling. That is, we know that the painter can paint himself into the scene any day he pleases, but the sense of "shared instant of time" is so much stronger that these reflexive pictures are disorienting. The one resolution, I think, is to back the viewer away from imagining himself as interacting in a scene of photographing and promoting a kind of detached analysis (and perhaps admiration) of the artifice--or amusement at what can easily come off as self-deprecating. Jonathan Miller's On Reflection includes a couple of pages (pp. 184-5) of photographer's self-portraits with taking camera; one, by Andreas Kertesz uses a distorting lens and model to suggest the queerness of the situation. But perhaps the most copious and now well-discussed body of such self-portraits is by Helmut Newton.

An introductionary essay by Urs Stahel to Helmut Newton: Selections from his Photographic Work ("Participating without Consequences: Rules and Patterns of Newton's Voyeurism, " pp. 19-30) discusses a number of Newton's pictures of himself at work photographing nudes. Among these is one ("Self Portrait with Wife June and Models," Paris, 1981) upon which Victor Burgin has lavished much semiotic and psychoanalytic attention. (see In/Different Spaces , University of California Press, 1996, cc. 2 and 3). Although Burgin begins with a textbook application of Barthesian semiotic analysis (first denotation--the non-codified description of the scene and then connotation--the cultural codes and associations of raincoats, FM spiked heels, pinup posture, followed by "rhetorical" patterning of antithesis and repetition), he moves toward explication of the feminist psychoanalytic argument of Laura Mulvey's work (and toward personal themes engaged by the picture). What both Burgin and Stahel ignore is Newton's opening up of the scene of the work and the consequences of glamour photography. This is a scene for dramatic imagining: what can the model be thinking as Newton's wife sits watching like a casting director? Is she turning toward him to receive instructions? What can Newton be thinking as he positions people (and make no mistake, they are all positioned) and dons a raincoat? Why does he make himself so short? What exactly might June be thinking? Is this a proper use of the Vogue Paris studio? Who's paying the model? and when we have finished all that, what about the other model? It seems to me this picture works exactly against Stahel's title: it drops the screens and baffles to expose relations that do have consequences--personal and material--that visual eroticism attempts to bracket and conceal.

The last of these pictures thematizing the acts of viewing, making, and seeing is a near contemporary of Newton's "Self Portrait," namely, Jeff Wall's equally wellknown "Picture for Women." Like many other Walls, it has a precursor in Manet, namely "The Bar at the Folies Bergère." This too appears to have a mirror, this time behind the subject, in which her reflection, along with that of a patron, appears. The geometry, as has been noted by a number of critics, does not seem to be quite right: if we are standing more or less directly in front of her (though not meeting her gaze), then it is hard to know where the other customer is located, or else where we are. (One critical cartoon of the times drew the scene up supplying what M. Manet had "forgotten" to put it, namely the figure of the other customer standing to the right, back to our view. In a sketch for the painting, Manet posed the girl looking sharply to her left across the viewer's gaze to the customer.) It is above all the woman's posture that echoes Manet. Here we note a bit of illusionism even in classic realism--it is hard to imagine, given the scene Manet wants to evoke, where he would set his easel, or how it would look if he chose to paint it in.

Wall, however, drops the illusion of being anywhere but his studio and also opens up the full apparatus of enhanced warehouse lighting and wiring, all of which set up superb parallel line grids to assist the eye in perspective. The light stands partition the composition into a triptych rather classically occupied by the the three principle persons: the subject, the photographer, and the camera eye/I (but the light favors her). The woman, once again reversing Manet, is looking directly at the viewer in as level a gaze as one could imagine--not challenging, or flirtatious, or submissive, supplicating, the list goes on. Well, of course she isn't looking at you, she's looking at the camera, but Wall stands a good distance away from the camera and farther forward (that is a very long cable release he has there). He appears to be looking, off the mirror, at her. But the effect of moving away from the camera is to vacate the space of the viewing eye, which is then free for the viewer to fill. The central protagonist is the camera, and the camera is you.

In his "Survey: The Mainstream and the Crooked Path" to Jeff Wall, (ed. Thierry de Duve, Arielle Pelenc, and Boris Groys, Phaidon, 1996) Thierry Duve celebrates this photograph as a breakthrough modernist photograph. For him this means broadly "self-critical and self-referential" and narrowly "conscious of the medium," which in this case is the transparency of the picture's surface (p. 29). But I do not think we are made aware of the materiality of the photograph's (or transparency's) surface; rather, I think that our awareness that we are looking at a photograph collapses. Our brain tells us the woman is posed in an utterly contrived position with her hands resting on the edge of a plywood sheet not more than 4 feet wide facing directly into a large plate glass mirror. But perceptually one or two (incompatible) conclusions seem evident: either she and her assistant Mr. Wall are waiting for you to come to the camera to take the shot, or they are about to take your picture. This completes the turning of the tables on the viewer, who becomes, finally, the viewee. Surely the title, "Picture for Women" is some sort of pointer. Then her remarkable gaze becomes The Gaze, the regard classically directed from the male observor toward the female object, now here reversed.

Conclusions

Throughout we have been working with the modern notion of art as de-automatization--as making conscious and evident the grounds of normal day to day viewing--through the violation of conventions, some of them conventions of practical graphics and some of them conventions of classical art appreciation. On the issues of rhetorical signfication, tension between text and image, and the scene of viewing, we have been able to tease out interpretations according to regular and one hopes transportable principles, using text and figures of rhetorical form, though with no hope of a syntax-semantics mapping strings of images onto logical forms. The general point seems fairly evident that insofar as a certain image does de-automatize, it obtrudes its own making and functioning in ways that would interfere with its use in advertising or instruction. Hence these are not the images and ways of signifying that will be found in your basic corpus of practical working images.

Over and over Tony Godfrey says of conceptualist art that its purpose is not to be beautiful but to make you think. Such art should resist adaptation to advertizing or instructional uses, especially the former, since the purpose of advertising is to focus your thoughts on the object for sale, not to make you think beneath the surface. But of course the industry employs many very clever people and its appetite for a new look is insatiable. Even locating the viewer as the maker of the image can be brought off, say, in a camera ad.

Here is one last image--an advert for Agfa's digital camera from the August 1999 edition of Wired. In broad outline, of course, this is conventional to and beyond the hackneyed point, selling the camera as a sex-appeal-enhancing possession. But there is a special twist--this happy encounter occurs as the camera is being used, not just displayed. Assuming the picture is what "you" see, "you look up" seems to refer to the moment when you look up through the camera's viewfinder to shoot the woman in the second story window (this is why the window casement is appears so tipped inward at the top); "she sees you" in the act of shooting , approves of your somewhat cyborgian mien (which of course is not depicted), and blows you a kiss. The crucial clue for this interpretation is the slight vertical pinching in the middle of the picture (i.e., the top and bottom edges are not straight but curve inward, then outward again). This gives "viewfinder" look. So you want us to think about the scene of shooting? OK, we can use that to sell cameras too! "incredibly easy to use ePhoto digital cameras."