The Sound and Voice of Violent Things: Against the Silence of Data Visualization

Fabio Lattanzi Antinori and Nathalie Miebach are worlds apart in the stylistic, thematic, and material choices of their data artwork.^[1] They speak different visual languages altogether: the former creates monumental pieces, minimalist in color and form, whereas the latter makes smaller, intricately shaped multicolored objects that evoke the playfulness of toys. Although they share an interest in the ripple effects of phenomena, and especially their calamitous results, they expose different domains of violence: while Lattanzi Antinori examines the malfunctions of algorithmic trading, Miebach deals with the cultural and emotional effects of fierce weather, and human agency in climate change. Their works align with many contemporary art forms and movements, but neither of their oeuvres fits neatly within any one box. However, considering their oeuvres side by side, especially with respect to how they employ and interrogate the materiality of data, brings out certain commonalities of approach. First, both artists tap into historical data and use that data in three ways: as a medium, a tool, and a subject of critique. Second, their multisensory treatment of data transcends current tendencies in artistic engagement of data representation, which emphasize visual media. Third, they experiment with what data may evoke and signify across the divide between art and science. To contest that divide, the artists treat data as a construct that can be experienced through various senses as much as it can be processed in logical terms—and yet never fully understood through either means due to the complexity that it harbors and evokes. Their method, bracketed here as data perceptualization, is to combine image, sound, touch, and other senses to engage with and represent the ways we perceive data. 1

The artists’ experientially expansive approach exhibits their epistemological discontent with visual biases and dominant sensory orders in knowledge production across the humanities and sciences, which privilege visual modes of capturing the real world with data. Although Western theory has already criticized this bias, and earlier art movements, including the avant-garde and neo-avant-garde of the twentieth century, have challenged it with multisensory experiments, the current trend of excessive datafication that we face in our everyday lives reinforces the tendency to see the world through data, and primarily visualized data.^[2] Considered in tandem, Lattanzi Antinori’s and Miebach’s work brings into focus the sensory diversity inscribed in the data experiences that they design. In grappling with the limitations of visual mediations of data, Lattanzi Antinori and Miebach develop artistic idioms and knowledge systems that offer an alternative to the monoculture of data visuality.2

Art Where Data Meets the Senses

It is difficult to situate either Lattanzi Antinori’s or Miebach’s oeuvre in any one box, but many features of their work are reminiscent of forms that contemporary art commonly takes. Their performative and interactive pieces evoke the happenings and installations of Fluxus, for example, although the artists do not consistently adopt the movement’s anti-art principles such as working with whatever materials are available at hand. Rather, conceptual considerations motivate their media choices. Miebach, for example, articulates environmentalist concerns through engagement with sustainable materials such as paper, reed, or colored pencil, and handwork techniques such as weaving. Similarly, Lattanzi Antinori’s “industrial” aesthetic deconstructs automated modes of producing and consuming data, and their effects, often hidden from the human eye. Yet these conceptual considerations do not ally them with Conceptual Art, which for the most part rejected narrow object-based notions of the artwork. Rather, both Lattanzi Antinori and Miebach still center the materiality of the art object, as a means of demonstrating that information can be experienced as stories through senses other than sight.3

Perhaps because of the artists’ interest in the materiality of their work as a vehicle for sensory experiences of data, its links with computer, cyber, generative, and data art are not necessarily obvious. Of the two artists, Lattanzi Antinori’s installations, however, have a clearer connection to what typically and immediately would be considered digital or generative art, as some of them involve AI-generated text or sound and time-based media. Not only does the artist make these works using software and electronic elements (as well as non-digital media), but they are interactive—in a very tactile manner. For example, by means of touch, viewers are allowed some control (or at least the illusion of control) over their experience, the creation of which produces the ultimate shape of Lattanzi Antinori’s art objects. Miebach prefers working with natural materials such as paper and thus, though she occasionally uses everyday digital devices to collect data to build sculptural objects, her use of the digital is not readily apparent. Technological experimentation, on the other hand, is important to both artists. The groundbreaking 1968 exhibition Cybernetic Serendipity, organized by the Institute of Contemporary Arts in London, explored the intersection of art and science, focusing on the kind of art the machines of that time could make (primarily two-dimensional images) and generally not inquiring into art’s potential critical function in relation to digital technologies.^[3] Since then, curiosity about technology’s ability to make art has largely given way to investigations motivated by a sense of frustration and anxiety around technology’s pervasiveness and often imponderable uses. For both artists, experimentation is not an end in itself but a means to raise difficult questions around the production and consumption of data.4

In particular, Lattanzi Antinori and Miebach investigate the narrow notion of data as an objective snapshot of the real world. On the one hand, the artists use historical data as a medium; on the other hand, they engage with data as a controversial subject that harbors tensions, uncertainties, and insecurities. Some of Lattanzi Antinori’s pieces, for example, explore data-driven phenomena, such as stock market crashes and their far-reaching real-life consequences. Based on her aesthetically informed interpretation of numeric values and scientific measurements, Miebach invents her own systems of representing weather data to elucidate personal and collective cultural and emotional responses to historical storms. Both artists complicate the ways in which their audiences can engage with data, moving beyond conventional modes of representing it (such as the geometric shapes—circles, triangles, and squares—or icons of infographics) by mixing media and translating data into touch, moving image, and sound in video art, sculpture, and music.^[4] Data perceptualization is therefore a more accurate description of what Lattanzi Antinori and Miebach do. Sound in particular plays an important role in their work, and thus we may speak of their work as specifically practicing data sonification, or translating data into sound. Both artists often use acoustic elements to foreground the violence, instability, and insecurities that are inherent in complex dynamic systems, such as the economy and the weather, and in turn transfer to the representation of such systems. By interacting with Lattanzi Antinori’s screens, viewers are able to “compose” a musical piece of algorithmically randomized harmonies, which raises questions around what is being created and who is creating: an artist? the viewer? an invisible algorithm? that algorithm’s maker? In her musical performances, Miebach stages a confrontation between a musical ensemble and the audience, which both mediates and embodies her perception of the unruly genesis of natural dynamic systems.  5

Data and the senses—especially the non-visual senses—may seem an oxymoronic pair, especially given their troubled history in Western science; although all the senses inform scientific observation, science has largely discounted them for their purported subjectivity.^[5] While sound and image have been traditionally associated with the expression of artistic and esoteric thought, scientific argument has consistently privileged text. Recently, the practice of data sonification has attempted to write sound into scientific discourse.^[6] Similarly, visual thought and its representation, not that long ago also stigmatized as unscientific, has become instrumental in depicting and advancing knowledge in the digital age.^[7] Digital technologies have made visual communication extremely productive, as huge amounts of data may be packed into simple charts and graphs. On the other hand, data visualization can help perpetuate the myth that knowledge is objective. Philosopher Bruno Latour has written widely about the construction of truth in science, criticizing in particular basing science on a nature-culture dualism.^[8] Latour’s discussion of how disciplines compartmentalize knowledge recalls crisis-management theorist Ian I. Mitroff’s work on the stereotypes propagated by physical scientists.^[9] Furthermore, historian of technology Eugene S. Ferguson has offered a compelling assessment of design’s challenge to the cliché that thinking with pictures is not scientific or that technology can be designed based on scientific knowledge alone.^[10] Despite such critiques of knowledge’s supposed objectivity in scientific discourse, however, the arrival of new technologies continues to be attended by renewed hope that numbers will capture the world as it is.^[11] There has been hope, too, that geometric shapes will communicate that truth.^[12] Yet visual communication has created a regime of its own that is becoming increasingly abstract and independent of human labor and participation.^[13]6

The ideas at play in the work of Lattanzi Antinori and Miebach both arise from and mediate their concerns that scientific explanations and representations of data are intrinsically insufficient and, at times, deficient. Their works—what we might call data in(ter)ventions—are epistemic by nature since their makers produce them by interrogating historical data cases and their visually informed conventions as a factor that limits our ability to grasp what that data is designed to convey. Both artists invent their own sensory languages to communicate how they reason with and about data. Creating these languages is design intensive and requires skills and methods that interface between art and science. Miebach, for example, attests that it is through her artistic treatment of weather data that she was able to engage with scientific knowledge of how storms arise, spread, and affect humans.^[14] Alongside other creatives who employ technologies to interrogate data in ways beyond mere aesthetic articulation of that data, both Lattanzi Antinori and Miebach create art around the edges of knowing. I will now consider in more detail how their works mediate, translate, and enact senses in different artistic registers.7

Fabio Lattanzi Antinori: Deconstructing the Oracles of Financial Data

Much of Lattanzi Antinori’s work revolves around the subject of economic data, its obscurities, and its consequences. Pieces such as Dataflags, Temporarily Enslaved Gods, Trackers, and others offer insight into how complex economic systems based on algorithmic trading are prone to break down and dissolve into chaos. Visually, these installations tend to be laconic and elegant, the minimalism of their restrained and mostly monochromatic color palettes hiding this chaos, the critique of which is encoded instead in the artist’s design of the experience of these pieces. The resulting interactions allude to the unforeseen complexities that underlie the functioning of economic systems. 8

Dataflags (fig. 1), installed at the Victoria and Albert Museum in London, is perhaps one of Lattanzi Antinori’s most emblematic works in that it narrates the story of data violence through visual, auditory, and tactile experiences. The work is a hanging screen that features the trading data of the last ten years of Lehman Brothers, the investment bank whose financial collapse in 2008 was the largest in American history and had wide-reaching consequences for the global economy. Dataflags provokes viewers to ask how we failed to anticipate the collapse despite ceaseless data streaming and monitoring.9

An enormous amount of historical data is condensed into a small image positioned at the bottom of Dataflags’s screen. The graphic design and the monochromatic color scheme of the fluctuating strands of white and gray in this piece create the illusion of a controlled narrative. The gray fluctuations never reach the right edge of the flag but are instead abruptly consumed by the black background, leaving nothing but the contrast between the white and black as a reminder of the global shock brought about by Lehman’s downfall. The flag mediates the artist’s critical insight into the obscure effects of algorithmic trading and its representation: symbolically, the black background engulfs the decorative image of data—an allusion to the aesthetic deceptions of its visualization. Suspended in the air, the screen also evokes the fragility of the entire trading system. Yet the flag mediates more than visual aesthetics: what is not obvious at first glance is that it is also a sound installation that allows viewers to interact with the data (see fig. 2). By touching the surface of the screen, which is covered in electrically conductive paint and connected to a sound recording via sensors, viewers “compose” a musical piece.10

The tactile and acoustic components of Dataflags bring forth the hidden discourse of the collapse of Lehman Brothers. The touch causes the sound, but because the sound is randomized, that touch has no control over the particularities of the acoustic output. The role of touch is thus only ceremonial: on the outside, it provides the appearance of control, while on the inside, other forces such as the voice recording, a specific algorithm, and conductive materials shape the dynamics of sound production. This depiction of visible yet limited human agency is a metaphor for how the activities of Lehman Brothers and other big actors in the datafied economy had unforeseen consequences that were not exactly the direct result of any individual actions. Created from the voice of an English soprano, the operatic component sings random numbers and financial symbols from the Lehman data, and it is difficult to understand what is being sung at the highest end of the soprano range, at which point the voice sounds almost hysterical. Lattanzi Antinori has explained that he used opera as a metaphor for high finance since both are associated with elitism.^[15] These recordings of the voice at the outer limit of its possibilities also suggest the danger of collapse and, as the high register borders on the supernatural, they also evoke the oracles of antiquity through which gods were believed to have spoken to people. Like those oracles, who transmitted messages from distant theological entities beyond human reach, Dataflags speaks in an obscure language of high finance that few people speak. The randomization in the piece further implies that coherence is not even relevant to digital communication. The mechanism of the installation thus embodies and at the same time exposes an alien logic arising from the confluence of various technologies.11

The theme of data as a theological entity also recurs in Temporarily Enslaved Gods (fig. 3), an installation that consists of three interactive screens, similar to the screen in Dataflags, but mounted on aluminum frames painted gold. Temporarily Enslaved Gods, as well as a similar piece called Trackers (fig. 4), employ techniques of visual deception. Both play with viewers’ perception of distance to highlight the extent to which algorithmic data manipulate our choices and communication. Despite their similar themes, however, the works rely on different haptic means: Temporarily Enslaved Gods uses touch as an interface for acoustic production, while the sound element of Trackers is triggered by movement.12

When any one of the three screens of Temporarily Enslaved Gods is touched, the other two engage in a conversation powered by bots around the themes of data policies, digital monopolies, free information, and fake news.^[16] The bots were trained on a database of the past five years of US Congressional hearings related to data economy giants such as Amazon, Facebook, and Google. The patterns printed on the screens look like standard data graphs, but a few irregularities in the graphic and structural design of the installation hint that something is amiss. The visual patterns of the flags are reminiscent of a piano whose keys have been strewn in disarray across its surface. Two of the three flags are hung askew.13

Trackers furthers Lattanzi Antinori’s exploration of the visual deceptions and obscurities of data representation. Unlike the other installations, it consists of two structures with no decorative features, giving the immediate impression that something is missing. The sculptures borrow their shape from empty advertising billboards, a choice inspired by the decision of Gilberto Kassab, the former mayor of São Paulo, to ban outdoor advertising in 2006 on a scale never before seen.^[17] This anti-advertising campaign removed fifteen thousand signs in one year alone, leaving empty structures to bear witness to the public fatigue caused by the omnipresence of visual information. The empty frames prompt theological questions like, “What is there beyond the visible and how does it affect human life?” Along with other elements, the sound design of Trackers exposes the invisible infrastructures that facilitate the digital mediations and imitations of human behavior and experience.14

When viewers walk around the frames, the proximity sensor attached to their surface makes the speakers broadcast an AI-generated dialogue enacted by the voices of two female customer service agents. Their conversation weaves together fake news, vacation fantasies, and random words displayed on the screen of another installation called The Cost of Your Words.^[18] Exhibited outside the room in which Trackers is installed, The Cost of Your Words visualizes both the words and their cost per click as generated by the Google Ads account that was set up to promote this piece. The physical separation of the two installations deliberately obscures the link between the multiple sources of the conversation, and the hidden yet tangible link among Trackers, The Cost of Your Words, and Google Ads at once creates and exposes the infrastructure of mining and exploiting human attention. With the synthetic quality of the female voices and their rambling talk, the Trackers installation presents the artist’s dystopian industrial vision of human culture without humans.15

Ultimately, Trackers is a metaphor for the tenacious conditioning of consumer culture. It expresses a strong sense of estrangement from our technologically connected and visually saturated world—a world in which the human mind, like the work’s empty frames, waits to be filled with data pollution. This motif of passive reception evokes the iconic image of visual terror from Stanley Kubrick’s 1971 film A Clockwork Orange: Trackers suggests a world in which we are each strapped to a chair, unable to do anything but watch what’s out of our control.16

Nathalie Miebach: In the Eye of the Data Storm 

Like Lattanzi Antinori, Miebach explores complex dynamic systems, although hers lie in the realm of nature. Miebach’s environmentalist concerns are articulated in the design, materials, and style of her synesthetic work, which employs the meteorological data of historical storms.^[19] Her approach to dealing with data is more tactile and analog than Lattanzi Antinori’s, however, and she often uses manual techniques to draw and weave as well as low-tech materials and tools to create an alternative means—outside the regimented explanations of scientific models—to understanding complex systems and their consequences, including those arising from contemporary society’s polarization of culture and nature. But as in Lattanzi Antinori’s work, sound is a formative element in Miebach’s, underpinning the structure of her pieces, whether they be visual, tactile, or performative.17

It is sometimes difficult to tell where Miebach’s pieces begin and end. Just like storms that build up and then spread out in chaotic ways, her ideas wander from one work to another in a cycle of self-generating creativity that defies disciplinary and semiotic boundaries. For example, the artist takes scientific data and turns it into map-like visuals that combine elements of scientific maps and of her own invention. These maps also double as musical scores that she then metaphorically recycles to create sculptures or musical performances. The resulting artifacts are thus related to each other in genetic and epistemological ways. Through such series of transmediations, Miebach unfolds complex narratives that interrogate the conflict between culture and nature, and how this conflict is represented across artistic and scientific discourses. 18

By artifying meteorological data in visual, tactile, and acoustic ways, the artist explores the emotional and cultural implications of violent weather. Science understands weather in terms of wind levels, pressure readings, and the like. Each storm, however, presents a unique sound to those who experience it. Miebach attempts to capture that uniqueness by interpreting weather variables as musical notes. The Ghostly Crew of the Andrea Gail, for example, focuses on the so-called Perfect Storm that swept over the Gulf of Maine in 1991.^[20] (It was named for the colloquial expression describing a confluence of aggravating factors that together produce an extremely bad situation). Miebach has rendered this storm as a three-act musical score, with each act represented as a separate cartographic visual. The second act of The Ghostly Crew of the Andrea Gail (fig. 5) is the most dramatic. It features a fragment of the weather readings from the fatal October night when Massachusetts-based swordfish boat the Andrea Gail sank. The deadly weather event, which is also known as the Halloween Storm or No-Name Storm, does not have an official, World Meteorological Organization–granted name because of its obscure origins. It started as a smaller storm that then absorbed Hurricane Grace and evolved into an unnamed hurricane itself as it moved along the eastern coast of the United States and Canada.19

To map meteorological readings such as wind levels, barometric pressure, and cloud cover, Miebach used a standard Cartesian coordinate system in an unconventional way: the vertical axis features a piano scale and the horizontal axis represents the times when the readings were taken. Miebach then enlisted Matthew Jackfert to create a musical piece based on the resulting map and the traditional shanty “The Ghostly Crew.” The composition, called Shifting Winds, was then performed live (for a recording, see fig. 6). Jackfert used the shanty as the main theme, which can be heard in the bass clarinet at the beginning of the composition, and surrounded this with the data from all three acts of Miebach’s musical score.^[21] The musical variations are based, for example, on the wind speeds from the first act and feature notes going up and down depending on the values of the wind speeds. The patterns of dissonance in the middle of the composition are built on the data featured in the second act.20

As if mimicking the entangled genesis of a perfect storm, Miebach expands her narrative across senses and media. For example, based on all three maps of The Ghostly Crew of the Andrea Gail, she has created a three-dimensional sculpture called And the Winds Kept Roaring Through the Night (fig. 7). It consists of an inner basket woven based on moon phase data, and spiky outer supporting structures based on wind data. Plaited all around is a so-called time band that emerges from the main structure and is woven back in at the position representing the time when the boat is thought to have sunk (between midnight and 2:00 a.m.). The color of each bead and string of the sculpture signifies weather elements, but they can also be interpreted as musical notes.^[22] A scaffold-like structure supports the lower part of the basket and translates the data mapped in acts 1 and 2, specifically those related to barometric pressure and wind, while act 3 is color coded on the surface of the diagonal spikes that hold the other side of the sculpture. Through her aesthetic mediations and transformations of data from the Perfect Storm, Miebach produces a new data set that is intimately bound to her own methods of visualization.21

The analogy between weaving and narrating is prominent in other of Miebach’s sculptures based on meteorological data, such as a recent series of woven pieces called Weaving Weather During Quarantine (2020). Built from woven watercolored paper strips, their shapes are flatter than those of the previous sculptures, and their simple materials reflect what Miebach could acquire during the early days of the COVID-19 pandemic. The colors of the strips represent specific weather data or feelings, and the resulting patterns evoke the principles of repetition and alteration at the heart of musical variation. These sustainable pieces speak to an abundance that comes not from material wealth but from creative use of limited resources.22

Miebach’s sculptures are reminiscent of ritualistic and ceremonial artwork found in visual cultures worldwide, such as the Tibetan mandala and the Lithuanian dangaus sodas (garden of heaven).^[23] What unites these disparate works is their use of organic materials, their elemental themes of air, earth, fire, and water, and their expression of the contemplative connection between handwork and the mind. Traditional ceremonial artwork of this sort, however, tends to have a unifying center around which the remaining body of a piece is organized in a symmetrical way. The dangaus sodas, for example, is a symbolic replica of the world whose highly symmetrical hanging structure (composed of smaller three-dimensional shapes such as triangles and cubes) is meant to invoke harmony. Its most typical design consists of two pyramids joined at their bases; the middle section represents the earth as the center of the world. Miebach’s sculptures, by contrast, convey a sense of turbulent motion in their shapes, which lack a clear center. Idiosyncratic in style, they capture an evolving interplay between the artist’s feelings about the environmental phenomena and their scientific explanations. 23

The link between the destructive forces of storms and human culture is a prominent theme in the artist’s more recent series focused on Hurricane Harvey, whose landfall caused catastrophic flooding first in Texas and then in Louisiana in August 2017. Miebach recalls that she “spent most of the days during Harvey glued to the local Houston TV stations, hearing how the city was responding to the ensuing chaos.”^[24] The diagrammatic narrative Harvey’s Twitter SOS (fig. 8) weaves together the artist’s memories, the weather readings, and the emotional Twitter messages sent by people trapped in their flooded homes in Houston.^[25] Miebach uses different colors and shapes to encode both scientific and perceptual aspects of the storm: the blue columns represent the amount of rain, the circles map the force of gusts, and the gray rectangles signify traffic incidents that took place on highways during the storm. The green and red stripes at the bottom represent segments of time as measured in hours and cloud cover, respectively. Miebach traces the intersections among the tweets, the weather data, and the traffic system of Houston. According to the artist’s notes—written specifically for Harrison Ponce and Jessie Lanier, who composed the music for the Harvey score (see fig. 10)—the map narrates the interplay between the meteorological, geographical, and cultural factors that contributed to the overall consequences of the storm.^[26]
24

Miebach is quick to note that while the natural causes of the flood garnered wide national media attention, the human factor was overlooked.^[27] That is, little attention was paid to how Houston’s urban sprawl into ecologically fragile areas has stretched the local water channel and reservoir system, thus exacerbating the effects of floods. A wall-mounted sculpture called Harvey’s Twitter SOS II (fig. 9) is designed to shed light on these ecological interdependencies among urban expansion, human decisions, and the weather. The work features a wealth of information, such as temperature readings, humidity, and Twitter messages, stitched onto an inner quilt that maps the income distribution of Houston. Surrounded by three-dimensional figures of houses, cranes, and the city’s traffic system, the inner quilt is a symbolic mediation of complex causes often oversimplified in the conventional depiction of critical weather situations. The resulting image speaks to these complexities in aesthetic ways, and its seeming playfulness disguises the serious subject in the background.25

In moments like these, tensions between reception and representation arise—that is, tensions between what the artist wants her viewers to see and how she delivers it, and between viewers’ expectations and their actual experiences. Miebach restlessly searches for more holistic treatments of meteorological data because she seeks to trigger emotional responses in her viewers. The ideas for the transmediation of her visual pieces come from the realms of music and theater, which inform her approaches to the sonification and theatricalization of data. While views on the relationship between music and emotions differ, there is a general agreement that music affects us on the emotional and physiological levels.^[28] Unlike visual attention, which is selective, attending to music makes the brain’s different parts, each responsible for a wide range of skills and perceptions, fire simultaneously.^[29] The more we process and engage with music, the more compelling and transformative its effect.^[30]26

Since 2009, as part of her ongoing Weather Score Project, Miebach has been organizing a series of immersive events that evolved from her visual and sculptural work. In 2019 she and New York–based ETHEL—a string ensemble with deep roots in improvisational performance—played the Harvey storm composition at Denison University in Granville, Ohio (see fig. 10). It was staged as an immersive theatrical experience wherein the audience was made to relive the storm from a first-person perspective. The space and everything that happened within it, including how the musicians moved and how the viewers were seated, were arranged to play a part in the musical reenactment of the Harvey storm. The seats were set up to resemble the Houston highway system, and all but one of the musicians circulated through the space as they performed, to produce the effect of the storm closing in on the audience from the different sides of the city. The stationary cello, whose role was to embody the city itself, provided the central musical line for the other instruments to follow as they enacted the escalating sounds of the storm. At the climax of the piece (3:53 in the video of the performance), the two violins and the viola intensify their pace, and their pitch reaches hysterical heights symbolic of the culminating point of both the storm and the human anxiety it produced. The audience itself enacted yet another musical line by reading aloud tweets posted during the storm when the city’s 911 system broke down. As the instruments’ sounds escalate in pitch, so, too, do the human voices rise in volume, trying to outshout the noise.27

The Harvey performance was designed based on the expanded notion of three-dimensional sound that replicates the way we hear sound in the real world. To mediate the storm in experiential ways, Miebach brought together interweaving melodic lines and their counterpoints as produced by musical instruments, as well as noise, words, and shouts produced by human voices, and physical movements that aided in the production of sound. Representation and reception—traditionally separated in the theorizing of art—converge here through the participation of humans who are both performers and receivers: they mediate and experience the soundscape installation, one that is as volatile and unpredictable as the weather. The effects are both aesthetic and emotional, in that the escalating sounds not only portray the raging storm and its experience metaphorically but also contribute to heightened levels of excitement.^[31]28

Data Experience in the Museum: Challenges of Digital Display and Multisensory Artwork 

Some works by Lattanzi Antinori and Miebach, including those discussed in this article, have entered the collections of various museums. Taken together, the material, sensory, processual, and conceptual features of these pieces raise questions around how museums can contain them in virtual space as much as in physical space. If we understand curation as more than mere possession, it is clear that their works present a combination of the curatorial challenges that three-dimensional, durational, and performance art pose to current museum infrastructures. Their sensory behavior actively resists digital replication and dissemination precisely because they require something more than merely visual spectatorship. In particular, the object-based notion of online curation relies heavily on two-dimensional media such as images and videos. These common forms of digital reproduction are inadequate to mediating the multisensory experiences that their artworks provoke when displayed or staged in physical spaces. Like performance art, Lattanzi Antinori’s installations and Miebach’s weather projects require direct human involvement and interaction. It is the viewer who determines how exactly the artwork unfolds over time and what experiential form it ultimately assumes. In addition, the durational aspects of Miebach’s work are both explicit, as in her musical events, and implicit, in their encoding in the visual media of her cartographic pieces and three-dimensional sculptures (from which the musical performances evolve). In these ways, their works may in fact pose a greater challenge than that posed by performance and time-based media art forms, which—arguably—have continued to privilege visual experience. 29

VR technology may offer some solutions. For example, it could enact the anthropomorphic qualities that Lattanzi Antinori lends to his installations, such as their ability to respond to or involve human presence. Similarly, if Miebach’s sculptures, such as And the Winds Kept Roaring Through the Night, were rendered three-dimensionally in the digital environment, their volumetric character would be more accessible when displayed online, and it would be possible to examine them from all sides to appreciate the position and interrelation of their elements. However, the basket-like structure and materials of her woven sculptures are challenging to capture with existing 3D technologies unless they are rebuilt digitally rather than scanned. And still, even if reconstructed as discrete objects, Miebach’s sculptures cannot be fully understood outside the whole chain of art objects that attest to how she devises her methods of understanding weather data—that is, without reference to map-like visuals from which the sculptures evolved and to the musical performances that they subsequently inform.30

The historicity and intimacy of an encounter is yet another challenge for digital presentation. Even when video documentation of the artists’ works records human interaction, as in a video of Dataflags (see fig. 2), or records viewer participation, as in the musical enactment of the Harvey storm (see fig. 10), watching these records is not itself a first-person experience of the art. The difference between first-person and third-person access is both epistemic and psychological.^[32] The lack of first-person access changes the very nature of multisensory artwork designed to provoke subjective experiences. When reproduced in two-dimensional media such as static images or videos, Miebach’s musical data performances, for example, cease being empathetic experiences, since the viewers are no longer participating in the aesthetic recreation of the storm to experience the anxiety, anger, fear, and other affects it caused in those who went through the actual storm. Analogously, the video of Lattanzi Antinori’s Dataflags is documentary in that it captures someone’s experience, but it is also somewhat theatrical and voyeuristic: it leaves us wondering what is summoned when we cross the boundaries that separate us from the object. The ultimate object (if I may use that term) of Lattanzi Antinori’s artwork is the unique sensory experience that engaging with it produces. In their digital reincarnation, his installations cannot produce the diversity of sensory experiences they do when museumgoers or gallery visitors engage with them in reality. Indeed, by comparison, the digital experience is a mode of sensory deprivation.31

It may, however, be possible to display interactive work like Lattanzi Antinori’s and Miebach’s in the digital environment in ways that would address the limitations sketched above: by re-authoring and transmediating the works with the same focus on preserving sensory richness with which the originals were produced. As the rapid prototype of Trackers (fig. 11) shows, for Lattanzi Antinori the interplay between the tactile and the auditory are central to the production of the first-person experience. The task at hand for effective digital display is therefore to rethink the semiotic boundaries and technical affordances of online curation, along with the questions of privacy and freedom that attend our future sensory interactions within the digital environment. As a form of creation, sensory-focused curation would offer the possibility for makers to re-author their works and for curators to co-author the process of transformation.32

Nonetheless, the digital version of an analog encounter with a multisensory artwork would obviously be very limited in what it can relay and how it can relay it.^[33] Lattanzi Antinori’s and Miebach’s multisensory artworks challenge not only the visual bias of science and data production but also that of knowledge-making with and about art in museums. The sensory diversity they inscribe in their artwork has epistemic value. The artists’ innovations are intimately bound to their own perceptions and subjective methods of interrogating data as an experiential construct (an understanding of data with which institutionalized domains of knowledge generally struggle to come to grips). Perhaps the most radical challenge Lattanzi Antinori’s and Miebach’s work poses to curatorial practice is the necessity to recognize their works’ epistemic, rather than aesthetic, nature, and to recontextualize their outputs not as objects but methods that cannot be reduced to currently dominant definitions and models of knowledge creation and communication.33

Banner image: Detail of fig. 7.34