Interpreting Information Visualization

Information visualization has become a prevailing part of our visual culture, a research field, and a line of work for designers. The literature on information visualization is diverse, dominated by handbooks aimed at designers and illustrated surveys, sometimes with an emphasis on historical examples. This chapter makes a survey of the field of information visualization in order to map out and assess its analytical vocabulary. First, there is a need to refer to some different definitions and general concepts, such as the naming of phases in the visualization process, the naming of different visualization types, and their components. In order for scholars of human sciences to be able to identify, understand, and interpret information visualization as visual objects, some tools for suitable visual analysis of these objects would be useful. To meet this end, the chapter explores two particular interpretative frameworks. The first framework discusses social semiotics as an analytical tool. The second one analyzes the ethics and emotional appeal of information visualization.

that appeared then, such as the line chart, the bar chart, and circle diagrams, have become visual conventions that keep reoccurring even at times when information visualization is produced and represented in digital environments. 3 The aim of this chapter is to map out and exemplify how information visualization is defined and analyzed as visual objects. These visual representations, if at all acknowledged as nontextual representations, occupy a position in-between image and text and their interpretations are methodologically hard to capture. 4 Visualizations of many different kinds are being used as tools in the humanities and the social sciences; however, these methods are not the focus of this chapter. 5 The aim of observing types of visual analysis of visualizations originates from my belief that the competence of interpreting visual material in the humanities, like in the subject areas of art history and visual studies, would benefit from more consciousness when it comes to interpretative tools and terminology for information visualization. What is at stake here is not only to be able to read charts and graphs critically but also to make information visualization more transparent as subject to analysis. My chapter provides a starting point for this greater goal: It makes a survey of some of the literature on information or data visualization in order to distinguish tools and terminology that has been previously used in the different branches of the field. These, of course, have different purposes, many with the objective of educating designers. Nevertheless, they may be found usable for other ends. After this brief literary overview, focusing on terminology, I exemplify how information visualization can be interpreted as visual objects of study with two different frameworks: one based on semiotics and the other directed at the emotions and ethics of information visualization.

Information Visualization Terminologies
Many different concepts and meanings circulate when addressing the phenomenon of information visualization, data visualization, or visual representations of numerical value. The term information visualization will be used here as a family name of a category of visual representations, both historical and more recent examples. 6 Information visualization, or "infoviz," is a research field, referring both to the discipline and the objects of study. 7 To work with information visualization is to work with the structuring, categorizing, and designing of data or information, as well as with technological tools. 8 Parallelly, the name data visualization, or "dataviz," is being used. 9 The subjects to be visualized-the concepts of information or data-are not entirely unproblematic. Information is a curious entity, an abstract concept that can 6 Stuart K. Card, Jock D. Mackinlay, and Ben Schneiderman, "Information Visualization," in Readings in  contain almost anything. The established conception of information as an entity or stream with exchangeable content is dependent on information or data translated to numerical value or units, and is often said to be contemporary with the visual communication model of Claude Shannon's "Mathematical Theory of Communication" (1948). 10 Scientific and medical visualization is sometimes seen as a branch on the tree of information visualization, sometimes separated with its own properties. These orientations can be described as "data that describes a physical phenomenon." 11 Lev Manovich has made the observation that scientific visualization evolved in the 1980s alongside 3D visualization techniques in scientific communities, while information visualization was developed in the design community in the 1990s, mainly using 2D graphic tools and techniques. 12 The input data can be derived from readings of technical equipment and instruments, from entities that are infinitesimally small or otherwise invisible to human senses, and the line between a "physical phenomenon" and information or data in general can be hard to draw. Here, it is also relevant to mention the field of knowledge visualization: emphasis on the use of visualizations as facilitation in collaborative, knowledge producing processes and contexts, for instance in management studies. 13 Visualizations have also been used in scheduling and in industry 10 N. Katherine  production planning since the early 1900s, for instance with the well-known Gantt chart and its many succeeding adaptations. 14 Only from the 1970s and onwards did the term "visualization" become synonymous with visual representations on paper or on screens. 15 Information visualization, as forms of nontextual representation, is closely related to "information graphics" or "graphical representations" and the concept of the diagram, referring to a written, drawn, or painted trace on a surface. 16 Certain two-dimensional pictures, like maps, possess the capacity to harbor large amounts of information in one single sheet of paper, or equivalent. 17 The pictorial units can be called "graphisms," inscriptions, or traces-graphic marks on a surface. 18 As I will return to, the mark is the established term to describe graphic units in information visualization, in handbooks, in academic discourse, and by practitioners. 19 Tamara Munzner's handbook and survey Visualization Analysis and Design (2014) defines visualization without any information or data prequel. Here, a user-and technology-oriented definition is given: "Computer-based visualization systems provide visual representation of datasets designed to help people carry out tasks more effectively." 20 Visualizations are seen as objects that can facilitate users' decision-making processes and thereby function as extensions of human cognitive skills or, in Munzner's words, to "augment human capabilities." 21 Some of the literature uses "data" instead of information visualization. Generally, data is seen as the raw material of information, which is processed and organized data, stepwise proceeding to a visualization. 22 Data, like information, as a concept, earned extended meaning and use in relation to the first digital revolution. 23 Data as a rhetoric term is derived from the Latin dare, something given in an argument. 24 Data is thus termed the units or the building blocks of information, as items that precede information. For instance, Andy Kirk's Data Visualisation (2019), a handbook aimed at designers, uses a pragmatic approach to the concept of data. In the introductory glossary, a problematized view of the notion of raw data is acknowledged and appreciated, but is not in focus since it is said not to affect user comprehension of visualizations. 25 Kirk uses a concise definition of data visualization: "The visual representation and presentation of data to facilitate understanding." 26 Apart from the pragmatic approach to data as something collected or given, not yet subject to any intervention, sorting or design choice, there is a distinction between representation and presentation within this definition. Presentation is seen as a way to "portray your data visually" and, for instance, the different options of chart types. Representation is all that falls beyond this: design choices making the visual representation accessible to the intended users. 27 20 Munzner, Visualization Analysis and Design, 1. 21 Kirk,Data Visualisation,11. 26 Kirk,Data Visualisation,15. 27 Kirk,Data Visualisation,[17][18][19]

Information into visualization
One theme in the literature when defining information visualization is to identify steps in the process of decoding and coding data or information into visualizations. In a survey of surveys of academic writing on information visualization, "The Information Visualization Pipeline model" is introduced: "The pipeline describes the transition of raw data into a visualization which is visible to a user." This consists of five identified steps: "1. Data Enhancement & Transformation, 2. Visual Mapping & Structure, 3. Exploration & Rendering, 4. Interactive Analysis and 5. Perception." 28 The process is one of transformations: Socalled "raw data" has to be "captured" and structured into tables. The second stage is "Visual Mapping & Structure," where choice of visualization technique, and questions of which "visual primitives" such as color, texture, and geometry, are mapped onto chosen data sets. Stage three, "Exploration & Rendering," also takes on the question of visual presentation to the user, but has more to do with viewpoints, scaling, and presentation to the user, how to handle distortions and projections. The fourth stage, "Interactive Analysis," consists of surveys that address the question of how feedback from users is recorded. The level of interactivity can diverge significantly from different information visualization types and projects. 29 The habit of dividing the process of visualizing information into different stages is part of how to define and interpret information visualization. Colin Ware uses it in Information Visualization: Perception for Design (2012), starting out with: "1) The collection and storage of data itself, 2) The preprocessing designed to transform the data into something we can understand, 3) The display hardware and the graphics algorithms that produce an image on the screen, 4) The human perceptual and cognitive system (the perceiver)." 30 Methods and tools that facilitate visualization to generate an overview, to expose trends and patterns in large amounts of data, have now become more available and more user friendly. 31 Johanna 28 McNabb and Laramee, "Survey of Surveys (SoS)," 590-591. 29 McNabb and Laramee, "Survey of Surveys (SoS)," 590-591. 30 Ware, Information Visualization, 4-5. 31 Johanna Drucker, "Graphical Approaches to the Digital Humanities," in A New Companion to Digital Humanities, eds. Susan Schreibman, Drucker also has critiqued the use of the word data as something already "given" and introduces the term "capta"-"constructed and not given." 32 She stresses that humanities scholars, who are used to subjects laden with "uncertainty and ambiguity," should be more aware of problems that can occur when transferring qualitative information into numerical data to be visualized. Such transfer risks creating unnecessary dichotomies. Some variables are disputed and problematic and would need to be represented as fuzzy and nuanced, not definitive. Drucker exemplifies with the concepts of gender and that of nation, both common dividing categories in demographical statistics, but not static if viewed across time, culture, and who makes the definition. Data or information visualization can, to follow Drucker, risk bringing in simplified formats into disciplines that are used to discussing problems of any given category taken as unproblematic. 33 Ideally, there is a discrete relation between information, or data, in the form of numerical value, and the visual elements-point, line, or surface-of a visualization. This is also the core argument of Manovich's preliminary definition of information visualization: "a mapping between discrete data and a visual representation." 34 Information visualization, then, forms a specific representational type which deviates from many other visual representations. In the essay that was essential to the development of visual semiotics, "Rhetoric of the Image" (1964), Roland Barthes  images are "polysemic." Meaning ascribed to images is arbitrary depending on context, time, and place, that is, images can mean different things to different viewers at different occasions. 35 There is a strong tradition within the design-oriented literature on information visualization of categorizing it as a more monosemic representational type than other visual representations. That means that, as opposed to the polysemic or multiple possible meanings of most visual representation, it is narrowed down to having only one possible meaning. In his major work Sémiologie graphique: Les diagrammes, les réseaux, les cartes from 1967, cartographer Jacques Bertin states that graphics-diagrams, networks, and maps-are monosemic sign systems, where each sign represents only one specified meaning. In maps this can be specified in a legend, but Bertin also stresses the agreement on visual meaning within a community. 36 Bertin's writings have been influential in the formation of the theoretical field of information visualization. 37 Thereby, a sometimes simplified view that information visualization forms a pictorial language of truth can be present in handbooks of design of information visualization. 38 When it comes to the interpretation of information visualization as visual objects, the specificity of the way that these objects represent can provide some difficulties. First, their truth aspect can stand in the way of scrutinizing them. Second, they demand of the viewer to be able to decipher their accuracy in their representation of data. These aspects are not at all insignificant, but they risk keeping scholars from other fields, such as art history and visual studies, away from interpreting their formal visual properties. 35 Roland Barthes, "Rhetoric of the Image," in Image, Music, Text, trans.
Stephen Heath (London: Fontana, 1977), 32-51. 36 (2014) are prominent examples that also include a historical perspective. 42 There are examples of scholarly exposés nurtured by the rich visual history of information visualization, or its earlier equivalents. 43 The aims of such work is not to provide the reader with an analytical framework but it is always hard to make this kind of visual odyssey transferable. It requires expert knowledge to move around in the history of visualization. As contextualizations, this kind of work is very welcome and elegantly made. Another direction to strengthen the design of information visualization is to focus strictly on human perception-how humans perceive visual information-as in Colin Ware's Visual Thinking for Design (2008), and the more accessible Information Visualization: Perception for Design (2000). 44 A predecessor in popularizing information or statistical graphics, not using the term information or data visualization, is design- . 45 Cairo,The Functional Art,64. in better ways. 46 Some of his terms have become commonplace in speaking about and assessing information visualization and capture a concentrate of some of his thought. Tufte advocates a minimalist, less-is-more standpoint to the design of statistical graphics. The concept of "chart-junk" denotes all superfluous visual elements a designer uses to embellish statistical graphics, out of which some can be derived from convention and routine. 47 The second widely disseminated concept of Tufte's, which also is a potentially analytical one, is "data-ink ratio." This concept denotes the amount of "ink" necessary to visually represent data, for instance records from measuring equipment, divided by "total ink used to print the graphic." The overall principle is to reduce and erase all unnecessary ink. 48 The impact Tufte has had on general knowledge on information graphics, statistical graphics, and information visualization, to the general public, visual journalism, and designers cannot be overlooked. There is, however, a sometimes overexplicit minimalist incentive of visual reductionism that can overshadow the usefulness of an analytical tool like the "data-ink ratio." Cairo, who has a background as a visual journalist and designer of information graphics, now in US higher education, is a prominent figure in design of information visualization through The Functional Art (2013) and The Truthful Art (2016), both mainly aimed at designers with lots of practical advice. 49 His latest book is a contribution to an ongoing debate on how to interpret and validate charts and graphs in news media, How Charts Lie (2019), in an era of "fake news." 50 Since the design handbook perspective 46 Edward R. Tufte, The Visual Display of Quantitative Information (Cheshire, CT: Graphics Press, 2001), 51. "Principles of graphical excellence" are for instance "that which gives the viewer the greatest number of ideas in the shortest time with the least ink in the smallest space." 47 Tufte, The Visual Display, 107. 48 Tufte,The Visual Display,[93][94][95][96][97]Data Feminism, 76-77 discuss Tufte's advocacy of minimalism as representative of a larger problem of a false and also gendered binary between reason and emotion in information graphics and visualization. 49  is in focus in his first two books, there is not much room for presenting tools for analysis. However, there is one such, "the information graphics wheel," which is developed to assess qualities in information graphics, to help the designer during the design process, and for stakeholders to value commissioned work. The wheel model presents ways to find a balance between antipodes like density and lightness, figuration and abstraction, functionality and decoration. Cairo calls it "a visualization for planning visualizations." Hereby, it is clearly stated that it is not meant to be used as an academic analytic tool. 51 However, since analytical tools for evaluating information visualization are so rare, this could also form a first step of developing such analytical categories. I have singled out terminology in naming different graphical elements in information visualization as an area of interest in analyzing information visualization. These also constitute the building blocks that designers have at their disposal. As earlier mentioned, the conceptual hierarchy and terminology in Jacques Bertin's Semiology of Graphics has laid a foundation for the naming of graphic building blocks upon which visualization literature still depends. 52 Bertin's full terminology is too extensive to recapitulate here, but the graphic mark is at the center of his system, as it is in Munzner's clarifying overview. The mark equals one item in visualization, forming its "basic visual element." Marks can, in turn, be defined by the number of dimensions they occupy on a two-dimensional plane: zero dimensions constitutes a point, one dimension is a line, and two is an area. 53 Other factors that determine a mark on the visual plane are called visual variables by Bertin, and channels by Munzner, which is the established term in the visualization literature. Bertin and Munzner refer to eight different channels, basically corresponding but with some variation in terminology. These channels regard the mark's horizontal or vertical placement on the plane, its size, value, texture, color, orientation, or shape. 54 In addition, there can be relations between marks, in visualization types like network diagrams. Here, a mark can be either an individual item, a node-a point, or a connector-usually a line. 55 These are only the most basic categorizations of building blocks in information visualization, following the established terminology extracted from Bertin. They are useful when understanding and assessing these visual objects, providing primary insights into the vocabulary of the field.

Interpretative Framework 1: Semiotics and Information Visualization
One way of analyzing information visualization is to establish a grammar of its visual building blocks by using terms and tools from semiotics. 56 The building block, or element, in semiotics is the sign, and its grammar is about "sign-making" and visual entities like direction, color, and placement. 57 Gunter Kress and Theo van Leeuwen do not address information visualization per se but "abstract visuals," or "diagrams" in terms of realistic and less realistic representations. I have for the purpose of this text focused on some representational types in the archive of examples, like diagrams, charts, and maps, with elements like geometric volumes and vector lines, which are also intrinsic parts of information visualization. The main example in this genre starts out with previously mentioned Claude Shannon's influential communication model from 1948. The recurrent use of simple geometrical shapes is discussed: boxes and arrows in this visual model 54 Bertin, Semiology of Graphics, 7; Munzner, Visualization Analysis and Design, 96. 55 Munzner, Visualization Analysis and Design, 100. 56 Kress and van Leeuwen, Reading Images, 1-2, 14. The term grammar is used with some reservations. The aim is to analyze combinations of elements and the meanings that can be ascribed to their aggregations in social interactions. It is less concerned with sets of rules. Illuminating power relations and ideological positions that images convey is also an aim, leading to the method of analysis presented to become a contribution to broadening the scope of critical discourse analysis in a visual direction. 57 Kress and van Leeuwen,Reading Images,[6][7] indicate abstract participants and their conjunctive processes and relations, respectively. 58 According to the authors, in a broader cultural field, the fascination with geometrical shapes draws on modernist, abstract art of the early 20th century, associated with being "pure, quasi-scientific 'atoms' of the visible world, a 'pure manifestation of the elements'." 59 These clean, basic forms are seen as the building blocks of the visual grammar, and are compared to linguistic structures and concepts, such as nouns and verbs. 60 Geometric shapes are also associated with "the world of human construction": buildings in modern cities, machines and roads, as opposed to organic or "natural," irregular shapes. 61 Different geometrical shapes, in semiotic terms, represent different categories of things, and/or actions. A vector line can also indicate direction, or the linkage and some kind of transfer between elements. In short, these geometrical shapes and lines help to form a visual narrative. 62 In laying out the differences between vectors and volumes, Kress and van Leeuwen draw on art historian Rudolf Arnheim, who stated that "motion and change" are usually represented as a sequence in Cartesian space either horizontally or vertically, connecting the function of geometrical shapes to the basic categories of being as static, and acting as motion. Being is thus tied to volumes and acting to vectors. 63 Kress and van Leeuwen also address another diagrammatic feature that is important in information visualization, and that is the capability of basic geometric forms to display taxonomy, that is, how different elements-or, in their terminology, "participants,"-relate to each other, hierarchically, in, for instance, tree diagrams, network diagrams, and flowcharts. 64 The whole idea of a "grammar of visual design" is an interesting attempt to identify and name visual elements, applied to many different examples, but the development of an actual method for analysis with some generality is still to be refined.
Linked to social semiotics is Wiebke Weber's framework of analysis of data visualization, with its theoretical base in multimodality and social semiotics. Again, visual variables with a semiotic substance that are identified are, for instance, points, lines, areas, and colors. The framework also takes into account the aesthetic form of the visualization. The analytical framework is tested on examples of information visualization in news media, out of a larger corpus. The analysis is divided into graphical and textual modes and ideational, interpersonal, and compositional metafunctions, that is: the types of messages in the visualization, the relation to the audience, roles and relations within the visualization, and, finally, its overall composition. 65 My own orientation in this present text, especially when categorizing previous research and writing on information visualization, owes a lot to Weber's study. The latter further exposes some of the complexity and arduousness of building an analytical framework for information visualization. In the shorter format, the description of the analytical framework and its terminology risks appearing heavy in comparison the case studies. However, it shows the kind of rigor that is necessary when analyzing information visualization, taking into consideration a multitude of interpretative layers.

Interpretative Framework 2: Emotions and Ethics in Information Visualization
In the late 2010s there appeared signs of a paradigmatic shift in the way information visualization was produced and received. Charles Kostelnick, a professor with a background in comparative literature, is one of the founders of writing about information visualization and other forms of diagrammatic representations, together with Michael Hassett in Shaping Information (2003). 66 He observed how "emotional appeal" has returned as a dominant aspect in data visualization (the term used) in the 2010s. The shift contrasts greatly to Tufte's preeminent minimalist ideal,65  and this ideal is the main antagonistic stance in Kostelnick's exposition. Kostelnick analyzes examples of data visualization using concepts derived from Aristotelian rhetoric such as pathos and enargeia, but also produces a background in form of a historical excursion, showing that the emotional appeal was vital in the late Victorian era, equivalent to data visualization, and diminished during 20th-century modernism. 67 The "emotional appeal," according to Kostelnick, not only appears in design features such as color and "design novelty"; recent technology has also made it possible to put the user in the center and add interactivity such as personalized displays, such as visualizing real-time data, personal data, and options of selection of which data and range to view. 68 Following the discourse of emotions in information visualization, both in the ranks of designers and as a response in users, there is an orientation toward ethical information visualization, which came to my knowledge via Katherine Hepworth. In the paper "Racism in the Machine" (2018), Hepworth, together with Christopher Church, analyzes and compares two examples of web-based, interactive mapping projects, both dealing with the atrocity of lynchings in the US in the 19th and 20th centuries. In the analysis, the projects and their visual and interactive characteristics are carefully described. 69 Also, it is quite unusual in the writings on information visualization that colors, contrast, shape, and typography is considered. However, the study also gives at hand some of the difficulties of conducting a formal analysis of interactive information visualization, where "dots," "zoom," and "click" are recurrent. Without being able to go into detail of the analysis, the main question is how these chosen projects manage to expose the victims of these cruel acts without dehumanization. dignity of the represented subjects" is not carefully defended. 70 The large claim in the paper is the proposition of an "ethical visualization workflow." To recall, for instance, the "the Information Visualization Pipeline model," literature on information visualization is preoccupied with describing processes of a visualization project. 71 Contrary to such plans and descriptions, the "ethical visualization workflow" starts on the premises that visualization of data or information is not neutral. Thereby, the work to ensure ethics in a visualization project start "pre-data collection," studying up-to-date academic work on the subject, to avoid bias, and similarly regarding the design choices. For instance, some conventional forms frequently used in information visualization have built-in problems, like the Mercator projection, proportionally favoring the northern hemisphere. 72 Lastly, the expanding field of digital humanities is addressed. The criticality inherent in humanist subjects is needed throughout the whole process. 73 Scholars of the human and social sciences do have a role to play in the understanding and shaping of information visualization.

Conclusion
This chapter was first an attempt to map out some parts of the scholarly fields and literature on information visualization in order to illuminate some of its tools and terminology to scholars outside of those fields. The writings on information visualization are disseminated across different areas of knowledge, aimed at different readers. There are handbooks aimed at designers, and there is the academic field, which can be broken down into converging orientations, for instance concerning the visual design and the technologies of transferring data or information into visualizations. From these fields I highlighted some elemental terminology of entities of information visualization from well-known and widely used sources, like Jacques Bertin and Edward Tufte. Second, to exemplify, I selected two interpretative frameworks or ways to analyze information visualization, a framework inspired by social semiotics represented by Gunther Kress and Theo van Leeuwen and Wiebke Weber, and a framework circling around the emotions and ethics of information visualization, represented by writings by Charles Kostelnick and Katherine Hepworth. Kostelnick uncovers what he sees as a trend in information visualization projects in the 2000s. Hepworth and Church focus on the safeguarding of the design process, making it ethical in every step, most notably in the phase of data or information collection.
As a visual object of study, information visualization becomes even more curious and ungraspable when we are faced with interactive graphics and real-time-generated visualizations, or interactivity graphics that can be altered by the user, and thereby personalize the experience. 74 Real-time-generated visualizations are ephemeral visual objects that challenge methods of capture and documentation, and this discussion has not been included in this chapter. When more decision-making is based on information that is made available to different users as visualizations, these are visual objects that researchers in the human and social sciences need to take into consideration, and to develop methods for analysis. Studying visual representations on screen is nothing new to art historians and scholars of visual culture, but there is a need for a methodological discussion on how to handle study objects that change over time or with the input of users.
Neither of these two identified frameworks is in itself a fully developed methodology for analyzing information visualization as visual objects, or aims to be so. But some components do provide valuable terminology and analytical concepts when working with information visualization in the humanities and the social sciences. They could constitute a welcome start of a broader discussion on how scholars of for instance art history and visual studies can influence these fields in offering more apt and subtle tools for visual analysis of visual representations that are not always seen as such but nonetheless have an impact on how we see and interpret the world around us. 74 Kostelnick, "The Re-Emergence of Emotional Appeals," 117, 125-127.