Computational reading of culture allows us to pose new questions or create new cultural forms supporting new forms of critical thinking. The resulting data and interfaces require new ways of understanding. Reflecting through a distant reading project, the aim of this paper is to suggest a media-specific approach to understanding how the techné affects critical practice and epistemology. In the first part, I attend to structures of feelings and models encapsulating the world and frame Williams’s definitions as a model. The second part opens up Berry’s (2014) iteracy as theory and the basis of practice to find meaning in the models. The third part presents a Derridean reading of the technical processes and how these can be used to create a space for meaning to exist.
Hayles’s media-specific analysis “attends both to the specificity of the form … and to citations and imitations of one medium in another” (“Print Is Flat, Code Is Deep” 69), altering our critical relationship with machines and raising a crucial role for questioning the medium itself as site of cognitive practice through remediation. The consideration of structures of feeling, I contend, requires not only human reading but also technical reading itself using models to understand the digital. I suggest that this develops Hayles’s cyborg reading, reading through a distributed cognitive system, a technique to interpret the media’s discourse.
I briefly want to frame the issues using the Next Rembrandt[i] project. Using Rembrandt’s portraits, machine learning algorithms pored over labelled data to extract features to create a new painting. These included the facial proportions,type of light, the tilt of the head, and the paint layers on the canvas. Through a machine reading of many aspects of one artist’s painting, the final printed painting revealed layered effects of brush work in digital and physical media.
In many aspects, this is a technically demanding reading. We might feel the sadness and warmth in the sitter’s eyes or the slightly worn look derived from the way the light plays on the features and through the layers of paint. At this moment, I contend that we are inside an interpretational loop. The machine uses aggregations of the models and the data to create a new set of data points derived through a model. Its surface is a visualisation, where numerical mappings mediate the data into a new point, which humans perceive as colour, at a location. From a human perspective, we note the stylistic similarities, the attention to detail in the style and the emotion in the face. There is a disjunction here between the two readings that reveals the need for conceiving about how this can be critically approached.
I begin with a consideration of Williams’s construction of structures of feeling as “a cultural hypothesis, actually derived from attempts to understand such elements and their connections in a generation or period, and needing always to be returned, interactively, to such evidence” (Marxism and Literature133). Computational reading derives features from the data based on human thought and interpretation of the hypothesis, either in the construction of algorithms or labelling of data. Once identified, the features may then be analysed or combined to create new structures and elements.
The element’s existence and its interpretative possibilities as part of an emerging discourse is problematised through this process. The process of translating a feature into a series of technical languages to create a new model and element alters the discourse and its specificities. Manovich’s (2013) claiming of the digital as a metamedium – capable of transforming existing media and creating new media and technologies – highlights the need for a media-specific analysis. Through the work on reading the paintings, algorithms search for common patterns and shapes as the close reading of the painting collection. This reading is enhanced through work on the demographics of Rembrandt’s subjects, placing the work within a social context and on the physicality of the paint through its layers. At each stage the media specificity of the elements is acknowledged and remediated into the required form such as digital or 3-dimensionally printed image, but these translations develop the dis-junction between the generated layers of meaning.
I want to turn to models as an integral part of these computational structures. McCarty (2014) considers computational systems as dependent on the models given to them to understand a conception of the world, echoing Weizenbaum’s conception of models as a “symbolic recreation of [the designer’s] world” (Computer Power and Human Reason18). The use of AI to create data sets and models problematises this by raising questions of who is the designer and whose world is being created? The model’s structure of an element rests on how the designer or implementer translates and transcodes it into their work. McCarty’s argument that the “[m]odelling of something readily turns into modelling for better or more detailed knowledge … modelling for something feeds or can feed into an improved version” (Humanities Computing 27) suggests a feedback or learning process based on an understanding. I read McCarty’s assertion that “modelling is making new knowledge by manipulating hypothetical constructs” (Humanities Computing71) through grammatology as a process of challenging the technical language’s specificity. Acknowledging that modelling “cultural artefacts treats them as something like the empirical objects of nature; … paradoxically modelling anything is just as clearly an imaginative act” (Humanities Computing72) surfaces the inbuilt issue of trying to define the abstract into the concrete. The emotional structure of feeling becomes an imperfect structure of feeling.
A key point is that the Williams’s issue with the specifics what constitutes an element in discourse is further problematised through translation and encoding required for the machine to understand them as hypothetical constructs. From this, a new discourse is created from the results, which require reading. The underlying computer model both makes and is made from the translation. This alters the location of epistemology from the reading and interpretation to within the digital. A necessary consequence is a potential change of the location of the element’s negotiation. Whilst it may happen as part of wider cultural discourse, it is happening within the algorithms and their models of the world.
I want to take a brief pause to consider the critical theoretical response to the position that we find ourselves in. At one remove, the process of creating the model of the image reduces the human to a set of constructs, such as average width between the eyes, which is then broadcast to the viewer. The digital provides the ability to replicate the image into a variety of forms from the same underlying data and the results of the imperfect structures pass into language. Benjamin’s assertion that “reproduction detaches the reproduced object from the domain of tradition” (2015:215) can be operationalised to interrogate the structure in its new tradition.
This paints an oppressive picture, but Berry’s iteracy, “the ability to read, write and understand processes” (Critical Theory and the Digital190), is a key to understanding the artefact and to interact with its relocation of epistemology. I want to use iteracy as a form of praxis to interact with the new tradition through either play or revealing glitches.
Reflecting on the roots of iteracy as iteration, I want to think about how it can be used to repeat a process, perhaps with alterations, to allow the algorithm to be the point of interaction. Through making changes, user meaning can be given to the machine to continue hypothesis testing. The repetition of these processes provides a space for the human thought to enter the process and realise the potential of Ramsay’s (2011) algorithmic criticism to reconceive both the form and criticism’s logics in a playful form. Tweaking the parameters and repeating the process not only reveals the process through which the picture is made but also allows humans into the iterative loop and realise the hypothetical nature of the work through experimentation. This site of interaction moves human cognition into the machine so as to embed the concept of thinking with the machine and its models.
Iteracy’s root as literacy provokes questions of how one might read or listen to the results as abstractions and patterns. The act of interacting with the process embeds a human element in part of it, suggesting that the object being read comes from thinking through a network. Next Rembrandt may be read as an image but to understand it, one needs to consider a new practice of reading.
Hayles’s cyborg reading, where the “reader necessarily is constructed as a cyborg,spliced into an integrated circuit with one or more intelligent machines”(“Print Is Flat, Code Is Deep” 85) encourages this new form. Using a machine to write data suggests that it is required to read and remediate it, so using it as part of the interpretation through the models encoded into the process. It may be mediated through visualisation or sonification processes, providing another area that needs to be understood. Instead of reading data, we read models that affect models as a strategy of not reading (Clement 2008). This practice accepts that the quantity of information cannot be read at a close level, but that broad patterns can be viewed and new questions asked. In cultural terms, this builds on Moretti’s concept that “distance is not an obstacle, but a specific form of knowledge: fewer elements, hence a sharper sense of their overall interconnection” (Graphs, Maps, Trees 1). These abstractions, allowing the reading of patterns over specificities, to be a media-specific reading of the digital structure representing culture that is enabled through algorithmic processes. Machine interpretation may also be fuzzy and not show outliers or emerging patterns if they are too slow and long,suggesting that the subtleties of emotion may be aggregated through counts into clusters of readings at the machine level.
Here I want to problematise the object, derived from Plato’s text, as the pharmakon,which Derrida suggests “acts as both remedy and poison” (Dissemination73). This suggests an alteration how we think the digital affects writing. Where Plato’s writing loses both access to memory and the underlying discourse, the object is central to both as the locus between humans and machine cognitive practices. It both creates and transforms the cultural forms, acting as memory and discourse to express them.
As human and machine discourses mix, they reconstruct their own context through translation and the processes. As the model is read and processed, its form is rewritten and recontextualised. Realising that the object is made up of these changes recognises Derrida’s différance, the gap between the signifier and the signified, allowing the reader to create their own meaning in the potential interpretations. Although based on a learned aggregate set of elements, Next Rembrandt’s eyes may be interpreted with emotion in a human reading. Where machine process may be limited in their meaning making,human readers may recognise the possibilities of the elements and to recognise emotions creates a series if interpretative gaps encoded into the machine processes.
Using Derrida’s argument that a “structure is a sort of writing” (Dissemination156), making the man expression of language and thought, then the location of meaning is constituted where the structures are written: the digital medium. Where these digital processes are normally invisible, they might be made partially visible through iteratic approach and mixing discourse.
Using Williams’s definition of structures of feeling as cultural hypotheses, this paper has argued that they might be seen models of thought that are translated into computational models. Through understanding the processes as machine discourse,we can think about how we can both interpret and interact with them. Iteracy encourages not only a different form of reading but also critical engagement with the underlying discourse, so considering the medium as the site of cognitive practice where discourses mix and create interpretative gaps. Rather than seeing the digital mediation of cultural forms as a machine-driven process, I contend that considering them using a media-specific analysis opens up new forms of interpretation and critical techniques that use the revealed discourse.
Benjamin, Walter. 2015 “The Work of Art in the Age of Mechanical Reproduction” Illuminations Bodley Head:London, 2015. Print
Berry, David M., Critical theory and the Digital. Bloomsbury Publishing: London, 2014. Print
Clement, Tanya E., n.d. How Not To Read A Million Books, http://people.virginia.edu/~jmu2m/hownot2read.html,last accessed 10.9.17. Web
Derrida, Jacques. Writing and Difference. Routledge: Abingdon, 2001
Derrida, Jacques. Dissemination. Bloomsbury: London, 2016
Hayles, N. Katherine, 2004. “Print Is Flat, Code Is Deep: The Importance of Media-Specific Analysis”. Poetics Today 25, 67–90. https://doi.org/10.1215/03335372-25-1-67
Manovich, Lev Software Takes Command. Bloomsbury: London, 2013. Print
McCarty,Willard, Humanities Computing, Palgrave Macmillan:Basingtoke, 2014. Print
Moretti, Franco. Graphs, Maps, Trees: Abstract Models for Literary History. Verso: London, 2007. Print
Plato,Waterfield, Robin (trans). Phaedrus, Oxford University Press: Oxford, 2002
Ramsay, Stephen. Reading Machines: Towards an Algorithmic Criticism. University of Illinois Press: Urbana, Chicago and Springfield, 2011. Print
Weizenbaum, Jacob. Computer Power and Human Reason: From Judgment to Calculation, WH Freeman & Company: San Francisco, 1976. Print
Williams, Raymond. Marxism and Literature. Oxford University Press:Oxford, 1977. Print