Iteracies of Feeling: A Media Specific Approach to Cultural Forms – Iain Emsley

Computational reading of culture allows us to pose new questions or create new cultural forms supporting new forms of critical thinking. The resulting data and interfaces require new ways of understanding.  Reflecting through a distant reading project, the aim of this paper is to suggest a media-specific approach to understanding how the techné affects critical practice and epistemology. In the first part, I attend to structures of feelings and models encapsulating the world and frame Williams’s definitions as a model. The second part opens up Berry’s (2014) iteracy as theory and the basis of practice to find meaning in the models. The third part presents a Derridean reading of the technical processes and how these can be used to create a space for meaning to exist.

Hayles’s media-specific analysis “attends both to the specificity of the form … and to citations and imitations of one medium in another” (“Print Is Flat, Code Is Deep” 69), altering our critical relationship with machines and raising a crucial role for questioning the medium itself as site of cognitive practice through remediation. The consideration of structures of feeling, I contend, requires not only human reading but also technical reading itself using models to understand the digital. I suggest that this develops Hayles’s cyborg reading, reading through a distributed cognitive system, a technique to interpret the media’s discourse.

I briefly want to frame the issues using the Next Rembrandt[i] project. Using Rembrandt’s portraits, machine learning algorithms pored over labelled data to extract features to create a new painting. These included the facial proportions,type of light, the tilt of the head, and the paint layers on the canvas. Through a machine reading of many aspects of one artist’s painting, the final printed painting revealed layered effects of brush work in digital and physical media.

In many aspects, this is a technically demanding reading. We might feel the sadness and warmth in the sitter’s eyes or the slightly worn look derived from the way the light plays on the features and through the layers of paint. At this moment, I contend that we are inside an interpretational loop. The machine uses aggregations of the models and the data to create a new set of data points derived through a model. Its surface is a visualisation, where numerical mappings mediate the data into a new point, which humans perceive as colour, at a location. From a human perspective, we note the stylistic similarities, the attention to detail in the style and the emotion in the face. There is a disjunction here between the two readings that reveals the need for conceiving about how this can be critically approached.

I begin with a consideration of Williams’s construction of structures of feeling as “a cultural hypothesis, actually derived from attempts to understand such elements and their connections in a generation or period, and needing always to be returned, interactively, to such evidence” (Marxism and Literature133). Computational reading derives features from the data based on human thought and interpretation of the hypothesis, either in the construction of algorithms or labelling of data. Once identified, the features may then be analysed or combined to create new structures and elements.

 The element’s existence and its interpretative possibilities as part of an emerging discourse is problematised through this process. The process of translating a feature into a series of technical languages to create a new model and element alters the discourse and its specificities. Manovich’s (2013) claiming of the digital as a metamedium – capable of transforming existing media and creating new media and technologies – highlights the need for a media-specific analysis. Through the work on reading the paintings, algorithms search for common patterns and shapes as the close reading of the painting collection. This reading is enhanced through work on the demographics of Rembrandt’s subjects, placing the work within a social context and on the physicality of the paint through its layers. At each stage the media specificity of the elements is acknowledged and remediated into the required form such as digital or 3-dimensionally printed image, but these translations develop the dis-junction between the generated layers of meaning.

I want to turn to models as an integral part of these computational structures. McCarty (2014) considers computational systems as dependent on the models given to them to understand a conception of the world, echoing Weizenbaum’s conception of models as a “symbolic recreation of [the designer’s] world” (Computer Power and Human Reason18). The use of AI to create data sets and models problematises this by raising questions of who is the designer and whose world is being created? The model’s structure of an element rests on how the designer or implementer translates and transcodes it into their work. McCarty’s argument that the “[m]odelling of something readily turns into modelling for better or more detailed knowledge … modelling for something feeds or can feed into an improved version” (Humanities Computing 27) suggests a feedback or learning process based on an understanding. I read McCarty’s assertion that “modelling is making new knowledge by manipulating hypothetical constructs” (Humanities Computing71) through grammatology as a process of challenging the technical language’s specificity. Acknowledging that modelling “cultural artefacts treats them as something like the empirical objects of nature; … paradoxically modelling anything is just as clearly an imaginative act” (Humanities Computing72) surfaces the inbuilt issue of trying to define the abstract into the concrete. The emotional structure of feeling becomes an imperfect structure of feeling.

A key point is that the Williams’s issue with the specifics what constitutes an element in discourse is further problematised through translation and encoding required for the machine to understand them as hypothetical constructs. From this, a new discourse is created from the results, which require reading. The underlying computer model both makes and is made from the translation. This alters the location of epistemology from the reading and interpretation to within the digital. A necessary consequence is a potential change of the location of the element’s negotiation. Whilst it may happen as part of wider cultural discourse, it is happening within the algorithms and their models of the world.

 I want to take a brief pause to consider the critical theoretical response to the position that we find ourselves in. At one remove, the process of creating the model of the image reduces the human to a set of constructs, such as average width between the eyes, which is then broadcast to the viewer. The digital provides the ability to replicate the image into a variety of forms from the same underlying data and the results of the imperfect structures pass into language. Benjamin’s assertion that “reproduction detaches the reproduced object from the domain of tradition” (2015:215) can be operationalised to interrogate the structure in its new tradition.

This paints an oppressive picture, but Berry’s iteracy, “the ability to read, write and understand processes” (Critical Theory and the Digital190), is a key to understanding the artefact and to interact with its relocation of epistemology. I want to use iteracy as a form of praxis to interact with the new tradition through either play or revealing glitches.

Reflecting on the roots of iteracy as iteration, I want to think about how it can be used to repeat a process, perhaps with alterations, to allow the algorithm to be the point of interaction. Through making changes, user meaning can be given to the machine to continue hypothesis testing. The repetition of these processes provides a space for the human thought to enter the process and realise the potential of Ramsay’s (2011) algorithmic criticism to reconceive both the form and criticism’s logics in a playful form. Tweaking the parameters and repeating the process not only reveals the process through which the picture is made but also allows humans into the iterative loop and realise the hypothetical nature of the work through experimentation. This site of interaction moves human cognition into the machine so as to embed the concept of thinking with the machine and its models.

Iteracy’s root as literacy provokes questions of how one might read or listen to the results as abstractions and patterns. The act of interacting with the process embeds a human element in part of it, suggesting that the object being read comes from thinking through a network. Next Rembrandt may be read as an image but to understand it, one needs to consider a new practice of reading.

Hayles’s cyborg reading, where the “reader necessarily is constructed as a cyborg,spliced into an integrated circuit with one or more intelligent machines”(“Print Is Flat, Code Is Deep” 85) encourages this new form. Using a machine to write data suggests that it is required to read and remediate it, so using it as part of the interpretation through the models encoded into the process. It may be mediated through visualisation or sonification processes, providing another area that needs to be understood. Instead of reading data, we read models that affect models as a strategy of not reading (Clement 2008). This practice accepts that the quantity of information cannot be read at a close level, but that broad patterns can be viewed and new questions asked. In cultural terms, this builds on Moretti’s concept that “distance is not an obstacle, but a specific form of knowledge: fewer elements, hence a sharper sense of their overall interconnection” (Graphs, Maps, Trees 1). These abstractions, allowing the reading of patterns over specificities, to be a media-specific reading of the digital structure representing culture that is enabled through algorithmic processes. Machine interpretation may also be fuzzy and not show outliers or emerging patterns if they are too slow and long,suggesting that the subtleties of emotion may be aggregated through counts into clusters of readings at the machine level.  

Here I want to problematise the object, derived from Plato’s  text, as the pharmakon,which Derrida suggests “acts as both remedy and poison” (Dissemination73). This suggests an alteration how we think the digital affects writing. Where Plato’s writing loses both access to memory and the underlying discourse, the object is central to both as the locus between humans and machine cognitive practices. It both creates and transforms the cultural forms, acting as memory and discourse to express them.   

As human and machine discourses mix, they reconstruct their own context through translation and the processes. As the model is read and processed, its form is rewritten and recontextualised.  Realising that the object is made up of these changes recognises Derrida’s différance, the gap between the signifier and the signified, allowing the reader to create their own meaning in the potential interpretations. Although based on a learned aggregate set of elements, Next Rembrandt’s eyes may be interpreted with emotion in a human reading. Where machine process may be limited in their meaning making,human readers may recognise the possibilities of the elements and to recognise emotions creates a series if interpretative gaps encoded into the machine processes.

Using Derrida’s argument that a “structure is a sort of writing” (Dissemination156), making the man expression of language and thought, then the location of meaning is constituted where the structures are written: the digital medium. Where these digital processes are normally invisible, they might be made partially visible through iteratic approach and mixing discourse.

Using Williams’s definition of structures of feeling as cultural hypotheses, this paper has argued that they might be seen models of thought that are translated into computational models. Through understanding the processes as machine discourse,we can think about how we can both interpret and interact with them. Iteracy encourages not only a different form of reading but also critical engagement with the underlying discourse, so considering the medium as the site of cognitive practice where discourses mix and create interpretative gaps. Rather than seeing the digital mediation of cultural forms as a machine-driven process, I contend that considering them using a media-specific analysis opens up new forms of interpretation and critical techniques that use the revealed discourse.  

Bibliography

Benjamin, Walter. 2015 “The Work of Art in the Age of Mechanical Reproduction” Illuminations Bodley Head:London, 2015. Print

Berry, David M.,  Critical theory and the Digital. Bloomsbury Publishing: London, 2014. Print

Clement, Tanya E., n.d. How Not To Read A Million Books, http://people.virginia.edu/~jmu2m/hownot2read.html,last accessed 10.9.17. Web

Derrida, Jacques. Writing and Difference. Routledge: Abingdon, 2001

Derrida, Jacques.  Dissemination. Bloomsbury: London, 2016

Hayles, N. Katherine, 2004. “Print Is Flat, Code Is Deep: The Importance of Media-Specific Analysis”. Poetics Today 25, 67–90. https://doi.org/10.1215/03335372-25-1-67

Manovich, Lev Software Takes Command. Bloomsbury: London, 2013. Print

McCarty,Willard, Humanities Computing, Palgrave Macmillan:Basingtoke, 2014. Print

Moretti, Franco. Graphs, Maps, Trees: Abstract Models for Literary History. Verso: London, 2007. Print

Plato,Waterfield, Robin (trans). Phaedrus, Oxford University Press: Oxford, 2002

Ramsay, Stephen. Reading Machines: Towards an Algorithmic Criticism. University of Illinois Press: Urbana, Chicago and Springfield, 2011. Print

Weizenbaum, Jacob. Computer Power and Human Reason: From Judgment to Calculation, WH Freeman & Company: San Francisco, 1976. Print

Williams, Raymond. Marxism and Literature. Oxford University Press:Oxford, 1977. Print


[i] Next Rembrandt, https://www.nextrembrandt.com/

Advertisements

5 Comments

  1. Iain,

    Thank you for this interesting article! Your idea of exploring new structures of feeling through cyborg or machine readings and manipulations of (Next) Rembrandt’s artworks is very intriguing to me.
    When I read your text, I had these thoughts: What kind of feeling is inherent in the artworks that are machinic remixes of humanly produced art? As you write, the human can manipulate the “remix” through iteracy again. This means, we first read feelings that Rembrandt painted. Moreover, the artwork can elicit human feelings that are not explicitly depicted (and thus cannot be read by the machine). Furthermore, if Next Rembrandt remixes some Rembrandt artworks, we have a new artwork with mechanically remixed depictions of feelings. Last, if a human manipulates the code and thus plays with the remix, the depicted and elicited feelings might change again. Maybe, could we get from this process some machinic interpretation of feeling that exceeds mere data points?
    Moreover, which structures of feeling (that concern the concept of feeling?) might come up through the different artwork readings?

    Looking forward to discussing with you,
    Maike

    Like

    1. Hi Maike, Thanks for the response and intriguing questions.

      I agree with you that if there is a re-reading through iteracy, it would alter the emotional responses and readings. It may even alter the painting and not necessarily for the better but I think it does support a critical reading of this painting’s emotions as a digital object created from a reading.

      I think that your question, “could we get from this process some machinic interpretation of feeling that exceeds mere data points?”, is a good one and perhaps I’ve gone down a route that is too reductive. In my reading, the algorithms are performing an intensely close reading of an artistic object but the (re)constructive algorithms see and fit the larger patterns to create the picture. In this picture, the machine is recreating existing emotions from an intense close reading in the underlying portrait set of form, texture and patterns. The algorithms learned patterns from the paintings before creating the new face using learned features from the composition, materials and the artist’s use of form to fit them into the picture. So perhaps the machine reads the human into its models before reprocessing them to understand the forms, such as the slightly lost eyes and the wistful mouth, as patterns.

      See you next week to discuss this further,

      Iain

      Like

  2. Hi Ian,

    Thank you for sharing your paper!

    I resonate with your suggestion to consider digital mediation of cultural forms by their media-specificity. And I ask myself, whether this results in regarding and understanding the cultures of computational reading or “iteracy” rather then merely “attuning” to a computational reading of culture?
    I am interested to find out more about your argument regarding the transition from an “interpretational loop” to a “iteracitc loop” and the relocation of epistemology – what gaps are there coming into play in addition to the interpretative gaps between “form and content”? In what way is the translative capability of the digital mediation process different to language or sign based translations? In what relation do you see epistemology and mediality?

    As you have mentioned Benjamin, I was wondering if his essay about “The Task of the Translator” is also relevant for your thoughts.

    Like

  3. Hey Ian!
    Thanks a lot for sharing your thoughts on machine reading and this really interesting project on the Next Rembrandt! I have more of an associative stream of thought on this:
    You have been adressing the media-analysis approach and the receptive side of this project‘s outcome. But what if we look at it‘s political economy? Interestingly the project has been funded by ING (a multinational bank and financial service company) and Microsoft (declining tech company). Both companies are using this project for boosting their reputations as innovative. In this context, the „machine reading“ of art is used to show the „power of data“ that can be applied „outside of business“, but as a business, for giving business a nicer and more artsy ‚face‘ (see [1]).

    Ironically, Rembrandt is one of the most copied artists in the world! He was copied even during his lifetime and is remarkably hard to distinguish from some of his fellow artists. And many of the works attributed to him are disputed in authorship (see the work of the Rembrandt Research Project which reduced the number of works assigned to Rembrandt from ~700 to ~350). For the dissemination of meaning and origin we did not have to wait for artificial intelligence. 😉

    I wonder, what your perspective is on other projects of artificial mimesis: check out this interesting article on the advances of AI-generated facial images: https://medium.com/@kcimc/how-to-recognize-fake-ai-generated-images-4d1f6f9a2842. Of course, producing a painting is a much more complex and intricated procedure. But then again: machine-reading the human face seems to have become banal reality, where only the glitches and distortions of automation will help us distinguish machinic and human outcomes, maybe…
    Looking forward to discuss further the Derridean perspective on this with you!

    Like

  4. Iain, thank you very much for your inspiring piece. Much brainpickings ahead at Cambridge, I imagine. To hop on the ongoing discussion, I would like to hear your thoughts on some areas:

    1) how do you conceptualize medium-specificity? If I follow your essay rightly, there are moments where the discussion seems to equally concern the materiality of media and the broader context, which involves factors of political economy, as mentioned in the earlier thread. Admittedly, my interest in this question is colored by my background in digital games research inflected by multimodality. Also, it is an important issue that my research group is problematizing. One (polemical) observation floating around is that medium-specificity might be a fixation, rather than a productive vector into empirical (media) analyses. (I am still processing/resisting this notion…)

    2) in your discussion regarding the integral role of models in computational structures, I find a statement “The emotional structure of feeling becomes an imperfect structure of feeling.” Could you please unpack that for me a little? Does that somehow mean that modeling inherently aims for false-proof replications, and this holds even for something potentially elusive like emotions and affects?

    (Note: I shared information of the Next Rembrandt project with my students of a seminar on critical game studies. We had interesting discussions. Thank you.)

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s