The Rise of the Affective Machine: And a Little Bit of Resistance – Anja Breljak

When in the 1990s computer scientist Rosalind Picard startet to envision the field of affective computing, she reports, the problem of „too little emotion“ (x) in computers has not been adressed properly. Instead, decision making (which is integral part of computing/artificial inteligence) has been based on rationality only, which, as Picard argues empathically, is not enough to make computers „genuinley intelligent, to adapt to us, and to interact naturally with us“ (Picard x). Today, it seems, this genuinley artificial intelligence is still not on its way. What we do face, instead, is the rise of affect-sensitive technologies that started to shape and change social relations and political institutions. It is not only by companies advertising these technologies such as Affectiva (the Startup founded by Picard herself), that affect recognition is being deployed. We can find less explicit but similar technologies in social media anlytics as well as in user experience design (and: what, today, is not user experience by design?).

In this paper I argue that it is precisely because of the strategic ambiguity of affect as a concept (and not: despite of it) that this field of technological innovation has been flourishing in the last decade or so. And I will make the (short!) case for rationality (though I will not argue for conceptual precision!) when it comes to resistance and critique of these new technologies and their effects.

Affect as a topic of theoretical inquiry has a long and complex tradition, but it is only during the 1990s that it became subject of international academic discussion dubbed as „The Affective Turn“ (Clough/Halley). The writings of Brian Massumi, who draws on GillesDeleuze’s interpretations of Spinoza, have been playing a prominent role in these discussions. It is his rich and paradox notion ofaffect (and his poetic writing) that have set the cornerstone for the Affect Studies.

Massumi clearly rejects the belief that affect can be extensively captured by technological means. In his writings he argues for a notion of affect that is to be regarded as a characteristic of bodily and inter-corporeal processes, which take place primarily in a pre-individual world of becoming (aka the „virtual“). Thus, affect for Massumi is a “nonconscious, never-to-be-conscious autonomic remainder (…), as disconnected from meaningful sequencing, from narration, as it is from vital function” (25). On the other hand – and this is where Massumi is trying to give a comprehensive idea on how to access affect, on how to theorize it in the first place – he has to locate affect somehow, and sees it as “most directly manifested in the skin – at the surface of the body, at its interface with things ” (25). For Massumi, it isprecisely this tension between non-conscious residual and physical manifestation that enables a potential to think of affect as ontological power and source of resistance. In this, the body becomes a “sensor of change”, and functions as a “transducer of the virtual” (135). And, contrary to common usage, the virtual is not in digital technologies, it is not to be equated with simulation (Baudrillard), but something, „a mode of thought“ that only occurs in the analog and that digital technologies seek to capture/connect to in order to actualize.

Picard’s notion of affect, interestingly, is not that far away from Massumi’s. In her 1997 book on Affective Computing she states:

„If computers are to utilize the natural channels of emotional communication used by people, then when computers learn to recognize human emotion, they will have to rely primarily on sentic modulation, as opposed to having people explicitly tell them the names of their emotional feelings. To give computers affect recognition requires understanding the physical manifestations of emotion.“ (Picard 26)


Three things are remarkable in this statement: First, it is noticeable that Picard makes no distinction between affect and emotion. On the contrary, affect seems to be a kind of terminus technicus, if not to serve as an umbrella term here in order to encompass emotion, feelings, bodily reaction, even communicational subtleties. Second, Picard assumes that the bodies („the natural“ in the channels) and their channels of communication themselves (whatever they are) bring about a sort of modulational process that can be (technologically) sensed („sentic modulation“). In other words: she states that whatever people communicate through their bodies (may it be their voice, their mimics, or their electrodermal activity) is saying something true about people, even if they can not or do not want to say it themselves (or if they are saying something different). And third, it becomes explicit here that, according to Picard, affect or emotion can be accessed trough the physical manifestations of the bodies – but the nature of this manifestation remains open.

 So, what are the possible manifestations? What manifestations are prevalent in nowadays affect-sensitive technologies? What channels exactly are there? What strategies do we find in this field of new innovations?

I think there are (at least) three main strategies (or: structures) that reside in the fields of the symbolic-dynamic, the expressive and the vital. They need to be considered when it comes to the ontological power of affect nowadays:

  • The processing of symbolic input dynamics: Big parts of communication today is based on symbolic input, such as writing/typing words, clicking on links, liking pages, sharing pictures, retweeting, using emoticons, etc. This kind of input is being used in order to analyze the polarity, the vector, the emotional valence and affective patterns of users, the virality of content, the intensity of relationships and the effects of certain user experience modes („A/B-Testing“). Some of this input is also being displayed to its users, for instance when counting clicks or likes or assessing reviews with stars or maybe even providing „sentiment analysis“ for the readers of your online magazine (check out german newspaper „DieZeit Online“).
  • The processing of expressive flows and movements: With ubiqutious computing came also ubiquitous camera and microphone usage usually recording non-stop. Given a certain quality of these recordings, they provide the basis for face and voice recognition. By analyzing the movement of muscles, of posture, the speed of speech or its tonality, even the rhythm of typing and clicking, extensive information on user’s reaction can be drawn from. This is not only used in marketing („Realeyes“, „Google Assistant“), but especially inpredictive policing.
  • The processing of vital functions: Smartphones and wearables come with sensors detecting, among other things, activity, pulse, body temperature, even blood volume and oxygen level or skin conductivity. The more we equip ourselves, our homes and public spaces with (smart) sensors, the more this „inner“ information of bodies themselves becomes instructive for intervention, assessment and self-reflection.

Of course, these three strategies are not being used separately, they intertwine in complex and often obscure ways since this kind of processing mostly takes place in R&D-departments of big tech-companies or is being offered by startups as special insight given they are provided with the relevant data sets.

Nevertheless, what stands out, is the sort of knowledge this technolgical complex is about to produce (or at least is promising): affect-sensitive technologies are about extracting a „truth“ about subjects/users that escapes their own attention, ability or consciousness, so it becomes a more reliable „truth“ about them. This is why affect comes into play: not only is it a dynamic and relational category, it also promises insight into the body before the subject can even name it (or is willing to do so) since it works like an „autonomic remainder“ (Massumi). And, also, affect opens up the field of situative on-going: when using the body as „sensor of change“ (Massumi), many interesting conclusions can be made, may it concern bodily effects of environmental change, social effects (such as anxiety, depression or agreeableness) or political upheavals (at borders for instance, where infrared cameras are being used to detect refugees). Today, our bodies (their symbolic action, their expressive flows, their vital functions), our usage of ubiquitious and wearable digital media technology, and the growing confluence of bodies and media, is at the center of technological and capitalist interest. In this, affect-sensitive technologies produce a new, situative knowledge of the human, that not only changes what it means to be human today, but how humans can be governed, mobilized and tweaked, how they can be observed and predicted, on an individual scale as well as on a collective scale. And this is not only unprecedented, it still is pretty unclear where this will lead us (but since it seems to be powerfull, it will not necesserily lead into utopia). What becomes clear is that Massumi is probably right: The digital will never capture extensively the virtual (the analog/affect). But that is, unfortunately, the reason why there will be ever more venture capital and innovative energy flowing into technological advancements trying and promising exactly that: capturing the virtual/affect/the analog.

This brings us to the following conclusions: First, this new kind of knowledge about affects does not only play a big role in the tech-industry, where it is being capitalized and used to produce new media formats, or by the police or intelligence services. This kind of knowledge is also becoming part of our everyday life, may it be through the usage of smartphone apps that promise to help managing your feelings („Woebot“) or in social meda dynamics where knowing what others think and feel about certain content in real-time changes our societal self-awareness. So, when we say that these technologies are changing our affect cultures today, it seems, this happens through a new form of (real-time) perception of individual and societal reactions that we could call a big-data-based episteme (Foucault).

Second, we can not assume that the concept of affect (or any concept), if it is formulated only ambiguously enough and if it possibly connects us to „the virtual“ or the „pre-individual“ is already politically charged in the „right“ way and full of potential for resistance. The field of (capitalist) digital technologies is very eager to open up new markets and enter fields full of promises and openings, where imaginaries and potentials can be envisioned and, in the end, capitalized. So, what then could be a strategy for resistance?

There, for sure, is not just one answer to this. But, one, I would like to discuss, we can get from Massumi who draws on something I would like to call „poetic rationalism“. With this term I have a twofold strategy in mind: On one hand, it is obvious we need to talk about these kind of technologies, even though they mostly remain obscure or are hard to understand. Talking about it, generating (transmedial!) discourse, that could make perceivable what is going on when we use digital media today and what social/societal effects they bring about. And this, obviously, has already started. But, what needs to be the goal with this is provoking more „Einsicht“ (understanding, insight, sense) among users. It is not about bringing knowledge to the people/masses, it is about finding ways to describe and to adress these transformations in ways that it relates to people, that it affects and provokes involvement, to form „ideas“about what is going on (Spinoza). So, it is all about the mode of describing and discussing. If poetics is the art of „density“ (german: „Dichten“), coming from the greek word „poesis“(englisch: making/creating), it is about creating dense/thick descriptions that are able to escape from „meaningful sequencing, from narration“ (Massumi) but nevertheless can make you contemplate an idea of what is going on.

References:

Baudrillard, Jean. Simulacra and Simulation, The University of Michigan Press. 1994.

Clough, Patricia T.; Halley, Jean. The Affective Turn. Theorizing the Social. Duke University Press. 2007.

Massumi, Brian. Parables for the Virtual. Movement, Affect, Sensation. Duke University Press. 2002.

Foucault, Michel. The Order of Things: An Archaeology of the Human Sciences.Vintage. 1994.

Picard, Rosalind. Affective Computing. MIT Press. 1997.

Spinoza, Baruch de. Ethics: Proved in Geometrical Order. Cambridge University Press. 2018.

4 Comments

  1. Hello Anja, thanks so much for your interesting work on affect and computing. I was curious about your take on biometrics and wearable devices, especially in relation to the quote you excerpted from Massumi:

    ” On the other hand – and this is where Massumi is trying to give a comprehensive idea on how to access affect, on how to theorise it in the first place – he has to locate affect somehow, and sees it as “most directly manifested in the skin – at the surface of the body, at its interface with things ” (25). For Massumi, it is precisely this tension between non-conscious residual and physical manifestation that enables a potential to think of affect as ontological power and source of resistance. In this, the body becomes a “sensor of change”, and functions as a “transducer of the virtual” (135)

    Do you think that biometric data and the information it collects about the body–e.g. elevated heart rate signalling an anxiety-state–is an attempt to make affect available to measurement and computation?

    I’m also curious about the implied syllogism or interchangeability of the virtual/analog/affect, which appears when you write: “The digital will never capture extensively the virtual (the analog/affect). But that is, unfortunately, the reason why there will be ever more venture capital and innovative energy flowing into technological advancements trying and promising exactly that: capturing the virtual/affect/the analog.” How do you see the relationship among these three things?

    Like

  2. Hi Anja, Thanks for your post.

    In the post, you mention the social media applications such as Woebot. It reminded me of the Samaritan’s Radar application (https://en.wikipedia.org/wiki/Samaritans_Radar) which had to be taken off Twitter as it did not act as desired when acting with humans. Does that failure says anything about how complex emotions can be misread?

    I’m intrigued by the poetic rationalism being ‘about creating dense/thick descriptions that are able to escape from „meaningful sequencing, from narration“ (Massumi)’. Would you be able to comment further on that?

    Like

  3. Anja,
    Thank you for this interesting article! I agree with you that it is also because of the “ambiguity of affect as a concept” that affectivity in artificial systems is getting more and more popular. If affectivity / emotion / feeling (…) were clearly definable – how could we understand machinic affectivity in its today’s realizations? As some kind of affectivity or something else?
    Furthermore, I am also concerned about the new kinds of data coming from human affectivity that is collected by machines. What concept of affectivity will be reproduced, and maybe in the end, dominant? How could this shape our understanding of affectivity?
    Looking forward to discussing with you,
    Maike

    Like

  4. Hi Anja,

    Thank you for your text!

    I am very interested to find out more about your position on truth and the role it plays in the discourse. The knowledge production by digital technologies in terms of sensing, recording, tracking that you describe aims at determining nonverbal, bodily signs and by that transferring them not only into computable but also verbal expressions of “the self” and its social relations. It seems to me that there might be a similarity to your ambition when it comes to cultivating as discourse, for finding ways of describing the phenomena that are not yet addressed or verbalized in a way that reveal real “truths” about “what is going on”. What is your position on truth in knowledge production (technologically and theory wise)? And in what way can theory become means of resistance in this context? Do you aim for a resistance against the determination of the undeterminable or against a capitalization on new modes of determination and subjectivisation?

    Though you have described some similarities between the conceptions of the affect in Massumi’s writings, Picard’s writings, and technological applications, I still ask myself what role does it play in your argument that they have been developed in different contexts, with different aims, working on different objects? Do all this conceptions hold an ambiguity in play for “affect” or is the ambiguity a result of these differences? Do they explicitly refer to each other? Is that important for you?

    And just an annotation: Your notion on “poetic rationalism” reminded me terminologically of “thick description” – an ethnographical method developed by Clifford Geertz (that problematizes the possibilities of misunderstandings in the process of reading and interpretering cultural articulations as symbols and also emphasizes the necessity of reflecting the researcher’s situatedness in the description process) and also of Joseph Vogel’s concept of “poetologies of knowledge” (that might relate to your emphasis on the mode of comprehension/description).

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s