Resisting Touchscreen Sensibilities: The Potential of Non-User-Friendly Interaction Design – Tomasz Hollanek

In the late 2000s, Apple released the original iPhone, a new kind of apparatus equipped with a sensory system of its own, a touchscreen, a built-in camera, an accelerometer, a proximity sensor, a gyroscope, and other sensors – a device that could at once display and register images, connect different users across a distance, and react to light intensity, movement, and speed; it was both a screen – but one that could gaze back at the viewer, respond to his or her touch, heartbeat, and position in space – and a controller, a remote for executing tasks. The controller-screen seemed like the ultimate remediation (to use Bolter and Grusin’s term), [1] realizing our desire for instantaneity and immediacy, mobility and interactivity, manipulability and control. Now, more than a decade later, each of us carries a personal touchscreen, a device that can respond to its master’s voice, recognize his or her face, or track its user’s steps; it is always there, always at hand. By contrast with the classical screen of cinema, the controller-screen moves with us, like a parasite, seamlessly transforming our perception, concealing its influence. 

The mobile touchscreen is not only a specific kind of frame that ends up machining human vision, as touchscreens have become extensions of our sensory system that allow us to feel technologically. Mark Hansen has recently remarked that the ‘becoming topological of culture’ – the forging of topological relations among ‘elements of worldly sensibility’ by contemporary media machines [2] – demands both our reconceptualization of sensibility and phenomenology, [3] as today’s topological machines ‘provide artificial access to a domain of sensibility that exceeds what humans can process as sensations.’ [4] The controller-screen and other wearables not only determine what and how we see, but indeed how we ‘calibrate’ our bodies, how we orient them in the new, augmented reality. Different devices ensure that our sense of orientation is closely tied up with an illusion of control; while visualization masquerades as comprehension, touch colonizes space. With a controller-screen at hand the viewer-user can directly manipulate the universe fashioned out of data. Wandering off course is not possible in this world with a home button.

Ulrik Ekman argues that in the reality of ubiquitous computing (a term adopted by Mark Weiser in 1988, describing a sociocultural turn to integrate computing pervasively into everyday objects and bodies), environments themselves begin displaying ‘intelligent attention’ to individuals and social groups: ‘natural setting turns highly artificial as it appears attentive rather than neutral or non-caring’ – it constantly interacts with the viewer-user, responding with a directedness ‘coming not from distant otherness,’ but ‘intimate sameness.’ [5]  As the dominant perceptual norm, touchscreen sensibilities necessitate a design that obfuscates mediation; ‘intuitive’ design must feel both intimate and natural to allow the interface to erase itself and pass as an extension of the organic. Friedrich Kittler expressed his concerns about modern media technologies that are ‘fundamentally arranged to undermine sensory perception.’ [6] According to Kittler, the computing community ‘places all its stock in hiding hardware behind software, and electronic signifiers behind human/machine interfaces.’ [7] He even refers to a ‘system of secrecy’ based on user interfaces that ‘conceal operations necessary for programming’ and thus ‘deprive users of the machine as a whole.’ [8] The question of perceiving the imitation, penetrating through the illusion of software ‘that appears to be human,’ becomes – as Kittler seems to suggest – a fundamental one in the world of automated, machinic perception that on many, invisible levels keeps the human users informed and entertained, but unaware.  Ongoing developments in user experience design rely on dynamic, fully customizable interfaces that automatically adapt to the viewer-user’s needs, seemingly responding to his or her desire before it is consciously articulated. With advances in user profiling, a process of generating statistical models from large amounts of data produced by users, the diverse mobile applications can now predict, and attune their messaging to, the users’ sexual orientation, political affiliations, or even their menstrual cycle. As the interface facilitates not only the consumption of digital goods, but also self-tracking, it seemingly invites the viewer-user to become self-conscious; self-tracking, however, serves only as a prosthesis of the project, an illusion of individuation aiming to collect ever more data.

Bernard Stiegler’s conception of individuation in the age of ‘hyper-industrial’ capitalism – formulated in Technics and Time and Symbolic Misery, two pivotal series of works written over the course of ten years spanning the late 1990s and early 2000s – revolves around the paradoxical relationship between the illusion of personalization and the massification of cultural consumption – the ways in which audiovisual technologies control ‘the conscious and unconscious rhythms of bodies and souls,’ by exploiting the aesthetic and treating consciousness as ‘raw material’ in the process of production. [9] Broadcast media, Stiegler argues, function as pervasive systems of synchronization, relying on temporal objects such as TV programs or songs (objects whose affective potential is inscribed in their very duration), that standardize the time of consciousness to format the consumer’s behavior and manufacture his or her desires. While broadcast media have in fact laid the groundwork for drawing ‘the time of consciousness’ into production, personalized entertainment of the digital era, available anytime and anywhere, might complete the project by soliciting our attention on a full-time basis.

Although the controller-screen seems to realize the promise of ‘control vision,’ unlimited mobility and haptic immediacy, it also becomes a means of capturing information about the preferences and habits of users and turning the collected data into profit for corporations like YouTube. The question I would like to ponder is whether we can still conceptualize user experience design that enables ‘non-machine’ ways of feeling? In the age of instagrammable art, food porn, and YouTube playlists, with the progressive confusion of meanings, media, and experiences, and with no distinct boundaries between image, video, and code, is there cultural potential in formats that rely on what one could call ‘non-user-friendly’ design – one that poses a challenge to the logic of immediacy, one that denies the viewer-user the power to manipulate the image and register reactions to it, one that stages an experience that cannot be immediately turned into data – is there potential to transform the existing feedback loops into a new system of commentary, to de-automatize choice? Can we conceive of interaction design that reconnect the viewer-user with his or her time of consciousness, or – in other words – attunes consciousness to the lived body?

Non-user-friendly interaction design would have to call our new, AI-enabled ‘structures of feeling’ into question and to mobilize the viewer-user by staging a defamiliarizing  experience of visual culture. To think of a model for such interaction, one could consider, for example, the cultural potential of the classical screen – a seemingly passive surface for receiving projections – in the era dominated by the controller-screen – an interactive, mobile touchscreen, an interface connecting the user to an algorithmically curated world of information, providing access to anything from bank accounts and YouTube clips, to our memories stored as data in the cloud.  In the attention economy, the classical screen – precisely because it is only a passive surface that cannot ‘gaze back’ at the viewer-user – could prove a means of resisting the culture of touchscreen sensibilities. In the age of big data profiling and personalization of entertainment, by staging a different, less intuitive, experience of visual culture, freeing cognition from the mobile ‘frame’ of the controller-screen, the older medium that requires physical engagement on the part of the spectator could prove a means of paradoxical ‘de-framing’ of contemporary perception and defamiliarizing the act of consumption of images. While the controller-screen, an everyday enhancer of sensation, conceals its influence, the classical screen reverts the logic of touchscreen sensibilities – responding to our desire for immediacy without a complete erasure of hardware through software.

I could refer to my own experience of seeing Björk’s 2015 retrospective at the Museum of Modern Art in New York to explain how our increasingly technologized sensorium can be confronted with the scale of the classical screen – how vulnerability can replace mastery, and machine vision give way to the sensuous body. I am thinking specifically of one exhibition room designed from scratch to house ‘Black Lake,’ an immersive music/film installation commissioned by MoMA for Björk’s new song from her album Vulnicura: the use of mobile devices is not allowed inside; the ‘viewer-user’ has to renounce the comfort of total immediacy provided by the controller-screen upon entering, moving from the luminous atrium to a completely dark screening room. The design of this experience is anything but ‘user-friendly.’ The contrast between seeing and non-seeing, light and darkness, elicit a sense of confusion and render the viewer vulnerable, as he or she cannot use the controller-screen to navigate the unknown; disorientation forms part of the experience. Inside, Björk’s music video is projected on two wide, opposing walls of the room. The projections do not mirror each other, but rather constitute complementary elements of the viewing. The spectators thus find themselves in between two screens competing for attention. The feeling of dizzying immersion is intensified by the acoustics, with a design based on a computer-generated sound map of the interior. Thanks to the interplay of multiple surfaces, projections, refractions, and reverberations, as well as the hypnotic style of the music video itself – barely noticeable vibrations of the camera, slow cutting, Björk’s ecstatic performance – the spectator’s body becomes an active instrument of artistic production in this quasi-organic, cave-like space; it is a part of the multi-sensorial process of creation; struck by intense, penetrating sounds, the body extends itself into the space.

In her book on the ‘virtual window,’ Anne Friedberg reminds us that ‘the frame of the moving-image screen marks a separation – an “ontological cut” – between the material surface of the wall and the view contained within its aperture.’ [10] To resist the dominance of touchscreen sensibilities, non-user-friendly interaction design should stage precisely such an ontological rupture: an experience that breaks the dataflow and challenges rather than satisfies our desire for immediacy. For the contemporary viewer-user, whose expectations of audiovisuality are shaped by touchscreen sensibilities, the ‘cut’ simulated by non-user-friendly design should go deeper, separating not only physical space from virtual reality, but also consciousness from technology, and memory from data. Bernard Stiegler has recently admitted that while the new, interactive screen could be ‘a threat, enacted through the mediation of the fully computational and automated system,’ it could also ‘constitute a chance, an opportunity to renew commentary, to reconnect with the “gloss,” through a completely rethought hermeneutics.’  To live a vita activa, he argues, we must hold on ‘to the promise of a new hermeneutic epoch borne by these screens.’ [11] The idea of non-user-friendly design modeled on the experience of the classical screen – a technology that cannot gaze back – could be a starting point in our thinking about how to resist the dominance of touchscreen sensibilities.

[1] Jay David Bolter and Richard Grusin, Remediation: Understanding New Media (Cambridge: MIT Press, 1999), 47.

[2] Mark B. N. Hansen, ‘Topology of Sensibility,’ Ubiquitous Computing, Complexity and Culture, edited by Ulrik Ekman, Jay David Bolter et al. (Routledge, 2016), 34.

[3] Ibid., 37.

[4] Ibid., 39.

[5] Ulrik Ekman, ‘Introduction: Complex Ubiquity-Effects,’ Ubiquitous Computing, op. cit., 1.

[6] Friedrich A. Kittler, ‘There is No Software,’ The Truth of the Technological World: Essays on the Genealogy of Presence, translated by Erik Butler, (Stanford: Stanford University Press, 2014), 221.

[7] Ibid., 223.

[8] Ibid., 224.

[9] Bernard Stiegler, Symbolic Misery. Volume 1: The Hyperindustrial Epoch, translated by Barnaby Norman (Cambridge: Polity, 2014), 2.

[10] Anne Friedberg, The Virtual Window: From Alberti to Microsoft (Cambridge: MIT Press, 2006), 5.

[11] Bernard Stiegler, The Neganthropocene,edited and translated by Daniel Ross (London: Open Humanities Press, 2018), 173.

 

4 Comments

  1. Thank you for the interesting read, Tomasz. Sensible observations on the pursuit of immediacy in common interaction/interface design and how we might reclaim embodied, lived experiences by de-habituation. And even (temporary) detachment from media, as your experience of Björk’s 2015 MoMA retrospective illustrated. Calls to mind one of Alexander Galloway’s political orientation toward interfaces (The Interface Effect, 2012). One of ”whatever”: disengagement, indifference, negation. Opting out to potentially something new. This points to one of several unifying motifs that I find across papers shared so far. Say:

    1) rupture as means to knowledge and revealing the apparent in the transparent. Contextualizing this in the (techno-)aesthetic realm, some papers (e.g. Carleigh’s on compression hacking) explore glitch as such spaces of rupture.

    2) ”ontological cuts” and ”agential cuts” (Irina Raskin): both seem to indicate some sort of boundaries, with the former apparently more definite and the latter more concerned with identifying elements and their dynamics in meaning-making, if I read it rightly.

    3) bodies as biomediated (Rebecca Uliasz), bioinformatic (Anja Breljak), and as described in your paper, raw material. Papers of the thematic session ”Alien Desire” evoke/invoke the Deleuzian control society to discuss this aspect.

    4) your proposal of older screens as subversive, paradoxical de-framing of contemporary perception and image consumption. Does that suggest a productive return to the ”analog-ish”? (This might be colored by my recall of Massumi’s chapter on the superiority of the analog.)

    Do you have any further thoughts on these? Hope this might spark some discussion/ideas.

    Like

  2. Tom, great piece! I wondered what you might make of the rise in monetised touchscreen apps that are designed to limit our screen time, acting as a technological prosthetic to enable us to exercise a modicum of self-imposed (or app-imposed) control over our screen usage. What do we make of screens as devices which propose to help us moderate the attention we pay to them–e.g. Apple’s latest iOS updates permits users to set caps on screen time and to track their attention as it is distributed across applications.

    You write: “In the attention economy, the classical screen – precisely because it is only a passive surface that cannot ‘gaze back’ at the viewer-user – could prove a means of resisting the culture of touchscreen sensibilities.” Could you elaborate on how such “disobedient devices” or “static screens” might operate, and to what critical end?

    Like

  3. dear tomasz, thx for sharing!

    here few thoughts:

    – given that at the beginning you reference to the parasite i wonder if you’e interested in Serres’ parasite as an interesting ways to look at information technologies and their way of working – gazing back as an abusive a-symmetrical relation between user and tech.

    – i also wonder how r you integrating hansen and kittler in your research. the latter occluding the possibility of human sensibility into what he calls the “optoelectronic future”, while the other seeing opportunity in the abusive / parasitic relation between human and technology as a way to re-invent human perception.

    – I’m personally working on the ways technologies are gazing back in relation to the Lacanian asymmetry between eye and gaze which breaks the Merleau-Pontian reversibility between seeing / seen (i wrote about this in section II and then again in V and VI of my paper).

    – i think the challenge you’re throwing is fundamental and yet i have a bit of difficulties in understanding the effective viability of returning to the passive screen as a model to regain human political agency over algorithmic governance. how is this even possible in pragmatic terms and given technological trends?

    – in relation to your challenge, I’m not convinced by Friedberg’s take over the ontological cut enabled by cinema, because if there’s a cut the cut is immediately bridged by the phenomenological nature of cinema (movement) and especially by the figure of POV which for me is the hidden (and forbidden) genetic element of cinema and the aesthetic and technological figure behind new algorithmic technologies of vision.

    – furthermore the passivization of the screen is actually undergoing as a strategy to mask the system of secrecy you refer to – automatic youtube play and similar gives a sensation of passivity which is actually founded on profiling and creation of what i try to refer to as POV-data doubles (data selfies).

    – i think de-framing is a great wording fo the project you’re undertaking, and i think it can be richly approach by looking at the issue of framing and affectivity Hansen is talking about in relation to what he defines as the DFI Digital Facial Image (you can find his paper easily online on muse) and to what you call non-machine ways of feeling. I’ve tried to show how de-framing (in your wording) is at stake with POV tech that harness non-machine feelings (in your wording) by transforming Hansen’s Digital Facial Image into AFI Algorithmic Facial Image, able to re-frame (rather then de-framing as in the case of the DFI) the affective production behind the functioning of this new type of image. https://www.academia.edu/37522968/The_Algorithmic_Facial_Image_and_the_Relation_between_Truth_Value_and_Money_Value.pdf

    looking forward to talk more!

    Like

  4. Hi Tomasz. Thank you for your productive post. I cannot help but read a lot of speculative & critical design into your piece, yet you do not mention this tradition. Are you distancing your work from that of SCD, or are you unfamiliar with it? If so, I think it could be generative for you to read into.
    Following this, I have a question about your example (Björk’s exhibition at MoMA). What is the role of art in the endeavour of ‘resisting touchscreen capabilities’? Or rather, what might be kinds of things that art (as exhibited in museums) cannot do, and what other strategies may be adopted to further the ‘resistance’ in a broader context, without reverting to the ‘app-imposed’ self-tracking that carleighmorgan refers to above? Could such a resistance exist outside of the museum context — and if so, where and how?

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s