What does the mouse see? (Day 1)

A computer mouse conference (see the official site) has no business being as good as this one. Activists, early career artists and scholars in semiotics, anthropology, and design, each sought to answer to the question, “what does the mouse see?” A number of the presentations embraced playfulness, vulnerability, and earnestness, which seems like an apt and appropriate way to run an online conference in 2021.

Conference organizer and artist-researcher emma rae brunl norton (alongside Ashley Jane Lewis) summed her thesis up straight away: “the mouse is a way to understand our relationships with computers.” Dissecting the mouse, as metaphors and objects, cracks open our relationships with information machines. The conference offered new ways “in” to complicate that relationship and generate new sets of questions.

Sound issues and an antsy dog sometimes made arguments and connections harder to follow. But the gaps that emerged as we moved from history, theory, and design were potent places for the imagination to thrive.

This week’s entire newsletter is a collection of notes from that conference — which had me glued to my screen whenever I wasn’t walking my dog. (Apologies to Daniel Shiffman whose talk I missed!)

This is Day 1. I also have a blog post over at Cybernetic Forests (the website), which includes notes on Day 2.

The Spinning Wheel

Ingrid Burrington presented a short visual history of the wait cursor: “the spinning wheel of death,” tracking its development from black and white circle to stopwatch to the swirling rainbow whirlpool. The swirling rainbow, Burrington suggests, is based on the refraction of light on a compact disc or hard drive. In a brief literature review, Burrington found the counterintuitive impulse to romanticize the swirl, as something signaling the resolution of an outcome, a wish about to be granted. On the flip side, Angels are also spinning wheels, full of eyes, telling those who see it, “Do not be afraid.” The appearance of the lagging disc is part terror, part hope.

All Sensation is Touch

Nabil Hassein tells us that Cleopatra lived closer to the invention of the computer mouse than to the construction of the Pyramids of Giza. Citing Gera Lerner, Hassein notes that “Knowledge of history is knowledge that things have changed and that things can change.” (The Creation of Patriarchy, p.3). Today, the keyboard and mouse work in tandem, with some coders preferring the keyboard and hotkeys to navigate — and this is often the interface of choice for a variety of accessible interfaces. The mouse works in tandem, but is less intuitive. Its physical location does not map 1:1 with the space on the screen. Hassein ends with the question: What does the mouse feel? In the end, Hassein suggests, all sensation is touch: your eyes make contact with light to see, your ears make contact with vibrations to hear. The mouse feels our intention, and processes that into action and movement.

What does the desktop want?

For Cezar Mocan, the green hills and booting chime of Windows XP was “a seductive invitation to possibility” while growing up. But it’s led to a question: why is every manufacturer using landscape photographs for their default desktops? As the metaphor on the screen moved from a terminal to a visual one, it relied on objects people were familiar with: office spaces. Later, menus and submenus moved users more toward memory: knowing what was possible, and how to find it, was the core compromise of the desktop metaphor, moving away from the complex possibility of the command line to the constrained simplicity of the desktop: a visual arrangement of pre-selected choices.

Drawing on a history of landscape photography, Mocan sees that the landscape desktop brings nature to that environment: the mountains and skies suggest a return to vast possibilities; but also a sense of orientation to space. They lack people and animals (“can’t have cows compete with desktop icons,” one commenter noted). This suggests timelessness — an unbroken moment in time — but also suggests an idea of nature without the complexity of life. In answer to the question, “What does the desktop want?” Mocan suggests the desktop wants to forget. It suggests to us a timeless ideal of a natural world, capable of sustaining us forever in its green lush grasses, and removing any sources of friction that might intervene with bliss.

What do we point at?

Kameelah Janan Rasheed is a learner and artist, presenting work using mouse coordinates while typing poetry to generate music. While this was algorithmic sound composition, it was achieved using analog means: recording the screen while she typed, she then replayed that video with a transparency on the screen, making marks where ever the mouse moved. This was translated, on paper, into a grid, which created a structure for generating sound compositions. Rasheed ended with a note on the mouse as a kind of divination tool, a tool for pointing at our thoughts.

Semiotics of the Click

David Bering Porter’s incredibly deep presentation focused on the “click.” We measure the life of a mouse in terms of our physical interactions with it: 60 million clicks, for example, is the lifespan of a mouse used by extreme gamers. I was surprised to be reminded that the click is a physical sound, not only a verb — I had to click my mouse a few times to remind myself that yes, there’s an audible sound in that tiny switch being moved on and off with the tap. Porter tells us a click is a gesture, a sound, a metric. It’s a relationship: we click when we find affinity, the result of feedback. (For me, it’s also an idea sliding into focus: it finally clicked.)

All of these tell us something about the way our relationships, physical gestures, and interactions have been flattened into a single space: the space of clicks and clicking.

What I found fascinating from Porter’s talk is that the click was accidental, but kept, and in some cases it is simulated. Looking further, there are ways to rig a system to play a different sound when you click the mouse button.

I’m dreaming of a surveillance capitalism mouse, which plays a terrible cash register “ka-ching!” anytime your click is collected by data brokers. If anyone wants to build that with me, let me know. (Really).

Computer Mouse X Magic Wand

CyX is “a black queer non-binary storyteller and cyber witch merging sound, video art, installation, and performance.” Bringing in the background of magic wands, tools designed for health and focus (look at the World Health Organization logo) CyX suggests that the mouse can be treated as connectors between energy forces, moving our intentions into the digital realm. In a workshop, we were guided through tasks such as braiding plants and flowers into the tail of the mouse, the scents providing a reminder of that connection across two spaces. The mouse is a tool that connects our and work and relationships; why wouldn’t we adorn it in welcoming, contemplative, and inspiring ways? If time, relationships, and creativity is sacred, why don’t we create more space for the sacred in digital spaces?

The Mouse House

The always-inspiring Shannon Mattern (whose writing on Plexiglass is my go-to answer when people ask me “so what do you want to do?”) offered us a tour of tiny houses occupied by mice: the mammalian ones, the ones that Beatrix Potter anthropomorphized into clumsy, but studious working-class complexity. From that imagining of mice living alongside us in tiny homes built from our debris, we moved to the medical mouse: standing in for the bodies of humans in medical tests, living in caged grids, the original “mouse pad.” The mouse pad — the subject of this talk — rose from the need to prevent dust from gumming up the tracking ball inside the original mice. The optical mouse arrived, and then lasers — “turning the whole world into a mousepad” — and finally trackpads, which were the mouse and the pad and neither. But for precision work, and the market for gaming, the industry of the mouse pad is still alive and well, in a constant search for the perfect surface.

Does the mouse see race?

Charlton McIlwain — whose book, Black Software, is high up on my reading list — asks if the mouse sees race. To begin with, he points out that the father of the mouse (Douglas Engelbart) was a white man who was trying to solve a series of relational problems and used his body to solve them. Mark Dean, a black IBM engineer, would two decades later create a device that allowed for a broad range of peripherals to work with the same machine. These developments “made computer hardware bend to the users will,” rather than people bending to it, McIlwain says (citing a Usenet discussion). These tools opened the world of computing to more people, creating more points of relationship, including for black and brown people.

McIlwain then pointed to the Implicit Associations test: a tool designed to reveal unconscious bias. The tool scrambles pictures of diverse faces and words with negative and positive associations. You are supposed to click and sort specified combinations, and changes in reaction times for certain tasks can be measured. This test is not a diagnostic tool, and isn’t mentioned as an endorsement, but because of how much this test has shaped ideas and conversations around race. But the mouse is an intermediary for this unconscious bias and the screen used to test it, suggesting that the way we have come to understand implicit bias is now informed by our relationship with the mouse. In the end, the mouse “sees” race through our interactions.

Thanks for reading! For notes on Day 2 of the conference see the website copy and scroll down. Otherwise, feel free to share, subscribe to the newsletter, or come find me on Twitter.