Category: Implicit Interaction
Last Wednesday I had my 50% PhD seminar! Sill processing the feedback from my opponent Lone Koefoed Hansen who Zoomed in from Aarhus University, but very thankful for an inspiring discussion.
A short overview of my research can be found here Careful Design: Implicit Interactions with Care, Taboo, and Humor. And an even shorter snapshot of 2.5 years in 16 seconds:
Having only recently returned to work full-time in March from ten months of parental leave, and despite working from home rather than in the lab due to COVID-19, it has felt wonderful to re-engage with my research with new energy and focus. Even more rewarding was to recently find out that my Doctoral Consortium proposal was accepted to DIS 2020. Very much looking forward to discussing my work with the Chairs and other participants – even if most likely virtual! Below is my title, abstract, and a link to the pre-print.
Today my paper Do you have to pee? A Design Space for Intimate and Somatic Data will be presented at DIS 2019 in San Diego by a colleague from KTH as I am currently on parental leave with my four week old daughter. Would love to be at the conference presenting and discussing this work, but am more grateful for the conference’s flexibility and Vasiliki Tsaknaki’s assistance in the presentation of this paper while I am instead home in Sweden!
Since this research includes an autobiographic approach and a reflection on my positionality as a design researcher, both of which are critical to the work, the presentation I prepared included both video and audio recording of myself intermixed among a script for Vasiliki (which is why the script includes a mix of first and third person). It was definitely a new challenge rethinking how to make a presentation that included my own voice while using my colleague’s presence for audience engagement! Below is the full written script and slides, and the full paper (which won an honorable mention award!) is available here.
Couldn’t be more please to find out today that I have a full paper accepted to DIS 2019! Very thankful for my amazing colleagues who provided invaluable support for and feedback on the paper.
Below is the title and abstract, will post a camera ready version in the coming weeks.
Do you have to pee? A Design Space for Intimate and Somatic Data
The management of bodily excretion is an everyday biological function necessary for our physiological and psychological well-being. In this paper, I investigate interaction design opportunities for and implications of leveraging intimate and somatic data to manage urination. This is done by detailing a design space that includes (1) a critique of market exemplars, (2) three conceptual design provocations, and (3) autobiographical data-gathering and labeling from excretion routines. To conclude, considerations within the labeling of somatic data, the actuating of bodily experiences, and the scaling of intimate interactions are contributed for designers who develop data-driven technology for intimate and somatic settings.
Today was our department’s annual PhD Supervisory Panel at KTH during which PhD students are given the opportunity to get feedback from senior researchers who act as “guest supervisors.” To prepare for my meeting with two Associate Professors I reworked my research abstract and research questions following my 30% seminar in October, during which Johan Redström from Umeå Institute of Design acted as my discussant. My goals in today’s supervisory panel were to get feedback on the new scope of my abstract and research questions relative to being only about 40% through my PhD (and I’m sure will continue to evolve), and identify important areas that I need to work on articulating to more firmly position my research in the context of how I am conceptually “furnishing” my design space. Considering I was presenting to senior researchers from different academic backgrounds than my own and each other, it was especially helpful to see within which aspect I felt misunderstood, i.e. where I need to sharpen my arguments. Below my poster summarizing my research thus far, followed by a few notes/reflections based on feedback received today, and a textual version of my abstract and research questions.
Yesterday I had the pleasure of spending the day at Umeå Institute of Design (where I did my MFA in Interaction Design). In the morning I spent a couple hours with upcoming IxD master thesis students to discuss my experience, and in the afternoon gave a talk titled “Crafting Humorous Fictions & Taboo Frictions” as part of the design school’s Wednesday lecture serious. Was great to be back, even if only for a day! Below is the abstract of my talk.
Karey Helms is a PhD Student at KTH Royal Institute of Technology researching smart implicit interactions, those that are unseen or unnoticed yet proactively operate on our behalf. Her research through design approach includes speculative and autobiographical methods in which she designs humorous fictions and taboo frictions with intimate and somatic data to surface the social and societal implications of data-driven systems. These include the designing of fictional devices that predict when and how badly one has to urinate, and the prototyping of a genuine system to spy on her partner.
In this talk, she traces back her playful approach to fiction and friction to her master’s thesis in Interaction Design at UID and how she employed this approach while working in industry delivering actualized services within enterprise IoT prior to beginning her PhD. The aim of this talk is to advocate for humor in design and to craft experiences that disrupt and disturb to not only provoke others to think, but also yourself as a designer.
For this year’s Advanced Project Course in Interaction Design at KTH, I’m fortunate and excited to have two groups of second-year master’s students working with me on two project briefs – Spying on Loved Ones and Tangible Designs for Effort, Exertion, and Exhaustion during Outdoor Experiences. Excited to see their resulting design work in December!
Spying on Loved Ones
To spy on someone is commonly thought of as a negative or harmful act during which one person secretly observes or collects information on another for malicious purposes. Yet, we also “spy” for many positive reasons. For example, these might include watching a sleeping child through a baby monitor, checking a partner’s calendar to plan a surprise birthday party, or browsing the fridge of an elderly parent to ensure a healthy diet is being consumed. What these examples have in common is that they are acts of care. This projects investigates how we care for others through technology, both with technologies explicitly designed to survey or observe others and through technologies designed for other purposes yet leak implicit information that enables spying. Within this investigation, various forms of spying will be classified and critiqued, the social and cultural implications of these positive intentions with be explored, and speculative prototypes will be designed to either further enable or inhibit spying.
The project will follow the suggested steps: (1) a review and critique of technologies explicitly designed for spying in private and public spaces, (2) investigation into ways in which people spy on others through technologies not designed for spying, possibly through workshops, interviews, or cultural probes, (3) the design of speculative prototypes to either further enable or inhibit spying, resulting in an ecology of physical artifacts. Required skills include strong design sensibilities and an interest in critical design and design fiction.
This project builds upon Leaky Objects: Implicit Information, Unintentional Communication.
Tangible Designs for Effort, Exertion, and Exhaustion during Outdoor Experiences
Multi-day outdoor activities such as hiking, skiing, cycling, and horseback riding involve full body engagement, during which bodies become sore and often endure physical transformations. Yet despite perceived discomfort and difficulties, outdoors experiences are often considered rewarding and relaxing. Thus, rather than designing to make these activities easier or more comfortable, this project investigates designing tangible artifacts and devices to celebrate or enhance these experiences of effort, exertion, and exhaustion. This project is part of a broader, ongoing outdoors project within the Smart Implicit Interaction project, and has the opportunity to build upon recent interviews with participants who have engaged in long-term outdoors experiences.
The outcome of the project is open-ended, but students are expected to engage in a design process to that result in novel artifacts that are either proposed objects of use (e.g. products) or result in interesting new learnings (e.g. cultural probes). Students should have a design background or be interested in following a design process.
Last week I presented my paper Design Methods to Investigate User Experiences of Artificial Intelligence at the 2018 AAAI Spring Symposium on the User Experiences of Artificial Intelligence. The picture above was taken by a Google Clips camera, which was also one of the presented papers. Below are my slides, which contain supplementary images to my paper on the three design methods I have engaged with relative to the UX of AI. Another blog post will soon follow with reflections on other papers presented.
Today I saw a meme on Instagram which said, “We are now entering the third month of January.” I couldn’t relate more! And looking back over the past few weeks, cannot believe all that has already happened in 2018.
Following a paper deadline in early January, my department at KTH (Media Technology and Interaction Design) kicked off 2018 with a writers’ retreat. What happens at a writers’ retreat? We book a venue in the Stockholm archipelago for three days and two nights, and write. And sauna and winter swim, but mainly write. The primary purpose of the retreat is to provide time and space away from everyday academic duties, from teaching to admin responsibilities, in order to focus on increasing the quality and quantity of our writing output. During the three days, we follow an agile framework in which junior/senior pairs write in ~45 minute sprints and then provide ~15 minutes of feedback. In addition to intense writing blocks, lunches, dinners, and evening activities provide ample opportunities to better know our colleagues professionally and personally. Though equally as exhausting as the writing, this social time I find incredibly valuable in creating a continued collaborative culture at work.
During this year’s writing camp I started a paper on a Pee-ometer, a recent project by Master’s students that I proposed and supervised in which they prototyped a wearable device that predicts when a user has to pee to investigate Machine Learning as a design material.
In mid January, the Smart Implicit Interaction project had a two day project offsite. As the project is composed of differing philosophical and methodological backgrounds – i.e. Artificial Intelligence, Social Sciences, and Interaction Design – the first day consisted of a beginners overview into reinforcement and representational learning in neural networks to introduce technical terminology and objectives. During the second day, all of the sub-projects presented their current status and goals for the year. I specifically presented two ongoing design projects, data-driven design methods and the Pee-ometer. In the former, I discussed early design activities and resulting concepts from investigating the implications of screenshots as a data source. In the latter, I discussed three high-level interests guiding future project directions, including Machine Learning as a design material, interactional loops, and critique and ethics. Overall, it was inspiring to share and strategize better collaborations while revisiting overarching project objectives.
Last week continued January’s streak of out-of-office research activities and into the forest. To kick off an new outdoors project, myself and three senior researchers went on a mid-week day hike 30 minutes outside of Stockholm. Not only was I surprised at a Professor’s ability to make a fire in the snow, but the excursion was both refreshing and constructive. More in the coming months!
And last but definitely not least, friend and fellow PhD student Vasiliki successfully defended her thesis Making Preciousness: Interaction Design Through Studio Crafts. Her opponent Ron Wakkary gave a much deserved brilliant presentation of her work before lengthy discussions with him and the committee. Admittedly, it is selfishly bittersweet to see her finishing as she has been a tremendous support and inspiration during the first year of my own PhD.
For an autobiographical project building off Leaky Objects, in which I’m allowing myself to undertake a significantly slow design process, this week I read Geography of Home: Writings on Where We Live by Akiko Busch. The book is a collection of beautiful musings organized by room and touches upon notions of comfort, dichotomies between possessions and spaces, clutter, deeply personal rituals, thresholds, processions, and changing functionalities. So much I love in this book while mapping the geography of our tiny home which defies all traditional expectations and conditions. Last year before moving from London to Stockholm, we read The Life-Changing Magic of Tidying Up, an amusing read that we nevertheless abided by and asked everything we own, “Do you bring me joy?” Not only did we end up giving away bags of stuff, but it also prompted many meaningful discussions regarding and aligning views on possessions and a home, in addition to musings around if someday everything will be connected, will all our possessions? And why do we keep them, and do we deserve them if possession implies care, custody, and guardianship? More and more through many of my ongoing projects and initial concepts, I realize I’m interested in overlaps between privacy and care, and perhaps comfort too.
A project I recently started with MSc students is investigating predicting pee habits, which to many is a taboo topic due to the invasive nature of the proposed interaction and an aversion to recognizing private bodily functions. A subsequent discussion with my supervisor on researching taboo topics led to the reading of the paper Accountabilities of Presence: Reframing Location-Based Systems and the website Between the Bars: Human Stories from Prison. In the paper, the authors research paroled sex offenders who are tracked via GPS to explore the intersection between mobility, presence, and privacy; while the website offers a digital platform to share handwritten content from people in prison who do not have access to internet. In both cases, what could be considered extreme or fringe users are researched or designed for, and equally interesting for different reasons, they also serve as interesting case studies that either directly learn from or address the messy realities of society. One of my biggest peeves – designers don’t address the messiness of life nearly enough. So while pee might not be prison, it is definitely an everyday and occasionally very messy, mundane, and universal reality that should not be ignored as I believe there is much to be gained from deviating from designing only delightful experiences.
On a fun, related note, I recently found out that Engelbart conceived what he called the “tinkle toy”, a small waterwheel in a toilet bowl that would spin when pee was run over it, serving as a potty-training aid for boys as the interaction was designed to be an incentive to pee in the toilet. From Markoff’s What the Dormouse Said: How the Sixties Counterculture Shaped the Personal Computer Industry.
A week or two ago, it was difficult to miss Algorithms as culture: Some tactics for the ethnography of algorithmic systems on Twitter. Found it incredibly informative and a nice compliment to Dourish’s The Stuff of Bits: An Essay on the Materialities of Information, which I started last week for implicit interaction book club. The paper also led me to these slides The algorithm multiple, the algorithm material: Reconstructing Creative Practice – in which I was thrilled to see my former UVa architecture professor and brief employer Jason Johnson of Future Cities Lab mentioned.
Trying something new with the aim of semi-regular updates regarding what I’m reading, working on, and thinking about.
Last week had a short chat with Ann Light from the University of Sussex regarding my work and attended her guest lecture Framing Wonder: Beyond Design for Existential Crisis (sketch notes above). Dark yet very inspiring talk and highly recommend her related paper Design for Existential Crisis from CHI17. Both the talk and her paper inspired me to look into care ethics and feminist theory regarding my work.
For a recent paper and my ongoing multi-disciplinary project work, have been thinking a lot about Terry Winograd’s Shifting viewpoints: Artificial intelligence and human–computer interaction, in particular how to work across philosophical and methodological differences.
Based on a colleague recommendation, recently listened to Our Robots, Ourselves: Robotics and the Myths of Autonomy by David Mindell, a very accessible contribution to discourse on the meaning of autonomy. Throughout the book, within each of many case studies, he repeatedly challenges listeners (readers) to ask, ‘Where are the people, which people are they, what are they doing, and when are they doing it?” as he proposes an emphasis on human-computer relationships rather than the idea or potential of replacement. My favorite of his examples regards remote control of Mars rovers from Earth bound operators and the coinciding debate if presence can be felt at such a distance and with significant time delays – is remote presence real presence?
Last week discussed Things That Keep Us Busy: The Elements of Interaction by Lars-Erik Janlert and Erik Stolterman in a research book club with a focus on Implicit Interaction. I found the book an easy read as it was very much like a collection of papers (which most of it draws from). While I appreciated the provision of definitions, frameworks and objective perspective – as in taking the object’s instead of the user’s perspective – overall there was much I disagreed with, in particular the notion that implicit interaction decreases interactivity, and found it difficult to maintain a narrow perspective that doesn’t regard the social or systemic implications. Perhaps also, it seemed to contrast philosophically with Dewey’s Art of Experience which I am reading for a PhD course on the theoretical foundations of UX, and as a design researcher situated in a technical university am over-resisting the application of frameworks upon a field dependent on embracing the subject, chaotic, and messy nature of the world.
Attended a great talk by my friend Lorenzo at IxDA Stockholm’s event Questions and answers in the design process with Nordic Morning. Great nuggets regarding the materialization of cultures to engage people in discourse and materialization of infrastructures to enable designers to transform them. Definitely calls for a revisit of his paper Materializing infrastructures for participatory hacking.
Lastly, in addition to a bunch of course grading, this week I’ll be continuing concepts derived from a recent project workshop on designing with data and starting a new project with three KTH master’s student on prototypes to probe the notion of training when approaching Machine Learning as a design material. A related paper includes UX Design Innovation: Challenges for Working with Machine Learning as a Design Material.
Two weeks ago I had a lovely time speaking about my current research interests and questions surrounding Smart Implicit Interactions with industry practitioners at EuroIA 2017 in Stockholm!
Thanks EuroIA for the INTENSE photo above and Inge Nahuis for the sketchnotes below!
As previously mentioned, I attended, and thoroughly enjoyed, the workshop at DIS17 on People, Personal Data and the Built Environment. Not only was the topic relevant to my project, interests, and background, but I also found the strict structure (and quick introductions) very effective and thus resulting in meaningful discussions at the end of the day.
The below image is a fraction of a future IoT system followed by a corresponding fictional narrative created with Albrecht Kurze.
2022 – five years after 2017 – a public space odyssey
It is the year 2022, five years after the government started to implement the dynamic waiting management system in public buildings. To simultaneously reduce waiting times while keeping visitors preoccupied, the system routes visitors on an adaptive, and often the least efficient, way through the building. As this has unsurprisingly resulted in many lost visitors, a place-based location-aware voice-controlled guiding and help-and-get-helped system was introduced, in which visitors can leave voice messages to aid other lost visitors.
Dave was born in 1950 and retired in 2017, the same year his wife died. Every time he enters public buildings, he is asked to confirm the usage and data processing terms of the building, as smart building are being classified as interactive data processing units by the 2019 extended GDPR. These temporary consents are based on minimum viable data and are thus only valid for a single visit as all data collected is automatically deleted or anonymized upon leaving.
Dave pretends to apply for a hunting license but actually just wants to hear his wife’s voice in a message she left in the help-and-get-helped system after the system’s implementation. As the system is gender intelligent, he needs a female to find his wife’s message. Furthermore, the message is not locationally linked nor directly addressable because of dynamic shuffling and the anonymization policy.
Claire was born in 2001 and has been applying for a family planning permit for the past five days. As the system is implicitly regulating family planning, prioritizing lonely widows and widowers for which Claire is not, it sends her on an impossible route. As she is not successful in her quest for a permit by the end of each day, the temporary data consent causes her to restart the whole process the following day, resulting in a never ending journey.
But this one special day Dave and Claire met, two lost visitors, trapped some way in and by the system. They decided to help each other and resolve their unfinished business.
Last week at DIS 2017 in Edinburgh I presented a poster (pdf) for my work-in-progress Leaky Objects: Implicit Information, Unintentional Communication. Very thankful for the discussions and feedback!
Yesterday I attended the workshop Workshop on People, Personal Data and the Built Environment at DIS 2017. Was incredibly inspiring and perhaps the best workshop format I’ve experienced as of yet! Below is my position paper.
Implicit Interaction: Information, Intention and Infrastructure (position paper PDF)
In this position paper I present three in-progress design projects that are initial explorations into Smart Implicit Interaction, which is investigating data as a design material and a new paradigm of interaction for the Internet of Things. The first project, Context Clues, critically examines our existing interactions and exchange of implicit information across digital mediums. The second project, Manuals of Misuse, is an Internet of Things design brief in which students are exploring the peripheral intentions embedded within everyday objects to design novel connections while exposing hidden patterns of behavior and engagement. The third project, Phygital Layers, is an architectural study seeking to understand the implicit relationships between physical, technological and social infrastructures within domestic environments. While all three projects differ across scale and medium, they offer potential avenues of investigation into designing for people, data and the built environment.
Very excited to be giving a 20 minute talk on September 30th at EuroIA in Stockholm on Implicit Interactions: Implied, Intangible and Intelligent! Below is my talk abstract.
As our physical and digital environments are ubiquitously embedded with intelligence, our interactions with technology are becoming increasingly dynamic, contextual and intangible. This transformation more importantly signifies a shift from explicit to implicit interactions. Explicit interactions contain information that demands our attention for direct engagement or manipulation. For example, a physical door with a ‘push’ sign clearly describes the required action for a person to enter a space. In contrast, implicit interactions rely on peripheral information to seamlessly behave in the background. For example, a physical door with motion sensors that automatically opens as a person approaches, predicting intent and appropriately responding without explicit contact or communication.
As ambient agents, intelligent assistants and proactive bots drive this shift from explicit to implicit interactions in our information spaces, what are the implications for everyday user experiences? And how do we architect dynamic, personal information in shared, phygital environments?
This talk aims to answer the above questions by first introducing an overview of explicit and implicit interactions in our mundane physical and digital environments. Then we will examine case studies in which unintended and unwanted consequences occur, revealing complex design challenges. Finally, we will conclude with example projects that explore a choreography between explicit and implicit interactions, and the resulting insights into architecting implied, intangible and intelligent information.
Both very unexpected and exciting to have a work-in-progress accepted to DIS 2017 in Edinburgh. I wrote about an ongoing autobiographical design project that was started this spring along side a few other explorations to kick-start my PhD at KTH this year. Very thankful to Kia Höök who talked me through my insight at a KTH Interaction Design writer’s camp in March and the reviewers who gave great feedback on how to continue developing it into a full paper.
Leaky Objects: Implicit Information, Unintentional Communication (PDF)
This paper introduces the concept of leaky objects to describe this phenomenon in which shared objects unintentionally reveal implicit information about individual or collective users. This leaking of implicit information changes our individual interactions with objects to through objects, enabling expressive communication and ambiguous speculation. The aim of this paper is raise awareness of this phenomenon through an ongoing autobiographical design probe in which remote interpersonal communication through a connected object is being explored, and raise questions regarding the potential implications for designers.
Below is the design brief that I wrote for the master’s level course in which I am a Teaching Assistant. The course is Interaction Design as a Reflective Practice.
Technological forecasts often predict that someday in the future everything will be connected. What is everything? And why is connected so often synonymous with tangible interactions being transferred to mobile applications or voice-based assistants? Additionally, these solutions primarily focus on efficiency and automation, signaling a shift from engagement to a frictionless relationship with technology. Is this shift necessary, or do our physical things have overlooked abilities, hidden meanings or magical uses?
The project Manuals of Misuse investigates these questions by examining our everyday interactions with faceless objects and reimagining how they might be misused to playfully control or meaningfully communicate with other things, people or places.
You will do this by:
- Reflecting upon the intended interactions of an everyday object of your choice
br>What is the essential use of the object and the ideal scenarios of interactions?
- Communicating how the object is meant to be used, interacted with and part of an ecosystem
br>What information is needed to communicate to a novice user, product partner, or manufacturer?
- Investigating how the object is or could be misused – used for purposes not intended by the designer
br>What are the object’s properties and/or affordances that result in these misuses?
- Generating novel design concepts of playful, meaningful or magical misuse
br>What modifications, interactions and technologies are required to make this possible?
- Creating a stand-alone, multimedia experience that conveys your reimagined misuse
br>What is needed to communicate the essence of your concept and prompt further investigation?
- Reflection: Upon our mundane, micro interactions with everyday objects. How to we use things and why do we use, or misuse, them? What can we learn from misuse and how does it inform how open or closed we design for appropriation?
- Communication: Of design intent, technical specifications and user experience to diverse audiences through various mediums.
- Playfulness: For engaged action and meaningful interaction.
- Documentation: For reflection and communication.
Very excited that I’ll be attending two workshops at CHI’17 in Denver! Below are links to the workshop descriptions and my position papers.
Workshop 1 – Making Home: Asserting Agency in the Age of IoT (description)
The Family Circuit: A New Narrative of American Domesticity (position paper PDF)
As the world endures and approaches a string of energy crises, The Family Circuit: A New Narrative of American Domesticity, aims to critique and challenge society’s relationship with energy by provoking individuals to examine their current habits of energy consumption, consider the future implications of these actions, and question their willingness to make sacrifices for a cleaner environment. This is accomplished through the development of a fictional near future society in which individuals are required to produce all the electrical energy that they need or desire to consume. Within the daily narrative of a fictional family of five, the events of their domestic lives have been extrapolated to create a liminal world where mundane, yet peculiar diegetic prototypes create tense situations, uncomfortable behaviors, and unforeseen consequences. The project raises questions regarding local to global contextual considerations, behavioral change within the home diegesis, and hyper-localized hacking.
Workshop 2 – Designing the Social Internet of Things (description)
Phygital Party Mode: A Relationship with Relationships (position paper PDF)
In this position paper, I present an exploratory autobiographical design project, Phygital Party Mode, in which visitors’ interactions with my website remotely control an Internet of Things light within my apartment. I reflect upon my relationship with the project as a ‘thing’ and explore the themes of active versus reactive agency, conditional relationships and designing the behaviors of objects. Finally, I end with discussion questions that address a transfer of agency due the democratization of the Internet of Things, the transformation of relationships with and because of connected ‘things,’ and the empowerment of people over objects.