It is hard to deny that our artifacts and environments are becoming more and more complex, more and more "alive," and as a consequence more and more demanding. We have to interact more. There seems to be no retreat or escape from interactivity. Some well-informed critics worry that the proliferation of interactions and interactive things has already gone too far. Their concerns raise many questions. Does interactivity in fact increase? How can we know? What does it really mean to claim that it does? And if indeed it is increasing, what does it mean? And why is this happening? And should something be done? Despite this development there seems to be no precise idea of what interaction is and what being interactive means, beyond a vague notion that it is some kind of interplay, usually optimistically understood as good-natured cooperation. In this talk I will present the work I have done for many years, together with my colleague Lars-Erik Janlert. As our approach, maybe best described as analytical and philosophical, we have examined properties and qualities of designed artifacts and systems; primarily those properties that are open for manipulation to designers, that is, properties that designers can and do intentionally affect by their design decisions (and thus in principle are possible to control). Rather than taking users and their subjective experiences of the artifacts and systems as the primary target for examination, unfashionable as it may be, we have chosen to be objective in the sense of focusing on the artifacts and systems. Apart from discussing our approach, I will briefly introduce some of our main results consisting of some developed definitions of existing (and some new) concepts, such as, interactivity, interactability, interactiveness. I will end with some comments on what this kind of investigation can tell us about the future by introducing the notions of faceless interaction, interactivity clutter, and interactivity fields.
This is an interesting era for design in terms of the diverse ways that designers need to work. The advent of cloud computing and the sensors in the mobile phones we carry means that there is a vast amount of data collected about what people do. There is a vast opportunity for designers to leverage this data to create products, services and systems that are tailored to individual needs. At the same time, designers at Google are designing products and services that millions of people use every day. This sets up an interesting dichotomy for today's designers, who need to take into account both personal and adaptive and global and scaled in how they think, design, and take action.
The notion of research in the field of design has a relatively short history compared to other well established academic disciplines, and therefore the relationship between design academia and design practice is not very closely intertwined. Particularly this disconnecting phenomenon becomes more serious as the rapid introduction of new technologies such as AI is challenging the validity of the existing body of knowledge, methods, and processes.
Faced with this new big changes the healthy cycle of coexistence of academia and practice is required more than ever. In addition, in order to establish a single academic discipline, mutual synergetic collaboration between academia and practice is inevitable. Practical designers regard research as something done in the pre-stage of the design concept development, whereas academia perceives research as an instrument for knowledge generation. Design research in practice and academia significantly differ in respect to goal, deliverables, applicability, methods, scope, venue, and other things.
What are the diversifying factors of design research in academia and practice, and how can they be interconnected while maintaining the advantages of being diverse?
The issue of diversity in design research will be addressed by some comparative research with interview with design researchers in industry and review of papers presented in design research conferences.
The act of designing technologies does not simply create functionality; it also offers possibilities for action, ways of looking at the world, and modes through which we can relate to one another. How we design technologies reflects what we value; who we think is important, and in what ways; which places, people and possibilities are in our imaginations, and which are not. Current ways of designing technologies frequently narrow these possibilities, in two ways. The first is that technology design is dominated by a narrow demographic: predominantly white and Asian, white collar, highly educated, urban. These designers' ways of imagining new technological worlds are shaped by the worlds they themselves know and value, which are only a small slice of global ways of being. The second is that even as technology design is being increasingly engaged in around the world by and for people outside this demographic, local adaptations are frequently judged and limited by what makes sense from the perspective of Silicon Valley and other urban high-tech centers. Supporting the rich diversity of human experience requires explicitly identifying and appreciating values and experiences outside of mainstream technology design logics.
Digital civics research seeks to understand how technology can create new forms of relationships and services between public officials and citizens in governance. To accomplish this, design in digital civics emphasizes the importance of relationships based on dialogue, empowerment, and participation; all of which are contingent upon the existence of trust. Currently, however, these relationships are most often characterized by entrenched distrust which problematizes opportunities for dialogue and participation. In this paper, we explore how design might support trust in the relational aims of digital civics. To do so, we led 13 public officials in a large US city through a design-based inquiry centered around the role of trust in their various efforts to engage communities. In our findings, we discuss four strategies for supporting trust in digital civics.
This paper describes the reparative Internet of Things (Riot), a project investigating the role of IoT devices in maintaining public resource accessibility. Drawing on a mix of interviews, technology development, and ethnographic engagements, we explore the distribution and stratification of menstrual hygiene resources in Seattle, WA. We redesigned menstrual product dispensers placed in public settings by outfitting them with networked sensor inserts to make them easier to stock by custodial staff and easier to access by members of the public. We use this case to show how such newly connected devices structure experiences of hygiene access and help expose important consequences of integrating those devices into the socioeconomic logics and infrastructure of public life. Our interventions further examine the role of public IoT devices once they pass the proof-of-concept stage, revealing their capacity to cultivate and maintain collective responsibility.
Modern Information and Communication Technologies (ICTs) support job searches, resume creation, career development, and professional self-presentation. However, these technological tools are often tailored to high-income, highly educated users and white-collar professionals. It is unclear what interventions address the needs of job seekers who have limited resources or education, or who may be underserved in other ways. We gathered insights from the literature and generated ten tangible design concepts to address the needs of underserved job seekers. We then conducted a needs validation and speed dating study to understand which concepts were most viable among our population. We found that the three most preferred concepts immediately addressed job seekers' social and personal needs, where addressing social needs meant mediating job seekers' connections to others and supporting our job seekers' limited access to strong ties.
In response to the recent call for a more intersectionally-aware field of human-computer interaction (HCI), we aim to operationalize intersectionality for technology design in HCI. We develop our lens of intersectionality by drawing on the work of Rita Kaur Dhamoon, and use it to analyze data collected from a multi-sited ethnographic study of seven low-resource learning environments in the Indian states of Maharashtra, Tamil Nadu, and West Bengal. Our research contributions are threefold. First, we extend conversations in Intersectional HCI by expanding its scope from understanding users to recognizing social processes. Second, we emphasize the importance of factoring in both penalties and privileges when conducting research in underserved contexts. Finally, we engage situated comparisons as a methodology to identify pathways for designing interactive systems across intersectionally diverse environments.
Physically walking in virtual reality can provide a satisfying sense of presence. However, natural locomotion in virtual worlds larger than the tracked space remains a practical challenge. Numerous redirected walking techniques have been proposed to overcome space limitations but they often require rapid head rotation, sometimes induced by distractors, to keep the scene rotation imperceptible. We propose a design methodology of seamlessly integrating redirection into the virtual experience that takes advantage of the perceptual phenomenon of inattentional blindness. Additionally, we present four novel visibility control techniques that work with our design methodology to minimize disruption to the user experience commonly found in existing redirection techniques. A user study (N = 16) shows that our techniques are imperceptible and users report significantly less dizziness when using our methods. The illusion of unconstrained walking in a large area (16 x 8m) is maintained even though users are limited to a smaller (3.5 x 3.5m) physical space.
Busy daily lives and ongoing distractions often make people feel disconnected from their bodies and experiences. Guided attention to self can alleviate this disconnect as in focused-attention meditation, in which breathing often constitutes the primary object on which to focus attention. In this context, sustained breath awareness plays a crucial role in the emergence of the meditation experience. We designed an immersive virtual environment (iVE) with a generative soundtrack that supports sustained attention on breathing by employing the users' breathing in interaction. Both sounds and visuals are directly mapped to the user's breathing patterns, thus bringing the awareness researched. We conducted micro-phenomenology interviews to unfold the process in which breath awareness can be induced and sustained in this environment. The findings revealed the mechanisms by which audio and visual cues in VR can elicit and foster breath-awareness, and unfolded the nuances of this process through subjective experiences of the study participants. Finally, the results emphasize the important role that a sense of agency and control have in shaping the overall quality of the experience. This can in turn inform the design specifications of future mindfulness-based designs focused on breath awareness.
Virtual reality can help realize mediated social experiences where distance disappears and we interact as richly with those around the world as we do with those in the same room. The design of social virtual experiences presents a challenge for remotely located users with room-scale setups like those afforded by recent commodity virtual reality devices. Since users inhabit different physical spaces that may not be the same size, a mapping to a shared virtual space is needed for creating experiences that allow everyone to use real walking for locomotion. We designed three mapping techniques that enable users from diverse room-scale setups to interact together in virtual reality. Results from our user study (N = 26) show that our mapping techniques positively influence the perceived degree of togetherness and copresence while the size of each user's tracked space influences individual presence.
Current approaches for locomotion in virtual reality are either creating a visual-vestibular conflict, which is assumed to cause simulator sickness, or use metaphors such as teleportation to travel longer distances, lacking the perception of self motion. We propose VRSpinning, a seated locomotion approach based around stimulating the user's vestibular system using a rotational impulse to induce the perception of linear self-motion. In a first study we explored the approach of oscillating the chair in different frequencies during visual forward motion and collected user preferences on applying these feedback types. In a second user study we used short bursts of rotational acceleration to match the visual forward acceleration. We found that this rotational stimulus significantly reduced simulator sickness and increased the perception of self-motion in comparison to no physical motion.
This paper presents a design case study from industry that explores designing for personal health ecosystems. Following on from previous work on ecologies that gives a predominantly theoretical perspective, we present a more applied and design-oriented perspective. To do so, we build on our previously-developed data-enabled design approach, which utilizes contextual, behavioral and experiential data from situated design experiments as creative material. This approach comprises two steps, of which this paper presents the first (contextual) step. We introduced a small adaptable ecosystem of multiple artifacts, in four family homes over a period of eight weeks, through which we explored and further expanded on valuable ecosystem relationships. The insights gained were translated into three design opportunities that inform a future second step. We highlight and discuss practical examples from our situated explorations, and discuss how our data-enabled design approach served in designing for the complexity and versatility inherent to these ecosystems.
Mental health wellness concerns are a significant issue among emerging adults due to the compounded stressors of life changes, new responsibilities, and career-related burdens. Due to a lack of support and the stigma associated with seeking help, they often do not receive adequate professional care. In this paper, we show how emerging adults seek help managing their mental wellness through social support. We conducted 19 interviews with students at a large university in the U.S. Midwest. Our study identifies that students maintain different social support groups, selectively sharing their concerns based on each group's role, benefits, and limitations. We call this support network the Mosaic of Social Support. We discuss opportunities for technology design that enhances the Mosaic of Social Support, allowing it to be more accessible, safe, and sustainable, thus providing tailored help to serve the students' unmet needs.
Self-tracked health data can help people and their health providers understand and manage chronic conditions. This paper examines personal informatics practices and challenges in migraine, a condition characterized by unpredictable, intermittent, and poorly-understood symptoms. To investigate how people with migraine track and use data related to their condition, we surveyed 279 people with migraine and conducted semi-structured interviews with 13 survey respondents and 6 health providers. We find four distinct goals people bring to tracking and data: 1) answering questions about migraines, 2) predicting and preventing migraines, 3) monitoring and managing migraines over time, and 4) enabling motivation and social recognition. Each goal suggests different needs for the design of tools to support migraine tracking. We also find needs resulting from an individual's goals evolving over time, their varied personal experiences, and their communication and collaboration with providers. We discuss these goals and needs in terms of opportunities for personal informatics tools to facilitate learning to: 1) avoid common pitfalls; 2) support customization and flexibility; 3) account for burden, negativity, and lapsing; and 4) support management with uncertainty.
Self-defining memories represent significant emotional events capturing the most important concerns in our lives. While much HCI work on memory technologies has focused on autobiographic memories and lifelogging technologies for capturing them, there has been little exploration of self-defining memories and how they may be supported by appropriate cues. This is important as such memories are key in the development and maintenance of the sense of self, particularly in old age. We report on interviews with 8 older adults in their homes. Findings advance the understanding of self-defining memories and their possible cues with new insights into their relationship with self-identities and cues' specific qualities supporting richer emotional recall. Our findings led to several design implications such as novel technologies for curating self-defining memories and their cues, for embedding layered meaning in such cues across the lifespan, for crafting them, and sensitive design for their physical handling.
Historically, women have been excluded from engineering and computer science disciplines, and interactive audio is no exception. Relatively few women are involved with the designing and building of embedded audio systems with traditional tools such as microprocessors, but when embedded audio systems are built using e-textiles, much larger proportions of women become engaged with technology. In this paper we review theories for this gender disparity and the barriers women face in working with audio technology, and then present a comparison of survey data between an e-textile audio workshop and an audio platform user group. Extrapolating from the case study and the surveyed literature, we propose that flexibility in learning, communal dissemination of knowledge, and gendering of tools are prominent reasons why women engage with technology via e-textiles.
This paper presents insights from an ethnographic study with a diverse population of makers in the city of Newcastle upon Tyne, UK. By engaging individuals, groups and communities who 'make' in different contexts, we reveal under-explored perspectives on 'making' and highlight points of intersection between different kinds of making across the city. We reflect on the dynamics of these intersections and connect our observations to emerging discourses around 'open design'. In doing so, we argue for a renewed focus on 'inclusivity' and highlight a need for new infrastructure to support iterative, collaborative making within -- and across -- interconnected networks of makers.
Maker culture has been increasingly pervasive in a variety of communities and contexts, in particular devoted spaces such as Makerspaces, Hackerspaces and Fab Labs. Several people, however, have pointed out that while one of the values of these spaces is radical inclusion, the general Maker culture can be exclusive to some based on gender, race, and socioeconomic status. With the goal of disrupting existing Maker culture by developing, diversifying, and empowering our own university Maker community, we created a semi-structured making experience that we call Statement Making. Statement Making is a Maker Fashion show that we invited anyone from the community to participate in by "making a statement" for them or a friend to wear in a runway show. We report outcomes and experiences of those who participated. We then discuss the key aspects of the event, especially surrounding its performative aspect, using design principles of Feminist HCI to argue that events with similar aspects might also be successful at disrupting existing culture.
The Quietude project uses making, participation and co-design to collectively imagine a more sustainable, aesthetically enriched future for deaf women, by developing wearables that respond to the women's needs and desires: those that are well known, and those that may be only dimly glimpsed. We present our motivation and process, and describe our first workshop that brought together deaf women, ethicists, makers, designers and technology experts. The workshop led to the design and development of an ecology of jewellery products: fashionable accessories that enhance the experience of deaf women by translating sounds into vibration, light patterns and shape change. We reflect on the opportunities and challenges of developing aesthetically rich wearables for deaf women, using experimental participatory design methods, and the value of considering disability as an opportunity for wearables design, rather than as an issue that needs to be addressed or solved.
While ordinary touchscreen-based interfaces on urban explorer applications draw much of a user's attention onto the screen, visual and audio-haptic augmented reality interfaces have emerged as the two main streams for enabling direct focus on the surroundings. However, neither interface alone satisfies users in the highly dynamic urban environment. This research investigates how the two complementary augmentation can coexist on one system and how people adapt to the situation by selecting the more suitable interface. A prototype was deployed in a field experiment in which participants explored points of interest in an urban environment with both interfaces. The engagement with the surroundings was compared with a touchscreen-based application. Most participants spontaneously switched between the two interfaces, which manifests the value of the availability of both interfaces on one system. The results point at the situated advantages of either interface and reveal the users' preferences when both interfaces are available.
Because gesture design for augmented reality (AR) remains idiosyncratic, people cannot necessarily use gestures learned in one AR application in another. To design discoverable gestures, we need to understand what gestures people expect to use. We explore how the scale of AR affects the gestures people expect to use to interact with 3D holograms. Using an elicitation study, we asked participants to generate gestures in response to holographic task referents, where we varied the scale of holograms from desktop-scale to room-scale objects. We found that the scale of objects and scenes in the AR experience moderates the generated gestures. Most gestures were informed by physical interaction, and when people interacted from a distance, they sought a good perspective on the target object before and during the interaction. These results suggest that gesture designers need to account for scale, and should not simply reuse gestures across different hologram sizes.
Head-mounted augmented reality (AR) enables embodied in situ drawing in three dimensions (3D). We explore 3D drawing interactions based on uninstrumented, unencumbered (bare) hands that preserve the user's ability to freely navigate and interact with the physical environment. We derive three alternative interaction techniques supporting bare-handed drawing in AR from the literature and by analysing several envisaged use cases. The three interaction techniques are evaluated in a controlled user study examining three distinct drawing tasks: planar drawing, path description, and 3D object reconstruction. The results indicate that continuous freehand drawing supports faster line creation than the control point based alternatives, although with reduced accuracy. User preferences for the different techniques are mixed and vary considerably between the different tasks, highlighting the value of diverse and flexible interactions. The combined effectiveness of these three drawing techniques is illustrated in an example application of 3D AR drawing.
Human body in HCI is often seen as an actuator for issuing commands and providing input to digital systems. We present the concept of the body as a canvas, in which the body acts as both an actuator and a display for information. Body as a canvas creates an interaction loop where interaction with information causes changes in the body, which in turn changes the display of information. Our qualitative study using an on-body projection system in a public exhibition investigates this concept with regards to body characteristics, types of body input, interactions between multiple bodies, and comparison to other display technologies. Our findings show that body as a canvas create connectedness between the body and information. Finally, we discuss how body characteristics and appearances can complement the information, when the body acts as a canvas.
Visual support (VS) is one of the effective ways of facilitating activities of children with neurodevelopmental disorder (ND). This paper reports on an interactive VS provided by a large-scale floor projection system in an augmented gymnasium called FUTUREGYM, designed for children with ND. The study focuses on students' cleaning, and two interactive VS activities-Mop Game, an exergame involving group cleaning, and Mop Guide, a VS for training about vocational cleaning-were designed with the teachers with the aim of motivating students toward cleaning and help them acquire fundamental cleaning skills. The study attempts to design a VS for cleaning that is suitable for the students by conducting an empathic design approach, which helps us understand what are the problems, obtain new perspectives, and gather ideas into demonstrative prototypes by sharing values and thoughts with the teachers and their students. This is a case study of deploying an empathic design approach in a special needs school setting.
A person with a disability has to assemble support services and technologies from different organisations in order to live well, which may require help from family. We call this assembling of services and technologies personal infrastructuring, the process of learning about how to navigate the world, what support is available, and how to obtain and design new support through various organisational infrastructures. Such infrastructures include disability services organisations, the health sector, community organisations, and friend and family networks. Our vision was to explore how a person with a disability might engage in design with volunteer designers to meet their unique needs that were not met by their existing infrastructure of organisations, products and services. Through codesign with two people and their families, we developed design artefacts such as user profiles and video stories to support communication, mutual learning, need finding and need expression. We discovered that these design artefacts were used beyond their immediate purposes of design to further support their personal infrastructuring. In this paper, we discuss how understandings of infrastructure and infrastructuring from Science and Technology Studies and Information Systems translate into familial contexts and the concept of personal infrastructuring.
Early Intervention Services support children with disabilities in their development from early age on. To this end, therapeutic toys are regularly employed within training sessions. These toys often draw on interactive elements to make exercising more appealing, and hence, to motivate the children. While there is some research about interactive therapeutic toys in HCI, these works are often standalone design deployments, exploring specific health or rehabilitation applications. In contrast, this paper offers different lenses for investigating qualities of therapeutic toys to highlight the following aspects: therapeutic and playful/motivational efforts, the potential of technology in supporting suitable affordances, ease of use, flexibility and improvisation. The lenses enable guided explorations of existing toys or novel design proposals, resulting from a thematic analysis of a) in-situ observations of therapeutic exercises (n=18), b) in-depth/informal interviews with Early Intervention Specialists (combined n=16), and c) demonstrations of their favorite toys (n=21).
Fidgeting involves interacting with objects using repetitive hand movements. Before you can study its effects, you must first study the objects with which people choose to fidget. We present the findings of our five-phase three-month study with 28 children, 24 parents, and 2 teachers examining fidget material qualities and inherent interactions children gravitate towards when fidgeting and what, if any, connections can be made between their emotional state or activity when fidgeting and their fidget interactions/materials preferences. Our study included structured interviews, observations during usage, and design workshops. We present novel insights concerning fidget object preferences, across factors including materials, interactions, stealth, durability and sound, which together can act as guidelines in the development of educational, experimental and utility tangibles for children. For example, children tend to prefer a fidget item with inherent squeezing interactions when they are angry and clicking/pressing/tapping interactions when they are bored.
We report on the design and deployment of a probe study aimed at understanding the values, practices, and perspectives of people that actively embrace living situations that could be considered 'alternative' to normative domestic dwellings. In response to the returned probes, the pictorial describes and unpacks speculative interpretations and design responses that (i) propose alternative ways technology could be designed for the home, (ii) embody different ideas of where home is located, (iii) explore how home is constructed, re-made, curated, and pursued, and (iv) productively question material, technological, and social boundaries between the home and the outside world.
Actuating, dynamic materials offer substantial potential to enhance interior designs but there are currently few examples of how they might be utilized or impact user experiences. As part of a design-led exploration, we have prototyped (Wizard-of-Oz) an actuating, dining table runner (ActuEater1), and then developed a fully-interactive fabric version that both changes shape and colour (ActuEater2). Four in-situ deployments of 'ActuEaters' in different dinner settings and subsequent 'design crits' showed insights into how people perceive, interpret and interact with such slow-technology in interesting (and often unexpected) ways. The results of our 'ActuEating' studies provide evidence for how an actuating artefact can be simultaneously a resource for social engagement and an interactive decorative. In response, we explore design opportunities for situating novel interactive materials in everyday settings, taking the leap into a new generation of interactive spaces, and critically considering new aesthetic possibilities.
This pictorial describes the design and design process of the IoT Sandbox: a (scale model of a) smart home equipped with actuators and a modular interface that controls the interactive aspects of the smart home. This modular interface can 'grow" together with the house it controls whereby it offers unique possibilities for designers to explore different avenues of physically rich interaction in domestic personal spaces. We demonstrate that also physically rich interfaces can respond adequately to the dynamics of home IoT systems. We conclude with insight from the design process and a concise discussion and outlook to future work.
In this paper, we present an exploratory study of hygge as a low-energy design vision for the smart home. Hygge is a Danish concept that embodies aesthetic experiences related to conviviality, often shaped by orchestrating atmospheres through low-level lighting. To explore this vision, we probe two Australian households that already live with smart home lighting technology. We report on household reflections of embedding hygge into everyday life. We conclude by outlining future directions for exploring desirable and sus¬tainable smart home visions.
Mobile application designers use onboarding task flows to help first time users learn and engage with key application functionality. Although some guidelines for designing onboarding flows have been offered by practitioners, a systematic, research-informed approach is needed. In this paper, we present the creation of a method for designing mobile application onboarding experiences. We used the minimalist instruction framework to engage twelve university students in an iterative set of design and evaluation activities. Participants interacted with a physical prototype of an educational badging mobile application through a semi-structured exploration and reflection activity, bookended by structured mini-interviews. We found that this method facilitated engagement with participants' meaning-making processes, resulting in useful design insights and the creation of an onboarding task flow. Research opportunities for integrating instructional design and learning approaches in HCI in the context of onboarding are considered.
Sketches are much more than marks on paper; they play a key role for designers both in ideation and problem-solving as well as in communication with other designers. Thus, the act of sketching is often enriched with annotations, references, and physical actions, such as gestures or speech---all of which constitute meta-data about the designer's reasoning. Conventional paper-based design notebooks cannot capture this rich meta-data, but digital design notebooks can. To understand what data to capture, we conducted an observational study of design practitioners where they individually explore design solutions for a set of problems. We identified sketching and non-sketching actions that reflect their exploration of the design space. We then categorized the captured meta-data and mapped observed physical actions to design intent. These findings inform the creation of future digital design notebooks that can better capture designers' reasoning during sketching.
Mobile devices are a substantial part of our lives, supporting communication, work, and play. However, situational visual impairments (SVIs) can make completing tasks a challenge (e.g., browsing online in bright sunlight) and poorly designed content can cause or exacerbate SVIs. We surveyed 43 mobile content designers and ran four follow-on interviews to understand what designers currently do regarding SVIs, what resources they know of, and what is required to best support them in designing to reduce SVIs. Our findings highlight key similarities and differences between accessibility and designing to reduce SVIs. Our participants requested improved guidelines, education, and digital design tools for SVIs. To accommodate the growing number of people affected by SVIs and improve the inclusion of accessibility in design, we introduce recommendations that leverage the overlap between accessibility and SVIs to minimise the effort required in extending current design processes.
We introduce the concept of hypercue, a complement to the hyperlink in the form of an interactive representation of real-world entities, e.g. persons, places, concepts, providing personalized access points to information. As a pendant to the hyperlink, hypercues create opportunities to flexibly discover, store and share information, organize one's thoughts and gain insights of the data. We explore the design space of interaction techniques supporting entity-based information exploration by reviewing recent examples of such work. We reflect on these through the lens of eight essential features of exploratory search systems, to devise generalizable design principles. Our main contribution is a design template describing the hypercue. It consists of a minimal set of affordances that ensure all important features for supporting exploratory search can be addressed, while leaving enough design space to facilitate integration within a variety of systems. We finally describe the rationale behind the design template and discuss its implications.
In this pictorial, we illustrate steps towards a novel approach that situates connected technologies for older people as resources. In contrast to mainstream approaches in gerontechnology that consider elderly as frail and passive, we aim to complement older people's vital competences by designing technologies that can be used in less prescriptive, and broader ways. The pictorial describes our design process in which resourceful strategies were identified through thing ethnography and used as inspiration to create a series of new connected objects conceived as resources.
This pictorial offers a visual diary of our qualitative fieldwork to understand ageing people's experiences in Saudi Arabia. It provides insights gained through conducting qualitative fieldwork with ageing Saudis. We present a range of cultural considerations that shaped the design of the fieldwork and highlight opportunities, challenges, and issues that we faced when conducting interviews and deploying research probes. In particular, we highlight the power and effectiveness of using probes to elicit participants' values, views and desires when working within the sociocultural norms of Saudi Arabia.
Older adults are increasingly engaging in online activities, including games, with other people. Many online environments require the user to create some form of self-representation, ranging from a simple user name through to a full body avatar. These self-representations not only enable access to online activities, but also provide an opportunity for expressing both the real and ideal identity. We wanted to better understand the impacts of later life on the construction of self-representations when playing online games. Our study used gameplay observations and semi-structured interviews with 10 older adult gamers aged from 65 to 95 years. We found they designed their player self-representations to project aspects of their lost (former) self and to embrace their present older selves. This engagement with self-representations as a form of self-expression suggests that designers need to consider older gamers, and their diverse preferences, when creating tools for customizable self-representations in online games.
Gamification, the use of game elements in non-game contexts, has been successfully used to motivate people to reach their goals more efficiently or turn unpleasant tasks into fun ones. However, most gamified systems are conceptualized for a younger audience and do not account for age-specific changes in the motivation to play or the perception of game elements. To inform the design of gamified applications targeting elderly people (aged 75+), we investigated their gaming experiences, what affects them positively while playing and their attitudes towards the most commonly used game elements. We report findings from semi-structured interviews and a storyboard-based game element assessment (N=18, mean age=84.61), indicating that the main motivation to play is socializing, that participants avoid competition and prefer collaboration and care-taking as well as that badges and points are considered meaningless and provide a level of visibility that puts participants under pressure.
In this paper, we argue for framing the crafting and studying of research products as doing philosophy through things. We do this by creating an annotated portfolio of such Research through Design (RtD) artifact inquiries as postphenomenological inquiries. In our annotated portfolio, we first provide an account of the postphenomenological commitments of 1) taking empirical work as the basis of the inquiry, 2) analyzing structures of human-technology relations and 3) studying technological mediation. Secondly, we trace these commitments across six RtD artifact inquiries. We conclude with a discussion on how research products can be seen as an experimental way of doing postphenomenology and how HCI design researchers can work with that. As a result, the presented philosophical framing can be leveraged in HCI research to form a deeper and more dimensional understanding of the human-technology relations we craft and study. This also adds a methodological path to moving beyond foci of use, utility, interaction, and human-centeredness.
Over the last decade, a number of craft-based approaches to research have emerged within the field of Human-Computer Interaction (HCI). In this paper, we examine the roots of crafting as they apply to these approaches, which blend analog crafts with digital technology, and we outline three defining characteristics: the integration of analog and digital crafting processes, the creation of highly refined products, and the creation of a deep and embodied knowledge. Moreover, we demonstrate how Richard Sennett's tripartite deconstruction of the crafting process can be applied to support analysis of the types and processes of knowledge generated in craft-based approaches to HCI.
Cubic shapes play an astonishing role in the design of tangible interactive devices. Due to our curiosity for this widespread design preference lasting over thirty years, we constituted a literature survey of papers, books and products since the late 1970s. Out of a corpus of fourty-seven papers, books and products that propose cubic shapes for tangible interactive devices we trace the origins of cubicle tangibles and highlight the rationale for their application. Through a comparative study, we analyze the properties of this shape for tangible interaction design and classify these along the themes of: Manipulation as Input, Placement in Space as Input, Arrangement, Multifunctionality, Randomness, Togetherness & Variations, Physical Qualities, Container, and Pedestal for Output. We propose a taxonomy for cubic shaped tangible interactive devices based on the reviewed contributions, in order to support researchers and designers in their future work of designing cubic shaped tangible interactive devices.
This pictorial explores the material resources that can unfold through taking things apart. We describe a workshop program and according exercises designed around four particular modes of disassembling (interactive) artefacts. These exercises aim to provide low-threshold engagements with artefacts and the materials those artefacts are composed of. Based on a series of workshops we conducted following this program, we depict nine different forms of material resources that are unveiled through taking things apart. With this distinction we aim to contribute to the understanding of how the material histories of components and the rules that were governing their previous relations, are carried over to new compositions during reuse or reinterpretation.
Illiteracy is a global problem impacting the growth and development of individuals and society. Studies indicate that picturebook reading within a facilitated storytime setting is an important tool for children's language acquisition. In the research reported here, we hypothesized that literacy, in an increasingly digital society, can be cultivated in a robot-embedded environment that is physical, digital and evocative of the picturebook being read. Words become worlds. To test our hypothesis, we designed, prototyped, and implemented the LIT ROOM, a cyber-physical room for literacy. As a Research through Design [RtD] exemplar for interactive systems at habitable scale, the LIT ROOM featured a multi-phase, iterative process of design and evaluation for usability and efficacy. Evaluations with 35 children and 6 librarians in a public library serving a population with grave literacy challenges suggest that our reconfigurable learning environment facilitates a diversity of children's literary responses during the dialogical reading of picturebooks.
Visual markers, in particular QR codes, have become widely adopted in museums to enable low cost interactive applications. However, visitors often do not engage with them. In this paper we explore the application of visual makers that can be designed to be meaningful and that can be created by visitors themselves. We study both the use of these markers as labels for portraits that link to audio recordings and as a mechanism for visitors to contribute their own reflections to the exhibition by drawing a marker and linking an audio comment. Our findings show visitors appreciated the use of the aesthetic markers and engaged with them at three levels -- physical placement, aesthetic content and digital content. We suggest that these different levels need to be considered in the design of future visiting systems, which make use of such markers, to ensure they are mutually supporting in shaping the experience.
Interpersonal touch, one of the most primitive social languages, is an excellent design element frequently used in interaction design. In this paper, we present a richer understanding of it by using spatial factors and social relations among people, which has rarely been explored in interactive systems. We designed an interactive installation called "TouchBranch" where players can move light between branches placed at various distances by connecting their bodies. The user studies were conducted with 21 groups consisting of intimates, acquaintances, and strangers. We observed a change in the interpersonal touch pattern and touch tolerance according to each factor. Interestingly, the effect of the social relation was dramatic, but that of the spatial factor was not quantitatively significant. Nevertheless, we discovered that spatial factor can influence the interpersonal touch experience. Based on the results, we discuss in this paper the influence of two factors on the interpersonal touch that stands out in the context of interactive systems.
In this pictorial we present a project case, where an interactive office environment was designed following concurrent prototyping embedded in an iterative design approach. The case illustrates how concurrent prototyping supports designing complex interactions between multiple people and multiple interactive objects, while innovating in both social and technological realm. Identified variables of the involved process allow steering the design towards a variety of possible solution qualities. We propose this approach as a viable strategy for dealing with the complexity of designing in the domain of Human-Building Interaction.
We present CORE-MI, an automated evaluation and assessment system that provides feedback to mental health counselors on the quality of their care. CORE-MI is the first system of its kind for psychotherapy, and an early example of applied machine-learning in a human service context. In this paper, we describe the CORE-MI system and report on a qualitative evaluation with 21 counselors and trainees. We discuss the applicability of CORE-MI to clinical practice and explore user perceptions of surveillance, workplace misuse, and notions of objectivity, and system reliability that may apply to automated evaluation systems generally.
Machine learning (ML) promises data-driven insights and solutions for people from all walks of life, but the skill of crafting these solutions is possessed by only a few. Emerging research addresses this issue by creating ML tools that are easy and accessible to people who are not formally trained in ML (non-experts). This work investigated how non-experts build ML solutions for themselves in real life. Our interviews and surveys revealed unique potentials of non-expert ML, as well several pitfalls that non-experts are susceptible to. For example, many perceived percentage accuracy as a sole measure of performance, thus problematic models proceeded to deployment. These observations suggested that, while challenging, making ML easy and robust should both be important goals of designing novice-facing ML tools. To advance on this insight, we discuss design implications and created a sensitizing concept to demonstrate how designers might guide non-experts to easily build robust solutions.
Machine learning (ML) plays an increasingly important role in improving a user's experience. However, most UX practitioners face challenges in understanding ML's capabilities or envisioning what it might be. We interviewed 13 designers who had many years of experience designing the UX of ML-enhanced products and services. We probed them to characterize their practices. They shared they do not view themselves as ML experts, nor do they think learning more about ML would make them better designers. Instead, our participants appeared to be the most successful when they engaged in ongoing collaboration with data scientists to help envision what to make and when they embraced a data-centric culture. We discuss the implications of these findings in terms of UX education and as opportunities for additional design research in support of UX designers working with ML.
As cities around the world become more diverse in culture and language, there is a growing need for learning foreign languages. To further this excitement, we have built a human-scale, immersive room with a virtual AI agent that aids foreign language learning. Our system aids the language learning process through task-completion exercises using multi-modal dialogue. The Cognitive and Immersive Room (CIR) is developed as an immersive Chinese restaurant to teach Mandarin, but the interaction challenges and solutions can be reasonably generalized to other languages taught using similar techniques. As users interact with the immersive environment and the virtual AI agent, they face several user interaction challenges. These challenges arise from new learners' lack of proficiency in the foreign language. By studying user interactions in the CIR, we were able to articulate some of the interaction challenges. We have enhanced the AI agent, virtual environment, and the on-boarding process for new users to mitigate these challenges. The enhancements and the results which show that they were effective are discussed here.
In this paper we explore how screen-based smartphone interaction can be enriched when designers focus on the physical interaction issues surrounding the device. These consist of the hand grips used (Symmetric bimanual, Asymmetric bimanual with thumb, Single handed, Asymmetric bimanual with finger), body postures (Sitting at a table, Standing, Lying down) and the tilting of the smartphone itself. These physical interactions are well described in the literature and several research papers provide empirical metrics describing them. In this paper, we go one step further by using this data to generate new screen-based interactions. We achieved this by conducting two workshops to investigate how smartphone interaction design can be informed by the physicality of smartphone interaction. By analysing the outcomes, we provide 14 new screen interaction examples with additional insights comparing outcomes for various body postures and grips.
We present empirical results about users' gesture preferences for smart rings by analyzing 672 gestures from 24 participants. We report an overall low consensus (mean .112, maximum .225 on the unit scale) between participants' gesture proposals, and we point to the challenges of designing highly-generalizable ring gestures across users. We also contribute to the practice of gesture elicitation studies by discussing how a priori conditions (e.g., participants' traits, such as creativity and motor skills), commitment and behavior during the experiment (e.g., their thinking times), but also a posteriori aspects (the experimenter's choice of criteria to group gestures into categories) affect agreement. We offer design guidelines for ring gestures informed by our empirical observations, and present a collection of gestures reflective of our participants' mental models for effecting commands using smart rings.
In this paper, we explored the application of human factor guidelines in personal fabrication. This is useful for several Do-It-Yourself (DIY) scenarios, including users adjusting workstation configurations or designing a desk to fit a single person. We identified a dependency map between the user's anthropometrics, ergonomic pose recommendations, and design dimensions. Based on this, we developed situated and interactive guidelines to assist users in design applications. We applied these guidelines in a Virtual Reality (VR) system that lets users customize their desk and provides real-time feedback and feedforward on pose and design. We evaluated the system with six participants, had each one design a personal desk, fabricated their desks, and let them work on their desks for four hours. The design and evaluation contribute to fabrication tools as it helped users be aware of their pose and ergonomic knowledge, and design for their bodies and needs.
Mastering fine motor tasks, such as playing the guitar, takes years of time-consuming practice. Commonly, expensive guidance by experts is essential for adjusting the training program to the student's proficiency. In our work, we showcase the suitability of Electromyography to detect fine-grained hand and finger postures in an exemplary guitar tutor scenario. We present EMGuitar, an interactive guitar tutoring system, that assists students by reporting on play correctness and adjusts playback tempi automatically. We report person-dependent classification utilizing a ring of electrodes around the forearm with an F1 score of up to 0.89 on recorded calibration data. Furthermore, our system was received well by neither diminishing ease of use nor being disruptive for the participants. Based on the received comments, we identified the need for detailed play accuracy feedback down to individual chords, for which we suggest an adapted visualization and an algorithmic approach.
In improvisational theatre (improv), actors perform unscripted scenes together, collectively creating a narrative. Audience suggestions introduce randomness and build audience engagement, but can be challenging to mediate at scale. We present Robot Improv Puppet Theatre (RIPT), which includes a performance robot (Pokey) who performs gestures and dialogue in short-form improv scenes based on audience input from a mobile interface. We evaluated RIPT in several initial informal performances, and in a rehearsal with seven professional improvisers. The improvisers noted how audience prompts can have a big impact on the scene - highlighting the delicate balance between ambiguity and constraints in improv. The open structure of RIPT performances allows for multiple interpretations of how to perform with Pokey, including one-on-one conversations or multi-performer scenes. While Pokey lacks key qualities of a good improviser, improvisers found his serendipitous dialogue and gestures particularly rewarding.
The design of social robots usually does not focus on their kinetic expression, and often follows the assumption that their appearance should be human or animal like. To encourage a broader understanding of the possibilities for design of social robots, and as an inquiry into alternative relations with them, we present two robots, the Lat-Sac and the Blo-Nut, which are purposefully moving away from typical social robot design. We present how we engaged performance experts in the choreographic sketching of their elastic expression, and how we staged the robots in a fictitious near-future scenario to create a discursive space for reflection on emerging relations. Based on these encounters we discuss how acknowledging the otherness of social robots can be valuable in designing as well as growing intriguing relations with them.
Tango is a form of partner dancing in which two bodies sense one another, and move accordingly, in a dynamic, physical dialogue that is known for its subtle complexities, beauty and intimate experience. In MoCap Tango, we explore how we can build on our skills as designers to highlight and unravel these embedded qualities and use them as inspiration in designing interactions. In this pictorial, we invite the reader to actively participate in the designerly engagement that turns objective data into subjective expressions; highlighting the qualities embedded in the movements of professional dancers.
As drone-based entertainment becomes more popular, researchers have explored different forms of expression and systems to support drone performances. However, most of these systems are pre-programmed and do not interact with the body movement of dancers in real-time. In response, some have presented drone performances using bulky camera systems to track the performer's body movement. We introduce Aeroquake, an augmented dance system that uses simple wearable microphones to enhance a dancer's body movement with sound and control the movement of drones in real-time. Dancers experience a simulation of "quaking" the space around them: upon stomping, the movement is translated into sound and vertical motion across multiple drones. Aeroquake allows dancers to improvise choreography and explore their creativity in the space in which they choose to dance. We worked with a dancer to validate our system by performing before a live audience.
This paper explores the contrasting notions of "permanance and disposability," "the digital and the physical," and "symbolism and function" in the context of interaction design. Drawing from diverse streams of knowledge, we describe a novel design direction for enduring computational heirlooms based on the marriage of decentralized, trustless software and durable mobile hardware. To justify this concept, we review prior research; attempt to redefine the notion of "material;" propose blockchain-based software as a particular digital material to serve as a substrate for computational heirlooms; and argue for the use of mobile artifacts, informed in terms of their materials and formgiving practices by mechanical wristwatches, as its physical embodiment and functional counterpart. This integration is meant to enable mobile and ubiquitous interactive systems for the storing, experiencing, and exchanging value throughout multiple human lifetimes; showcasing the feats of computational sciences and crafts; and enabling novel user experiences.
Some recently emergent themes in HCI include adapting to changing conditions by simplifying life, learning skills of adaptation, and finding balance between the digital world and an authentic physical world. These themes imply that design is best understood as ontological-that is, design concerns lifestyles and ways of being. This pictorial celebrates the delight the authors take in nuclear family and the wisdom of their parents' simpler lifestyle. This case may be understood more generally as an instrument of inspiration to help find balance between the potentials of technology and traditional notions of a life worth living. Our images are accompanied by inspirations for thinking about interaction design differently.
We report upon the conduct and findings of an investigation into technology design for long-distance relationships (LDRs), where South Korean culture raises specific challenges. Through two qualitative studies we explore inter-generational LDRs from the perspective of South Korean students based in the United Kingdom. We identify and document the particular nuances within, and challenges that arise from, these relationships, before turning to the pragmatics of technology design for LDRs. Through both an extended diary study and interviews with students, we illustrate the impact of Korean familial obligations on intergenerational LDRs, and the mistrust and anxiety on both sides (parents and students) arising from limitations in communication channels. From our findings, we develop the notion of 'respectful disconnection' which we propose as a framework for designing interactions that appropriately support LDRs within this specific South Korean context.
Focusing on the design of technology for mourning and memorialization, we describe the emergence of Automatic Conveyor-belt Columbaria, locally developed in Japan, as an example of an interactive system combining physical and digital remains, and discuss its user experiences and social influences. It concludes with implications for future HCI research and practice with a focus on future gravesites and memorialization sites in dense urbanized regions.
While self-usage has long been regarded as a questionable approach in human-computer interaction (HCI) research, recent projects have shown the successful use of autobiographical design as a method to investigate long-term and intimate relations between people and technologies in everyday life. In an effort to continue the development of methodological best practices, we need to acknowledge with more nuance the tensions that arise in use. In this paper, we articulate such tensions by examining two first-hand accounts of using autobiographical design and four autobiographical design projects of other HCI researchers. Our findings address: genuine needs, design participation, intimacy, reflexivity, and authorial voice. Our contribution is constituted of critical insights into the complexities of using autobiographical design and recommendations for researchers interested in using this method.
This paper presents an autoethnography of my experiences living without a mobile phone. What started as an experiment motivated by a personal need to reduce stress, has resulted in two voluntary mobile phone breaks spread over nine years (i.e., 2002-2008 and 2014-2017). Conducting this autoethnography is the means to assess if the lack of having a phone has had any real impact in my life. Based on formative and summative analyses, four meaningful units or themes were identified (i.e., social relationships, everyday work, research career, and location and security), and judged using seven criteria for successful ethnography from existing literature. Furthermore, I discuss factors that allow me to make the choice of not having a mobile phone, as well as the relevance that the lessons gained from not having a mobile phone have on the lives of people who are involuntarily disconnected from communication infrastructures.
We use photography as a research method to cultivate a designerly sensibility of the theoretical concept natureculture, a provocation to transgress the dichotomy of nature and culture. We investigate the visual language of natureculture through an iterative practice of creating, editing, organizing, and reflecting on images. Specifically, we explore natureculture as spatiotemporal movements, sediment-like layers, heterogeneous gatherings, formal homonyms, emotional experiences, and aestheticized expressions of style. Each of these has a materioformal concreteness and symbolic density that supports design ideation on topics such as environmental sustainability, agroecological systems, human-animal cohabitation, urban informatics, and more.
While numerous design methods used in industry help designers rapidly brainstorm design ideas, few help them to use theory in the design process. Behavior change theories can support such design activities as understanding, ideating, sketching, and prototyping. We present the Behavior Change Design Sprint (BCDS), a design process for applying behavior change theories to the design process and for prototyping behavior change technologies. BCDS facilitates the application of theories into the design process through a series of exercises that help designers identify intervention placement and project behavioral outcomes, conduct more focused ideation, and advocate for their design rationale. We present our process to create the sprint and findings from a series of sprint deployments.
Viking VR is a Virtual Reality exhibit through which viewers can experience the sights and sounds of a 9th Century Viking encampment. Created as part of a major museum exhibition, the experience was developed by an interdisciplinary team consisting of artists, archaeologists, curators and researchers. In this paper, approaches to the design of authentic, informative and compelling VR experiences for Cultural Heritage contexts are discussed. We also explore issues surrounding interaction design for the long-term deployment of VR experiences in museums and discuss the challenges of VR authoring workflows for interdisciplinary teams.
Immersive design fiction is a novel approach that embeds speculative interactions within a rich virtual reality (VR) storyworld. Immersive design fictions use VR to translate new design opportunities into story-driven, embodied experiences by positioning the participant as a character in a narrative world. This paper presents a case study of an immersive design fiction that depicts a fictionalized reimagining of an industry partner's work practices. This VR experience explores speculative interfaces for creative work and collaboration in the context of a fictional workplace environment. By placing design fictions within rich immersive contexts such as room-scale VR, researchers and practitioners can go beyond prototyping imagined interfaces to also speculate about the interaction rituals and surrounding social context within an experiential storyworld. This approach makes methodological and theoretical contributions to design fiction research by demonstrating a toolkit for exploring and reflecting upon the intersections between speculation, embodiment, and narrative context.
Recent subject matters of design have become complex systems involving various elements and multiple users' interactions in large-scale spaces. In addition, visualizing these complex design subjects without preparing heavy resources in the early design phases is challenging. In this paper, we present SketchStudio, a prototyping tool for generating, sharing, and reviewing an animated design scenario involving complex design subjects. The tool allows designers to define the user experience by simply creating a node graph of the user's journey over time and space. SketchStudio instantly generates unique 2.5D animated design scenarios by blending 2D sketches and 3D characters. In addition, the tool supports the virtual reality mode for immersive views of the created scenario prototype. We also report on the result of a user study with nine potential users in the design practice and education fields. Based on our tool-developing experiences as well as on the study results, we discuss possible applicable areas and points for further improvement.
Field-of-view limitation has been a long-standing issue in video communication systems. With the advancement of omnidirectional panoramic technology, the omnidirectional camera, which can provide a 360° field of view, has become increasingly popular in the last few years. Previous research indicated that one-way video communication systems with a wider field of view improve task efficiency. Therefore, we propose to utilize omnidirectional cameras in a symmetrical video communication system and study how this configuration affects remote collaboration. In this study, we conducted experiments based on two conditions, which are an omnidirectional camera with a spherical display and an omnidirectional camera with a horizontally placed 2D flat display. Under these conditions, we analyzed how the display types affected remote collaboration. Our results show that participants marginally preferred the spherical display to the 2D flat display. We also show the advantages and disadvantages of each display. The findings contribute to our understanding of how to design an environment for remote collaboration that captures and shows a 360° panoramic view of a remote site.
In-home, place-based, conversational agents have exploded in popularity over the past three years. In particular, Amazon's conversational agent, Alexa, now dominates the market and is in millions of homes. This paper presents two complementary studies investigating the experience of households living with a conversational agent over an extended period of time. First, we gathered the history logs of 75 Alexa participants and quantitatively analyzed over 278,000 commands. Second, we performed seven in-home, contextual interviews of Alexa owners focusing on how their household interacts with Alexa. Our findings give the first glimpse of how households integrate Alexa into their lives. We found interesting behaviors around purchasing and acclimating to Alexa, in the number and physical placement of devices, and in daily use patterns. Participants also uniformly described interactions between children and Alexa. We conclude with suggestions for future improvement for intelligent conversational agents.
While digital personal assistants (DPAs) are moving into our homes, managing our everyday lives and providing help in the household, we have barely begun to understand them. Design fiction can be a method for contextualizing the social and cultural implications for adoption of future technologies like DPAs. In this paper, we present an analytical perspective on gender issues arising when a DPA moves into our home. Through a critical and feminist design methodology, the design fiction project "Intimate Futures" focuses on how a DPA's character and functions are often gendered and what it means for the design and adoption of a DPA. We argue that the gender issues of DPAs are interwoven with our collective imaginings of DPAs, and that design fiction is a method to explore and "trouble" our collective imaginings of DPAs. The paper contributes with an analysis of gender issues of DPAs, and a methodological way of "staying with the trouble" of future technologies through design fiction.
Conversational agents stand to play an important role in supporting behavior change and well-being in many domains. With users able to interact with conversational agents through both text and voice, understanding how designing for these channels supports behavior change is important. To begin answering this question, we designed a conversational agent for the workplace that supports workers' activity journaling and self-learning through reflection. Our agent, named Robota, combines chat-based communication as a Slack Bot and voice interaction through a personal device using a custom Amazon Alexa Skill. Through a 3-week controlled deployment, we examine how voice-based and chat-based interaction affect workers' reflection and support self-learning. We demonstrate that, while many current technical limitations exist, adding dedicated mobile voice interaction separate from the already busy chat modality may further enable users to step back and reflect on their work. We conclude with discussion of the implications of our findings to design of workplace self-tracking systems specifically and to behavior-change systems in general.
Text messaging-based conversational agents (CAs), popularly called chatbots, received significant attention in the last two years. However, chatbots are still in their nascent stage: They have a low penetration rate as 84% of the Internet users have not used a chatbot yet. Hence, understanding the usage patterns of first-time users can potentially inform and guide the design of future chatbots. In this paper, we report the findings of a study with 16 first-time chatbot users interacting with eight chatbots over multiple sessions on the Facebook Messenger platform. Analysis of chat logs and user interviews revealed that users preferred chatbots that provided either a 'human-like' natural language conversation ability, or an engaging experience that exploited the benefits of the familiar turn-based messaging interface. We conclude with implications to evolve the design of chatbots, such as: clarify chatbot capabilities, sustain conversation context, handle dialog failures, and end conversations gracefully.
Interactive technology has become integral part of daily life for both humans and animals, with animals often interacting with technologized environments on behalf of humans. For some, animals' participation in the design process is essential to design technology that can adequately support their activities. For others, animals' inability to understand and control design activities inevitably stands in the way of multispecies participatory practices. Here, we consider the essential elements of participation within interspecies interactions and illustrate its emergence, in spite of contextual constraints and asymmetries. To move beyond anthropomorphic notions of participation, and consequent anthropocentric practices, we propose a broader participatory model based on indexical semiosis, volition and choice; and we highlight dimensions that could define inclusive participatory practices more resilient to the diversity of understandings and goals among part-taking agents, and better able to account for the contribution of diverse, multispecies agents in interaction design and beyond.
Despite the popularity of fish as pets, there is little knowledge available about the fishkeeping experience and the related interactions. In this regard, this study aims to look into the experience of fishkeeping by supporting people's actions through a tech-mediated system. Based on the results, an interactive system called BubbleTalk was developed to help people to convey their actions using bubbles into a fish tank. A user study was conducted with BubbleTalk, and the results showed that the interaction through BubbleTalk varied people's behavior, prolonged their interaction and thus reshaped their relationship with fish. Beyond the implications for fishkeeping, we believe that our findings could serve as insight and further motivation for overcoming interactions limited by this physically disconnected environment.
Bats are often disliked and feared by people. How might we enable the general public to learn more about the true nature of these creatures, and even to like them? In this paper, we introduce PlayBat, a physical public display, which combines a multi-modal interface, a constrained narrative structure and real-time IoT environmentally sensed bat call data. The aim of our research is to investigate whether promoting curiosity and discovery through enabling people to explore real-life data, answer quiz-like questions and engage with a multi-modal interface, is effective at engaging people and confronting their fears. We report on the design process and implementation of PlayBat, and the findings from an in-the-wild study. We discuss how tapping into multiple senses can draw people in, evoke curiosity and even change their views.
This paper describes a methodology for conducting interaction design research workshops within wilderness locations.
In biological field expeditions, scientists travel to areas with minimal infrastructure to conduct research in environments featuring unique, naturalistic interactions. Digital interaction design is growingly important to field biologists as a way to develop new forms of scientific exploration and experimentation. Ideally field biologists would create their own interactive, scientific tools based upon their developing research questions. In practice, however, time and training constraints mean design is typically outsourced to specialized practitioners in dedicated laboratories.
The Hiking Hack model unites biologists and designers in collaborative, outdoor workshops. Hiking Hacks combine experiences and techniques from biological expeditions with Research Through Design methodologies.
This model has been refined and analyzed throughout several Hiking Hack expeditions. The result is an adaptable workshop structure considering gear, practices, and syllabi for exploring interaction design situated within wild environments.
Social media platforms have often been described as online spaces supporting political discourse. However, online discussions are often polarized; people tend to commune with those who are ideologically similar to them. The HCI response to this phenomenon has been to purposefully expose people to diverse viewpoints. This common design agenda is supported through analysis of link sharing, yet little attention has been paid to how users discuss these links. Therefore, the common design agenda may not mitigate polarization. We study the emergent discourse in 10 Finnish migration-related Facebook groups and examine how the same links are shared and discussed across anti- and pro-migration camps. Qualitative analysis of the posts and comments revealed that shared media links do not bridge polarized groups with regard to worldviews and opinions. We then demonstrate alternative design opportunities to resolve this issue and begin to develop a new design agenda to mitigate polarization.
This research explored ways of utilizing social media to engage non-activist users in public policy development using a design intervention. The intervention was designed to enable the users to derive policy needs from their daily life contents. We implemented the intervention in Facebook by posting stimuli on a Facebook page and collecting user comments. A qualitative comment analysis shows that the users' playful grumbles on daily life could evolve into collective needs on public policies through the intervention. We also identified social media functions that facilitated the intervention. Based on the findings, we discussed how social media systems could be developed to engage non-activist users for bottom-up public policy development.
Emergency services in North America have relied on the use of audio calls to the phone number, 9-1-1, since the late 1960s. In the coming years, 9-1-1 services will move to integrate media-rich calling capabilities such as video-based calling. We explore how video calling services should be designed through an interview study with people who have called 9-1-1 in the past. Our results show the potential for video calling to help people who are calling 9-1-1 describe their location to call takers, show the situation at hand, receive video-based instructions, and assist in cases with language barriers. Yet video calling raises issues around anonymity, consent, culture and gender-based biases, and camera work. 9-1-1 video calling is best thought of as a collaborative act where camera work is negotiated between callers and call takers where callers are willing to hand over control of the call if their privacy concerns can be met.
Recent years demonstrate an increased interest in Passive Acoustic Monitoring (PAM) applications when studying cetaceans. However, they remain expensive underwater systems and targeted for industrial and military purposes. While the usage of smartphones as acoustic sensors has been observed in terrestrial environments, ocean and nautical PAM applications remain greatly unexplored. This paper presents the design, deployment and testing of a POSEIDON system, used for real-time augmentation of whale-watching experiences. We collect and use cetaceans' vocal call acoustic samples (clicks, moans and whistles) and apply machine learning for offline model training and prediction. When discriminating the calls, we find that Extra Trees and Gradient Boosting outperform other classifiers (>0.95 confidence threshold). Collected samples are at disposal to citizen scientists and marine biologists. Future studies involve real-time on-boat user testing.
Cognitive skills and their assessment gain increasing importance in soccer. In the past, athletes' perceptual-cognitive capabilities were only assessed using 2D media with its limitations. We used a virtual reality 360° video environment, where 15 high-skilled and 15 low-skilled soccer players (age 24 ± 3 years) experienced nine real-life soccer scenes from varying perspectives. The experience was frozen at crucial time points where they had to decide for one of three soccer actions. The recognition-action time and risk-level for decision were determined. Furthermore, six different interaction concepts were evaluated with respect to user experience, presence and immersion to find the most adequate and appealing one for assessment and training in soccer. Results show that high-skillers had a significantly lower overall recognition-action time. Risk-values for decisions did not differ significantly between skill levels.
Flow state is a psychological state of optimal performance. To experience flow state, one needs to receive unambiguous feedback. Previous studies have described activities with internalized feedback modalities (e.g. visual). However, they do not offer any appropriate feedback modality for the activities that may benefit from external feedback, such as opponent-based sports. Addressing the issue, we adopted a research through design process and considered tennis as our case, in which players can benefit from attaining flow [1]. This pictorial reveals our approach to design 6 wearable device concepts under 3 design themes as future directions for design practitioners and researchers.
The use of technology to assist in instructed fitness training in collocated social settings is underexplored. Here we focus on how technology can be designed to fit within, leverage, and be part of the strategies and tools that fitness trainees and instructors use to detect and correct performance errors. Drawing on ethnomethodological approaches and using the concept of correction to focus our analysis, we scrutinize the interaction between instructor and trainees in two fitness activities, AntiGravity Fitness and Pilates. We identify social configurations and resources employed in instances of correction. We also present an analytical tool useful for deconstructing such correction processes, highlighting the strategies and resources used by the social actors, and their impact on performance. Based on insights gained from our analyses, we propose design recommendations and identify design opportunities that capitalize on existing tools and collaborative correction strategies, such as scaffolding the correction process.
Thing-centered design has suggested analyzing our product-scape through the metaphor of agency. However, interaction design with plant-related agency usually animates plants to simulate human behavior. We intend to make interactive things with the ontological nature of plants. Through workshops including guided annotation of six artifacts and situated probes, the senses of the "plantness" of an artifact emerge. Drawing on these understandings, we built and deployed Botanical Printer, which lives with us slowly responding to natural and electronic climates. We present rich results including conceptual, situated, and interactive plantness. Empirical data allow us to explore the future of object-oriented speculation in greater depth.
Both a provocative artistic object and research artefact, Blo-nut is part of a project where we explore novel robotic behaviour away from the mimicry of complex human expressions more commonly associated with robotic form and movement. In this pictorial we start by outlining our underlining design principles for the design process and use images to describe the making, real-time GUI and the "choreographic sketching' of Blonut's movement characteristics. Finally, we argue for embracing novel and provoking 'otherness' in form and material when exploring interaction and communication with social robots.
We derive a theory of multiscale design through a study of a landscape architecture studio classroom. We find that processes of designing, to meet a site's situated needs, involve creating and connecting representations across levels, such as overview and detail. We introduce multiscale design theory, which works to understand how designers explore, juxtapose, and synthesize relationships across levels of scale. We identify three design strategies landscape architecture students use to work with scale: multiply, map, and shift perspective. We combine these strategies with prior literature, across fields, to initiate a theory of multiscale design.
The growing makers' community demands better supports for designing and fabricating interactive functional objects. Most of the current approaches focus on embedding desired functions within new objects. Instead, we advocate repurposing the existing objects and rapidly authoring interactive functions onto them. We present Plain2Fun, a design and fabrication pipeline enabling users to quickly transform ordinary objects into interactive and functional ones. Plain2Fun allows users to directly design the circuit layouts onto the surfaces of the scanned 3D model of existing objects. Our design tool automatically generates as short as possible circuit paths between any two points while avoiding intersections. Further, we build a digital machine to construct the conductive paths accurately. With a specially designed housing base, users can simply snap the electronic components onto the surfaces and obtain working physical prototypes. Moreover, we evaluate the usability of our system with multiple use cases and a preliminary user study.
Industrial design focuses on minimizing fabrication variability, aiming for identical products, while craft practice often results in unpredictable outcomes. We rely on crackle, an explicit pottery phenomenon that renders a pattern of cracks in ceramic glazing, to produce craft-unique outcomes in a moderately controlled design process. With the help of a dedicated CAD tool and a laser machine, we embed artificial decorations in a crackle pattern. By pre-processing the clay and post-processing the glaze, we demonstrate a technique to partially control the typical size of cracks in a given area, thus embedding visual forms in the glaze.
Generative tools contribute new possibilities to traditional design, yet formal representation of digital procedures can be counterintuitive to some makers. We envision the use of catalogues in parametric design, replacing abstract design procedures with a given set of visual options to select from and react to. We review the research challenges in realizing our catalog vision. We contribute an embryonic catalog generated from a formal list of parameters, demonstrated on a parametric mushroom. We also present a simple user study where students engaged in a design task relying on a catalog of prototypes generated by parametric design.
Laser cutters are 2D tools, but their speed and compatibility with a variety of affordable materials also makes them a frequent choice to create 3D objects. We propose CutCAD, a tool to easily construct simple 3D objects from 2D faces, inspired by the process of paper modeling and magnetic construction kits. The user creates her 3D model by drawing or loading existing 2D shapes, and connecting their edges in the software. CutCAD then automatically resolves the resulting constraints, and folds the faces up into a 3D model that is previewed live. CutCAD also automatically creates the required finger joints based on thickness of the material and dihedral angles, for smooth assembly. Cutouts are easy to add by importing their outlines as vector drawings, and placing them onto faces. After the faces have been cut, CutCAD provides assembly instructions. Observations and feedback from using CutCAD show the resulting process to be easier to understand than traditional 3D modelling. CutCAD is open-source, and has been downloaded over 2,000 times.
An increasing number of non-profit groups and organizations have formed "libraries" of shared things to leverage the collaborative use of underutilized resources (e.g., power tools) for the benefit of local communities. Their key challenges are the transience and anonymity of their members, and how to nurture creative interactions among them. We designed and developed Roaming Objects, an interactive system aimed at supporting the capture and sharing of equipment-use experiences among these members. We deployed the system for two months in a tool-sharing cooperative to explore how it may help to address these challenges. We offer insights into how resource sharing cooperatives and collectives could be better supported, by proposing design opportunities that facilitate sharing both physical objects and digital information about their use.
Indirect resource exchange (IRE), where individuals share physical items with one another but do not receive direct benefits (e.g. payment), has the potential to increase communities' access to resources, reduce consumption and waste, and bootstrap social ties. Although social technologies could play a key role in realizing this potential, significant barriers have emerged to the adoption of IRE services, including concerns related to trust, reciprocity, and coordination. To explore these issues, we designed and iterated on a concept called ShareBox, a system that enables IRE through a smart lockbox. We developed ShareBox as a technology probe following a set of design guidelines including: creating a physical-virtual system, enabling asynchronous and anonymous exchange, allowing for low-entry-barrier interactions, and emphasizing affordability and flexibility. We explore the benefits and trade-offs of these design guidelines through short deployments and semi-structured interviews with community members, and present findings that highlight both the potential and the remaining challenges of our design.
Existing platforms for sharing locative digital content rely on the use of mobile phones for accessing the content. This can be a major deterrent to wider public access and also hinders immediacy and 'in the moment' discoverability. Building on previous work in situated public installations, we developed Pinsight, a novel platform for enabling end-users, such as local communities, to create and share digital content in-situ with public audiences through physical interactive devices. Pinsight is based on a set of design principles that focus on supporting both the expressiveness of content creators and the appeal to public audiences. This paper describes the design of the platform and how it supports sharing knowledge in ways different to conventional media. Through preliminary evaluations and two in-the-wild studies, we explore how such a situated technology can be used by different user groups (content designers, history communities, local residents) for sharing content with public audiences (visitors, pedestrians, residents) in different contexts.
In this paper we examine ways to make data more meaningful and useful for citizens in participatory sensing. Participatory sensing has evolved as a digitally enabled grassroots approach to data collection for citizens with shared concerns. However, citizens often struggle to understand data in relation to their daily lives, and use them effectively. This paper presents a qualitative study on the development of a novel approach to Community Level Indicators (CLIs) during two participatory sensing projects focused on noise pollution. It investigates how CLIs can provide an infrastructure to address challenges in participatory sensing, specifically, making data meaningful and useful for non-experts. Furthermore, we consider how this approach moves towards an ambition of achieving change and impact through participatory sensing and discuss the challenges in this way of working and provide recommendations for future use of CLIs.
An increasing number of domains, including aeronautics, are adopting touchscreens. However, several drawbacks limit their operational use, in particular, eyes-free interaction is almost impossible making it difficult to perform other tasks simultaneously. We introduce GazeForm, an adaptive touch interface with shape-changing capacity that offers an adapted interaction modality according to gaze direction. When the user's eyes are focused on interaction, the surface is flat and the system acts as a touchscreen. When eyes are directed towards another area, physical knobs emerge from the surface. Compared to a touch only mode, experimental results showed that GazeForm generated a lower subjective mental workload and a higher efficiency of execution (20% faster). Furthermore, GazeForm required less visual attention and participants were able to concentrate more on a secondary monitoring task. Complementary interviews with pilots led us to explore timings and levels of control for using gaze to adapt modality.
We present WristOrigami, an origami-inspired design concept and system extending the interaction with smartwatches through a foldable structure with multiple on-wrist displays. The current design provides extra affordances via folding, flipping, and elastic pulling actions on a multi-display smartwatch. To motivate the design of WristOrigami, we developed a taxonomy that could be useful for analyzing and characterizing the origami-inspired multi-display smartwatch interaction. Through a participatory-design study with a set of prototypes with different levels of fidelity, we investigated users' perception of WristOrigami in a wide range of applications with the presented features, and summarized a list of common shape configurations. We summarized our findings into seven design recommendations, to inform the future design of foldable smartwatch interactions. We further developed a set of application demonstrations as proofs-of-concept.
Despite considerable prior work exploring foot-based interaction techniques, direct comparisons of the performance of these approaches have been lacking. Here, we compare the performance of the two most common approaches found in previous studies: rocking (applying pressure to different parts of the foot) versus rotating and sliding, considering the use case of a hands-free interface intended for seated musicians. Participants performed a number of representative operations, such as setting the tempo of a metronome, using the two strategies. Results indicate superiority of the rotating and sliding approach, both in completion time and responses to NASA TLX questionnaires, although rocking was preferred by some participants due to its ergonomics and subtle movements required for parameter-controlling tasks. Beyond the comparison itself, the decisions we faced related to menu design and feedback for our use case may offer helpful insight for the design of future foot-based interfaces.
We introduce Traffico, a tangible timetable representing dematerialized schedule and transportation information. It delivers a user's schedules in chronological order along with transportation information between schedules. Placed on the user's desk, Traffico suggests required transportation times using four options-walking, bicycling, bussing, and driving a car-and through an e-ink display. To investigate the advantages that Traffico provides to users, we conducted an in-field study of 10 participants over five days. The results revealed that Traffico supports the planning of moving times in a day through displaying transportation options on each schedule. We also found that Traffico provides better schedule reminders with event notifications in a sequential order, along with rotating interaction for checking events. Through this type of tangible interaction, Traffico provides possibilities to reflect dematerialized digital information into a physical form and to adopt a new way of scheduling and handling time.
Creativity has been a growing topic in the ACM community since the 1990s; however, no clear overview of this trend has been offered. We present a thorough survey of 998 creativity-related publications in the ACM Digital Library collected using keyword search to determine prevailing approaches, topics, and characteristics of creativity- oriented Human-Computer Interaction (HCI) research. A selected sample based on yearly citations yielded 221 publications, which were analyzed using constant comparison analysis. We found that HCI is almost exclusively responsible for creativity-oriented publications; they focus on collaborative creativity rather than individual creativity; there is a general lack of definition of the term 'creativity'; empirically based contributions are prevalent; and many publications focus on new tools, often developed by researchers. On this basis, we present three implications for future creativity-oriented HCI research: develop and employ clearer definitions of creativity; go beyond in-vitro studies of novel tools; and move toward interdisciplinary research collaborations.
Failure, whether it be "complete-and-utter" or "a minor setback", occurs in a variety of different creative practices, yet how it is perceived, handled, and recovered from is a lesser explored design space. Failing to address these perceptions of failure can have psychological repercussions, discourage users from continuing a practice, and form cultural stigma such as those associated with STEM fields. However, mediating practices to develop a culture of resiliency and perseverance is key to sustaining a (lifelong) practice and reshaping pedagogical strategies. In this work, we outline the design space of "guardians", or elements of a creative practice that mitigate the psychological effects of failure. Through contextual inquiry, we contribute an inventory of failure-mitigation strategies from a variety of creative disciplines. We synthesize guidelines for the design of new guardians and present a preliminary exploration of guardians for the lasercutting practice -- effigies and test tags.
This paper aims to understand interactions at creative hubs, and how this understanding can be used to inform the design of virtual creative hubs -- i.e., social-technical infrastructures that support hub-like interactions amongst people who aren't spatially or temporally co-located. We present findings from a qualitative field study in UK creative hubs, in which we conducted seventeen observations and ten interviews in three sites. Our findings reveal a range of key themes that define interactions within creative hubs: smallness of teams; neutrality of the hubs; value of the infrastructure; activities and events; experience sharing; and community values and rules. These interactions together form a network and elements that influence one another to make a creative hub more than just physical space. We employ the concept of Assemblage introduced by Deleuze and Guattari to explore this network of interactions and, in doing so, reveal implications for the design of virtual creative hubs that seek to replicate them.
Graphic designers often use the Web to collect images to use as inspiration and references for their work. Their resulting collections of images, however, typically do not retain important aspects of their visual research, such as their thought process when searching and all explored design avenues. Guided by an exploratory study with 14 expert graphic designers, we developed Prism - a system that supports a graphic designer's visual research on the Internet by automatically capturing all inspected images and annotating them with the designer's search trails. We evaluated Prism through a two-week field study with 11 expert designers. Our findings suggest that Prism's capture and display capabilities helped the designers to reify their design thinking, to better reflect on and compare alternative design ideas, and to collaborate with their colleagues and clients.
Conversations among people involve solving disputes, building common ground, and reinforce mutual beliefs and assumptions. Conversations often require external information that can support these human activities. In this paper, we study how a spoken conversation can be supported by a proactive search agent that listens to the conversation, detects entities mentioned in the conversation, and proactively retrieves and presents information related to the conversation. A total of 24 participants (12 pairs) were involved in informal conversations, using either the proactive search agent or a control condition that did not support conversational analysis or proactive information retrieval. Data comprising transcripts, interaction logs, questionnaires, and interviews indicated that the proactive search agent effectively augmented the conversations, affected the conversations' topical structure, and reduced the need for explicit search activity. The findings also revealed key challenges in the design of proactive search systems that assist people in natural conversations.
A learnable system allows a user to know how to perform correctly any task of the system after having executed it a few times in the past. In this paper, we propose an approach to measure the learnability of interactive systems during their daily use. We rely on recording in a user log the user actions that take place during a run of the system and on replaying them over the system interaction models, which describe the expected ways of executing system tasks. Our approach identifies deviations between the interaction models and the user log and assesses their weight through a fitness value. By measuring the rate of the fitness value for subsequent executions of the system we are able not only to understand if the system is learnable with respect to its tasks, but also to quantify its degree of learnability over time and to identify potential learning issues.
To ensure the playability of Multiplayer Online Battle Arena (MOBA) games, designers strive to balance different game occurrences. Although machine learning (ML) can help classify matches into different occurrence categories, designers demand more flexible input, interpretable output, and interactive collaboration with ML to facilitate analysis in breadth and depth. To this end, we work closely with a game company to design a visual occurrence analytics system through a stepwise co-design process. We first identify bottlenecks in game designers' conventional practices and their concerns about ML via an observational study. Then, we develop the single-match module of the visualization system to familiarize users with interactive analytics. Next, we incorporate ML models to recommend match segments of interest during occurrence classification and streamline the cross-match analysis. Empirical studies confirm the efficacy of our system. Experts' feedback suggests that our stepwise co-design process indeed helps them better embrace collaboration with machines.
While shoppers increasingly value sustainable products, considering sustainability can be difficult and time-consuming while shopping. In an expert workshop with 22 stakeholders, we gathered requirements for an assistance system supporting customers in identifying the sustainability of products at the point of sale. We integrated the resulting demands in a first mockup prototype, which was tested and discussed with a focus group. From the workshop and the focus group discussion, we deduced a set of ten guidelines for sustainability-oriented assistance systems. These guidelines were transferred into a prototypical mobile application, which allows customers to specify their personal understanding of multiple dimensions of sustainability. According to this profile, they receive easily understandable ratings for scanned products while shopping. A user study in a real supermarket strengthens the deduced guidelines and indicates that such a system can support customers to make more sustainable product choices.
Privacy policies are critical to understanding one's rights on online platforms, yet few users read them. In this pictorial, we approach this as a systemic issue that is part a failure of interaction design. We provided a variety of people with printed packets of privacy policies, aiming to tease out this form's capabilities and limitations as a design interface, to understand people's perception and uses, and to critically imagine pragmatic revisions and creative alternatives to existing privacy policies.
In this paper, we use design fiction to explore the social implications for adoption of brain-computer interfaces (BCI). We argue that existing speculations about BCIs are incomplete: they discuss fears about radical changes in types of control, at the expense of discussing more traditional types of power that emerge in everyday experience, particularly via labor. We present a design fiction in which a BCI technology creates a new type of menial labor, using workers' unconscious reactions to assist algorithms in performing a sorting task. We describe how such a scenario could unfold through multiple sites of interaction: the design of an API, a programmer's question on StackOverflow, an internal memo from a dating company, and a set of forum posts about laborers' experience using the designed system. Through these fictions, we deepen and expand conversations around what kinds of (everyday) futures BCIs could create.
Generative metaphorical design while rich is possibility, is not easy to do. In response, we have developed Metaphor Cards, a toolkit for supporting metaphorical design thinking. In this pictorial, we introduce Metaphor Cards and provide a how-to-guide for design researchers to make and use their own sets. To demonstrate this process, we provide a case study documenting our development of a set of Metaphor Cards for designing information systems for international justice. We conclude with reflections on the benefits and limitations of the Metaphor Card toolkit and suggestions for how to adapt Metaphor Cards to other domains and technologies.
Within sustainable HCI research, we have witnessed a growing interest in studying interaction designs that support households to 'shift' energy usage to times when it is sustainably favourable. In this paper, we investigate shifting through a purposely provocative and scripted design, which challenges the idea that renewable electricity is an always-available resource for households to consume. To do so, we made electricity for washing laundry either free or not available. We conducted a detailed qualitative study with four families that experienced our intervention for a month. We present five themes that illustrate how families adapted, reflected, and formed new routines and expectations related to washing practices. We discuss the broader implications of combining scripting and provocation as a means to intervene, disrupt and understand energy consuming practices within the home.