IDC '20: Proceedings of the Interaction Design and Children Conference

Full Citation in the ACM Digital Library

SESSION: Learning about computational concepts and data

Decoding design agendas: an ethical design activity for middle school students

If we expect our children to be driving technology design agendas in the future, we must first help them recognize that opinions and beliefs are baked into the technologies that we create and that these opinions may serve some groups of people more than others. In this paper, we discuss an ethical design activity completed by 19 middle school-aged children. The activity encourages students to see technical systems as socio-technical systems, to engage them in stakeholder analysis, and to apply ethical design tools in order to redesign YouTube. Results indicate students are capable of transforming into critical users and ethical designers of technology. They are able to recognize the underlying design agendas for popular technologies such as YouTube, identify stakeholders who shape those design agendas, and apply an array of tools to reimagine technologies in a more inclusive manner.

Coming to your senses: promoting critical thinking about sensors through playful interaction in classrooms

Learning through exploration is assumed to be a powerful way of introducing children to computer science concepts. However, it is uncertain how exploring physical computing toolkits can promote movement between conceptual knowledge and abstract reflection, and lead to critical thinking about technology. We investigated how children aged 9-11 years explored and reasoned about personal and environmental data sensors, using a playful exploration-based physical toolkit in their classroom. We report on the ways in which critical thinking about sensor accuracy and reliability developed through reflective dialogue and playful interaction, taking into account the support structures embedded in the classroom. Finally, we discuss strategies for designing exploration-based learning for classroom settings, to promote critical thinking about data sensing.

SmileyCluster: supporting accessible machine learning in K-12 scientific discovery

There is an increasing need to prepare young learners to be Artificial Intelligence (AI) capable for the future workforce and everyday life. Machine Learning (ML), as an integral subfield of AI, has become the new engine that revolutionizes practices of knowledge discovery. Making ML experience accessible to young learners, however, remains challenging due to its high demand for mathematical and computational skills. This research focuses on designing novel learning environments that help demystify ML technologies for K-12 students, and also investigating new opportunities for maximizing ML accessibility through integration with scientific discovery in STEM education. We developed SmileyCluster - a hands-on and collaborative learning environment that utilizes glyph-based data visualization and superposition comparative visualization to assist learning an entry-level ML technology, namely k-means clustering. Findings from an initial case study with high school students in a pre-college summer program show that SmileyCluster leads to positive change in learning ML concepts, methods and sense-making of patterns. Findings of this study also shed light on understanding ML as a data-enabled approach to support evidence-based scientific discovery in K-12 STEM education.

Investigating children's spontaneous gestures when programming using TUIs and GUIs

Spontaneous gestures produced during mathematics learning have been widely studied, however, research on the role of gesture in computing education is limited. This paper presents an investigation into children's use of spontaneous gestures when learning programming using either a tangible user interface (TUI) or a graphical user interface (GUI). The study explored the relationship between spontaneous gestures, interface type and learning outcomes in a programming lesson for primary school students aged 6-7. In the study, 34 participants engaged in a learning activity lasting approximately 37 minutes, using a TUI or a GUI. The study used a between-subjects design, and mixed methods. Pre-test and post-test data were collected, and sessions were video recorded and subsequently coded and analysed. A video analysis scheme, adapted from mathematics education research, was used to code the spontaneous gestures produced during the learning session. We found a statistically significant difference between the mean learning gains of high-frequency gesturers and low-frequency gesturers, with the top quartile showing significantly greater learning gains. There was no significant difference in the frequency of gestures between interface types. A qualitative analysis of representational gestures showed that some children use spontaneous hand gestures to demonstrate abstract computational concepts, providing evidence for the embodiment of children's offline thinking in the computing domain.

'Phone apps know a lot about you!': educating early adolescents about informational privacy through a phygital interactive book

Children's usage of online services expands as they transition into adolescence. However, many are unaware or ill-equipped about how their informational privacy is compromised through continuous tracking and data collection online. This paper explores how to enable early adolescents be more aware of informational privacy online and make informed decisions, taking a research-through-design approach. Based on technology probes with adolescents, the four-stage framework was developed and a smartphone-embedded book was chosen as a medium for defamiliarization yet relatability. Through the iterative design and testing with early adolescents, we investigated the novelty of blended interaction between physical and digital, and the effectiveness of various narratives based on the four stage framework. We found the importance for privacy educational materials to incorporate multiple levels of information complexity and discuss how to achieve that.

Youth making machine learning models for gesture-controlled interactive media

Machine learning (ML) technologies are ubiquitous and increasingly influential in daily life. They are powerful tools people can use to build creative, personalized systems in a wide variety of contexts. We believe ML has vast potential for young people to use to make creative projects, especially when used in conjunction with programming. This potential is understudied. We know little about what projects youth might create, or what computational practices they could engage in while building them. We combined a beginner-level ML modeling toolkit with a beginning programming tool and then investigated how young people created and remixed projects to incorporate custom ML-based gestural inputs. We found that (1) participants were able to build and integrate ML models of their own gestures into programming projects; (2) the design of their gestures ranged from coherent to disjoint with respect to the narratives, characters, and actions of their interactive worlds; and (3) they tested their projects by assessing the programmed vs. modeled aspects of them as distinct units. We conclude with a discussion of how we might support youth in combining code and ML modeling going forward.

Out of tune: discord and learning in a music programming museum exhibit

Museum visitors often come into the museum space receptive to exploring new ideas, and this may encourage members of visitor groups to be supportive and cooperative when engaging together with exhibits. However, as participant groups explore the concepts of the exhibit, interruptions, conflicts, or disagreements may result. We collectively label this social tension as discord. This paper studies discord among family groups interacting with TuneTable, a museum exhibit designed to promote middle school students' interest in and learning of basic computing concepts (e.g. loops, conditionals) through music programming. We analyzed video recordings of each participant group and found that discord often appears alongside three markers of high engagement: a) complex physical manipulation of exhibit components; b) conversation demonstrating an in-depth understanding of how the exhibit works; and c) instances of collaboration between group members. Our findings suggest that certain types of discord could potentially be indicators of productive learning experiences at museum exhibits related to computing. In addition, when designing informal learning experiences for computing education, our findings suggest that discord is a potential trigger for deeper engagement that warrants further exploration.

A designerly approach as a foundation for school children's computational thinking skills while developing digital games

This paper contributes to the contemporary debate on the increasing use of computational thinking (CT) in primary schools. It is based on an empirical study in which 28 Swedish third-grade school children (9-10 years of age) participated in a creative workshop where they were challenged to design a digital game using stop-motion film technique, working in groups. The study applies a designerly approach to game design activities to investigate what aspects of computational skills can be identified when children employ stop motion filmmaking as a means to envision a digital game design idea and how a designerly approach can enable them to enact dimensions of their computational skills? The data included video observations, casual conversations, and stop-motion videos representing the children's game design ideas. The analysis identified three aspects of computational thinking strategies while children produced stop-motion films: step-by-step procedural skills; design and arrangement skills; and computational perspectives.

SESSION: Diversity and ethics

Can design documentaries disrupt design for disability?

This paper shows how design documentaries can motivate new perspectives for design and disability. We critically consider the ways in which design documentaries can foreground children's lived experiences and priorities, in cases where it is not always possible to involve children early on in the design process. By presenting a design case for supporting communication that involves children with severe speech and physical impairments and their social peers, we discuss how this narrative method can evoke designer empathy and guide new interpretations. Our findings show that design documentaries can convey to designers rich and multifaceted accounts of children's communication experiences. Although this is found to be generative, we also identify a tension with a bodily impairment understanding of disability. Drawing on reflections from our case study, we propose new methodological implications for embedding design documentaries in the design process of technologies for disability.

Designing an online sex education resource for gender-diverse youth

Transgender and gender-diverse youth deserve proper sex education, but current educational and clinical structures largely ignore their developmental experiences. As a result, many of these teens go online to seek crucial information. Designers and researchers alike can benefit from an understanding of the design needs of gender-diverse youth for sex education online resources. We recruited 19 gender-diverse youth, ages 15 to 21, to participate in a mix of in-person and online design methods. This research makes three contributions; 1) identification of preferences for where gender-diverse teens prefer to get certain kinds of sexual health information, 2) design considerations for an online resource, 3a) a new method for eliciting preferences, the Four Corners Exercise, and 3b) a new method for combining the Asynchronous Remote Community (ARC) method with in-person sessions. Through this research, we provide key considerations in developing an online sex education resource for gender-diverse youth.

"It's your private information. it's your life.": young people's views of personal data use by online technologies

Children and young people make extensive and varied use of digital and online technologies, yet issues about how their personal data may be collected and used by online platforms are rarely discussed. Additionally, despite calls to increase awareness, schools often do not cover these topics, instead focusing on online safety issues, such as being approached by strangers, cyberbullying or access to inappropriate content. This paper presents the results of one of the activities run as part of eleven workshops with 13-18 year olds, using co-designed activities to encourage critical thinking. Sets of 'data cards' were used to stimulate discussion about sharing and selling of personal data by online technology companies. Results highlight the desire and need for increased awareness about the potential uses of personal data amongst this age group, and the paper makes recommendations for embedding this into school curriculums as well as incorporating it into interaction design, to allow young people to make informed decisions about their online lives.

Not on any map: co-designing a meaningful bespoke technology with a child with profound learning difficulties

We present a case study of co-designing digital technology that offers positive and meaningful experiences with and for a child with profound learning difficulties and his family. We combine Method Stories and Participatory Evaluation to capture the process of engaging with Archie, a seven year old boy affected by brain damage, in performing independently an activity he enjoys and in the learning of communication skills. Our co-design process led to the creation of "Not On Any Map", an interactive physical device comprised of a custom made box, smart buttons and an Android application. Participatory evaluation with Archie showed that he was highly engaged with the technology, giving him more agency with an overall positive impact on his life and that of his direct family. We thus contribute an outline of an approach to co-designing bespoke technology, and present accounts of interaction in which a non-verbal child with profound learning difficulties can convey meaning through actions and joint attention. These findings open up new spaces for rethinking the function of Augmentative and Alternative Communication system and shape new directions for co-design in this context.

Exploring accessible programming with educators and visually impaired children

Previous attempts to make block-based programming accessible to visually impaired children have mostly focused on audio-based challenges, leaving aside spatial constructs, commonly used in learning settings. We sought to understand the qualities and flaws of current programming environments in terms of accessibility in educational settings. We report on a focus group with IT and special needs educators, where they discussed a variety of programming environments for children, identifying their merits, barriers and opportunities. We then conducted a workshop with 7 visually impaired children where they experimented with a bespoke tangible robot-programming environment. Video recordings of such activity were analyzed with educators to discuss children's experiences and emergent behaviours. We contribute with a set of qualities that programming environments should have to be inclusive to children with different visual abilities, insights for the design of situated classroom activities, and evidence that inclusive tangible robot-based programming is worth pursuing.

18 Years of ethics in child-computer interaction research: a systematic literature review

Recent years have seen growing interest in 'ethics' within the Child-Computer Interaction (CCI) community. In this paper, we take stock of 18 years of CCI research by conducting a systematic literature study exploring how and to what extent ethics has been dealt with in the community's leading venues: the Interaction Design and Children (IDC) conference and the International Journal of Child-Computer Interaction (CCI). Searching all papers in the IDC conference proceedings and IJCCI, 157 papers were found that use the word stem 'ethic*'. Based on our analysis of these papers, our study demonstrates that while ethics is frequently mentioned, the literature remains underdeveloped in a number of areas including definition and theoretical basis, the reporting of formal ethical approval procedures, and the extent to which design and participation ethics is dealt with. Based on our study we provide five avenues of future research in the interests of developing a more explicit discourse on ethics in CCI.

Enabling children to design for others with expanded proxy design

We explore the question of how child designers can provide design ideas for technology that might reduce the marginalisation that can be experienced by some of their peers. To do this, we introduce the idea of Expanded Proxy Design that moves beyond the notion of "proxies as people" in design, to guide methods for engaging children into thinking about design ideas for a group that exists at some distance from their own experience. We outline three case studies where we made use of such methods. First, we consider expanded proxies in the context of technology and newly immigrant children who are unable to speak in English. Second, we consider the case of designing technology for children with and without visual impairments. Finally, we consider designing playful experiences for children with different temperaments. We reflect on the extent to which this expanded notion of proxies can be used as a meaningful vehicle for overcoming marginalisation and exclusion when children with different abilities design for each other. And we suggest ways to characterise, develop and refine expanded proxy design methods in this broader sense.

Designing dyadic caregiver-child personas for interactive digital media use

This paper provides a framework that adapts a design tool known as personas to better capture the ways that caregivers mediate their children's use of interactive media. Interactive digital media has become a more pervasive part of families' lives and tensions have increased around children's engagement with digital media. Historically, caregivers have enacted various tactics to mediate their children's practices around digital media. However, the design of technologies for children fails to account for different approaches to caregiving and the relationships between caregivers and children, instead focusing on the caregiver and the child as separate entities. The goal of our framework is to enable designers to consider the caregiver-child relationship by adapting the persona design tool to account for the relationship between caregivers and children. Drawing from prior research from user experience on persona development and communication on parental mediation theory, the framework outlines five phases to be used as a guide to develop caregiver-child dyadic personas. A dyadic approach to persona design explicitly highlights the relationship between two individuals (in this case, caregiver and child). We suggest that designing with dyadic personas enables designers to be more aware of nuances in caregiver-child relationships and can surface opportunities to facilitate collaboration between caregivers and children around interactive media.

Another decade of IDC research: examining and reflecting on values and ethics

Examining the values inherent in papers published at IDC provides a lens to our research community and informs the path of future research needs and opportunities. We conducted a content analysis of the values expressed in all full IDC papers between 2011 and 2019 and a survey with the first authors of 20% of these papers. We examine the types of IDC research contributions, the qualities and behaviors the research seeks to support in children, the role of the child and other stakeholders in the design process, the theories that inform IDC research, and the criteria that guide the technical design choices. We discuss the research contributions and the core value trends over the past two decades of IDC full published papers. We also present the ethical considerations central to the surveyed authors' work. Based on our analysis, we discuss implications and opportunities for future contributions, such as explicit attention to inclusivity in research, encouraging multidisciplinary collaborations, and expanding the qualities our community aims to support in children. These qualities include: focusing on children's sense of autonomy, agency, and empowerment; and children's participation in research as active creators of technology.

SESSION: Conversation agents and robots

Exploring young children's engagement in joint reading with a conversational agent

Joint book reading is a highly routinized activity that is nearly universal among families. Conversational agents (CAs) can potentially act as joint-reading partners by engaging children in story-related, scaffolded conversations. In this project, we develop a CA reading partner that incorporates components of effective conversational guidance (i.e., questions to stimulate thinking, specific feedback, and adaptive scaffolding) and examine children's interactions with this CA. We identify patterns in children's language production, flow maintenance, and affect when responding to the CA. We then lay out a set of affordances and challenges for developing CAs as conversation partners. We propose that, rather than attempting to develop CAs as an exact replicate of human conversational partners, we should treat child-agent interaction as a new genre of conversation and calibrate CAs based on children's actual communicative practices and needs.

Investigating family perceptions and design preferences for an in-home robot

Child-robot interactions in educational, developmental, and health domains are widely explored, but little is known about how families perceive the presence of a social robot in their home environment and its participation in day-to-day activities. To close this gap, we conducted a participatory design (PD) study with six families, with children aged 10--12, to examine how families perceive in-home social robots participating in shared activities. Our analysis identified three main themes: (1) the robot can have a range of roles in the home as a companion or as an assistant; (2) family members have different preferences for how they would like to interact with the robot in group or personal interactions; and (3) families have privacy, confidentiality, and ethical concerns regarding a social robot's presence in the home. Based on these themes and existing literature, we provide guidelines for the future interaction design of in-home social robots for children.

Better together: young children's tendencies to help a non-humanoid robot collaborator

In child-robot collaborations, a robot may fail to accomplish its part of a task. In this situation, the robot is reliant on the child to recover. Inherently prosocial, a child is inclined to help the robot collaborator if the child can properly identify the robot failure and infer how to help correct it. In this study, we investigate how a non-humanoid robot can solicit the help of a child-collaborator using only its motion path. We conducted a study with twenty-two children, ages 3-7, who participated in a collaborative building task with a non-humanoid mobile robot. We found that autonomous motion of a non-humanoid robot elicited prosocial behavior from 59% of children, and that young children were willing to engage with the robot as an animate partner despite its limited capabilities and form. This finding has implications for robot design striving to encourage prosocial behavior in children of different ages.

What is a robot?: an artistic approach to understand children's imaginaries about robots

The article describes an ideographic study conducted with 10 to 11 years old students to investigate their perceptions, ideas and imaginaries about robots. Its objective is to use this understanding to expand the ways of thinking the pedagogy of educational robotics. The study employed an art-based research approach and focused on involving students in the process of producing a fictional audiovisual narrative about robots. We analyzed their creative process and the resulting video through a multimodal approach. This analysis allowed identifying the different imaginaries, discourses and ideas that the participants have around the concept of "robot". These axes are used as cornerstones to begin a reflexive process to problematize and enable new perspectives to the pedagogy of educational robotics.

"Whom would you like to talk with?": exploring conversational agents for children's linguistic assessment

The dramatic increment of communication impairments among children increases the demand for intensive, highly accessible and low-cost interventions as well as new assessment and therapeutic tools. Our research aims at exploring the use of Conversational Agents (CAs) to support linguistic assessment and training among children with language impairment. One of the open research issues in this arena concerns the identification of the most appropriate form of "embodiment" of the CA for children to interact with. To this end, we evaluated the linguistic performance of 14 neuro-typical children and 3 children with language impairment comparing different CAs - physical object and virtual character - with "traditional" human interaction. Based on our analysis, we identify insights for the design of CA: the physicality does influence the performance of linguistic tasks for children with linguistic impairment. In addition, children seem to show a preference for the physical CA and perceived it as smarter than the virtual one.

Teachers' perspectives on social robots in education: an exploratory case study

Research has shown that social robots carry potential to be used in an educational setting. The possibility to have multiple roles carried out by one tool does not only instigate curiosity but also raises concerns. Whereas practical challenges get tackled by rapid technological advances, the moral challenges often get overlooked. In this study, we examined the moral values related to educational robots from a teachers' perspective, by first identifying concerns and opportunities, and subsequently linking them to (moral) values. We conducted focus group sessions with teachers to explore their perceptions regarding concerns and opportunities related to educational robots. Teachers voiced several considerations ranging from having concerns towards privacy to seeing opportunities in adding friendship and attachment a robot could emanate.

Using tabletop robots to promote inclusive classroom experiences

Geometry and handwriting rely heavily on the visual representation of basic shapes. It can become challenging for students with visual impairments to perceive these shapes and understand complex spatial constructs. For instance, knowing how to draw is highly dependent on spatial and temporal components, which are often inaccessible to children with visual impairments. Hand-held robots, such as the Cellulo robots, open unique opportunities to teach drawing and writing through haptic feedback. In this paper, we investigate how these tangible robots could support inclusive, collaborative learning activities, particularly for children with visual impairments. We conducted a user study with 20 pupils with and without visual impairments, where they engaged in multiple drawing activities with tangible robots. We contribute novel insights on the design of children-robot interaction, learning shapes and letters, children engagement, and responses in a collaborative scenario that address the challenges of inclusive learning.

SESSION: Science and sustainability

"When is the pressure zero inside a container? Mission impossible": 7th grade students learn science by constructing computational models using the platform

The paper explores students' learning about gases in chemistry through constructing computational models of complex systems with the new platform (MMM; [7]). The design of MMM is based on the agent-based modelling approach to complex systems. The interface is governed by an epistemological structure into which programming blocks are integrated. They are inserted into one of three sections: properties, actions and interactions, for each population of entities. 22 Seventh-grade students' learning of science and systems concepts and modeling practices using the MMM are compared with 28 students' learning with a normative curriculum. Findings shows that conceptual learning is deeper and more integrated; the system's levels are better distinguished and related. Students' modeling expresses a gradual increase in explorative-ness and most importantly, show a gradual shift from relying on the external modeling platform to activating and developing their own internal mental models on the fly.

Phenomenological programming: a novel approach to designing domain specific programming environments for science learning

There has been a growing interest in the use of computer-based models of scientific phenomena as part of classroom curricula, especially models that learners create for themselves. However, while studies show that constructing computational models of phenomena can serve as a powerful foundation for learning science, this approach has struggled to gain widespread adoption in classrooms because it not only requires teachers to learn sophisticated technological tools (such as computer programming), but it also requires precious instructional time to introduce these tools to students. Moreover, many core scientific topics such as the kinetic molecular theory, natural selection, and electricity are difficult to model even with novice-friendly environments. To address these limitations, we present a novel design approach called phenomenological programming that builds on students' intuitive understanding of real-world objects, patterns, and events to support the construction of agent-based computational models. We present preliminary case studies and discuss their implications for STEM content learning and the learnability and expressive power of phenomenological programming.

Gathering garbage or going green?: shifting social perspectives to empower individuals with special needs

Digital technologies are increasingly used with individuals with special needs for skill-building, social inclusion, and empowerment. However, different stakeholders involved in raising an individual with special needs have different and sometimes conflicting understanding of empowerment and motivations towards it. Using an empowerment framework, we collaboratively analyzed and critically reflected on the design and outcomes of three user studies conducted at a special needs school in New Delhi, India. The studies focused on learning how to make compost out of everyday kitchen waste, to assemble a solar lantern, and to buy groceries from a local store. Findings from the analysis provide insights into the complex socio-cultural conditions that enable, and in some cases limit, empowerment of individuals in an underserved and special needs context. We contribute to discussions on empowerment of children through design and use of digital technologies.

Designing to engage children in monitoring indoor air quality: a participatory approach

Indoor air quality (IAQ) is especially important for children because they are more susceptible to the deleterious impacts of poor air quality compared to adults. While devices to monitor IAQ are increasingly available, these are designed primarily for adults, and little attention has been paid to their potential use by children. This paper describes an effort to engage children directly in the design of an IAQ visualization interface for children. In engaging children in participatory workshops, we found that they rely heavily on visual, olfactory, and thermal cues to perceive and assess IAQ. Reflecting on these findings and based on design principles for technology for children, we created and tested child-friendly interface prototypes for IAQ visualization. Based on children's input, we designed a final set of visual interfaces that will be implemented in the IAQ monitor. The next study will test and deploy the monitor in the real world.

CCI in the wild: designing for environmental stewardship through children's nature-play

The recent UN Global Assessment report on Biodiversity and Ecosystem Services identified human-induced biodiversity loss as one of the greatest threats facing humanity today. Despite this, the field of Child-Computer Interaction (CCI) has so far paid little attention towards how it might directly contribute to biodiversity conservation efforts. This paper begins to address this gap by exploring how digital applications may be designed to support the nature-play experiences known to instill a value for nature and motivate environmental stewardship behavior in children. It describes a co-envisioning process carried out with nine children (7-11 years) within their nature-play contexts to understand how they would appropriate abstract 'digital functions' to enhance or support their situated nature-play. The functions envisioned by children are then reinterpreted as five design openings for CCI researchers to pursue to support nature-play opportunities for children. This research provides an initial framework for CCI researchers to contribute to global biodiversity conservation efforts by supporting the nature-play experiences known to promote long-term environmental stewardship in children.

Augmented scientific investigation: support the exploration of invisible "fine details" in science via augmented reality

Augmented reality (AR) has great potential to radically change science education by making abstract science concepts visible and interactive. In this paper, we describe initial investigations into high school students' perceptions of learning science with an AR technology (i.e., SmartIR) through analyzing semi-structured interviews. SmartIR is an app that supports the investigation of science, such as thermodynamics. Specifically, it can show changes in thermal imaging over time and provides a data analytics function that visualizes data for analyzing and interpreting the changes. Our analysis of 31 interviews shows that students perceived the exploration of science phenomena with "fine details", including a full vision of second-by-second changes in thermal imaging, as helpful and engaging to understand science concepts. In future work, these findings will be triangulated with logging data of their interactions with SmartIR and student-generated lab reports.

What do atoms feel?: understanding forces and energy in chemical bonding through the ELI-Chem environment

We developed and explored four degrees of bodily engagement - video, mouse, joystick and haptic - to support chemistry students in grasping the attraction-repulsion forces and energy changes involved in chemical bonding. These topics are difficult to grasp as there are no analogues from everyday life for opposing forces happening simultaneously. Our theoretical framework is based on embodied learning theory by relating conceptual learning to bodily experiences. The study uses quantitative methods with 48 high-school students in a pretest-intervention-posttest design. Findings showed an increase in accuracy and a decrease in response time in all conditions. Only the haptic condition showed a significantly larger increase in accuracy, however, not a corresponding decrease in response-time compared to other conditions. This study highlights the states in which embodied learning uniquely contributes to understanding: absence of prior embodied experience, learning about a nonvisual concept related to forces, and a high congruence with the concept learned.

SESSION: Literacy and language

A content analysis of voice-based apps on the market for early literacy development

Voice-based applications powered by conversational agents can potentially support young children's literacy development in informal settings. Yet, to realize such potential, designers must consider young users' typical communication and learning patterns. In this paper, we developed a framework of seven design dimensions across three aspects (i.e., learning content and goals, interactions and gamifications, and conversation design) that could influence the educational benefits young children receive from voice-based apps. We then used this framework to conduct a content analysis of 535 literacy-focused voice apps on the market to examine whether the prevalent design features of these apps meet the unique needs of young learners. Lastly, we discussed the implications of current design tendencies with the aim of encouraging future voice-based app designers to bridge the gap between research and practice.

Spelling their pictures: the role of visual scaffolds in an authoring app for young children's literacy and creativity

Children as authors and creators need to be supported at all stages of literacy development. This paper presents Picture-Blocks (PB) - a constructionist mobile app that allows children (ages 5-9) to create personally meaningful digital pictures while exploring spelling and vocabulary concepts in an open-ended manner. In PB, children can spell any number of picture objects (sprites) into existence, that they can use to make a picture composition and share with friends. PB also suggests semantically similar sprites, allowing children to explore related objects and discover new words. We evaluated the app by running an exploratory pilot with 14 children over a two weeks in-the-wild deployment. Qualitative and quantitative examples suggest that our design of the visual scaffolding interactions facilitated (i) high engagement and a sense of authorship via created pictures, (ii) instances of spelling corrections and vocabulary explorations (iii) digitally mediated social interaction and remixing. We present our findings of children's interactions and creations, while discussing implications for designers and developers of literacy technologies.

An english language learning study with rural chinese children using an augmented reality app

Augmented reality (AR) apps have the potential to support early English learning for children. However, few studies have investigated how children from rural low socio-economic status (SES) schools, who learn English as a foreign language (EFL) used and perceived an AR app in language learning. In this paper, we present an exploratory case study of 11 EFL children and four school teachers from a Chinese rural county who used an AR app (called AR PhonoBlocks), for one week. The goal of the app is to support children to learn the alphabetic principle of English. The key features are overlaid dynamic colour cues on 3D physical letters. We present the results including themes related to children's interactional behaviours and motivations, and rural teachers' feedback on the opportunities and concerns around using an AR app in a rural school context. We suggest design implications and future research directions for designing AR apps to support EFL children from low SES schools in early English learning.

Guiding the selection of child spellchecker suggestions using audio and visual cues

Spellchecking functionality embedded in existing search tools can assist children by offering a list of spelling alternatives when a spelling error is detected. Unfortunately, children tend to generally select the first alternative when presented with a list of options, as opposed to the one that matches their intent. In this paper, we describe a study we conducted with 191 children ages 6-12 in order to offer empirical evidence of: (1) their selection habits when identifying spelling suggestions that match the word they meant to type, and (2) the degree of influence multimodal cues, i.e., synthesized speech and images, have in prompting children to select the correct spelling suggestion. The results from our study reveal that multimodal cues, primarily synthesized speech, have a positive impact on the children's ability to identify their intended word from a list of spelling suggestions.

"Try your best": parent behaviors during administration of an online language assessment tool for bilingual Mandarin-English children

The world is becoming increasingly multilingual. In the U.S., despite rapid growth in linguistic diversity, there is a complete lack of multilingual language assessment tools and a severe shortage of multilingual clinicians to detect language impairments among children who speak minority languages. To develop accessible child language assessment tools, we designed MECO-LAB, a web-based bilingual Mandarin-English assessment that uses parents as one of the potential groups of test administrators. We analyzed 16 videos of child-parent dyads and found that with minimal instructions, the majority (11 out of 16) of parents were capable of administering MECO-LAB to their children. We identified 296 interference and 381 support behaviors from parents that are influenced by linguistic, cognitive, emotional, and technical factors that researchers should consider when designing online language assessments. We proposed design recommendations for supporting child-parent interactions in similar applications that enable parents to administer online bilingual language assessments to their children.

Ding- dong: the Storybell and its wizard

In this note the Storybell robot is described as the main touch-point of a remote reading aloud service which enables senior readers to read stories from their home to children. The Storybell acts as a tangible interface for children to interact with the community of readers as well as with the single reader. A research strategy based on Wizard of Oz (WoZ) and experience prototyping methods is described with the aim of probing parents' trust towards the remote readers as well as making parents reflect on their parental responsibility into guiding respectful interactions of their child with the community of readers. The note highlights the potential of the Storybell as both a tangible interface for children and a domestic embodiment for remote people and suggests involving parents as the wizards in WoZ experiments with their children to overcome trust issues and inspire strategies to guest the embodied remote readers at home.

Collaborative comic-based digital storytelling with primary school children

This work explores how comic-based digital storytelling can support primary school children in reflecting on situations involving conflict in the classroom. In particular, we focus on investigating three specific aspects: (1) the potential of digital story composition conducted collaboratively or individually, (2) the children's perception on the use of digital storytelling for reflecting on conflicts that might arise in class and, (3) the teachers' experience of introducing a digital tool for collaborative storytelling and comics composition in an educational context. In this paper, we explored these aspects by developing a case study. A class of 12 children and 2 teachers explore the use of a digital tool, named Communics, aimed at creating digital narratives individually and collaboratively. The results show that digital narratives created from collaborative storytelling are longer, more structured, and richer with meaning compared to stories from individual work. Moreover, it emerged that children prefer to work collaboratively, even if it meant compromising, going slower and waiting for their turn. Finally, teachers appreciated the collaborative use of Communics, and in particular, the turn-based feature as children can practice the narrative re-elaboration with a peer while waiting for their turn.

Ebook presentation styles and their impact on the learning of children

As technology advances, more and more learning materials can be expected to be available as ebooks. In spite of this, there is limited understanding as to how their content should be presented to enhance learning. In this experiment, 11-year-old children interacted with an ebook that contained either representational pictures, decorative pictures, seductive pictures or no pictures at all. They then took part in both a retention and transfer test. The results of these tests showed that the children attained their highest scores in both tests with the ebook that contained representational pictures and that they benefitted from the decorative style more than text only in retention test. The findings of this experiment should be of value to people involved in the design of children's ebooks and those who work on multimedia instructional materials.

SESSION: Wellbeing and health

"Otter this world": can a mobile application promote children's connectedness to nature?

Can a mobile application encourage children to spend more time outdoors and promote their connectedness to nature? In this paper, we present results from a three-week experimental deployment study of NatureCollections, a mobile application that allows users to build, curate, and share nature photo collections. Twenty-eight children (aged 9-12) and their parents participated in the study; 15 used the NatureCollections app, and 13 used a basic Photo app. We found that the NatureCollections app significantly increased the time children spent outdoors compared to the Photo app. Children in both groups said they felt happy and excited about spending time in nature. However, children in the NatureCollections group reported that time spent outside with the app increased their curiosity about the types of species and plants they saw and photographed. Children in the NatureCollections group also engaged in nature-based conversations with their parents, and even sought to look up information online about the plants and animals they observed. In contrast, children in the basic Photo app group did not display this level of curiosity about what they saw in nature, and the photos they took were driven largely by the aesthetic qualities of nature elements. Our results suggest that NatureCollections promotes and supports children's connectedness to nature.

"The thinking cap 2.0": preliminary study on fostering growth mindset of children by means of electroencephalography and perceived magic using artifacts from fictional sci-fi universes

Interventions aimed at promoting a growth mindset in children range from teaching about the brain's ability to change to playing computer games. In this work, we explore a novel approach to foster a growth mindset by means of interaction with a "magic hat" system which consists of using objects from sci-fi and pop-cultural references like Avengers or Star Wars. The artifacts are "enhanced" with embedded Electroencephalography (EEG) electrodes. In an initialization phase, the "magic hat" uses established Brain-Computer Interface algorithms to recognize certain mental processes of the child and the child is then able to use their brain signals to control a robot. We report on an experiment that validates the system with children who were asked to solve math problems. We evaluated their mindset before and after use of the system. In comparison with a control group, the children who used the system self-reported having a stronger growth mindset.

Pengunaut trainer: a playful VR app to prepare children for MRI examinations: in-depth game design analysis

We present the concept, design, and evaluation of a playful mobile virtual reality (VR) app for children to reduce anxiety and stress during MRI examinations. The Pengunaut Trainer aims to help children to familiarize themselves with the medical environment so that they can be examined without fear, rendering sedation unnecessary. The young patients learn about the procedure and train to lie still during a virtual MRI scan. We conducted a clinical trial focusing on an in-depth analysis of the game design. 29 children trained over 14 days on average before their MRI examination. The participants were impressed by the VR experience and motivated to train. They reported high levels of immersion and positive affect. Anxiety and negative feelings towards the upcoming MRI examination were significantly reduced after the training period. Moreover, our results indicate that the Pengunaut Trainer could be effective in reducing anxiety and stress during the MRI scan. Our results and the positive feedback from parents and medical professionals prove the validity of our approach.

Therapist vibe: children's expressions of their emotions through storytelling with a chatbot

Storytelling can develop children's emotional intelligence when they are asked to freely talk about their emotions. While parents are responsible for teaching emotional intelligence, studies in using affective technologies to help people become aware of their emotions have also been explored. In this paper, we investigate the opportunity of this technology in enabling children to recognize and express their emotions. We describe a chatbot that leverages storytelling strategies to listen to children as they share emotional events they experienced, then guides them through reflective discipline to devise the next course of action. We report the types of emotions children choose to share with the chatbot, the kinds of support that the chatbot provided, the challenges during the conversation and children's perception of the chatbot. From our findings, we suggest design considerations for a conversation flow that anchors on storytelling to support child-agent interaction.

Identifying opportunities and challenges: how children use technologies for managing diabetes

Type 1 diabetes (T1D) is one of the most common chronic diseases for children in the United States. Once diagnosed, the family must manage the disease to prevent further complications. Recent health technologies, like mobile phone health applications (mHealth apps), have become useful tools to support health monitoring, particularly for children living in the digital era. We used a narrative story to prompt 10 children with T1D to describe their diabetes management routines during semi-structured video interviews. Our results show that while the children feel a sense of ownership over their management routine, especially if they use newer monitoring technologies (e.g., continuous glucose monitor), they experience many challenges in managing their blood glucose (BG) levels, particularly during school. They also expressed emerging fears of security and privacy issues associated with technology use. By capturing these themes in T1D management from the children's perspective, we discuss design implications for developing supportive mHealth interventions.

Exploring an augmented reality social learning game for elementary school students

Social learning games can enhance students' learning engagement in the field of STEM. However, students do not always have the chance to experience social interactions during their learning activities. Augmented reality (AR) games have the ability to enable social interactions among students and allow them to interact with the virtual content while engaging in natural communication in the real world. However, little is still known on how to design social AR games for learning. In this paper, we presented a textbook-based AR social learning game for elementary school students to practice math together. We designed and developed the game concepts based on previous studies and co-design sessions and conducted a user study to explore how students would behave and interact with each other in the game. Our findings extend the understanding of students' social patterns in both collaboration and competition conditions under AR settings. Based on the findings, we propose design implications for designing AR social learning games in the future.

LifeMosaic: co-design of a personal informatics tool for youth

While a growing number of technologies offer personal data to the user, little is known about how such tools can be harnessed for and by adolescents. Prior work has focused on implementations, in which youth experience is subordinated to the prescribed aims of adults. Few studies have engaged with the concerns and motives of the young people themselves. Co-design can be a powerful method for exploring beyond such limits, informing design that reflects the voices and values of the designed-for population. This paper presents a case study exploring the design of a personal informatics app (LifeMosaic) by a group of 14 to 16-year-olds. LifeMosaic lets users set any focus they personally care about and track it using colours and stickers. The design was shaped through youth concerns around privacy, social support, flexibility and self-expression as well as the desire to support mental wellbeing. This work illustrates new opportunities and understandings for personal informatics with youth: framing the data as mediating transindividual meaning making.

Puppy island: theory-driven design of a serious game for young children with cystic fibrosis

This paper outlines the theory-driven design and development of Puppy Island, a serious game for children aged three to five who are living with the chronic illness cystic fibrosis. Puppy Island differs from typical serious games, as its central learning and design objectives relate to long-term wellbeing and empowerment rather than short-term knowledge transfer. The Early Years Learning Framework and Self-Determination Theory informed the design and development of the iPad prototype, while domain experts from industry, academia and the broader cystic fibrosis community were consulted to establish user requirements and evaluate the iterative designs. We analyse the development process, domain expert feedback, and how the game's features relate to core theoretical pillars in order to highlight key lessons that can be applied to future work on serious games with comparable aims or target audiences.

SESSION: Designing for learning and engagement

Using sensing technologies to explain children's self-representation in motion-based educational games

Motion-Based Touchless Games (MBTG) are being investigated as a promising interaction paradigm in children's learning experiences. Within these games, children's digital persona (i.e, avatar), enables them to efficiently communicate their motion-based interactivity. However, the role of children's Avatar Self-Representation (ASR) in educational MBTG is rather under-explored. We present an in-situ within subjects study where 46 children, aged 8--12, played three MBTG with different ASRs. Each avatar had varying visual similarity and movement congruity (synchronisation of movement in digital and physical spaces) to the child. We automatically and continuously monitored children's experiences using sensing technology (eye-trackers, facial video, wristband data, and Kinect skeleton data). This allowed us to understand how children experience the different ASRs, by providing insights into their affective and behavioural processes. The results showed that ASRs have an effect on children's stress, arousal, fatigue, movement, visual inspection (focus) and cognitive load. By exploring the relationship between children's degree of self-representation and their affective and behavioural states, our findings help shape the design of future educational MBTG for children, and emphasises the need for additional studies to investigate how ASRs impacts children's behavioural, interaction, cognitive and learning processes.

Engaging families around museum exhibits: comparing tangible and multi-touch interfaces

Over the last decade, large multitouch displays have become commonplace in museums and other public spaces. While there is preliminary evidence that exhibits based on tangible technologies can be more attractive and engaging for visitors than displays alone, very little empirical research has directly compared tangible to large multitouch displays in museums. In this paper, we present a study comparing the use of a tangible and a multitouch tabletop interface in an exhibit designed to explore musical rhythms. From an observation pool of 791 museum visitors, a total of 227 people in 82 groups interacted with one of the two versions of our exhibit. We share the exhibit design, experimental setup, and methods and measures. Our findings highlight advantages of tangible interaction in terms of attracting and engaging children and families. However, the two exhibits were equally effective at supporting collaborative interaction within visitor groups. We conclude with a discussion of the implications for museum exhibit design vis-à-vis visitor engagement and learning.

Supporting children's math learning with feedback-augmented narrative technology

A key challenge in education is effectively engaging children in learning activities. We investigated how a narrative story impacts engagement and learning, as well as how feedback can provide further benefits. To do so, we created an interactive, tablet-based learning platform with a multi-step math task designed using Common Core State Standards. Subjects completed a pretest and then were assigned to a condition, either one of three variations of the system (narratives, narratives with hints, and narratives with a tutoring chatbot using wizard-of-oz techniques) or a control system that has children complete the same learning task without narratives nor feedback, before the subjects completed a post test. 72 children in U.S. grades 3--5 participated. Our results showed that embedding learning activities into narratives boosted children's engagement as evaluated by coding video responses and surveys, and the integration of a tutoring chatbot improved learning outcomes on the assessment. These results provide evidence that a narrative-based tutoring system with chatbot-mediated help may support effective learning experiences for children.

How do we design for concreteness fading?: survey, general framework, and design dimensions

Over the years, concreteness fading has been used to design learning materials and educational tools for children. Unfortunately, it remains an underspecified technique without a clear guideline on how to design it, resulting in varying forms of concreteness fading and conflicting results due to the design inconsistencies. To our knowledge, no research has analyzed the existing designs of concreteness fading implemented across different settings, formulated a generic framework, or explained the design dimensions of the technique. This poses several problems for future research, such as lack of a shared vocabulary for reference and comparison, as well as barriers to researchers interested in learning and using this technique. Thus, to inform and support future research, we conducted a systematic literature review and contribute: (1) an overview of the technique, (2) a discussion of various design dimensions and challenges, and (3) a synthesis of key findings about each dimension. We open source our dataset to invite other researchers to contribute to the corpus, supporting future research and discussion on concreteness fading.

Techniques for augmented-tangibles on mobile devices for early childhood learning

Integrating physical learning materials with mobile device applications may have benefits for early childhood learning. We present three techniques for creating a hybrid tangible-augmented reality (T-AR) enabling technology platform. This platform enables researchers to develop applications that use readily available physical learning materials, such as letters, numbers, symbols or shapes. The techniques are visual marker-based; computer-vision and machine-learning; and capacitive touches. We describe details of implementation and demonstrate these techniques through a use case of a reading tablet app that uses wooden/plastic letters for input and augmented output. Our comparative analysis revealed that the machine-learning technique most flexibly sensed different physical letter sets but had variable accuracy impacted by lighting and tracking lag at this time. Lastly, we demonstrate how this enabling technology can be generalized to a variety of early learning apps through a second use case with physical numbers.

Blue whale street art as a landmark: extracting landmarks from children's cognitive maps for the design of locative systems

In this paper, we present design implications for the creation of digital maps in the context of wayfinding systems for children. In the process, we engaged 70 children in drawing the cognitive maps of their journey from home to school. Conducted in 2017, the study involved fifth-grade students (9-12 years old), at three different public schools in Funchal, Portugal. This paper offers reflections on their drawings and the resulting implications for the design of geographic technologies for children. Our contribution consists in highlighting ten types of landmarks extracted from children's 'maps' clustered into four categories, namely: Newness, Cultural personalization, Infrastructure, and Natural Landscapes.

Giggle gauge: a self-report instrument for evaluating children's engagement with technology

The Giggle Gauge offers a quick and simple way for researchers to evaluate the engagement of systems designed for children. This self-report metric is based on prior work delineating the components of engagement and was designed to address the limitations of children's cognitive development (e.g., by focusing on simple language and rapid administration). Through a process of iterative design (N = 23, ages 4 -- 10) and co-design (N = 8, ages 7 -- 11), we refined the items of this metric to ensure children's comprehension. A validation study with 26 children, ages 4 -- 7, confirmed the validity and reliability of the Giggle Gauge through the assessment of three properties: known-groups validity, criterion validity, and test-retest reliability. We simultaneously developed a bifurcated response type, intended to reduce the cognitive load of traditional ordinal response, and show through participant quotes that it may decrease the cognitive load of self-report questions for children.

"It's just too much": exploring children's views of boredom and strategies to manage feelings of boredom

Boredom is a universal phenomenon: everyone has experienced the sense of disengagement and apathy that comes when "there's nothing to do" Children are especially quick to grumble, "I'm bored!", despite an increase in pre-scheduled activities and interactive technologies designed to capture their attention. Are today's children losing an ability to generate their own antidotes to boredom, instead growing more dependent on external sources of increasingly digital and ubiquitous forms of entertainment? Current research indicates that young minds benefit from learning strategies to overcome feelings of boredom. How do children conceptualize boredom and find ways to transform feeling bored into creative play? We conducted five cooperative inquiry sessions with youth (7-13 years old) to explore the role that boredom plays in their lives and techniques to spark imagination and enhance their efforts to overcome boredom. Our findings suggest that boredom in children includes (1) a time dimension, (2) the sense that they lack control or agency, and (3) tangible interactions that prompt transitions from boredom to a more engaged, not-bored state.