What is the nature of design, and the meaning it holds in human life? What does it mean to design well-to design ethically? How can the shaping of technology reflect our values as human beings? Drawing from the unconventional book Artful Design: Technology in Search of the Sublime , this keynote breaks down the designs of everyday tools, software, toys, musical instruments, and social experiences, examining the ways in which we shape technology and how technology shapes our society, and us in turn. This is a meditation for the "engineer with a soul" as well as for anyone curious (or concerned) about technology - not only what it does for us, but also what it does to us.
Tacit knowledge is a type of knowledge often existing in one's subconscious or embodied in muscle memory. Such knowledge is pervasive in creative practices yet remains difficult to observe or codify. To better understand tacit knowledge, we introduce a design method that leverages time-series data (interaction logs, physical sensor, and biosignal data) to isolate unique actions and behaviors between groups of users. This method is enacted in Eluent, a tool that distills hundreds of hours of dense activity data using an activity segmentation algorithm into a codebook - a set of distinct, characteristic sequences that comprise an activity. The results are made visually parsable in a representation we term process chromatograms that aid with 1) highlighting distinct periods of activity in creative sessions, 2) identifying distinct groups of users, and 3) characterizing periods of activity. We demonstrate the value of our method through a study of tacit process within computational notebooks and discuss ways process chromatograms can act as a knowledge mining technique, an evaluation metric, and a design-informing visualization.
Motivated by prior work on everyday creativity, we adopt a design-oriented approach seeks to move beyond designing for explicit interactions to also include the implicit, incremental and, at times even, unknowing encounters that slowly emerge among people, technologies, and artifacts over time. We contribute an investigation into designing for slowness grounded in the practice of making a design artifact called Slow Game. We offer a detailed critical-reflective accounting of our process of making Slow Game into a research product. In attending to key design moves across our process, we reveal hidden challenges in designing slow technology research products and discuss how our findings can be mobilized in future work.
We investigate engaging a computer science conference audience in sketching responses to the event as it occurs. In particular, we explore the response to inviting those present to engage in what is essentially an off-line, co-located, attendee-sourcing experience. Sketchnoting is a popular practice for documenting events, but these sketched records can be limited in scope at multi-track conferences, and paid professionals can be unaffordable at smaller events. Our challenges included: working with an audience with little or no experience of sketching or working with imagery; who were unaware of the possible benefits; and whose attendee engagement was variable - with individuals often working on laptops rather than actively listening during sessions. In order encourage engagement we hosted a pre-conference workshop, developed a conference-specific set of visual icons, and created prompt materials. This resulted in a remarkable visual record of the event, and also an increase in active listening and engagement.
Advances in biology and computational power have led to the availability of large biological data sets, yet these advances raise new design challenges. Designers must build effective tools that cater to the needs of biologists and data scientists in order to visually explore and manipulate data for modeling and analysis. We present the Tangible Tensors system, a new tensor-based visualization and tangible manipulation tool that serves to improve functionality over previous data analytics approaches. We designed a platform that supports iterative exploration of the solution space and better interpretation tools for biologists. User study results indicate that our system is easy to learn and use, and useful for data modeling and analysis tasks.
We present Mutation: performing arts based approach that can help decrease the cognitive load associated with cyborg transitioning. Cyborgs are human-machine hybrids with organic and mechatronic body parts that can be implanted or worn. The transition into and out of experiencing additional body parts is not fully understood. Our goal is to draw from performing arts techniques in order to help decrease the cognitive load associated with becoming and unbecoming a cyborg. Actors constantly shift between states, whether from one character to another, or from pre- to post- performance. We contribute a straightforward adaptation of classic performing art practices to cyborg transitioning, and a study where actors used these protocols in order to enter a cyborg state, perform as a cyborg, and then exit the cyborg state. Our work on Mutation suggests that classic performing art practices can be useful in cyborg transitioning, as well as in other technology augmented experiences.
The development of digital advertising capabilities has drawn attention toward the benefits of advertising automation and digital consumer analytics, including the appropriate technical skills to support these capabilities. By focusing on constraint and provocation in otherwise innocuous requests from advertisers, I uncover collaborative creative practices undertaken by the digital advertising workers responding to those requests when conceptualizing creative digital branding campaigns. The findings highlight some of seen but unnoticed interpretive methods these workers use to marshal a variety of technical and non-technical resources to develop successful and creative campaign ideas and plans from these requests. Outcomes can be used to improve the technology used to plan and launch digital advertising campaigns by helping teams better consider the user and advertiser, and enable better communication of campaign differentiation.
Shared Virtual Environments (SVEs) have been extensively researched for education, entertainment, work, and training, yet there has been limited research on the creative aspects of collaboration in SVEs. This raises questions about how to design virtual working spaces to support collaborative creativity in SVEs. In this paper, we outline an SVE named LeMo, which allows two people to create music collaboratively. Then we present a study of LeMo, in which 42 users composed music together using three different virtual working space configurations. Results indicate that (i) two types of territory and working configurations emerged during collaborative composing (ii) when made available to them, personal working spaces were extensively used, and were considered to be essential to successful collaborative music making and (iii) a publicly visible personal working space was preferable to a publicly invisible one. Based on these findings, three corresponding design implications for Shared Virtual Environments focusing on supporting collaborative creativity are given.
People express their creativity through something that they enjoy doing. Through engagement in creative work, creators cultivate their identity, enhance problem-solving skills, and speak their thoughts and feelings. In this paper, we look into how a community of practice supports members in legitimizing their creativity. We conducted a user study with members of a home brewing community to understand their creative beer-crafting practice. We first interviewed 11 home brewers, nine of which are members of a home brew club and two who did not belong to any home brewing organization. We then observed club members as they participated in club activities. Our findings suggest that two types of resource sharing are key in supporting different aspects of creativity in a community of practice. Based on these findings, we propose design strategies for supporting sharing in a creative community of practice.
Online crowds, with their large numbers and diversity, show great potential for creativity. Research has explored different ways of augmenting their creative performance, particularly during large-scale brainstorming sessions. Traditionally, this comes in the form of showing ideators some form of inspiration to get them to explore more categories or generate more and better ideas. The mechanisms used to select which inspirations are shown to ideators thus far have not taken into consideration ideators' individualities, which could hinder the effectiveness of support. In this paper, we introduce and evaluate CrowdMuse, a novel adaptive system for supporting large-scale brainstorming. The system models ideators based on their past ideas and adapts the system views and inspiration mechanism accordingly. We evaluate CrowdMuse over two iterative large online studies and discuss the implication of our findings for designing adaptive creativity support systems.
By necessity as well as desire, people move and act in space. Our feet carry us from place to place along paths, just as our minds carry us from idea to idea along paths. The same brain structures that underlie places and paths underlie ideas and relations. We talk that way, too; I just did. Our hands place, raise, lower, put together, take apart, turn over push, pull, and throw objects. Our minds and our talk do the same with thoughts. Our minds allow us to take different perspectives on the paths in space, on the relations among ideas, on actions on objects. That turns out to be the key to creativity.
Facilitating audience participation in a music performance brings with it challenges in involving non-expert users in large-scale collaboration. A musical piece needs to be created live, over a short period of time, with limited communication channels. To address this challenge, we propose to incorporate social interaction through mobile music instruments that the audience is given to play with, and examine how this feature sustains and affects the audience involvement. We test this idea with an audience participation music system, Crowd in C. We realized a participation-based musical performance with the system and validated our approach by analyzing the interaction traces of the audience at a performance. The result indicates that the audience members were actively engaged throughout the performance, with multiple layers of social interaction available in the system. We also present how the social interactivity among the audience shaped their interaction in the music making process.
This paper presents an observational study of collaborative spatial music composition. We uncover the practical methods two experienced music producers use to coordinate their understanding of multi-modal and spatial representations of music as part of their workflow. We show embodied spatial referencing as a significant feature of the music producers' interactions. Our analysis suggests that gesture is used to understand, communicate and form action through a process of shaping sounds in space. This metaphor highlights how aesthetic assessments are collaboratively produced and developed through coordinated spatial activity. Our implications establish sensitivity to embodied action in the development of collaborative workspaces for creative, spatial-media production of music.
The question whether deafness is a disability causes controversial and emotionally charged debates. The experience of hearing loss is different from the experience of being born deaf, as well as the need and willingness of using hearing aids. Following a medical model of disability, deafness has been traditionally viewed as a physical impairment. This paper points out highly debated issues related to the design of assistive technologies for people with hearing impairment. It illustrates the design case of a suite of smart jewels designed to address their emotional and socio-cultural needs beyond the functional goal of supporting hearing. The design case calls for a shift in the attitude toward disability - from a medical model which sees impairment as a personal deficiency to be "normalized", to a socio-cultural model which views disability as a socially constructed concept defined by the obstacles of a hearing-oriented world.
Many artists livestream their creative process, allowing viewers to learn and be inspired from the decisions -- and mistakes -- they make along the way. This paper presents the first broad look at the range of creative activities people stream. Through content analysis of livestream archives, interviews with 8 streamers, and online surveys with 165 viewers, we study current practices and challenges in creative livestream communities and compare them with prior observations of livestreaming in other domains. We observed four common types of creative livestreams: teaching, making, socializing, and performing. We identify three open questions for the research community around how to better support the goals of creative streamers and viewers: how to support richer audience interactions at scale, how to support all parts of the creative process, and how to support watching livestream archives.
AI art tools based on interactive evolutionary computation provide a form of creative collaboration between the user and machine. While some see the involvement of the user in the process as indication of the failure to truly develop computational creativity, another perspective would view it as an instance of a distributed system leveraging the strength of both the human and the computer. ImageSpace was developed to explore this dynamic further, providing an embodied interface based on spatial layout to make the communication more fluid. This paper discusses the way information is exchanged through interaction with the space, and how it can improve the experience of creating artwork with the system.
The idea of the 'model user' is well established in Computer Science. However, the usual practice of establishing a model user might not be appropriate within the context of personal data where such models are potentially limiting and exclusory. The domain of self-tracking data art offers an alternative view on the concept of the Quantified Self. The model user then becomes that of the data artist. An 'edge case' where the user's representation of 'self' is determined more by creative skills and less by the tools used. Moreover, it is argued that models of creativity could be useful in order to understand the concept of Quantified Self as an on-going process. The Mace and Ward model is given particular attention and examples of artists work, including the author's own, are discussed in order to highlight the complexity of the issues they face and the ways in which they overcome them.
Virtual Reality (VR) headsets have made immersive 3D drawing available to the general public. However, compared to 2D drawing, the presence of an additional dimension makes sketching in VR challenging, since creating precise strokes that are positioned as intended in all three dimensions imposes higher demands on the users' perception, motor and spatial skills. Another challenge users face is creating accurate shapes in which strokes are positioned correctly relative to previous ones, as they may need to use different views to plan their next hand movement. In this paper, we analyze the behaviours of users with different spatial abilities while drawing in VR. Our results indicate that there are different types of behaviours that affect different aspects of the sketches. We also found that the user's spatial ability affects the shape of the drawing, but not the line precision. Finally, we give recommendations for designing 3D drawing interfaces.
Visual metaphors are a creative technique used in print media to convey a message through images. This message is not said directly, but implied through symbols and how those symbols are juxtaposed in the image. The messages we see affect our thoughts and lives, and it is an open research challenge to get machines to automatically understand the implied messages in images. However, it is unclear how people process these images or to what degree they understand the meaning. We test several theories about how people interpret visual metaphors and find people can interpret the visual metaphor correctly without explanatory text with 41.3% accuracy. We provide evidence for four distinct types of errors people make in their interpretation, which speaks to the cognitive processes people use to infer the meaning. We also show that people's ability to interpret a visual message is not simply a function of image content but also of message familiarity. This implies that efforts to automatically understand visual images should take into account message familiarity.
We present DrawMyPhoto, an interactive system that can assist a drawing novice in producing a quality drawing by automatically parsing a photograph in to a step-by-step drawing tutorial. The system utilizes image processing to produce distinct line work and shading steps from the photograph, and offers novel real-time feedback on pressure and tilt, along with grip suggestions as the user completes the tutorial. Our evaluation showed that the generated steps and real-time assistance allowed novices to produce significantly better drawings than with a more traditional grid-based approach, particularly with respect to accuracy, shading, and details. This was confirmed by domain experts who blindly rated the drawings. The participants responded well to the real-time feedback, and believed it helped them learn proper shading techniques and the order in which a drawing should be approached. We saw promising potential in the tool to boost the confidence of novices and lower the barrier to artistic creation.
On the basis of a qualitative study of five domains of creative work, this paper analyzes two recurring strategies in the use of digital tools, 'margins' and 'view-shifts'. These strategies are commonly employed by creative professionals across five different domains. Based on video from observational studies of music producers, video production, industrial design, graphic design, and service design, we conduct a thematic analysis to arrive at the two strategies. We furthermore examine the two strategies in relation to existing research into creativity and cognition, and discuss how this can inform future studies of the use of digital tools in creative work.
This paper reports the evaluation of a new digital support tool designed to increase journalist creativity and productivity in newsrooms. After outlining the tool's principles, interactive features and architecture, the paper reports the installation and use of the tool over 2 months by 12 journalists in the newsrooms of 3 newspapers. Results from this evaluation revealed that tool use was associated with published news articles rated as more novel but not more valuable than published articles written by the same journalists without the tool. However, tool use did not increase journalist productivity. The evaluation results were used to inform future changes to the digital creativity support tool.
We investigate new media to improve how teams of students create and organize artifacts as they perform design. Some design artifacts are readymade-e.g., prior work, reference images, code framework repositories-while others are self-made-e.g., storyboards, mock ups, prototypes, and user study reports. We studied how computer science students use the medium of free-form web curation to collect, assemble, and report on their team-based design projects. From our mixed qualitative methods analysis, we found that the use of space and scale was central to their engagement in creative processes of communication and contextualization.
Multiscale design curation involves collecting readymade and creating self-made design artifacts, and assembling them-as elements, in a continuous space, using levels of visual scale-for thinking about, ideation, communicating, exhibiting (presenting), and archiving design process. Multiscale design curation instantiates a constructivist approach, elevating the role of design process representation. Student curations are open and unstructured, which helps avoid premature formalism and supported reflection in iterative design processes. Multiscale design curation takes advantage of human spatial cognition, through visual chunking, to support creative processes and collaborative articulation work, in integrated space.
Co-creative (i.e. collaboratively creative) activities involving physical interaction are becoming more prevalent in museums as a way of promoting opportunities for exploratory learning-through-doing. However, there is still a need for new techniques for understanding how physical interaction relates to engagement and creative expression in order to both evaluate exhibits and iterate on their design. This article reports on a study of how family groups physically interact in a museum environment with a specific co-creative exhibit--TuneTable. We relate observable markers of physical interaction with stages of engagement/expression based in the literature and identify several different trajectories of participant engagement and creative expression as they navigate the exhibit. We explore what these trajectories tell us about the types of inquiry and experimentation that TuneTable supports and discuss design implications. This paper's main contribution is a deep study of how physical markers reveal trajectories of creative engagement within a specific co-creative installation.
"Late twentieth-century machines have made thoroughly ambiguous the difference between natural and artificial, mind and body, self-developing and externally designed, and many other distinctions that used to apply to organisms and machines. Our machines are disturbingly lively, and we ourselves are frighteningly inert." - Donna Haraway. Simians, Cyborgs, and Women: The Reinvention of Nature.
The boundaries that Haraway lamented in 1990 were further eroded by the World Wide Web, wireless networks, smartphones, and today by social media, fake news, claims about Artificial Intelligence and an impending "Singularity". Are all boundaries truly illusionary? Or can we question illusions of illusion by actively asserting boundaries between ourselves and our machines? Artistic collaborators Shlain and Goldberg will describe how their art projects and experiences with technology are leading them to rediscover old barricades.
This paper presents and evaluates a new method for inspiring creativity in a co-creative design system. The method uses a computational model of aconceptual shift based on clustering of deep features from a database of sketches. The co-creative sketching tool maps a user's sketch to a sketch of a distinct category that has high, medium, or low visual and semantic similarity. We hypothesize that the degree of similarity between the user's and the system's sketches is associated with a range of cognitive models of creativity in a design context. We report on the findings of an empirical study that analyzes different design scenarios in which the user sketches in response to a proposed conceptual shift. The findings show that how similar the computational agent's sketch is to the user's original sketch is related to the presence of three types of design creativity in the user's response: combinatorial, exploratory, and transformational.
Artificial intelligence (AI) is becoming increasingly pervasive in our everyday lives. There are consequently many common misconceptions about what AI is, what it is capable of, and how it works. Compounding the issue, opportunities to learn about AI are often limited to audiences who already have access to and knowledge about technology. Increasing access to AI in public spaces has the potential to broaden public AI literacy, and experiences involving co-creative (i.e. collaboratively creative) AI are particularly well-suited for engaging a broad range of participants. This paper explores how to design co-creative AI for public interaction spaces, drawing both on existing literature and our own experiences designing co-creative AI for public venues. It presents a set of design principles that can aid others in the development of co-creative AI for public spaces as well as guide future research agendas.
Creative idea generation, the generation of original yet effective ideas, can be supported in virtual environments when interactions are facilitated via avatars with an appearance similar to their user, i.e. self-similar avatars. However, it is not known how self-similar avatars support creative idea generation. We propose that self-similarity supports the generation of original ideas, because it (i) increases the identification a user has with its avatar, which (ii) increases positive affect, and (iii) influences the positive affect-original idea generation link. To test this conjecture, an experiment was conducted where people composed their own avatar to be either self-similar or non-self-similar, which they then used to engage in two idea generation tasks presented within a custom virtual environment. The results suggest that using a self-similar rather than a non-self-similar avatar positively influences the generation of original ideas; and that this depends on the influence of self-similarity on the link between identification and positive affect. Thus, this paper contributes a mechanism that explains how self-similar avatars support the generation of original ideas.
With increasingly pervasive integration of technologies in everyday life, more data are generated around individuals' behaviors. Some of these data are accessible to individuals for reflection yet mostly presented in numbers or graphs, or represented by common metaphorical items, like virtual badges. Grounded in embodied cognition theories including conceptual metaphor and blending, and insights from social psychology, the idea of "blended causality" argues that behavioral data should be represented in virtual terms through blending behavioral consequences with users' existing knowledge of comparable causality. This paper emphasizes elaboration of blended causality into multiple imaginative narratives for reflective user experiences and reports the application of the extended guidelines as a creative design support tool in a series of workshops for designing representations of behavioral data. The concepts developed from the workshops vary in topics and blends, showing the effectiveness of the tool and informing a language of blended causality. Designers can use it to delineate representational mappings in terms of embodied experiences for examination and communication with team members like engineers.
Exposing people to concepts created by others can inspire novel combinations of concepts, or conversely, lead people to simply emulate others. But how does the type of exposure affect creative outcomes in online collaboration where dyads interact for short tasks? In this paper, we study the creative outcomes of dyads working together online on a slogan writing task under different types of interactions: providing both the partner's idea and their explanation for that idea, enabling synchronous chat, and only exposing a person to their partner's idea without any explanation. We measure the creative outcome and define text-similarity-based metrics (e.g., mimicry, convergence, and fixation) to disentangle the interactions. The results show that having partners explain their ideas leads to largest improvement in creative outcome. In contrast, participants who chatted were more likely to reach convergence on their final slogans. Our work sheds lights on how different online interactions may create trade-offs in creative collaborations.
Metaphors are important at multiple levels within design and society-from the specifics of interfaces, to wider societal imaginaries of technology and progress. Exploring alternative metaphors can be generative in creative processes, and for reframing problems strategically. In this pictorial we introduce an inspiration card workshop method using juxtaposition (or bisociation) to enable participants to explore novel metaphors for hard-to-visualise phenomena, drawing on a provisional set of inspiration material. We demonstrate the process through illustrating creative workshops in France, Portugal, Chile, and the USA, and reflect on benefits, limitations, and potential development of this format for use within interaction design.
We picture various forms of engagement with Tea (and Coffee). The images are selected from a large and ongoing collection of street photography and posed images along this theme. The images as presented here are prompts to interaction design inspirations, after the style of . The text is deliberately minimal, as part of a focus on visually rich content for and advancement of the form of primarily photographic pictorials in HCI.
The current work investigates how creativity manifests when designers use data work in the early phase of design. Designers are increasingly interested in utilizing the massive amounts of data surrounding our everyday lives. However, data work is still challenging to incorporate into the design process. In this paper, we present a case study with three novice design teams who were tasked to integrate data work into their design process. During the study, we observed how creativity took place in framing a design problem. We present and discuss their actions from a creativity process perspective, highlighting how they used and rationalized data-inspired inquiries creatively in the early phase of design. The current results inform the development of a design framework to structure data work methodologically and coherently into design processes. We coin this design framework Exploratory Data Inquiry.
Children's creativity contributes to their learning outcomes and personal growth. Standardized measures of creative thinking reveal that as children enter elementary school, their creativity drops. In this work, we evaluated whether a social robotic peer can help 6-10-year-old children think creatively by demonstrating creative behavior. We designed verbal and non-verbal behaviors of the social robot that constitute interaction patterns for artificial creativity. 51 participants played the Droodle Creativity Game with the robot to generate creative titles for ambiguous images. One group of participants interacted with the creative robot, and one group interacted with the non-creative robot. Participants that interacted with the creative robot generated significantly higher number of Droodle titles, expressed greater variety in titles, and scored higher on the Droodles' creativity. We observe that children can model a social robotic peer's creativity, and hence inform robot interaction patterns for artificial creativity that can foster creativity in children.
Here, we present the visual development of a digital manipulative that promotes creative storytelling among young children. The tool addresses multiculturalism, by presenting children with story elements from different cultures. We reflected on how to create visual elements that both represent a given culture and foster the children's imagination and creativity. It is key to create eye-catching and imagination-inducing visuals, that can be achieved through an exploratory and iterative process. It is crucial to undertake extensive research to learn about and understand the history and meanings behind each culture, to avoid stereotypes and other representation clichés. Whenever possible, it is important to involve people raised in each culture in the design process, since their contribution is grounded in real life experience and will translate into valuable insights.
This pictorial summarizes a qualitative study of children's perceptions of mobility through their own cognitive map drawings of the journey from home to school. We engaged 27 sixth-grade students (11 to 12 year-old) from Lisbon, Portugal, and analyze their drawings extracting 12 different design recommendations. We use these recommendations to provide design suggestions in terms of i) existing functionalities of mapping applications (that can be re-used); ii) improvements for existing map and wayfinding systems; iii) children's original design suggestions. The qualitative study reported here provides contributions that could help promote sustainable mobility for children in particular, with regards to innovation but also autonomy and supervision by caregivers.
This paper presents and discusses the preliminary findings of a research focusing on the design and use of Mobeybou, a digital manipulative aimed at promoting the collaborative creation of multicultural and cross-cultural stories among young children. We frame the research in the wider discussion of the 21st century skills and competencies, and present the digital manipulative as the product of a participatory design methodology. We discuss narratology, multiculturalism, embodiment and collaboration as the cornerstone ideas underpinning the design of Mobeybou. We also show how storytelling is envisioned as implying the creative use of the multicultural and multimodal information that is offered to users in the form of constraints and open-ended options. A pilot study, carried out with twelve 8-year-old children, has validated these design options and offered interesting insights for its further development. Here, we present the interface and discuss the feedback provided in the user study.
Using games to model, measure and increase creativity in children and adults would be a very engaging path to social impact and empirical progress. This paper proposes an approach to reframe the new and popular board game Codenames using associative creativity principles. The Remote Associates Test (RAT) is a test measuring creativity as a function of associative ability. comRAT-C is a previous computational cognitive system that can solve the RAT using associative and convergence principles. In this paper, we formalise Codenames using associative principles from comRAT-C. A way to computationally model and measure the difficulty of Codenames is proposed. We discuss whether Codenames or a future variant of it can be used in creativity research.
This paper explores "You Are the Ocean," an interactive art installation where a participant's brainwaves control a projected ocean simulation. Using an EEG headset, the approximate attention and meditation levels of the participant are measured. As the participant increases her attention and focus, the ocean and sky become stormier. When the participant calms her mind, the ocean too becomes calm. This paper describes the concept, implementation, and participant interaction surrounding the installation. While many existing artworks have used EEG headsets, "You Are the Ocean" is novel in its use of interactive, photorealistic real-time rendering to communicate the intrinsic connection between humans and the planet.
ICARUS is a game/performance for drum-kit and live electronics. The work explores improvised musical structures within a game setting, where the player navigates between five musical chapters. Each of these levels defines a particular set of musical interactions, sonorities, and performance possibilities through distinct mappings and level design. The performer is free to improvise, fail, explore, and through trial and error understand what the game rules are and complete each chapter. This results into a dynamic audio-visual performance, where while the rules are fixed, each level can be completed in multitude of ways. The game is designed specifically for the augmented drum-kit and the instrument's affordances; performance minutiae, digital electronics and musical parameters are analyzed and used as control input to the game. As the musical instrument offers a much wider range of expressive possibilities compared to a conventional game controller, the result is a musically expressive game play performance where the game acts as the mediator to the improvised drum-kit performance, while the music becomes the live soundtrack of the game.
This project experiments with techniques of smart product collages and schematic scenarios to investigate smart home technologies. Focusing on smart cameras, the artist presents three scenarios and a range of design research artifacts. These scenarios and collages are presented as conceptual tools to help understand and anticipate present and future issues related to digital privacy, security, trust, accountability, and fairness.
We introduce an audience participation system for a music piece, Crowd in C. A distributed musical instrument is implemented entirely on a web browser for an audience to easily participate in music making with their smartphones and to generate sound on their palms. This web-based instrument is designed to encourage an audience to play music together and to interact with other audience members. Each participant composes a short musical tune that will be a musical profile of oneself. Once a profile is submitted, they can browse other people's profiles and play a pattern they like in pairs as if one improvises with another musician. The use of musical profiles is a metaphor for online dating websites and the network created by the instrument mimics such social interaction where a user browses profiles, likes someone, and mingle with other online users.
Natura Machina: Teenage Meadow symbiotically combines plants and robotics in a harmonious relationship as part of a long-term goal of creating art that is simultaneously a new life form that can exist in nature.
The work consists of silicone rubber elastomer, air pumps, sensors, and reindeer moss lichen. It inflates and deflates rhythmically like the movement of breathing. Similar to a teenager's nature, it will react to a stranger's presence, especially when touched.
An essential function of architecture is to control the environment around us. In practice, interior climates are discretized into self-contained units, where wetness is designated to wet spaces, and dryness is kept to dry spaces. Contrary to nature's changing weather patterns, architecture is often static and binary, with no diffusion in between. As a result, weather conditions in nature are not experienced inside. Current installations using vapor geometries in architecture are limited to creative showcases. With Diffusive Geometries, we are proposing vapor as a medium to bring microclimates into architectural spaces. The unique characteristics of vapor as tectonic elements allow users to modulate visibility, create cooling gradients, and produce spatial patterns with three main elements: vapor vortex ring, vapor tornado, and vapor wall.
ReadingRites are human + A.I. participatory poetry readings. Poets and audience members read onscreen poetry written by artificial intelligence (A.I.). They read at the rate that the machine writes, -- which is sometimes very fast, and often confusing. Venues upcoming in 2019 include Beinecke Library at Yale University and the Barbican in London. ReadingRites was premiered at Brown University's Interrupt Festival on Feb 8th 2019. At each venue, local participants join digital-poet Jhave in reading rapid A.I.-generated text, -- playing their wits and voices against an evocative infinite deep-learning muse.
reSpire lets people bring tangibility to their invisible physiological state through shape-changing fabric deformed by airflow. We explore a way to support mental wellness via improving a self-interaction and interpersonal connectedness. reSpire encourages not only people to focus on their connection to inner body but also to interact with others through playful tangible interactions in the same location and develop a empathy. We created a non-machine like interface responsive to users' respiration patterns and hand gestures using a fabric and its deformation by airflow control. We also introduce a computational model to simulate the deformation of fabric by the variance of airflow pres-sure and direction. Various interaction scenarios highlight its applications not only to health but also to interactive art installation.
Bodies are interfaces, thresholds, vestibules, and gateways capable of hosting, carrying and birthing other life forms. Occupying indeterminable positions, interfacing with multidimensional borders, cultures, media, and ecologies; mediating internal and external inputs/relations - bodies feel vibrations, perceive and collect information, remediate, respond, negotiate expectations, and make connections through the limbs, eyes, ears, skin, cells and beyond. Scholars, critics, and theorists describe this complex interfacing as a multispecies relationship consisting of deep histories, continually re-forming and transitioning into something new (Dooren et al., 2016). What Eben Kirksey deems as the microorganism, Wolbachia, a post-human actor that meshes a variety of species together that exist in different time frames and realities (Kirksey, 2018). For this exhibition, we introduce Mothering Bacteria: A Speculative Forecast of the Body as an Interface (2018), a multichannel artwork with an embedded AI that interacts with bacteria, as it grows in real-time.
Pictures of Jap Girls in Synthesis, is a machine that creates live, visual poetry. The machine reads, or listens to poetry in a variety of languages. It synthesizes in real-time, visual images illustrating its understanding and interpretation of the text. As an interactive installation, poets can perform live poetry, with the machine simultaneously translating it into visual imagery, allowing a synesthetic and a more universal experience.
A Machine for Living In is a multi-layered digital media artwork using newly available computational and sensing tools to study the home as a site of intimate life. The project has two distinct phases: the construction and inhabitation of a functional smart home system, followed by an exhibition of processed data as a multi-part digital art installation. For ACM Designing Interactive Systems / Creativity and Cognition 2019, A Machine for Living In is installed as a multi-element piece in the gallery space consisting of a sensor narrative video, overhead camera video, and digital audio. All data were originally recorded in the artist's home over a period of weeks, and here are filtered through diverse processes of machine perception. In a process of joint human-machine authorship, this system produces a complex portrait of the home: as a space of language, intimacy, bodily practice, and quotidian narrative.
Rover is a mechatronic imaging device inserted into quotidian space, transforming the sights and sounds of the everyday through its peculiar modes of machine perception. Using computational light field photography and machine listening it creates a kind of cinema following the logic of dreams: suspended but mobile, familiar yet infinitely variable in detail. Rover draws on diverse traditions of robotic exploration, landscape and still life depiction, and audio field recording to create a hybrid between photography and cinema. Rover won the Best Art Paper award at SIGGRAPH 2017. For ACM Designing Interactive Systems / Creativity and Cognition 2019, Rover is installed on the 70 megapixel VRoom display in the black box theater in CalIT2 at UC San Diego. We have developed new functionality where image synthesis is distributed across the panels of the tiled display, visual and sonic trajectories are shaped using the unique hardware and software capabilities at UCSD. The result is an ultra-high resolution audio-visual experience with 4 channel surround sound.
A search journey in the imagined visions of a neural network. Given a photo, an artificial intelligence painter tries to recreate its likeness, and takes us through a visual journey in its search for the perfect reproduction. The spectator can intervene in the process, and focus on areas of her interest in the intermediate imagery. The painter will then shift its efforts to recreate the chosen impression. The emerging experience may resemble wandering within a vision or a dream.
The Burden of Selfhood is an interdisciplinary performance artwork exploring the intersection of feminism, identity and technology. By connecting methods from cognitive science, music, poetry, video and performance art, we investigate the experience of viewing and being viewed as a gendered body. Technology has accelerated the recursive gaze to the point that we continually perform and project back onto each other our internalized expectations for unattainable perfection. This poly-vocal performance uses large-scale data visualizations and live performers to make explicit both the collective gaze and the implicit impact of being seen. Select portions of the performance can be found at: https://www.youtube.com/watch?v=UPk2JSt-e9Q https://www.youtube.com/watch?v=j1DXmiBU_3w
Projected Horizons is an installation for the at the California Institute for Telecommunications and Information Technology (Calit2) comprised of visualizations of arctic horizons and an accompanying sound composition. Footage of the jagged horizons of arctic glaciers will slowly pan as an audio composition unfolds, comprised of hydrophone recordings of arctic aquatic life. The rise and fall of the linear line that comprises the horizon will be mirrored in the sonic arc of the accompanying composition.
In 1979 Rosalind Krauss laid bare the root cause or conditions of possibility that led to the transitory shift to post modernism. Envisioned through a series of Klein diagrams, a mathematical model borrowed from the social sciences her seminal work proved an inspiring model. Last year preliminary post-graduate research posed the question "Where do our bodies begin and end in a networked world?" Adopting a similar approach to Krauss I examine the sociological shift in primary communication from the physical (face-to-face) to the virtual (text-messaging) across a pre/post digital timeframe. Focusing upon Maurice Merleau Ponty's concept of intercorporeality I reimagine, via what may be termed 'extended positioning', interaffectivity through the mapping of the affective dimension. The concluding position, presented at the AAANZ conference in Melbourne last year, continues to inform my research and creative practice.
A tiny program for the big screen; a classic computer driving the latest, cinematic display. My contribution to the Art Exhibition for Creativity & Cognition / Design Interactive Systems 2019 is a 256-byte executable for the Commodore 64, written in 6502 assembly, for the auditorium's 8K display. This non-interactive program will produce graphical effects using only the character-based facilities of the Commodore 64's VIC-II and producing sound via the system's SID. I describe its relationship to my practice and concrete poetry, retrocomputing, sizecoding, the demoscene, and platform studies. The program's output is suitable for presentation as a "short" with other short experimental motion pictures/executable artworks or prior to one or more longer works. Bridging the border of time and different eras of computing, the program will be run on an original hardware NTSC Commodore 64 (not in emulation) with the video output upscaled.
We describe a series of short music pieces that is generated in a semi-improvised manner by a computer, using deep representations and temporal memory model constructed from learning a corpus of piano works by Sergei Prokofiev. Inspired by Prokofiev's piece of a similar name, this work explores imaginative capabilities of generative music machine through a series of passing-by sonic visions triggered by fleeting activations of the underlying musical network from a musician performer input. Unlike most other common machine learning and neural music compositions that explore stylistic imitation, the impetus here is to provide a rainbow of unimagined possibilities enabling creative human intervention and interaction with a complex system, realized in a series of improvisations, each with a different form, texture and character.
This paper describes the evolution of a "hybrid violin" developed and played by the author and reflects on the process of developing its sensorimotor dynamics as a salient affordance of real-time digital signal processing. The hybrid violin consists of a technical ensemble of hardware and software elements, including an acoustic violin, microphone, custom sensor glove (alto.glove), and customized ergonomic shoulder rest embedded with voice coils for haptic feedback coupled to digital audio output. Through trial-and-error revision by the designer, sensory feedforward and feedback paths are effectively symmetrized, thereby engaging the claim of the "enactive" approach to cognition as a creative practice: perception drives action and action drives perception. The hybrid violin catalyzes improvised performance insofar as it is responsive to all gesture and embeds no assumptions about expressive intention. The system allows for experimentation with violin technique by continuously and intensively tracking gestural and auditory inputs.
In this demonstration, I present an interactive prototype of the Drumball system, an embodied learning environment that allows for drum patterns to be turned into and used as letters, words or phrases. The system acts as a transducer of rhythmic input into multimodal output, and was designed to investigate the affordances of this embodied learning approach on the early literacy skills acquisition of children. The project follows a design thinking process (empathize, define, ideate, prototype, test) to explore cultural systems as a grounding for learning technologies design. The session provides a space to think beyond the traditional keypad interface and its reliance on alphabetic input, to explore how the application of the talking drum cultural system in the domain of human-computer interaction can be used to transform children's early experiences with literacy. I will demonstrate creating sequences of letters and words using the system by playing drum tones of varying pitches (tone, slap and bass).
Pixeldust (An Algorithmic Collective Memory Project) is a contributory, interactive video/audio/text engine that can be output to many different forms of exhibition-from small monitors to large-scale public video projections. Written in the Processing language, it accepts photographic portraits, disassembles them into component pixels that lie like dust in the bottom of the screen, then dramatically sweeps them up into the complete portrait as the viewer listens to a short autobiographical statement by the portrayed person. In a world increasingly threatened by erosion of civil rights, environmental destruction and economic disparity, Pixeldust presents a visual/aural display of hope, faith and resoluteness through short autobiographical texts and visual portraits from inspirational figures in world history.
The post-graduate enrollment statistic for deaf and hard of hearing (DHH) in Science, Technology, Math and Engineering (STEM) fields is very low compared to the hearing population. This drastically reduces DHH representation in the Information Technology (IT) workforce or academic research. DHH students generally use sign language interpreters to understand lecture materials but technically qualified interpreters are rare. These days, traditional in-person classes are being replaced with Massive Open Online Courses (MOOC). MOOCs improve access to materials, but hinder opportunities for collaboration which is vital for the DHH population. In this work, we propose DAVEE, a Virtual Reality (VR) classroom experience that facilitates live interpretation. During live sessions, DHH students can ask questions, receive feedback and have interactions with other students. The lectures and the interpretations can also be recorded for offline viewing.
Improvisational theatre (improv) has been proposed as a grand challenge for general artificial intelligence (AI)~\citemartin2016improvisational. Current state-of-the-art conversational intelligence models lack proper grounding, language understanding, and generate meaningless meandering responses~\citedziri2018augmenting. Utilizing them as improvised comedy partners (improvisors) is doomed to fail - curiously, this limitation makes their use particularly appealing. Improv theatre celebrates risk taking and failure by inviting performers to express themselves without hesitation or fear of being judged~\citejohnstone1979impro. Our installation is an interactive improv workshop for a group of interested participants, culminating in a live public performance. Attendees are invited to observe and interact with AI-based improvisational theatre technology. The workshop is facilitated by two improv theatre professionals with a combined 30 years of experience in teaching, training, and touring. The performance features various AI tools for augmented creativity.
Tabletop board games are a collaborative creative process that can benefit from new technical tools to augment game sessions. This demo is an interactive map creator made to create and display dungeon maps for people who play tabletop board games. It has an editing component for people to make maps, and a display component to view and interact with those maps. Users can engage with the demo on two levels, as a creator making dungeon maps, or as a player exploring them.
This demonstration is meant to educate the community about a topic called The Spanish Black Legend. The demonstration will feature interactive components meant to engage the general public and pique their interest in learning more about the source and history of Anti-Hispanic Rhetoric. The demonstration will include interactive features through a timeline-based strategy where the user will visualize the events of the Spanish Black Legend as they move through their story. The timeline-based strategy will be used to clearly demonstrate the history of Anti-Hispanicism as it began and has manifested into the way society presently speaks about and treats Chicanx and Latinx peoples. This demonstration will be displayed on a tablet with the hopes that conference attendees interact with the story from beginning to end and walk away with a deeper understanding of a topic that is very pertinent in today's political climate. The goal of this platform is to educate about this topic while providing an opportunity for users to share their thoughts and experiences with each other both during and after the demonstration.
Receiving feedback on open-ended creative work is quintessential for success. Receiving early-stage feedback often leads to higher quality results by increasing iteration. However, when designers in the "wild" seek feedback, the majority do so towards a later stage in their design process. Moreover, online communities geared towards sharing in-progress work often fail. This paper explores why designers wait to ask for feedback until later in their design process. We guided 21 designers through two peer feedback exchanges (early- and late-stage) and used pre- and post-surveys to capture expectations and reflections, respectively. Participants viewed both stages being similarly valuable, but opt for late-stage feedback because the design space is larger and less defined, which makes the "script" for early-stage feedback less clear in practice, leading participants to avoid it. Furthermore, participants had misunderstandings regarding idea selection and the prototype fidelity necessary to elicit effective feedback. We conclude with design implications for feedback system builders.
A business model describes the mechanisms whereby a firm creates, delivers, and captures value. Following the steadily growing interest in business model innovation, software tools have shown great potential in supporting business model development and innovation. Yet, understanding the cognitive processes involved in the generation of business model ideas is an aspect of software design-knowledge that has so far been neglected. To investigate whether providing stimuli - in this case, brainstorming questions - can enhance individual creativity in this context, we conduct an exploratory experiment with over 100 participants. Our study is the first to systematically investigate the process of idea generation using a software-based business model development tool with stimuli. Our preliminary findings have the potential to support the future development of business model development tools and to refine the research design used to evaluate such tools.
Sketching is one of the most accessible techniques for communicating our ideas quickly and for collaborating in real time. Here we present a web-based environment for collaborative sketching of everyday visual concepts. We explore the integration of an artificial agent, instantiated as a recurrent neural network, who is both cooperative and responsive to actions performed by its human collaborator. To evaluate the quality of the sketches produced in this environment, we conducted an experimental user study and found that sketches produced collaboratively carried as much semantically relevant information as those produced by humans on their own. Further control analyses suggest that the semantic information in these sketches were indeed the product of collaboration, rather than attributable to the contributions of the human or the artificial agent alone. Taken together, our findings attest to the potential of systems enabling real-time collaboration between humans and machines to create novel and meaningful content.
The internet and digital technologies have had a strong influence on how art is created, distributed and perceived. Museums and art galleries, however, are still predominantly places of passive consumption of art. In our project, we explore new forms of communication and feedback between artists and their audience, mediated by public displays. In this work-in-progress paper, we present a situated feedback system for giving feedback on artworks in a public setting. We present a preliminary evaluation of the system with artists and potential audience on their reactions to the system and eight different types of feedback.
The concept of flow is central to many fields including video game design, but the phenomenon is poorly understood scientifically, with no agreement on the cognitive components underlying it. In this paper, we present a cognitive science-based model of the flow experience that puts the processing of information at its core. For this, we identify key theories from the literature and propose two hypotheses about the cognitive processes?the "conditions"?underlying flow's phenomenology. We then describe the mechanisms that facilitate the fulfillment of these conditions and propose a small number of design heuristics for eliciting flow in video games. Finally, we consider some of our model's implications for the traditional interpretation of the phenomenon and discuss the need to test our framework with empirical studies.
Misunderstandings of science affect many lives. Novices commonly misunderstand explanations by overly relying on surface details instead of evaluating underlying logic. Prior work has found adding a patina of neuroscience leads readers towards positively assessing explanations. How might we help people better understand science explanations? A between-subjects experiment tested whether asking readers to reconstruct experiments leads them to focus more on underlying logic. Participants relied less on irrelevant surface details when reconstructing experiments. However, this did not impact their subsequent assessment of explanations. Our results suggest that reconstruction is a useful strategy for understanding explanations but is not readily transferred towards evaluating explanations.
In this paper we present ZenG, a neurofeedback ARapplication concept based on Zen Gardening to fostercreativity, self-awareness, and relaxation through embodiedinteractions in a mixed reality environment. We developedan initial prototype which combined physiological sensingthrough EEG with AR visualisation on the Magic LeapDisplay. We evaluated the prototype through preliminaryuser testing with 12 adults. Results suggest users found theexperience to be enjoyable and relaxing, however theapplication could be improved by including more featuresand functionality. ZenG shows the potential for AR toprovide immersive and interactive environments that couldpromote creativity and relaxation, providing solid groundsfor further research.
Effective creative work requires both "hot" (exploratory) and "cool" (exploitative) thinking. Unfortunately, many people (especially novices) under-explore, jumping to the "cool'' part too quickly, because they assume their current thinking "has to be" the path. This paper presents empirical results of how metaphorical problem framing scaffolds can influence creative performance. The task used De Bono's "Thinking Hats." In a between-subjects experiment comparing exploratory to exploitative problem frames, the exploratory problem frame led to more original designs and more diverse ideas during brainstorming. This work provides an empirical baseline of how -- even for short tasks -- assigning people responsibility for broad thinking leads to better creative work.
Writers regularly use a thesaurus to help them write well; the thesaurus is one of the few widespread writing support tools and many writers find it integral to their writing practice. A normal thesaurus is hand-crafted and structured around strict synonymy for a given word sense. However, writers rarely look for a perfectly synonymous word -- instead they have additional ideas or constraints, such as words that are less cliche, more specific, or less gendered. Poets describe their usage as searching for words that "hold more interesting connotations." We present a machine learning approach to thesaurus generation, using word embeddings, that leverages stylistically distinct corpora -- such as naturalist writing, novels by a particular author, or writing from a technical discipline. We show examples of how stylistic thesauruses differ from each other and from a regular thesaurus, as well as preliminary responses from two writers who are given multiple stylistic thesauruses. Writers describe these thesauruses as reflective of style, unique from each other, and more exploratory and associative than a regular thesaurus. They also describe an increased attention to connotation. We outline plans for quantitative evaluation of stylistic thesauruses, and user studies to understand their impact on specific tasks.
This study explored the impacts of a Virtual Reality (VR) design tool on designers' cognitive actions and creativity. Results revealed that the conceptual action in VR stimulated physical and perceptual action, and enhanced flexible cognitive thinking, to a greater degree than did the 2D digital design tool. Additionally, flexibility among the different levels of cognitive action in VR reinforced divergent design thinking, leading to more creative outcomes. Finally, we discuss the use of VR as an educational tool to heighten designers' creativity and divergent thinking.
This study seeks to provide insight into two-dimensional (2D) artists' approach to three-dimensional (3D) creativity within a virtual environment. Specifically, this research investigates the widespread assumption that Virtual Reality (VR) provides a natural interface by which traditionally 2D-centric artists may explore 3D content creation. Using a prototype of Canvox, a VR tool that uses voxels to represent 3D space with a single stroke , we performed a qualitative study in which artists were observed producing familiar assets in a virtual space. Here, we present data on 2D artists' expectations for the User Interface (Space, Navigation, Scale, Tool Accessibility, Lighting), Functionality (Tool Functionality, Materials, Brushes, Primitives), Applications (Communication Mechanisms, Use Cases), and User Mindset (Ideation, Creation, Opportunities) for VR creation. Our results offer insights into the bifurcations between artists' mental models of 2D and VR creativity: in VR, artists expect to create content, while in 2D, artists expect to render representations of content. We also demonstrate the potential use cases of this emerging creative platform.
Craft practitioners rely on reflection to train and improve their skills. The lack of media to represent and communicate their tacit knowledge makes reflection difficult, especially if this reflection happens after the fact. This paper presents a set-up that tracks and visualizes muscle activities of potters when throwing on the wheel. We use this set-up to investigate the effect of the visualization of potters' muscle activities on their retrospective reflections on their own practice. We conducted a preliminary user study with three expert potters. The preliminary results suggest that externalizing the once implicit muscle activities alters potters' reflection of their own practice and can potentially serve as a medium to communicate the tacit knowledge involved in throwing.
Many designers and artists in India, because of limited domestic patronage, are interested in international opportunities. However, they experience difficulties in discovering such opportunities. Finding relevant grants, funds, residencies, jobs, and collaborators takes significant time and effort. Typically, this is best done via a human matchmaker - someone who knows of opportunities and makes introductions. However, this person might be limited by their own social circles and knowledge. The few websites available are usually intimidating and not personalized. Our vision is to design and develop Dara, a conversational chatbot that asks a series of questions to its users and is able to suggest relevant opportunities. As a first step we built a Wizard-of-Oz prototype and interviewed 9 users about their experience. We then designed the personality, conversation flow and referral nomination system for our next-level prototype. We conducted a preliminary evaluation with 14 users in Bangalore, India. We discuss the implications for design of chatbots targeted at the creative community in the developing world.
This paper introduces BlockArt - a colorful visualization tool designed to let users discover their own unique programming patterns on Scratch, the leading block-based online creative coding platform for children. Existing tools and dashboards often utilize data about the types of blocks used in children's projects to generate a narrow quantitative assessment of a project's computational complexity. BlockArt serves both as an alternative approach and an artistic provocation that challenges this view. Rather than datafying children's creations, the tool utilizes data to reveal a hundred ways children create and express their ideas with code. For any given username, the tool dynamically generates colorful visualizations representing the diversity of blocks used in each of their shared projects over time. I discuss the design rationale and illustrate the functionality through examples. I conclude by discussing future work exploring the social potential of the tool in helping young creators connect with others in the community based on their creative computational styles.
To effectively support individual contributors in large scale ideation settings, we need a computational understanding of their cognitive processes. Although related work exists that references models from psychology, there are two shortcomings in existing approaches: First, they only analyse ideas on a statistical level. Second, they lack a notion of individual ideator differences. This work proposes a new user model for ideation, based on process-models from psychology and knowledge-graph based information extraction from submitted idea-texts. By building a computational model of ideation, this work aims to enable new graph-based analysis methods of cognitive ideation processes and new approaches to adaptive ideation support systems. These systems could be used to detect and counter fixation in individuals and to guide group efforts based on categories that should be explored.
Online feedback systems have gained in popularity in recent years. These systems have the potential to provide vast amounts of feedback from the crowd. My dissertation explores how creative workers can be assisted in evaluating this crowdsourced feedback. I qualitatively explored and studied this issue in three different feedback systems. Future work will develop a framework for recommending feedback evaluation strategies for different feedback types in the context of creative work and potentially massive amounts of crowdsourced feedback.
Feedback is a staple of the design process, but little is known about why designers delay or refuse feedback collection. To fill this knowledge gap, my dissertation identifies triggers and deterrents to feedback seeking behaviors. Based on our findings, I propose and test two interventions to promote feedback seeking behavior: 1) helping designers plan when they seek feedback to increase commitment and 2) generating feedback templates based on design stage to reduce the effort of feedback seeking. We envision a future where creativity support and educational tools use our interventions to encourage designers to seek feedback earlier, more frequently, learn faster, and eventually create better designs.
With the increased popularity of cameras, more and more people are interested in learning photography. People are willing to invest in expensive cameras as a medium for their artistic expression, but few have access to in-person classes to help improve upon their artistic skills. Inspired by critique sessions common in in-person art practice classes, we propose design principles for creative learning. We focus on applying these principles to design new interfaces that provide contextual in-camera feedback to aid users in learning visual elements of photography. We interactively visualize results of image processing algorithms as additional information for the user at capture-time. In this paper, we describe our design principles, and apply these principles in the design of three guided photography interfaces: one to explore lighting options for a portrait, one to highlight overall composition, and one to aid in de-cluttering.
The goal of this research is to identify the key factors that influence the engagement for the elderly with interactive technology in public and personal platforms. Such engagement factors can be used as design goals for the elderly interaction with technology and as a measurement tool for evaluating the engagement of the elderly's user experiences. We conducted two pilot studies and proposed an engagement framework for the elderly. In our proposed work we will examine the effect of creative expression and emotional attachment on the engagement of the elderly with interactive technology in public and personal platforms through a longitudinal study.
Creative work combines exploratory thinking to find novel solutions and exploitative thinking to refine those solutions. People often assume that the idea they come up with has to be the "correct solution," leading to under-exploration and preemptive exploitation. Despite advances in the practitioner literature, a cognitive and empirical basis of exploration strategies remains sparse. My dissertation examines scaffolding methods to enhance exploration in creative tasks. I investigate this through two interventions at different stages of the creative process. First, interactive guidance and adaptive suggestions embodied in the CritiqueKit system to improve evaluation of creative work. Second, problem-framing scaffolds that attune people towards the phases of exploration and exploitation during ideation. My research demonstrates content and process scaffolds with applications in the design of creativity support tools and creative education pedagogy.
The effect of feedback as an educational tool to nurture creativity varies via circumstances. My dissertation will propose an authentic-performance-based investigation of effective practices of feedback on creativity in an interior design studio. The primary work found that students with high feedback receptivity outperformed in creativity. The next step is examining the smaller percentage of those with low feedback receptivity but still achieved decent creative outcomes.
This extended abstract outlines the research plans and previous work of a first year PhD student applying to the Creativity and Cognition Graduate Student Symposium. The applicant's research interest is in designing and implementing motion-based interactions with robots, where the study participant is primed for the interactions through various narratives. Past work is described as well as an outline for future progress. The applicant is particularly interested in generating expressive motion for robots originating with human movers, captured by sensors and video.
Creativity has become a crucial topic in several disciplines such as computational creativity. Co-creativity is a rapidly growing research field in designing creative systems and artificial intelligence (AI) agents. Co-creative systems can use neuroscience to detect temporal activations to both control aspects of the user interaction and validate creativity theories. This work briefly introduces co-creative and computational creativity, presents the recent works of co-creative agent. It also introduces my future study of investigating divergent and convergent thinking in developing co-creative systems, reports my progress in research and doctoral program
This proposed research investigates new media and interaction techniques to address creativity challenges facing design teams. I propose role-based layers within zoomable spaces, as a novel technique for reducing collaborative fixation, while maintaining shared awareness and shared visual context. My research will enable design teams to create layered, multiscale design curation spaces which can modulate individual and collaborative creative processes.
The Open Science Hardware (OSH) movement has been gaining momentum to make scientific instruments widely accessible through open-source blueprints or DIY kits which can be built for a fraction of the cost of their commercial counterparts. Even though the designs of OSH are freely and openly available over the internet, in most cases, building or assembling science equipment from online designs require knowledge and skills in areas such as digital fabrication and basic electronics. Due to this, some open science practitioners face a multitude of challenges when replicating or customizing those designs to be used in their science experiments.To this end, through ethnographic field work and participatory design, my doctoral research explores such practical challenges faced by open science practitioners and tries to create new mediums and workflows to support efficient and inclusive dissemination of OSH.
The Maker and DIY craft movements promote values of democratization, creativity, and customization . While the associated informal learning structures - makerspaces, workshops, and DIY tutorials - support fabrication, they do not necessarily scaffold design learning . Here, I present my work on design learning in formal and informal contexts and propose the re-design of DIY instructions to better support and understand informal design learning.
In this workshop, the focus will be on providing a forum for researchers in the various interdisciplinary fields to engage in and explore research outcomes and discussions on sketching interactions, specifically for interfaces that promote creativity and cognition. Attendees of the workshop will actively experience sessions related to the presentation, discussion, and sharing of discoveries and innovations that target or leverage sketching interactions for interfaces in domains that include the learning sciences, education, artificial, and engineering. The workshop will serve as a platform to encourage opportunities of new collaborations and inspire growth for leveraging sketching interactions towards creativity and cognition.
Our objective is to explore distributed forms of creativity that arise in play to help guide and foster supportive research, game design, and technology. This workshop seeks to bring together researchers, game designers, and others to examine theories of creativity and play, game design practices, methods for studying creativity in play, and creative play experiences. Participants will present work, video prototype, discuss topics, and contribute to outcomes.
Crowdsourcing is a powerful approach for tapping into the collective insights of diverse crowds. Thus, crowdsourcing has potential to support designers in making sense of a design space. In this hands-on workshop, we will brainstorm and conceptualise new user interfaces and crowdsourcing systems for supporting designers in the design process. The workshop consists of developmental discussions of ideas contributed by the participants. In brainstorming and design sessions in groups, the participants will ideate new crowd-powered systems and user interfaces that support the designer's divergent and convergent thinking.