MHFI'18- Proceedings of the 3rd International Workshop on Multisensory Approaches to Human-Food Interaction

When we are babies we put anything and everything in our mouths, from Lego to crayons. As we grow older we increasingly rely on our other senses to explore our surroundings and objects in the world. When interacting with technology, we mainly rely on our senses of vision, touch, and hearing, and the sense of taste becomes reduced to the context of eating and food experiences. In this paper, we build on initial efforts to enhance gaming experiences through gustatory stimuli. We introduce TasteBud, a gustatory gaming interface that we integrated with the classic Minesweeper game. We first describe the details on the hardware and software design for the taste stimulation and then present initial findings from a user study. We discuss how taste has the potential to transform gaming experiences through systematically exploiting the experiences individual gustatory stimuli (e.g., sweet, bitter, sour) can elicit.

Film makers, producers, and theaters have continuously looked at ways to embody and/or integrate multiple sensory cues in the experiences they deliver. Here, we present a reflection on past attempts, lessons learnt, and future directions for the community around multisensory TV, film, and multimedia as a historical, though renewed, space of content creation. In particular, we present an overview of what we call "tasty film", that is, film involving taste, flavor, and more broadly food and drink inputs, to influence the audience experience. We suggest that such elements should be considered beyond "add-ons" in film experiences. We advocate for experimentation with new kinds of storytelling taking inspiration from multisensory design research and work on sensory substitution. We position this article as a starting point for anyone interested in multisensory film involving taste, flavor, and foods.

Social robots become increasingly important in the domain of healthcare and maintenance. Nutrition is not an exception: research robots are used in several experiments to teach people about healthy nutrition. Moreover, robotic products, whose task is to keep track of the user's nutrition, to provide tips, reminders, and to recognize anomalies in health-related behaviors are on the way to market. To convince users of a robot's recommendations, speech is an important interaction modality. However, automatically adapting a robot's spoken advise depending on users' behaviors is still a challenge. We address this issue by building Drink-O-Mender, an interactive installation, which includes a Reeti robot augmented with additional sensing and adaptation abilities. The robot is designed to offer drinks in a social setting. It aims to convince users of consuming healthy drinks while adapting its spoken advices depending on the users' selected beverage choices. The installation is equipped with custom hardware including a smartscale to sense the type and quantity of consumed drinks. We describe the interactive installation in detail and demonstrate feasibility of generating adaptive spoken advices by reporting on insights gained from exhibiting the installation during a public event and observing interactions of 78 users with the robot.

Food craving is one of the fundamental desires of human nature. Many HCI researchers used to define this desire as a problem and has developed its corrective technology. However, positive aspects of food craving and food reward has rarely been in the scope of research. Few studies made its efforts to evaluate how food positively interacts with mental aspects of humans but have shown inconsistent results because of user subjectivity and environmental variations. Therefore, we have evaluated the human-food interactions with an EEG as objective indicator to track the mental activity. We found that when participants have a high craving for the certain food, which makes them feel comfort (hereafter comfort food), their working memory performance and related theta signal increase and stress related high beta signal decrease. The methodology adopted in this study will contribute to the progress in food-related celebratory technologies in HCI research field.

This work introduces the Virtual Cafeteria, a novel virtual reality simulation of a buffet-style cafeteria for use in food portion-size education of adolescents as an alternative to established approaches such as verbal education and education with fake food replicas. As user behavior is automatically recorded to the millisecond, the application also serves as a tool for assessment of food portion-size perception. The Virtual Cafeteria offers a wide range of energy and nutrient densities in 64 food and 7 beverage options. Addressing the limitations and shortcomings of the immersive environments used in previous studies on food portion-size, the Virtual Cafeteria features not only a considerably wider variety of food but also multisensory feedback coupled with a high level of interaction and photorealism, and imperceptible latency.

Food preference varies from person to person and is not easy to verbalize. This study proposes a dialogue system that elicits the user's food preference through human-robot interaction. First, as the default knowledge of the dialogue system, we determined the ingredients of each dish from a large-scale recipe database, and collected the taste and texture of each dish and its ingredients by analyzing a large number of Twitter messages. Subsequently, the dialogue system asks questions to elicit the user's preferred taste/texture of the food by using the default knowledge base, while employing frame-based dialogue management. Finally, we created a food vector space that represents the relationship between the dish names, ingredients, and taste/texture expressions. We also discuss the possibility of using this vector space in dish recommendation.

In this paper, we present Flavorlens, a mobile application for sharing dish observations. Each observation consists of one or more photographs, a title, a location tag, a description, a rating, a sensoring experience reporting about flavors, textures, and odors of a particular dish. Finally, we describe the design and implementation of our system which is based on a requirement analysis.

This paper introduces the methodological exploration of a design led approach towards utilizing human-food-interactions in teaching art appreciation. In workshops participants express emotional and cognitive responses towards works of art, then interpret and re-imagine those responses into the creation of representational-food dishes. From this we deconstruct and analyze the multi-sensory and multi-modal experience of food in its complex network of taste, colour, texture, and emotions.