Scientific Publications

What cognitive and affective states should technology monitor to support learning?

Olugbade, Temitayo; Cuturi, Luigi; Cappagli, Giulia; Volta, Erica; Alborno, Paolo; Newbold, Joseph; Bianchi-Berthouze, Nadia; Baud-Bovy, Gabriel; Volpe, Gualtiero; Gori, Monica
This paper discusses self-efficacy, curiosity, and reflectivity as cognitive and affective states that are critical to learning but are overlooked in the context of affect-aware technology for learning. This discussion sits within the opportunities offered by the weDRAW project aiming at an embodied approach to the design of technology to support exploration and learning of mathematical concepts. We first review existing literature to clarify how the three states facilitate learning and how, if not supported, they may instead hinder learning. We then review the literature to understand how bodily expressions communicate these states and how technology could be used to monitor them. We conclude by presenting initial movement cues currently explored in the context of weDRAW.

The Influence of Auditory Information on Visual Size Adaptation

Alessia Tonelli, Luigi F. Cuturi and Monica Gori
Size perception can be influenced by several visual cues, such as spatial (e.g., depth or vergence) and temporal contextual cues (e.g., adaptation to steady visual stimulation). Nevertheless, perception is generally multisensory and other sensory modalities, such as auditory, can contribute to the functional estimation of the size of objects. In this study, we investigate whether auditory stimuli at different sound pitches can influence visual size perception after visual adaptation. To this aim, we used an adaptation paradigm (Pooresmaeili et al., 2013) in three experimental conditions: visual-only, visual-sound at 100 Hz and visual-sound at 9,000 Hz. We asked participants to judge the size of a test stimulus in a size discrimination task. 

Developing a pedagogical framework for designing a multisensory serious gaming environment

Sara Price, Sam Duffy, Monica Gori
The importance of multisensory interaction for learning has increased with improved understanding of children’s sensory development, and a flourishing interest in embodied cognition. The potential to foster new forms of multisensory interaction through various sensor, mobile and haptic technologies is promising in providing new ways for young children to engage with key mathematical concepts. However, designing effective learning environments for real world classrooms is challenging, and requires a pedagogically, rather than technologically, driven approach to design. This paper describes initial work underpinning the development of a pedagogical framework, intended to inform the design of a multisensory serious gaming environment. It identifies the theoretical basis of the framework, illustrates how this informs teaching strategies, and outlines key technology research driven perspectives and considerations important for informing design. An initial table mapping mathematical concepts to design, a framework of considerations for design, and a process model of how the framework will continue to be developed across the design process are provided.


Sam Duffy, Sara Price, Gualtiero Volpe, Paul Marshall, Nadia Berthouze, Giulia Cappagli, Luigi Cuturi, Nicolo Balzarotti, David Trainor, Monica Gori

WeDRAW aims to mediate learning of primary school mathematical concepts, such as geometry and arithmetic, through the design, development and evaluation of multisensory serious games, using a combination of sensory interactive technologies. Working closely with schools, using participatory design techniques, the WeDRAW system will be embedded into the school curricula, and configurable by teachers. Besides application to typically developing children, a major goal is to examine this multisensory approach with visually impaired and dyslexic children. 


A multimodal serious-game to teach fractions in primary school

Simone Ghisio, Paolo Alborno, Erica Volta, Gualtiero Volpe

Multisensory learning is considered a relevant pedagogical framework for education since a very long time and several authors support the use of a multisensory and kinesthetic approach in children learning. Moreover, results from psychophysics and developmental psychology show that children have a preferential sensory channel to learn specific concepts (spatial and/or temporal), hence a further evidence for the need of a multisensory approach. In this work, we present an example of serious game for learning a particularly complicated mathematical concept: fractions. The main novelty of our proposal comes from the role covered by the communication between sensory modalities in particular, movement, vision, and sound. The game has been developed in the context of the EU-ICT-H2020 weDRAW Project aiming at developing new multimodal technologies for multisensory serious-games on mathematical concepts for primary school children.

The Effect of Visual Experience on Perceived Haptic Verticality When Tilted in the Roll Plane

Luigi F. Cuturi, Monica Gori

The orientation of the body in space can influence perception of verticality leading sometimes to biases consistent with priors peaked at the most common head and body orientation, that is upright. In this study, we investigate haptic perception of verticality in sighted individuals and early and late blind adults when tilted counterclockwise in the roll plane. Participants were asked to perform a stimulus orientation discrimination task with their body tilted to their left ear side 90° relative to gravity. Stimuli were presented by using a motorized haptic bar. In order to test whether different reference frames relative to the head influenced perception of verticality, we varied the position of the stimulus on the body longitudinal axis. Depending on the stimulus position sighted participants tended to have biases away or toward their body tilt. Visually impaired individuals instead show a different pattern of verticality estimations. 

An open platform for full-body multisensory serious-game to teach geometry in primary school

Simone Ghisio, Erica Volta, Paolo Alborno, Gualtiero Volpe

Recent results from psychophysics and developmental psychology show that children have a preferential sensory channel to learn specific concepts. In this work, we explore the possibility of developing and evaluating novel multisensory technologies for deeper learning of arithmetic and geometry. The main novelty of such new technologies comes from the renewed understanding of the role of communication between sensory modalities during development that is that specific sensory systems have specific roles for learning specific concepts. 

Angle Discrimination by Walking in Children

Luigi Cuturi, Giulia Cappagli, Monica Gori

In primary school, children tend to have difficulties in discriminating angles of different degrees and categorizing them either as acute or obtuse, especially at the first stages of development (6-7 y.o.). In the context of a novel approach that intends to use other sensory modalities than visual to teach geometrical concepts, we ran a psychophysical study investigating angle perception by spatially navigating in space. Our results show that the youngest group of children tend to be more imprecise when asked to discriminate the walking angle of 90°, pivotal to learn how to differentiate between acute and obtuse angles. These results are then discussed in terms of the development of novel technological solutions aimed to integrate locomotion in the teaching of geometrical concepts.

Using force-feedback devices in educational settings: a short review

Gabriel Baud-Bovy, Nicolò Balzarotti

In this short review, we aim at providing an update about recent research on force-feedback devices in educational settings, with a particular focus on primary school teaching. This review describes haptic devices and education virtual environments before entering into the details of domain-specific applications of this technology in schools. Currently, the number of studies that investigated the potential of haptic devices in educational settings is limited, in particular for primary schools. 

Opportunities and Challenges of Bodily Interaction for Geometry Learning to Inform Technology Design

Sara Price, Sam Duffy

An increasing body of work provides evidence of the importance of bodily experience for cognition and the learning of mathematics. Sensor-based technologies have potential for guiding sensori-motor engagement with challenging mathematical ideas in new ways. Yet, designing environments that promote an appropriate sensori-motoric interaction that effectively supports salient foundations of mathematical concepts is challenging and requires understanding of opportunities and challenges that bodily interaction offers. This study aimed to better understand how young children can, and do, use their bodies to explore geometrical concepts of angle and shape, and what contribution the different sensori-motor experiences make to the comprehension of mathematical ideas.

Exploring how children interact with 3D shapes using haptic technologies

Nikoleta Yiannoutsou, Rose Johnson, Sara Price

Haptic devices have the potential to enhance the learning experience by foregrounding embodied, sensory and multi-modal elements of learning topics. In this paper, we report on-going work investigating a game prototype with haptic feedback for seven year old children’s engagement with geometrical concepts as part of an iterative design study. Our findings include a new game play mode adopted by the children, that empowers the use of haptic feedback in game play and has the potential to enable the enactment of shape properties in the game play process.

Effects of Chai3D Texture Rendering Parameters on Texture Perception.

Balzarotti N, Baud-Bovy G (2018)

Touching objects in a virtual environment is a challenge that has yet to be addressed convincingly, in part because haptic technology and, in particular, low-cost haptic technology have strong limitations. This work aimed at assessing the impact of Chai3D texture rendering parameters on texture perception. We used Multidimensional Scaling techniques to build psychological scales for the texture level, stiffness, dynamic friction and several texture patterns. Two perceptual dimensions were in general necessary to fully account for the one-dimensional parameter change. The scales for the texture level, dynamic friction and texture pattern parameters were markedly larger than the stiffness scale, indicating the potential of these parameters to generate well differentiated textures.

WeDraw has received funding from the European Union's Horizon 2020 Research and Innovation Programme under Grant Agreement No. 732391.
This website reflects only the author's view and that the Commission is not responsible for any use that may be made of the information it contains.
© WeDraw