What we did:We built a dual-stream architecture that leverages physiological signals for action control and utilizes residual effort-coupled variability as perceptual feedback in games.
What we found:We found that integrating tongue pressure with head-gaze for selection significantly improved performance compared to using tongue-only input, and we demonstrated that rendering residual variability as feedback can lead to a phenomenon we call physiological ventriloquism.
Takeaway:We contribute a design perspective that encourages the use of physiological variability as an expressive element rather than a source of error, promoting more innovative interaction methods in Human-Computer Interaction.
Abstract
Physiological input enables hands-free, embodied interaction in games, but its inherent signal variability complicates precise control. Most systems suppress this variability to achieve stability, leaving fine-grained variability of user effort underutilized. In this paper, we propose a dual-stream architecture that uses physiological signals for action control and residual effort-coupled variability as perceptual feedback to shape game feel. We investigate this approach using tongue-based input sensed via non-invasive surface electromyography (sEMG) in two studies. In Study 1 (n = 24), we evaluated the feasibility of tongue pressure in primary and modifier roles. Integrating head-gaze for selection with tongue pressure for confirmation significantly outperformed tongue-only input on performance measures, suggesting that separating spatial intent from physical effort can yield more stable control. In Study 2 (n = 12), we examined the experiential impact of residual variability in tongue pressure control as feedback. Rendering this residual jitter as a perceptual response led participants to misattribute their own effort to external resistance or animacy, a phenomenon we describe as physiological ventriloquism\rev, where one’s effort feels externally owned. We contribute a design perspective that reframes physiological variability not as error to be eliminated, but as expressive material when architecturally separated from control.
Video
Playing videos on this website requires sharing information with YouTube and
accepting cookies.
We designed a method to non-invasively infer Energy Metabolism Zones and Lactate Accumulation using smartphone touchscreen input combined with built-in sensors.
We built PneumoVolley, a Virtual Reality experience that utilizes pneumatic actuation to provide pressure-based haptic feedback on the head during gameplay.
We conducted a controlled experiment to assess the feasibility of using toe movements as an input modality for computer systems, focusing on the influence of five factors on interaction performance.