Publications · Florian Müller

TonguePlay: Non-Invasive Tongue Pressure Input for Gaming

Sun, 01 Nov 2026 00:00:00 +0000

Pooria Ghavamian, Natalia Sempere, Asreen Rostami, Martin Hedlund, Xiaoyan Zhou, Oskar Hokkanen Eriksson, Florian Müller, Andrii Matviienko
Proc. ACM Hum.-Comput. Interact.

TL;DR: We built a dual-stream architecture that leverages physiological signals for action control and utilizes residual effort-coupled variability as perceptual feedback in games.

Physiological input enables hands-free, embodied interaction in games, but its inherent signal variability complicates precise control. Most systems suppress this variability to achieve stability, leaving fine-grained variability of user effort underutilized. In this paper, we propose a dual-stream architecture that uses physiological signals for action control and residual effort-coupled variability as perceptual feedback to shape game feel. We investigate this approach using tongue-based input sensed via non-invasive surface electromyography (sEMG) in two studies. In Study 1 (n = 24), we evaluated the feasibility of tongue pressure in primary and modifier roles. Integrating head-gaze for selection with tongue pressure for confirmation significantly outperformed tongue-only input on performance measures, suggesting that separating spatial intent from physical effort can yield more stable control. In Study 2 (n = 12), we examined the experiential impact of residual variability in tongue pressure control as feedback. Rendering this residual jitter as a perceptual response led participants to misattribute their own effort to external resistance or animacy, a phenomenon we describe as physiological ventriloquism\rev, where one’s effort feels externally owned. We contribute a design perspective that reframes physiological variability not as error to be eliminated, but as expressive material when architecturally separated from control.

Stars Without Steps: Bridging Observatory Access Based on Physical Proximity Through Educational XR

Mon, 01 Jun 2026 00:00:00 +0000

Clara Sayffaerth, Jennifer Meiler, Crystal McArdle-Ventura, Atakan Çoban, Christoph Hoyer, Florian Müller, Arno Riffeser, Jochen Kuhn, Albrecht Schmidt
Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems

TL;DR: We developed a mobile Extended Reality application that allows users to explore and interact with a mountain observatory virtually, addressing physical accessibility barriers.

Remote research facilities such as mountain observatories offer unique scientific and educational value, yet their physical inaccessibility limits participation for many audiences. This paper presents a location-independent, mobile Extended Reality (XR) application that extends the experiential reach of a mountain-based observatory beyond its geographic constraints. The application allows users to virtually explore a research telescope and interactively learn about the observatory’s scientific work, instrumentation, and operational principles. Following interviews with on-site visitors to identify access barriers and learning needs, we developed an XR experience and evaluated it in an in-the-wild study with three school classes (N = 35). We deployed the application across three contexts (remote location, intermediate plateau, mountain summit) and assessed factors such as mental load, presence, motivation, perceived access, and quiz results. Our findings indicate that context-dependent physical proximity influences all factors except the quiz outcome, highlighting the importance of proximity-aware design for future XR systems.

ShadAR: Real-Time Prompting on-Demand Visual Augmentation in Augmented Reality

Yanni Mei, Samuel Hartenstein, Jonas Wombacher, Florian Müller, Jan Gugenheimer — Mon, 01 Jun 2026 00:00:00 +0000

Yanni Mei, Samuel Hartenstein, Jonas Wombacher, Florian Müller, Jan Gugenheimer
Proceedings of the 39th Annual ACM Symposium on User Interface Software and Technology

TL;DR: We built ShadAR, an Augmented Reality system that allows users to issue verbal commands for on-demand visual augmentations in real-time.

Augmented Reality can reshape our visual perception (e.g., simulating color blindness) yet such effects are typically pre-programmed by experts, making visual augmentation static and inflexible. We introduce ShadAR, an AR system that allows users to describe expected visual augmentations (e.g., “Blur all people”) for changing their vision in real time. ShadAR can graphically manipulate AR passthrough real-time via shader injection. When receiving a user’s prompt, ShadAR uses LLMs (GPT5.2) to interpret the user’s intent, analyzes the user’s view with object detection (YOLOv26), generates a pixel shader, and applies it to the targeted region on AR passthrough. To understand usage pattern, we invited 21 HCI researchers to use ShadAR, identifying four application domains (design, health, productivity, and social interaction) and two prompt-framing patterns: procedural prompts specifying what to modify and how (“Highlight the book.”), and end-goal prompts conveying desired mental/behavioral states (“Make me laugh.”). Such end-goal prompts suggest a future in which users specify outcomes (e.g., “Make me not miss my flight”) while the system determines how to realize them, though often with inherently indeterminate results. We discuss design opportunities for ShadAR and ethical concerns.

No Needles Attached? Inferring Energy Metabolism Zones and Lactate Accumulation from Touchscreen Input

Dominik Schön, Uwe Gropengießer, Max Mühlhäuser, Florian Müller — Mon, 01 Jun 2026 00:00:00 +0000

Dominik Schön, Uwe Gropengießer, Max Mühlhäuser, Florian Müller
Proceedings of the Augmented Humans International Conference 2026

TL;DR: We designed a method to non-invasively infer Energy Metabolism Zones and Lactate Accumulation using smartphone touchscreen input combined with built-in sensors.

Recreational athletes increasingly adopt quantified-self practices to track and advance their training, recovery, and fitness. Blood lactate is a key biomarker in this context, but testing remains invasive and costly, limiting its use to professional sports and clinics. For everyday exercisers, even a coarse distinction, such as whether they are training below or above key thresholds, already provides actionable insight. We investigate whether commodity smartphones can classify these thresholds non-invasively using swipe input and built-in sensors. In a data-collection study, participants completed touchscreen tracing tasks at varying physiological states during their workout, while collecting blood samples. We analyzed touch, pressure, motion, and task features to understand their role in classifying the Energy Metabolism and Lactate Accumulation Zones and trained a Support Vector Machine and a Recurrent Neural Network. Our results demonstrate the feasibility of estimating these zones from short smartphone interactions, suggesting a path toward accessible, non-invasive on-device training guidance.

Meme, Myself and AR: Exploring Memes Sharing in Face-to-Face Conversation Using Augmented Reality

Yanni Mei, Samuel Wendt, Florian Müller, Jan Gugenheimer — Mon, 01 Jun 2026 00:00:00 +0000

Yanni Mei, Samuel Wendt, Florian Müller, Jan Gugenheimer
Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems

TL;DR: We explored how AR memes influence face-to-face conversations through a survey of meme users and a multi-user prototype study in which pairs of friends designed and used their own AR meme visualizations.

Internet memes are central to online communication, yet their visual humor is often lost in face-to-face (F2F) conversations. Augmented reality (AR) offers new ways to bring memes into F2F interactions, but it is unclear how memes can be integrated into F2F conversations using AR, and how they impact conversational dynamics. We surveyed meme users (N=29) to understand motivations and challenges in visualising memes in F2F conversations. With these insights, we developed an AR meme-sharing prototype and invited 12 pairs of friends to design AR visualizations for their memes and use them in conversations. Our analysis reveals two AR-unique visualizations: merging memes with one’s body (The-Meme-OnMe) and situating oneself in meme environment (Me-In-The-Meme). We observed two integration patterns: using speech as setup before a meme punchline, and showing memes simultaneously with speech to amplify humor. We report users’ reactions toward AR memes, showing how it enables playful social interaction.

If It's Safe, Don't Ask: Decreasing Frustration through User Involvement for Risky Robot Behaviors

Jan Leusmann, Sarah Schömbs, Maximilian Diedrich, Florian Müller — Mon, 01 Jun 2026 00:00:00 +0000

Jan Leusmann, Sarah Schömbs, Maximilian Diedrich, Florian Müller
Proceedings of the 21st ACM/IEEE International Conference on Human-Robot Interaction (HRI '26)

TL;DR: We conducted a user study to investigate how the speed-accuracy trade-off and user involvement affect frustration during human-robot collaboration in an assembly task.

Human–robot collaboration faces a speed–accuracy trade-off (SAT): higher speed lowers latency but increases errors; lower speed improves accuracy but extends waiting time. Both pathways can frustrate users in research and real-world deployments. Despite this importance, the impact of SAT on frustration and how to mitigate it remains underexplored. We conducted a user study (N = 24) in which participants collaborated with a robot in an assembly task. We investigate three levels of SAT (conservative, moderate, risky), and examine how uncertainty communication and offering decision autonomy affect user frustration. Our results show that user frustration is highest for risky robot behavior. User involvement decreased frustration for risky behaviors, but increased it for conservative ones, while verbal uncertainty communication had no effects. We further found that perceived transparency, agency, intelligence, and utility of the robot increase with conservative SAT, with user workload decreasing. We propose that user involvement is advisable in higher-risk settings to mitigate user frustration, whereas autonomous operation is preferable in lower-risk scenarios.

Engineering Cross-Compatible Personalization for Interactive AI-Driven Services

Mon, 01 Jun 2026 00:00:00 +0000

Florian Müller, Andrii Matviienko, Alessandro Bozzon, Carmen Santoro, Larbi Abdenebaoui, Andrea Lavazza, Andreas Komninos, Nicolas Sklavos
Companion Proceedings of the 18th ACM SIGCHI Symposium on Engineering Interactive Computing Systems

TL;DR: We conducted a workshop aimed at engineering cross-compatible personalization for interactive AI-driven services.

AI-driven services continuously infer “models-of-users” to personalize content, automate decisions, and optimize interaction. Yet, most user models remain opaque, fragmented across platforms, and hard to correct, transfer, or revoke. As a result, personalisation in AI-driven services is commonly experienced as a trade-off between privacy and platform lock-in. Users must stay loyal to platforms if a model of adequate utility is to be built around them. This tension is sharpened by emerging regulatory obligations around transparency and user oversight (e.g., GDPR and the EU AI Act). We propose a full-day EICS 2026 workshop that focuses on engineering cross-compatible personalization: a vision for future interactive systems in which users can make their digital self-representations inspectable, editable, and selectively shareable. The workshop brings together researchers and practitioners from interactive systems engineering, HCI, AI/ML, and security/privacy to (1) map the design and engineering space of user-sovereign personalization; (2) derive reusable artifacts for the EICS community, such as a reference architecture, protocol sketches, and a pattern language of interaction techniques; and (3) seed a community around building interoperable toolchains for AI personalization that is not only accurate, but also controllable, auditable, and context-adaptive.

Do It Fast, Forget It Fast: How Timing and Limb Visualizations Affect First-Person Augmented Reality Instructions

Mon, 01 Jun 2026 00:00:00 +0000

Clara Sayffaerth, Ehbal Ablimit, Annika Köhler, Jonas Wombacher, Albrecht Schmidt, Florian Müller
Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems

TL;DR: We conducted a controlled experiment to investigate the effects of imitation timing (parallel vs. sequential) and limb visualizations (hand vs. full arm) on user performance and experience in first-person Augmented Reality instructions.

Acquiring tacit knowledge and practical skills often depends on direct observation and in situ training. AR offers an alternative by overlaying first-person step-by-step instructions that guide users through tasks such as assembly and repair. Previous work demonstrates the effectiveness of AR instruction for specific applications. In our experimental work, we systematically explore aspects of the broader design space. We conducted a controlled experiment (n = 40) to investigate three key factors identified in learning theory and XR embodiment research: imitation timing (parallel vs. sequential), limb visualization (hand vs. full arm), and limb visibility (opaque vs. semi-transparent). Across all conditions, participants followed AR instructions and afterward repeated the tasks from memory. We assessed performance, user experience, and retention. Our results show that parallel imitation is faster and increases embodiment, whereas sequential imitation enhances memory retention and comfort. Our findings provide guidance for the temporal and visual design of first-person AR tutorials.

Context Is Cue-Cial: Assessing the Interpretation of Social Signals from Non-Anthropomorphic Robots in Different Contexts

Mon, 01 Jun 2026 00:00:00 +0000

Aparajita Chowdhury, Ana Carrasco, Florian Müller, Aino Ahtinen, Kaisa Väänänen, Albrecht Schmidt, Jan Leusmann
Social Robotics + AI

TL;DR: We conducted a study examining how humans interpret social signals from non-anthropomorphic robots in a hospital setting, testing 103 cues with 1,545 participants.

Humans excel at understanding social cues in communication, but robots struggle. Social cues are crucial for humans to interpret the intentions of their communication partners. Research indicates that we typically interpret the actions of anthropomorphic robots analogously to their human counterparts, paving a clear path to the design of appropriate social cues. For non-anthropomorphic robots, however, it is an open question how humans interpret social cues with different output modalities and in different contexts. Our study investigates whether social cues signaled by typical non-anthropomorphic modalities such as lights, sounds, and gestures are consistently interpreted across people and contexts. We, therefore, conducted a contextual investigation in a hospital, derived scenarios from co-design workshop, and tested 103 cues collected from the literature in a large online survey (N = 1545). Our results demonstrate that most human interpretations vary by context, highlighting the need to design dynamic and adaptive social cues for interactive robotic systems.

Comfort Is in the Air: Investigating the Perceived Comfort of Rotational and Translational Mid-Air Interactions

Mon, 01 Jun 2026 00:00:00 +0000

Dominik Schön, Thomas Kosch, Martin Schmitz, Sebastian Günther, Max Mühlhäuser, Florian Müller
Proceedings of the Augmented Humans International Conference 2026

TL;DR: We conducted a user study to analyze the perceived comfort and performance of rotational and translational mid-air interactions in Extended Reality.

The degrees of freedom of our arms allow us to easily translate and rotate our hands in front of our bodies. As such, most of the common mid-air input in Extended Reality (XR) builds on these two 3D transforms. However, current research primarily focuses on specific forms and expressions of mid-air input, thereby not providing insights into the comfort and performance of the two gestural primitive building blocks: rotation and translation. To design ergonomic mid-air XR gesture input, we need a thorough understanding of the comfort and performance intrinsic to these two ground-laying building blocks. In this work, we investigate the effects of multiple influences of gestural input, such as interaction placement and complexity, on perceived comfort and performance. Our results reveal comfort preferences for translational interaction, while also highlighting opportunities for rotational-based input. We discuss how these findings can guide designers in creating more comfortable XR experiences and situate our findings within previous work.

Anticipation Without Acceleration: Benefits of Shared Gaze in Collocated Augmented Reality Collaboration

Mon, 01 Jun 2026 00:00:00 +0000

Julian Rasch, Vladislav Dmitrievic Rusakov, Jan Leusmann, Florian Müller, Albrecht Schmidt
Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems

TL;DR: We conducted a user study to investigate the effects of shared gaze on collaboration in collocated Augmented Reality by varying gaze visibility, task types, and interaction spaces across three collaborative tasks.

Knowing what collaborators attend to is essential. Previous studies demonstrated that shared gaze enhances coordination and social connectedness in remote settings. In collocated settings, gaze can be both naturally observable and technologically augmented. AR enables gaze cues to be rendered explicitly in the environment. To investigate if and how such cues are beneficial in collocated AR collaboration, we examined both qualitative and quantitative effects across three task types (puzzle, negotiation, search) and two spatial setups (plane, room), focusing on task completion time and the collaborative experience. In our user study with 24 dyads (n=48), we varied gaze visibility and measured task performance, user preference, social connectedness, and shared attention. Our results show that sharing gaze in collocated collaborative AR can increase shared attention, is perceived as helpful, and improves the user experience, similar to remote collaboration, but has a limited impact on the actual task completion time across the chosen tasks.

ShadAR: LLM-Driven Shader Generation to Transform Visual Perception in Augmented Reality

Yanni Mei, Samuel Wendt, Jonas Wombacher, Florian Müller, Jan Gugenheimer — Wed, 01 Apr 2026 00:00:00 +0000

Yanni Mei, Samuel Wendt, Jonas Wombacher, Florian Müller, Jan Gugenheimer
Proceedings of the Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems

TL;DR: We built ShadAR, an Augmented Reality prototyping pipeline that enables real-time creation of visualizations through large language models and object detection.

Augmented Reality (AR) can visually transform a user’s world by rendering virtual content on top of reality. However, developing such AR apps and visualizations remains a complex process that requires an understanding of computer vision and programming skills. We present ShadAR, an AR prototyping pipeline that enables real-time creation of small AR visualizations and applications using large language models (LLMs) and object detection. ShadAR allows users to express their visual intent (e.g., pixelate every person around me) via natural language, which is interpreted by an LLM to generate corresponding shader code. This shader is then compiled in real-time and applied to the passthrough video stream.

Manipulating Stiffness Perception of Compliant Objects While Pinching in Virtual Reality

Yannick Weiss, Steeven Villa, Moritz Ziarko, Florian Müller — Mon, 01 Dec 2025 00:00:00 +0000

Yannick Weiss, Steeven Villa, Moritz Ziarko, Florian Müller
Proceedings of the 2025 31st ACM Symposium on Virtual Reality Software and Technology

TL;DR: We designed an experimental setup to investigate the manipulation of stiffness perception of compliant objects in Virtual Reality by altering the Control-to-Display ratio of participants' index finger and thumb movements during pinching.

Providing users with realistic sensations of object stiffness in virtual environments remains challenging due to the intricacies of our haptic sense. We investigate the use of a visuo-haptic illusion to alter the perceived stiffness of hand-held objects in virtual reality. We manipulate the Control-to-Display ratio of the index finger and thumb movements during pinching to make virtual objects feel softer or harder. We evaluated this approach on a variety of haptic representations and visualizations we selected through a pre-study survey (N=24). Results of our user study (N=20) demonstrate that this method effectively and reliably modifies stiffness perception, bridging gaps of 50% in physical stiffness without adversely affecting the visuo-haptic experience. Our findings offer insights into how different visual and haptic presentations impact stiffness perception, contributing to more effective and adaptable future haptic feedback systems.

BroomBroom! Evaluation of Leaning and Controller-Based Locomotion for Flying in Virtual Reality

Mon, 01 Dec 2025 00:00:00 +0000

Martin Hedlund, Florian Müller, Martin Schmitz, Cristian Bogdan, Remy Rey, Pooria Ghavamian, Deirdre Tobin, Andrii Matviienko
Proceedings of the 2025 31st ACM Symposium on Virtual Reality Software and Technology

TL;DR: We conducted a user study comparing headset-leaning and controller-based locomotion methods for flying in Virtual Reality, utilizing a broom metaphor for intuitive interaction.

Virtual Reality (VR) locomotion methods are mainly ground-based, room-scale, or discrete, making them ill-suited for flying experiences. Although leaning- and controller-based techniques are promising for flying in VR, we lack empirical evidence of their advantages. We compared combinations of leaning- and controller-based methods for steering and velocity in a user study (N = 24) using a broom metaphor to integrate these methods into an understandable locomotion reference. The steering methods were: 1) controller-pointing (CP) and 2) headset-leaning (HL); and for velocity control: 1) controller linear displacement (CLD) and 2) headset linear displacement (HLD). Results indicate that HL increase presence compared to CP. However, combining HL with CLD worsens coin collection rate, completion time, mental load, control factor ratings, and enjoyment. In contrast, HLD worked well when paired with either steering method. CP-CLD led to the highest coin collection rate and lowest mental load. All methods had comparable feelings of flying.

Falling into Fears: Exploring Virtual Reality as a Fear-Reduction Training Tool for Climbers

Remy Rey, Florian Müller, Francesco Chiossi, Andrii Matviienko — Sat, 01 Nov 2025 00:00:00 +0000

Remy Rey, Florian Müller, Francesco Chiossi, Andrii Matviienko
Proceedings of the First Annual Conference on Human-Computer Interaction and Sports

TL;DR: We conducted an exploratory study to investigate the effectiveness of Virtual Reality as a fear-reduction training tool for climbers, involving 20 participants who performed multiple falls from a height of 5 meters with and without a VR headset.

Fear of falling limits climbers’ performance and progression, particularly for beginners who must develop trust in their safety equipment and falling techniques. Traditional methods of overcoming this fear through repeated practice are time-consuming and stressful. In this paper, we explore the potential of using Virtual Reality (VR) to assist climbers’ training. We conducted an exploratory experiment in which 20 climbers performed multiple falls from a height of 5 meters both with and without a VR headset, experiencing 1-2 meters of free fall. We collected qualitative data through interviews conducted before, after, and following each fall. Our findings suggest that VR is a promising tool for reducing the fear of falling, though the exact mechanisms and effectiveness require further investigation through longitudinal studies. This study opens new possibilities for using VR as a training tool in climbing education, potentially accelerating the process of overcoming the fear of falling.

Through the Expert's Eyes: Exploring Asynchronous Expert Perspectives and Gaze Visualizations in XR

Clara Sayffaerth, Annika Köhler, Julian Rasch, Albrecht Schmidt, Florian Müller — Wed, 01 Oct 2025 00:00:00 +0000

Clara Sayffaerth, Annika Köhler, Julian Rasch, Albrecht Schmidt, Florian Müller
2025 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)

TL;DR: We conducted a user study to explore the impact of asynchronous first- and third-person perspectives and gaze visualizations on the efficiency, embodiment, and social connectedness during manual tasks in Extended Reality.

Transferring knowledge across generations is fundamental to human civilization, yet the challenge of passing on complex practical skills persists. Methods without a physically present instructor, such as videos, often fail to explain complex manual tasks, where spatial and social factors are critical. Technologies such as eXtended Reality and Artificial Intelligence hold the potential to retain expert knowledge and facilitate the creation of tailored, contextualized, and asynchronous explanations regardless of time and place. In contrast to videos, the learner’s perspective can be different from the recorded perspective in XR. This paper investigates the impact of asynchronous first- and third-person perspectives and gaze visualizations on efficiency, feeling of embodiment, and connectedness during manual tasks. The empirical results of our study (-) show that the first-person perspective is better in quantitative measures and preferred by users. We identify best practices for presenting preserved knowledge and provide guidelines for designing future systems.

ShadAR: LLM-Driven Shader Generation to Transform Visual Perception in Augmented Reality

Yanni Mei, Samuel Wendt, Florian Müller, Jan Gugenheimer — Wed, 01 Oct 2025 00:00:00 +0000

Yanni Mei, Samuel Wendt, Florian Müller, Jan Gugenheimer
2025 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)

TL;DR: We designed ShadAR, an Augmented Reality application that enables real-time transformation of visual perception through shader generation using large language models.

Augmented Reality (AR) can simulate various visual perceptions, such as how individuals with colorblindness see the world. However, these simulations require developers to predefine each visual effect, limiting flexibility. We present ShadAR, an AR application enabling real-time transformation of visual perception through shader generation using large language models (LLMs). ShadAR allows users to express their visual intent via natural language, which is interpreted by an LLM to generate corresponding shader code. This shader is then compiled real-time to modify the AR headset’s viewport. We present our LLM-driven shader generation pipeline and demonstrate its ability to transform visual perception for inclusiveness and creativity.

From Pegs to Pixels: A Comparative Analysis of the Nine Hole Peg Test and a Digital Copy Drawing Test for Fine Motor Control Assessment

Mon, 01 Sep 2025 00:00:00 +0000

Dominik Schön, Thomas Kosch, Martin Schmitz, Sebastian Günther, Max Mühlhäuser, Florian Müller
Proc. ACM Hum.-Comput. Interact.

TL;DR: We conducted a comparative analysis of the Nine Hole Peg Test and a Copy Drawing Test to assess Fine Motor Control using digital surfaces.

User interaction with digital systems requires Fine Motor Control (FMC), especially if the interfaces are complex or require high fidelity and fine-grained interactions. Despite its importance, Fine Motor Control is often overlooked in interactive system design, partly because of its complex assessment. Measuring changes in fine motor abilities due to prolonged use or fatigue currently requires repeated manual testing. This paper analyzes the concept of using the digital mobile devices’ input behavior to assess the user’s Fine Motor Control. For this, we show that Fine Motor Control can be assessed for touch and stylus-based interaction with a digital mobile system. We conducted a user study, where participants performed a Nine Hole Peg Test and a predefined Copy Drawing Test before and after exercises that affect fine motor skills. Based on this data, we investigated how metrics such as pressure, velocity, and entropy for touch and stylus input can be used to predict Fine Motor Control.

Pull Requests From The Classroom: Co-Developing Curriculum And Code

Dennis Zyska, Ilia Kuznetsov, Florian Müller, Iryna Gurevych — Fri, 01 Aug 2025 00:00:00 +0000

Dennis Zyska, Ilia Kuznetsov, Florian Müller, Iryna Gurevych
Proceedings of Mensch Und Computer 2025 - MuC '25

TL;DR: We conducted a case study on the co-development of curriculum and a custom-built peer feedback system for a university course on scientific writing.

Educational technologies often misalign with instructors’ pedagogical goals, forcing adaptations that compromise teaching efficacy. In this paper, we present a case study on the co-development of curriculum and technology in the context of a university course on scientific writing. Specifically, we examine how a custom-built peer feedback system was iteratively developed alongside the course to support annotation, feedback exchange, and revision. Results show that while co-development fostered stronger alignment between software features and course goals, it also exposed usability limitations and infrastructure-related frustrations, emphasizing the need for closer coordination between teaching and technical teams.

Social MediARverse: Investigating Users' Social Media Content Sharing and Consuming Intentions with Location-Based AR

Linda Hirsch, Florian Müller, Mari Kruse, Andreas Butz, Robin Welsch — Tue, 01 Jul 2025 00:00:00 +0000

Linda Hirsch, Florian Müller, Mari Kruse, Andreas Butz, Robin Welsch
Virtual Reality

TL;DR: We conducted an online survey with 110 participants to investigate how spatial contexts, dimensionality, and dynamics affect user engagement with location-based Augmented Reality content in social media.

Augmented Reality (AR) is evolving to become the next frontier in social media, merging physical and virtual reality into a living metaverse, a Social MediARverse. With this transition, we must understand how different contexts—public, semi-public, and private—affect user engagement with AR content. We address this gap in current research by conducting an online survey with 110 participants, showcasing 36 AR videos, and polling them about the content’s fit and appropriateness. Specifically, we manipulated these three spaces, two forms of dynamism (dynamic vs. static), and two dimensionalities (2D vs. 3D). Our findings reveal that dynamic AR content is generally more favorably received than static content. Additionally, users find sharing and engaging with AR content in private settings more comfortable than in others. By this, the study offers valuable insights for designing and implementing future Social MediARverses and guides industry and academia on content visualization and contextual considerations.

Investigating the Effects of Haptic Illusions in Collaborative Virtual Reality

Yannick Weiss, Julian Rasch, Jonas Fischer, Florian Müller — Sun, 01 Jun 2025 00:00:00 +0000

Yannick Weiss, Julian Rasch, Jonas Fischer, Florian Müller
IEEE Transactions on Visualization and Computer Graphics

TL;DR: We conducted a user study to investigate the effects of visual shape and size illusions on user performance, experience, and behavior during collaborative object handover tasks in Virtual Reality.

Our sense of touch plays a crucial role in physical collaboration, yet rendering realistic haptic feedback in collaborative extended reality (XR) remains a challenge. Co-located XR systems predominantly rely on prefabricated passive props that provide high-fidelity interaction but offer limited adaptability. Haptic Illusions (HIs), which leverage multisensory integration, have proven effective in expanding haptic experiences in single-user contexts. However, their role in XR collaboration has not been explored. To examine the applicability of HIs in multi-user scenarios, we conducted an experimental user study (N=30) investigating their effect on a collaborative object handover task in virtual reality. We manipulated visual shape and size individually and analyzed their impact on users’ performance, experience, and behavior. Results show that while participants adapted to the illusions by shifting sensory reliance and employing specific sensorimotor strategies, visuo-haptic mismatches reduced both performance and experience. Moreover, mismatched visualizations in asymmetric user roles negatively impacted performance. Drawing from these findings, we provide practical guidelines for incorporating HIs into collaborative XR, marking a first step toward richer haptic interactions in shared virtual spaces.

Don't They Really Hear Us? A Design Space for Private Conversations in Social Virtual Reality

Josephus Jasper Limbago, Robin Welsch, Florian Müller, Mario Di Francesco — Sun, 01 Jun 2025 00:00:00 +0000

Josephus Jasper Limbago, Robin Welsch, Florian Müller, Mario Di Francesco
IEEE Transactions on Visualization and Computer Graphics

TL;DR: We conducted a study on the design of private conversation methods in Social Virtual Reality applications, focusing on user needs and interaction methods.

Seamless transition between public dialogue and private talks is essential in everyday conversations. Social Virtual Reality (VR) has revolutionized interpersonal communication by creating a sense of closeness over distance through virtual avatars. However, existing social VR platforms are not successful in providing safety and supporting private conversations, thereby hindering self-disclosure and limiting the potential for meaningful experiences. We approach this problem by exploring the factors affecting private conversations in social VR applications, including the usability of different interaction methods and the awareness with respect to the virtual world. We conduct both expert interviews and a controlled experiment with a social VR prototype we realized. We then leverage the outcomes of the two studies to establish a design space that considers diverse dimensions (including privacy levels, social awareness, and modalities), laying the groundwork for more intuitive and meaningful experiences of private conversation in social VR.

CreepyCoCreator? Investigating AI Representation Modes for 3D Object Co-Creation in Virtual Reality

Julian Rasch, Julia Töws, Teresa Hirzle, Florian Müller, Martin Schmitz — Sun, 01 Jun 2025 00:00:00 +0000

Julian Rasch, Julia Töws, Teresa Hirzle, Florian Müller, Martin Schmitz
Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems - CHI '25

TL;DR: We conducted a Wizard-of-Oz study to investigate how different representation modes of an artificial intelligence influence user perception and interaction during 3D object co-creation in Virtual Reality.

Generative AI in Virtual Reality offers the potential for collaborative object-building, yet challenges remain in aligning AI contributions with user expectations. In particular, users often struggle to understand and collaborate with AI when its actions are not transparently represented. This paper thus explores the co-creative object-building process through a Wizard-of-Oz study, focusing on how AI can effectively convey its intent to users during object customization in Virtual Reality. Inspired by human-to-human collaboration, we focus on three representation modes: the presence of an embodied avatar, whether the AI’s contributions are visualized immediately or incrementally, and whether the areas modified are highlighted in advance. The findings provide insights into how these factors affect user perception and interaction with object-generating AI tools in Virtual Reality as well as satisfaction and ownership of the created objects. The results offer design implications for co-creative world-building systems, aiming to foster more effective and satisfying collaborations between humans and AI in Virtual Reality.

AR You on Track? Investigating Effects of Augmented Reality Anchoring on Dual-Task Performance While Walking

Julian Rasch, Matthias Wilhalm, Florian Müller, Francesco Chiossi — Sun, 01 Jun 2025 00:00:00 +0000

Julian Rasch, Matthias Wilhalm, Florian Müller, Francesco Chiossi
Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems - CHI '25

TL;DR: We investigated the effects of different Augmented Reality content anchoring strategies (hand, head, torso) on user performance in a dual-task scenario involving walking and a visual working memory task.

With the increasing spread of AR head-mounted displays suitable for everyday use, interaction with information becomes ubiquitous, even while walking. However, this requires constant shifts of our attention between walking and interacting with virtual information to fulfill both tasks adequately. Accordingly, we as a community need a thorough understanding of the mutual influences of walking and interacting with digital information to design safe yet effective interactions. Thus, we systematically investigate the effects of different AR anchors (hand, head, torso) and task difficulties on user experience and performance. We engage participants (n=26) in a dual-task paradigm involving a visual working memory task while walking. We assess the impact of dual-tasking on both virtual and walking performance, and subjective evaluations of mental and physical load. Our results show that head-anchored AR content least affected walking while allowing for fast and accurate virtual task interaction, while hand-anchored content increased reaction times and workload.

The Bitter Taste of Confidence: Exploring Audio-Visual Taste Modulation in Immersive Reality

Thu, 01 May 2025 00:00:00 +0000

Pooria Ghavamian, Jan Henri Beyer, Sophie Orth, Mia Johanna Nona Zech, Florian Müller, Andrii Matviienko
Proceedings of the 2025 ACM International Conference on Interactive Media Experiences

TL;DR: We conducted a study to explore how augmented reality visual filters and synchronized audio cues influence taste perception in Extended Reality environments.

Extended Reality (XR) technologies present innovative ways to augment sensory experiences, including taste perception. In this study, we investigated how augmented reality (AR) visual filters and synchronized audio cues affect gustation through a controlled experiment with 18 participants. Our findings revealed unexpected crossmodal interactions: while pink visual filter typically associated with sweetness reduced perceived bitterness in isolation, it paradoxically enhanced bitterness perception when combined with sweet-associated audio cue. Furthermore, we observed an inverse correlation between participant confidence levels and their perception of taste intensities across multiple dimensions, highlighting confidence as an overlooked factor in sensory experience design. These findings inform the design of nuanced multisensory experiences in immersive media, where subtle crossmodal interactions significantly influence user perception.

Assessing Visualization and Interaction Techniques to Support Comparison Tasks in Virtual Reality

Henrike Weingärtner, Julian Rasch, Nils Rothamel, Florian Müller — Tue, 01 Apr 2025 00:00:00 +0000

Henrike Weingärtner, Julian Rasch, Nils Rothamel, Florian Müller
Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems

TL;DR: We conducted a controlled experiment to assess various visualization and interaction techniques for comparison tasks in Virtual Reality environments.

Desktop screens are effective for supporting comparison tasks, but as the scale increases to room-sized or larger structures, context is lost. Users are forced to focus on isolated details through panning, zooming, and scrolling, making it difficult to maintain an overview while exploring finer details. Virtual Reality (VR) potentially offers a solution to this problem by immersing users in 3D spaces and enabling more intuitive comparisons. While related work has proposed many solutions for visualizing and interacting for comparison tasks in desktop environments, knowledge regarding the efficacy of supporting such tasks in VR environments is still lacking. We investigated varying visualization and interaction techniques in a controlled experiment with 24 participants. Our findings provide valuable insights for designing VR systems that improve usability, reduce workload, and enhance performance in comparison tasks.

A Vision for AI-Driven Adaptation of Dynamic AR Content to Users and Environments

Julian Rasch, Florian Müller, Francesco Chiossi — Tue, 01 Apr 2025 00:00:00 +0000

Julian Rasch, Florian Müller, Francesco Chiossi
arXiv

TL;DR: We designed an AI-driven system for dynamic Augmented Reality content placement that adapts to user movement and environmental changes.

Augmented Reality (AR) is transforming the way we interact with virtual information in the physical world. By overlaying digital content in real-world environments, AR enables new forms of immersive and engaging experiences. However, existing AR systems often struggle to effectively manage the many interactive possibilities that AR presents. This vision paper speculates on AI-driven approaches for adaptive AR content placement, dynamically adjusting to user movement and environmental changes. By leveraging machine learning methods, such a system would intelligently manage content distribution between AR projections integrated into the external environment and fixed static content, enabling seamless UI layout and potentially reducing users’ cognitive load. By exploring the possibilities of AI-driven dynamic AR content placement, we aim to envision new opportunities for innovation and improvement in various industries, from urban navigation and workplace productivity to immersive learning and beyond. This paper outlines a vision for the development of more intuitive, engaging, and effective AI-powered AR experiences.

One Does Not Simply Meme Alone: Evaluating Co-Creativity Between LLMs and Humans in the Generation of Humor

Zhikun Wu, Thomas Weber, Florian Müller — Sat, 01 Mar 2025 00:00:00 +0000

Zhikun Wu, Thomas Weber, Florian Müller
Proceedings of the 30th International Conference on Intelligent User Interfaces

TL;DR: We conducted a user study to explore the potential of Large Language Models in co-creating internet memes with human participants.

Collaboration has been shown to enhance creativity, leading to more innovative and effective outcomes. While previous research has explored the abilities of Large Language Models (LLMs) to serve as co-creative partners in tasks like writing poetry or creating narratives, the collaborative potential of LLMs in humor-rich and culturally nuanced domains remains an open question. To address this gap, we conducted a user study to explore the potential of LLMs in co-creating memes—a humor-driven and culturally specific form of creative expression. We conducted a user study with three groups of 50 participants each: a human-only group creating memes without AI assistance, a human-AI collaboration group interacting with a state-of-the-art LLM model, and an AI-only group where the LLM autonomously generated memes. We assessed the quality of the generated memes through crowdsourcing, with each meme rated on creativity, humor, and shareability. Our results showed that LLM assistance increased the number of ideas generated and reduced the effort participants felt. However, it did not improve the quality of the memes when humans were collaborated with LLM. Interestingly, memes created entirely by AI performed better than both human-only and human-AI collaborative memes in all areas on average. However, when looking at the top-performing memes, human-created ones were better in humor, while human-AI collaborations stood out in creativity and shareability. These findings highlight the complexities of human-AI collaboration in creative tasks. While AI can boost productivity and create content that appeals to a broad audience, human creativity remains crucial for content that connects on a deeper level.

Decoding Fatigue: Analyzing Offline Handwriting with Machine Learning to Detect Perceived Exhaustion

Sun, 01 Dec 2024 00:00:00 +0000

Dominik Schoen, Thomas Kosch, Till Becker, Godfred Antwi-Boasiako, Merret Jung, Ana Laura Chioca Vieira, Max Mühlhäuser, Florian Müller
Proceedings of the International Conference on Mobile and Ubiquitous Multimedia

TL;DR: We built a database of handwriting and drawing samples from 17 participants to investigate perceived exertion using machine learning techniques.

The quality and readability of an individual’s handwriting and drawing can be influenced by various factors, including their level of physical exertion. This enables us to explore the quantification of exertion by observing an individual’s handwriting. To test this hypothesis, we collected data from 17 participants, building a database of handwriting and drawing samples and their corresponding Borg 10 exertion ratings at the time of drawing. In this paper, we investigate using machine learning techniques to estimate perceived exertion before, during, and after physical activity based on handwriting and drawings. We apply a regression model to compare different drawing tasks and demonstrate that perceived exertion can be predicted using simple line drawings. However, more complex sketches and handwriting demand further research. Our findings suggest that interactive systems could use handwriting and drawing to intervene when users experience excessive discomfort.

Embracer: A Wearable Encountered-Type Haptic Controller for 3 DoF Input and Feedback

Tue, 01 Oct 2024 00:00:00 +0000

Dennis Dietz, Steeven Villa, Moritz Ziarko, Michael Bonfert, Florian Müller, Andreas Butz
Proceedings of the 2024 ACM International Symposium on Wearable Computers

TL;DR: We built Embracer, a wearable encountered-type haptic controller that allows for three degrees of freedom input and feedback.

The lack of haptic sensations beyond very simple vibration feedback diminishes the feeling of presence in Virtual Reality. Research suggested various approaches to deliver haptic sensations to the user’s palm. However, these approaches are typically limited in the number of actuation directions and only focus on enhancing the system’s output, ignoring haptic input. We present Embracer, a wrist-mounted encountered-type haptic controller that addresses these gaps by rendering forces along three axes through a sphere-shaped end effector within the user’s palm. Using modified servo motors, we sense user-performed manipulations of the end effector as an input modality. In this paper, we contribute the design and implementation of Embracer together with a preliminary technical evaluation. By providing a more comprehensive haptic feedback system, Embracer enhances the realism and immersion of haptic feedback and user control.

Exploring Redirection and Shifting Techniques to Mask Hand Movements from Shoulder-Surfing Attacks during PIN Authentication in Virtual Reality

Sun, 01 Sep 2024 00:00:00 +0000

Yannick Weiss, Steeven Villa, Jesse W Grootjen, Matthias Hoppe, Yasin Kale, Florian Müller
Proc. ACM Hum.-Comput. Interact.

TL;DR: We designed and evaluated several masking techniques to obscure hand movements during PIN authentication in Virtual Reality to prevent shoulder-surfing attacks.

The proliferation of mobile Virtual Reality (VR) headsets shifts our interaction with virtual worlds beyond our living rooms into shared spaces. Consequently, we are entrusting more and more personal data to these devices, calling for strong security measures and authentication. However, the standard authentication method of such devices - entering PINs via virtual keyboards - is vulnerable to shoulder-surfing, as movements to enter keys can be monitored by an unnoticed observer. To address this, we evaluated masking techniques to obscure VR users’ input during PIN authentication by diverting their hand movements. Through two experimental studies, we demonstrate that these methods increase users’ security against shoulder-surfing attacks from observers without excessively impacting their experience and performance. With these discoveries, we aim to enhance the security of future VR authentication without disrupting the virtual experience or necessitating additional hardware or training of users.

Touch It Like It's Hot: A Thermal Feedback Enabled Encountered-Type Haptic Display for Virtual Reality

Steeven Villa, Kenji Ishihara, Moritz Ziarko, Sebastian Günther, Florian Müller — Sat, 01 Jun 2024 00:00:00 +0000

Steeven Villa, Kenji Ishihara, Moritz Ziarko, Sebastian Günther, Florian Müller
IEEE International Symposium on Mixed and Augmented Reality (ISMAR'24) 2024

TL;DR: We designed and implemented a novel Encountered-type Haptic Display that incorporates thermal feedback to enhance user experiences in Virtual Reality.

In recent years, the community has presented various novel solutions to address the lack of haptic feedback in virtual reality experiences. Yet, it remains a major challenge for Virtual Reality applications. Encountered-type Haptic Displays (ETHDs) have emerged as a promising alternative to enable haptic feedback in VR without requiring the user to wear any device while allowing for sensorily rich experiences such as texture, kinaesthetic feedback, and even ultrasonic tactile feedback. Nevertheless, as important as thermal feedback is for daily life interactions, such as assessing the temperature of a mug or knowing if the microwave is on, thermal feedback in ETHD has remained largely unexplored. In this paper, we present a novel ETHD that provides thermal feedback and explore its potential in VR. We describe the design of our ETHD, and we report the results of a user study that compares different thermal feedback settings in VR. Our results show that thermal feedback can significantly enhance the user immersion and haptic experience in VR, and we discuss the implications of our findings for the design of ETHD and VR experiences.

Decide Yourself or Delegate - User Preferences Regarding the Autonomy of Personal Privacy Assistants in Private IoT-Equipped Environments

Wed, 01 May 2024 00:00:00 +0000

Karola Marky, Alina Stöver, Sarah Prange, Kira Bleck, Paul Gerber, Verena Zimmermann, Florian Müller, Florian Alt, Max Mühlhäuser
Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems

TL;DR: We conducted a series of studies to investigate user preferences regarding the autonomy of Personalized Privacy Assistants in Internet of Things-equipped environments, establishing a method for assigning users to privacy profiles based on their needs.

Personalized privacy assistants (PPAs) communicate privacy-related decisions of their users to Internet of Things (IoT) devices. There are different ways to implement PPAs by varying the degree of autonomy or decision model. This paper investigates user perceptions of PPA autonomy models and privacy profiles – archetypes of individual privacy needs – as a basis for PPA decisions in private environments (e.g., a friend’s home). We first explore how privacy profiles can be assigned to users and propose an assignment method. Next, we investigate user perceptions in 18 usage scenarios with varying contexts, data types and number of decisions in a study with 1126 participants. We found considerable differences between the profiles in settings with few decisions. If the number of decisions gets high (> 1/h), participants exclusively preferred fully autonomous PPAs. Finally, we discuss implications and recommendations for designing scalable PPAs that serve as privacy interfaces for future IoT devices.

“Tele” Me More: Using Telepresence Charades to Connect Strangers and Exhibits in Different Museums

Clara Sayffaerth, Julian Rasch, Florian Müller — Wed, 01 May 2024 00:00:00 +0000

Clara Sayffaerth, Julian Rasch, Florian Müller
Extended Abstracts of the CHI Conference on Human Factors in Computing Systems

TL;DR: We designed and tested a telepresence station that enables visitors in different museums to engage in Charades-like interactive exhibits using voiceless and touchless communication methods.

The museum is changing from a place of passive consumption to a place of interactive experiences, opening up new ways of engaging with exhibits and others. As a promising direction, this paper explores the potential of telepresence stations in the museum context to enhance social connectedness among visitors over distance. Emphasizing the significance of social exchange, our research focuses on studying telepresence to foster interactions between strangers, share knowledge, and promote social connectedness. To do so, we first observe exhibitions and then interview individual visitors of a technical museum about their experiences and needs. Based on the results, we design appropriate voiceless and touchless communication channels and test them in a study. The findings of our in-situ user study with 24 visitors unfamiliar with each other in the museum provide insights into behaviors and perceptions, contributing valuable knowledge on seamlessly integrating telepresence technology in exhibitions, with a focus on enhancing learning, social connections, and the museum experience in general.

Just Undo It: Exploring Undo Mechanics in Multi-User Virtual Reality

Julian Rasch, Florian Perzl, Yannick Weiss, Florian Müller — Fri, 01 Mar 2024 00:00:00 +0000

Julian Rasch, Florian Perzl, Yannick Weiss, Florian Müller
Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems

TL;DR: We conducted a study to explore the influence of different undo techniques for object manipulation in multi-user Virtual Reality in various collaboration modes.

With the proliferation of VR and a metaverse on the horizon, many multi-user activities are migrating to the VR world, calling for effective collaboration support. As one key feature, traditional collaborative systems provide users with undo mechanics to reverse errors and other unwanted changes. While undo has been extensively researched in this domain and is now considered industry standard, it is strikingly absent for VR systems in research and industry. This work addresses this research gap by exploring different undo techniques for basic object manipulation in different collaboration modes in VR. We conducted a study involving 32 participants organized in teams of two. Here, we studied users’ performance and preferences in a tower stacking task, varying the available undo techniques and their mode of collaboration. The results suggest that users desire and use undo in VR and that the choice of the undo technique impacts users’ performance and social connection.

ThermoFeet: Assessing On-Foot Thermal Stimuli for Directional Cues

Fri, 01 Dec 2023 00:00:00 +0000

Bettina Eska, Jeff-Owens Iyalekhue, Sebastian Günther, Jasmin Niess, Florian Müller
Proceedings of the 22nd International Conference on Mobile and Ubiquitous Multimedia

TL;DR: We built ThermoFeet, a system that provides thermal feedback on the feet to convey directional cues using four thermal actuators.

Thermal feedback has been studied for navigation purposes with directional cues and a variety of other use cases. Yet, to date, systems providing thermal feedback were primarily designed for the upper body, targeting hands and arms in particular. As these parts are often occupied with other tasks, there is a need to extend the design space of thermal feedback to other body parts. To close this gap, we assess thermal feedback on the user’s feet. This research explores if creating stimuli representing any direction on a circle with only four actuators is possible. To evaluate this concept, we conducted a user study asking the participants to indicate the perceived direction after getting a hot or cold stimulus by direct actuation using one actuator or phantom actuation using two actuators. The results indicate that the detection accuracy was higher for cold signals. In addition, the results showed higher recognition for stimuli linked to actuator distribution than phantom sensation due to spatial summation.

My Heart Will Go On: Implicitly Increasing Social Connectedness by Visualizing Asynchronous Players' Heartbeats in VR Games

Sun, 01 Oct 2023 00:00:00 +0000

Linda Hirsch, Florian Müller, Francesco Chiossi, Theodor Benga, Andreas Martin Butz
Proceedings of the ACM on Human-Computer Interaction

TL;DR: We designed a virtual escape room game that visualizes asynchronous players' heartbeats using various types of data visualizations.

Social games benefit from social connectedness between players because it improves the gaming experience and increases enjoyment. In virtual reality (VR), various approaches, such as avatars, are developed for multi-player games to increase social connectedness. However, these approaches are lacking in single-player games. To increase social connectedness in such games, our work explores the visualization of physiological data from asynchronous players, i.e., electrocardiogram (ECG). We identified two visualization dimensions, the number of players, and the visualization style, after a design workshop with experts (N=4) and explored them in a single-user virtual escape room game. We spatially and temporally integrated the visualizations and compared two times two visualizations against a baseline condition without visualization in a within-subject lab study (N=34). All but one visualization significantly increased participants’ feelings of social connectedness. Heart icons triggered the strongest feeling of connectedness, understanding, and perceived support in playing the game.

Location-Aware Virtual Reality for Situational Awareness On the Road

Jingyi Li, Alexandra Mayer, Florian Müller, Andrii Matviienko, Andreas Butz — Sun, 01 Oct 2023 00:00:00 +0000

Jingyi Li, Alexandra Mayer, Florian Müller, Andrii Matviienko, Andreas Butz
Proceedings of the 2023 ACM Symposium on Spatial User Interaction

TL;DR: We designed a location-aware Virtual Reality system that incorporates real-world cues during mobile interactions in vehicles, using two visualizations based on points of interest.

When future passengers are immersed in Virtual Reality (VR), the resulting disconnection from the physical world may degrade their situational awareness on the road. We propose incorporating real-world cues into virtual experiences when passing specific locations to address this. We designed two visualizations using points of interest (POIs), street names alone or combined with live street views. We compared them to two baselines, persistently displaying live cues (Always Live) or no cues (Always VR). In a field study (N=17), participants estimated their locations while exposed to VR entertainment during car rides. The results show that adding environmental cues inevitably degrades VR presence compared to Always VR. However, POI-triggered Text&Live preserves VR presence better than Always Live and attracts user attention to the road more than POI-triggered Text. We discuss situational awareness challenges for using mobile VR on the road and potential incorporation strategies across transport contexts.

DensingQueen: Exploration Methods for Spatial Dense Dynamic Data

Sun, 01 Oct 2023 00:00:00 +0000

Julius von Willich, Sebastian Günther, Andrii Matviienko, Martin Schmitz, Florian Müller, Max Mühlhäuser
Proceedings of the 2023 ACM Symposium on Spatial User Interaction

TL;DR: We designed and evaluated two interaction modalities, flashlight and 3D cutting planes, for exploring Spatial Dense Dynamic Data in Virtual Reality.

Research has proposed various interaction techniques to manage the occlusion of 3D data in Virtual Reality (VR), e.g., via gradual refinement. However, tracking dynamically moving data in a dense 3D environment poses the challenge of ever-changing occlusion, especially if motion carries relevant information, which is lost in still images. In this paper, we evaluated two interaction modalities for Spatial Dense Dynamic Data (SDDD), adapted from existing interaction methods for static and spatial data. We evaluated these modalities for exploring SDDD in VR, in an experiment with 18 participants. Furthermore, we investigated the influence of our interaction modalities on different levels of data density on the users’ performance in a no-knowledge task and a prior-knowledge task. Our results indicated significantly degraded performance for higher levels of density. Further, we found that our flashlight-inspired modality successfully improved tracking in SDDD, while a cutting plane-inspired approach was more suitable for highlighting static volumes of interest, particularly in such high-density environments.

Digital Modeling for Everyone: Exploring How Novices Approach Voice-Based 3D Modeling

Giuseppe Desolda, Andrea Esposito, Florian Müller, Sebastian Feger — Thu, 01 Jun 2023 00:00:00 +0000

Giuseppe Desolda, Andrea Esposito, Florian Müller, Sebastian Feger
Human-Computer Interaction – INTERACT 2023

TL;DR: We conducted a high-fidelity Wizard of Oz study with 22 novice participants to explore how they approach voice-based 3D modeling.

Manufacturing tools like 3D printers have become accessible to the wider society, making the promise of digital fabrication for everyone seemingly reachable. While the actual manufacturing process is largely automated today, users still require knowledge of complex design applications to produce ready-designed objects and adapt them to their needs or design new objects from scratch. To lower the barrier to the design and customization of personalized 3D models, we explored novice mental models in voice-based 3D modeling by conducting a high-fidelity Wizard of Oz study with 22 participants. We performed a thematic analysis of the collected data to understand how the mental model of novices translates into voice-based 3D modeling. We conclude with design implications for voice assistants. For example, they have to: deal with vague, incomplete and wrong commands; provide a set of straightforward commands to shape simple and composite objects; and offer different strategies to select 3D objects.

When XR and AI Meet - A Scoping Review on Extended Reality and Artificial Intelligence

Sat, 01 Apr 2023 00:00:00 +0000

Teresa Hirzle, Florian Müller, Fiona Draxler, Martin Schmitz, Pascal Knierim, Kasper Hornbæk
Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

TL;DR: We conducted a scoping review of 311 publications to explore the intersection of Extended Reality and Artificial Intelligence research published between 2017 and 2021.

Research on Extended Reality (XR) and Artificial Intelligence (AI) is booming, which has led to an emerging body of literature in their intersection. However, the main topics in this intersection are unclear, as are the benefits of combining XR and AI. This paper presents a scoping review that highlights how XR is applied in AI research and vice versa. We screened 2619 publications from 203 international venues published between 2017 and 2021, followed by an in-depth review of 311 papers. Based on our review, we identify five main topics at the intersection of XR and AI, showing how research at the intersection can benefit each other. Furthermore, we present a list of commonly used datasets, software, libraries, and models to help researchers interested in this intersection. Finally, we present 13 research opportunities and recommendations for future work in XR and AI research.

Using Pseudo-Stiffness to Enrich the Haptic Experience in Virtual Reality

Yannick Weiss, Steeven Villa, Albrecht Schmidt, Sven Mayer, Florian Müller — Sat, 01 Apr 2023 00:00:00 +0000

Yannick Weiss, Steeven Villa, Albrecht Schmidt, Sven Mayer, Florian Müller
Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

TL;DR: We designed a novel technique for altering the perceived stiffness of objects in Virtual Reality by manipulating the Control-to-Display ratio while users press down on a fixed stiffness object.

Providing users with a haptic sensation of the hardness and softness of objects in virtual reality is an open challenge. While physical props and haptic devices help, their haptic properties do not allow for dynamic adjustments. To overcome this limitation, we present a novel technique for changing the perceived stiffness of objects based on a visuo-haptic illusion. We achieved this by manipulating the hands’ Control-to-Display (C/D) ratio in virtual reality while pressing down on an object with fixed stiffness. In the first study (N=12), we determine the detection thresholds of the illusion. Our results show that we can exploit a C/D ratio from 0.7 to 3.5 without user detection. In the second study (N=12), we analyze the illusion’s impact on the perceived stiffness. Our results show that participants perceive the objects to be up to 28.1% softer and 8.9% stiffer, allowing for various haptic applications in virtual reality.

UndoPort: Exploring the Influence of Undo-Actions for Locomotion in Virtual Reality on the Efficiency, Spatial Understanding and User Experience

Florian Müller, Arantxa Ye, Dominik Schön, Julian Rasch — Sat, 01 Apr 2023 00:00:00 +0000

Florian Müller, Arantxa Ye, Dominik Schön, Julian Rasch
Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

TL;DR: We built UndoPort, an extension of the point-and-teleport locomotion technique that incorporates undo actions for reverting position and orientation changes in Virtual Reality.

When we get lost in Virtual Reality (VR) or want to return to a previous location, we use the same methods of locomotion for the way back as for the way forward. This is time-consuming and requires additional physical orientation changes, increasing the risk of getting tangled in the headsets’ cables. In this paper, we propose the use of undo actions to revert locomotion steps in VR. We explore eight different variations of undo actions as extensions of point&teleport, based on the possibility to undo position and orientation changes together with two different visualizations of the undo step (discrete and continuous). We contribute the results of a controlled experiment with 24 participants investigating the efficiency and orientation of the undo techniques in a radial maze task. We found that the combination of position and orientation undo together with a discrete visualization resulted in the highest efficiency without increasing orientation errors.

TicTacToes: Assessing Toe Movements as an Input Modality

Sat, 01 Apr 2023 00:00:00 +0000

Florian Müller, Daniel Schmitt, Andrii Matviienko, Dominik Schön, Sebastian Günther, Thomas Kosch, Martin Schmitz
Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

TL;DR: We conducted a controlled experiment to assess the feasibility of using toe movements as an input modality for computer systems, focusing on the influence of five factors on interaction performance.

From carrying grocery bags to holding onto handles on the bus, there are a variety of situations where one or both hands are busy, hindering the vision of ubiquitous interaction with technology. Voice commands, as a popular hands-free alternative, struggle with ambient noise and privacy issues. As an alternative approach, research explored movements of various body parts (e.g., head, arms) as input modalities, with foot-based techniques proving particularly suitable for hands-free interaction. Whereas previous research only considered the movement of the foot as a whole, in this work, we argue that our toes offer further degrees of freedom that can be leveraged for interaction. To explore the viability of toe-based interaction, we contribute the results of a controlled experiment with 18 participants assessing the impact of five factors on the accuracy, efficiency and user experience of such interfaces. Based on the findings, we provide design recommendations for future toe-based interfaces.

Tailor Twist: Assessing Rotational Mid-Air Interactions for Augmented Reality

Sat, 01 Apr 2023 00:00:00 +0000

Dominik Schön, Thomas Kosch, Florian Müller, Martin Schmitz, Sebastian Günther, Lukas Bommhardt, Max Mühlhäuser
Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

TL;DR: We conducted a systematic evaluation of the ergonomics of rotational mid-air interactions in Augmented Reality environments through a controlled experiment with 19 participants.

Mid-air gestures, widely used in today’s Augmented Reality (AR) applications, are prone to the “gorilla arm” effect, leading to discomfort with prolonged interactions. While prior work has proposed metrics to quantify this effect and means to improve comfort and ergonomics, these works usually only consider simplistic, one-dimensional AR interactions, like reaching for a point or pushing a button. However, interacting with AR environments also involves far more complex tasks, such as rotational knobs, potentially impacting ergonomics. This paper advances the understanding of the ergonomics of rotational mid-air interactions in AR. For this, we contribute the results of a controlled experiment exposing the participants to a rotational task in the interaction space defined by their arms’ reach. Based on the results, we discuss how novel future mid-air gesture modalities benefit from our findings concerning ergonomic-aware rotational interaction.

In Sync: Exploring Synchronization to Increase Trust Between Humans and Non-Humanoid Robots

Sat, 01 Apr 2023 00:00:00 +0000

Wieslaw Bartkowski, Andrzej Nowak, Filip Ignacy Czajkowski, Albrecht Schmidt, Florian Müller
Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

TL;DR: We built a prototype of a simple non-humanoid robot that tracks and synchronizes with the upper body movements of humans to explore the impact of synchronization on trust.

When we go for a walk with friends, we can observe an interesting effect: From step lengths to arm movements - our movements unconsciously align; they synchronize. Prior research found that this synchronization is a crucial aspect of human relations that strengthens social cohesion and trust. Generalizing from these findings in synchronization theory, we propose a dynamical approach that can be applied in the design of non-humanoid robots to increase trust. We contribute the results of a controlled experiment with 51 participants exploring our concept in a between-subjects design. For this, we built a prototype of a simple non-humanoid robot that can bend to follow human movements and vary the movement synchronization patterns. We found that synchronized movements lead to significantly higher ratings in an established questionnaire on trust between people and automation but did not influence the willingness to spend money in a trust game.

Going, Going, Gone: Exploring Intention Communication for Multi-User Locomotion in Virtual Reality

Julian Rasch, Vladislav Dmitrievic Rusakov, Martin Schmitz, Florian Müller — Sat, 01 Apr 2023 00:00:00 +0000

Julian Rasch, Vladislav Dmitrievic Rusakov, Martin Schmitz, Florian Müller
Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

TL;DR: We designed and implemented three intention communication methods for multi-user locomotion in Virtual Reality, comparing them with two baseline methods in a controlled experiment.

Exploring virtual worlds together with others adds a social component to the Virtual Reality (VR) experience that increases connectedness. In the physical world, joint locomotion comes naturally through implicit intention communication and subsequent adjustments of the movement patterns. In VR, however, discrete locomotion techniques such as point&teleport come without prior intention communication, hampering the collective experience. Related work proposes fixed groups, with a single person controlling the group movement, resulting in the loss of individual movement capabilities. To close the gap and mediate between these two extremes, we introduce three intention communication methods and explore them with two baseline methods. We contribute the results of a controlled experiment (n=20) investigating these methods from the perspective of a leader and a follower in a dyadic locomotion task. Our results suggest shared visualizations support the understanding of movement intentions, increasing the group feeling while maintaining individual freedom of movement.

Understanding Stationary and Moving Direct Skin Vibrotactile Stimulation on the Palm

Wed, 01 Feb 2023 00:00:00 +0000

Hesham Elsayed, Martin Weigel, Florian Müller, George Ibrahim, Jan Gugenheimer, Martin Schmitz, Sebastian Günther, Max Mühlhäuser
arXiv

TL;DR: We conducted two user studies to investigate the effects of stationary and moving direct skin vibrotactile stimulation on the palm.

Palm-based tactile displays have the potential to evolve from single motor interfaces (e.g., smartphones) to high-resolution tactile displays (e.g., back-of-device haptic interfaces) enabling richer multi-modal experiences with more information. However, we lack a systematic understanding of vibrotactile perception on the palm and the influence of various factors on the core design decisions of tactile displays (number of actuators, resolution, and intensity). In a first experiment (N=16), we investigated the effect of these factors on the users’ ability to localize stationary sensations. In a second experiment (N=20), we explored the influence of resolution on recognition rate for moving tactile sensations.Findings show that for stationary sensations a 9 actuator display offers a good trade-off and a 3×3 resolution can be accurately localized. For moving sensations, a 2×4 resolution led to the highest recognition accuracy, while 5×10 enables higher resolution output with a reasonable accuracy.

Hands-On 3D Printed Electronics

Julian Rasch, Florian Müller, Thomas Kosch, Martin Schmitz, Sebastian S. Feger — Wed, 01 Feb 2023 00:00:00 +0000

Julian Rasch, Florian Müller, Thomas Kosch, Martin Schmitz, Sebastian S. Feger
Proceedings of the Seventeenth International Conference on Tangible, Embedded, and Embodied Interaction

TL;DR: We conducted a hands-on workshop focused on best practices and design recommendations for 3D printed electronics.

The parallel improvements in multi-material 3D printers and the quality of conductive filament open new possibilities for the fabrication of tangible and functional objects. In this studio, we discuss best practices for 3D printed electronics, talk about encountered problems, and derive design recommendations. We will guide the participants through a fabrication process by practically designing and printing objects. Consequently, we contemplate individual functional fabricated components, including small printed circuits and multi-material prints. We aim to spark a discussion about individually experienced challenges participants encountered during their design and fabrication process. This discussion includes problemsolving strategies, whose insights benefit other participants. Finally, we show the potential of printed electronics and discuss encouraging new opportunities in this field.

Rethinking Smart Objects: The International Workshop on Interacting with Smart Objects in Interactive Spaces

Tue, 01 Nov 2022 00:00:00 +0000

Martin Schmitz, Sebastian Günther, Karola Marky, Florian Müller, Andrii Matviienko, Alexandra Voit, Roberts Marky, Max Mühlhäuser, Thomas Kosch
Companion Proceedings of the 2022 Conference on Interactive Surfaces and Spaces

TL;DR: We conducted the Rethinking Smart Objects workshop, focusing on the interaction paradigms and societal implications of smart objects in interactive spaces.

The increasing proliferation of smart objects in everyday life has changed how we interact with computers. Instead of concentrating computational capabilities and interaction into one device, everyday objects have naturally integrated parts of interactive features. Although this has led to many practical applications, the possibilities for explicit or implicit interaction with such objects are still limited in interaction spaces. We still often rely on smartphones as interactive hubs for controlling smart objects, hence not fulfilling the vision of truly smart objects. The workshop Rethinking Smart Objects invites practitioners and researchers from both academia and industry to discuss novel interaction paradigms and the integration and societal implications of using smart objects in interactive space. This workshop will include an action plan with leading questions, aiming to move the research field forward.

TrackItPipe: A Fabrication Pipeline To Incorporate Location and Rotation Tracking Into 3D Printed Objects

Sat, 01 Oct 2022 00:00:00 +0000

Dominik Schön, Thomas Kosch, Martin Schmitz, Florian Müller, Sebastian Günther, Johannes Kreutz, Max Mühlhäuser
Adjunct Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology

TL;DR: We built TrackItPipe, a fabrication pipeline that allows users to semi-automatically integrate location and rotation tracking capabilities into 3D printed objects.

The increasing convergence of the digital and physical world creates a growing urgency to integrate 3D printed physical tangibles with virtual environments. A precise position and rotation tracking are essential to integrate such physical objects with a virtual environment. However, available 3D models commonly do not provide tracking support on their composition, which requires modifications by CAD experts. This poses a challenge for users with no prior CAD experience. This work presents TrackItPipe, a fabrication pipeline supporting users by semi-automatically adding tracking capabilities for 3D printable tangibles tailored to environmental requirements. TrackItPipe integrates modifications to the 3D model, produces the respective tangibles for 3D printing, and provides integration scripts for Mixed Reality. Using TrackItPipe, users can rapidly equip objects with tracking capabilities.

NotiBike: Assessing Target Selection Techniques for Cyclist Notifications in Augmented Reality

Thu, 01 Sep 2022 00:00:00 +0000

Thomas Kosch, Andrii Matviienko, Florian Müller, Jessica Bersch, Christopher Katins, Dominik Schön, Max Mühlhäuser
Proceedings of the ACM on Human-Computer Interaction

TL;DR: We conducted a study to evaluate three notification selection techniques—Dwell Time, Gestures, and MAGIC Pointing—specifically for cyclists using Augmented Reality in a simulated biking scenario.

Cyclists’ attention is often compromised when interacting with notifications in traffic, hence increasing the likelihood of road accidents. To address this issue, we evaluate three notification interaction modalities and investigate their impact on the interaction performance while cycling: gaze-based Dwell Time, Gestures, and Manual And Gaze Input Cascaded (MAGIC) Pointing. In a user study (N=18), participants confirmed notifications in Augmented Reality (AR) using the three interaction modalities in a simulated biking scenario. We assessed the efficiency regarding reaction times, error rates, and perceived task load. Our results show significantly faster response times for MAGIC Pointing compared to Dwell Time and Gestures, while Dwell Time led to a significantly lower error rate compared to Gestures. Participants favored the MAGIC Pointing approach, supporting cyclists in AR selection tasks. Our research sets the boundaries for more comfortable and easier interaction with notifications and discusses implications for target selections in AR while cycling.

MuC '22: Proceedings of Mensch Und Computer 2022

Thu, 01 Sep 2022 00:00:00 +0000

Max Mühlhauser, Christian Reuter, Bastian Pfleging, Thomas Kosch, Andrii Matviienko, Kathrin Gerling, Sven Mayer, Wilko Heuten, Tanja Döring, Florian Müller, Martin Schmitz
Association for Computing Machinery

TL;DR: We hosted the Mensch und Computer 2022 conference at the Technische Universität Darmstadt, gathering nearly 600 participants from academia and industry.

Die Mensch und Computer 2022 fand vom 4.-7. September 2022 an der Technischen Universität Darmstadt statt. Die 2001 ins Leben gerufene „Mensch und Computer“ (MuC) ist die größte Tagungsreihe der Mensch-Computer-Interaktion in Europa. Wir sind dankbar, dass wir im Jahr des 50jährigen Bestehens des Fachbereichs Informatik der TU Darmstadt fast 600 Teilnehmende aus Wissenschaft und Industrie live, in Farbe, physisch, also real in Darmstadt begrüßen konnten – passend zu unserem Konferenzmotto: „Facing Realities“. Träger der Konferenz waren der Fachbereich Mensch-Computer-Interaktion der Gesellschaft für Informatik (GI e.V.) und die German UPA – der Berufsverband der Deutschen Usability und User Experience Professionals. Die Technische Universität Darmstadt und insbesondere der Fachbereich Informatik fungierte als Gastgeber.

Baby, You Can Ride My Bike: Exploring Maneuver Indications of Self-Driving Bicycles Using a Tandem Simulator

Andrii Matviienko, Damir Mehmedovic, Florian Müller, Max Mühlhäuser — Thu, 01 Sep 2022 00:00:00 +0000

Andrii Matviienko, Damir Mehmedovic, Florian Müller, Max Mühlhäuser
Proceedings of the ACM on Human-Computer Interaction

TL;DR: We designed and evaluated four types of maneuver indications for self-driving bicycles using a tandem simulator to create a realistic cycling experience without control by the rider.

We envision a future where self-driving bicycles can take us to our destinations. This allows cyclists to use their time on the bike efficiently for work or relaxation without having to focus their attention on traffic. In the related field of self-driving cars, research has shown that communicating the planned route to passengers plays an important role in building trust in automation and situational awareness. For self-driving bicycles, this information transfer will be even more important, as riders will need to actively compensate for the movement of a self-driving bicycle to maintain balance. In this paper, we investigate maneuver indications for self-driving bicycles: (1) ambient light in a helmet, (2) head-up display indications, (3) speech feedback, (4) vibration on the handlebar, and (5) no assistance. To evaluate these indications, we conducted an outdoor experiment (N = 25) in a proposed tandem simulator consisting of a tandem bicycle with a steering and braking control on the back seat and a rider in full control of it. Our results indicate that riders respond faster to visual cues and focus comparably on the reading task while riding with and without maneuver indications. Additionally, we found that the tandem simulator is realistic, safe, and creates an awareness of a human cyclist controlling the tandem.

Proper Posture: Designing Posture Feedback Across Musical Instruments

Bettina Eska, Jasmin Niess, Florian Müller — Sun, 01 May 2022 00:00:00 +0000

Bettina Eska, Jasmin Niess, Florian Müller
arXiv

TL;DR: We designed a haptic feedback system to support musicians in maintaining proper hand posture while playing musical instruments, using vibrotactile and thermal cues.

There is a recommended body posture and hand position for playing every musical instrument, allowing efficient and quick movements without blockage. Due to humans’ limited cognitive capabilities, they struggle to concentrate on several things simultaneously and thus sometimes lose the correct position while playing their instrument. Incorrect positions when playing an instrument can lead to injuries and movement disorders in the long run. Previous work in HCI mainly focused on developing systems to assist in learning an instrument. However, the design space for posture correction when playing a musical instrument has not yet been explored. In this position paper, we present our vision of providing subtle vibrotactile or thermal feedback to guide the focus of attention back to the correct posture when playing a musical instrument. We discuss our concept with a focus on motion recognition and feedback modalities. Finally, we outline the next steps for future research.

Squeezy-Feely: Investigating Lateral Thumb-Index Pinching as an Input Modality

Martin Schmitz, Sebastian Günther, Dominik Schön, Florian Müller — Fri, 01 Apr 2022 00:00:00 +0000

Martin Schmitz, Sebastian Günther, Dominik Schön, Florian Müller
Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

TL;DR: We conducted a systematic evaluation of lateral thumb-index pinching as an input modality through a controlled experiment comparing various grip spans, counterforces, and directions in a visual targeting task.

From zooming on smartphones and mid-air gestures to deformable user interfaces, thumb-index pinching grips are used in many interaction techniques. However, there is still a lack of systematic understanding of how the accuracy and efficiency of such grips are affected by various factors such as counterforce, grip span, and grip direction. Therefore, in this paper, we contribute an evaluation (N = 18) of thumb-index pinching performance in a visual targeting task using scales up to 75 items. As part of our findings, we conclude that the pinching interaction between the thumb and index finger is a promising modality also for one-dimensional input on higher scales. Furthermore, we discuss and outline implications for future user interfaces that benefit from pinching as an additional and complementary interaction modality.

Smooth as Steel Wool: Effects of Visual Stimuli on the Haptic Perception of Roughness in Virtual Reality

Fri, 01 Apr 2022 00:00:00 +0000

Sebastian Günther, Julian Rasch, Dominik Schön, Florian Müller, Martin Schmitz, Jan Riemann, Andrii Matviienko, Max Mühlhäuser
Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

TL;DR: We conducted an experiment assessing how visualizations affect the haptic perception of roughness in Virtual Reality, using a prototype designed for arm stroking.

Haptic Feedback is essential for lifelike Virtual Reality (VR) experiences. To provide a wide range of matching sensations of being touched or stroked, current approaches typically need large numbers of different physical textures. However, even advanced devices can only accommodate a limited number of textures to remain wearable. Therefore, a better understanding is necessary of how expectations elicited by different visualizations affect haptic perception, to achieve a balance between physical constraints and great variety of matching physical textures. In this work, we conducted an experiment (N=31) assessing how the perception of roughness is affected within VR. We designed a prototype for arm stroking and compared the effects of different visualizations on the perception of physical textures with distinct roughnesses. Additionally, we used the visualizations’ real-world materials, no-haptics and vibrotactile feedback as baselines. As one result, we found that two levels of roughness can be sufficient to convey a realistic illusion.

SkyPort: Investigating 3D Teleportation Methods in Virtual Environments

Fri, 01 Apr 2022 00:00:00 +0000

Andrii Matviienko, Florian Müller, Martin Schmitz, Marco Fendrich, Max Mühlhäuser
CHI Conference on Human Factors in Computing Systems

TL;DR: We designed six new 3D teleportation methods in virtual environments by combining linear and parabolic aiming with different transitioning types.

Teleportation has become the de facto standard of locomotion in Virtual Reality (VR) environments. However, teleportation with parabolic and linear target aiming methods is restricted to horizontal 2D planes and it is unknown how they transfer to the 3D space. In this paper, we propose six 3D teleportation methods in virtual environments based on the combination of two existing aiming methods (linear and parabolic) and three types of transitioning to a target (instant, interpolated and continuous). To investigate the performance of the proposed teleportation methods, we conducted a controlled lab experiment (N = 24) with a mid-air coin collection task to assess accuracy, efficiency and VR sickness. We discovered that the linear aiming method leads to faster and more accurate target selection. Moreover, a combination of linear aiming and instant transitioning leads to the highest efficiency and accuracy without increasing VR sickness.

Reducing Virtual Reality Sickness for Cyclists in VR Bicycle Simulators

Fri, 01 Apr 2022 00:00:00 +0000

Andrii Matviienko, Florian Müller, Marcel Zickler, Lisa Alina Gasche, Julia Abels, Till Steinert, Max Mühlhäuser
Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

TL;DR: We conducted two controlled lab experiments to evaluate steering methods and countermeasures aimed at reducing virtual reality sickness in bicycle simulators.

Virtual Reality (VR) bicycle simulations aim to recreate the feeling of riding a bicycle and are commonly used in many application areas. However, current solutions still create mismatches between the visuals and physical movement, which causes VR sickness and diminishes the cycling experience. To reduce VR sickness in bicycle simulators, we conducted two controlled lab experiments addressing two main causes of VR sickness: (1) steering methods and (2) cycling trajectory. In the first experiment (N = 18) we compared handlebar, HMD, and upper-body steering methods. In the second experiment (N = 24) we explored three types of movement in VR (1D, 2D, and 3D trajectories) and three countermeasures (airflow, vibration, and dynamic Field-of-View) to reduce VR sickness. We found that handlebar steering leads to the lowest VR sickness without decreasing cycling performance and airflow suggests to be the most promising method to reduce VR sickness for all three types of trajectories.

E-ScootAR: Exploring Unimodal Warnings for E-Scooter Riders in Augmented Reality

Fri, 01 Apr 2022 00:00:00 +0000

Andrii Matviienko, Florian Müller, Dominik Schön, Régis Fayard, Salar Abaspur, Yi Li, Max Mühlhäuser
Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems

TL;DR: We conducted an outdoor experiment comparing three unimodal warnings for E-Scooter riders using Augmented Reality notifications, auditory signals, and vibrotactile feedback in a simulated environment.

Micro-mobility is becoming a more popular means of transportation. However, this increased popularity brings its challenges. In particular, the accident rates for E-Scooter riders increase, which endangers the riders and other road users. In this paper, we explore the idea of augmenting E-Scooters with unimodal warnings to prevent collisions with other road users, which include Augmented Reality (AR) notifications, vibrotactile feedback on the handlebar, and auditory signals in the AR glasses. We conducted an outdoor experiment (N = 13) using an Augmented Reality simulation and compared these types of warnings in terms of reaction time, accident rate, and feeling of safety. Our results indicate that AR and auditory warnings lead to shorter reaction times, have a better perception, and create a better feeling of safety than vibrotactile warnings. Moreover, auditory signals have a higher acceptance by the riders compared to the other two types of warnings.

BikeAR: Understanding Cyclists' Crossing Decision-Making at Uncontrolled Intersections Using Augmented Reality

Fri, 01 Apr 2022 00:00:00 +0000

Andrii Matviienko, Florian Müller, Dominik Schön, Paul Seesemann, Sebastian Günther, Max Mühlhäuser
Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

TL;DR: We designed two Augmented Reality visualizations, X-ray and Countdown, to assist cyclists in making crossing decisions at uncontrolled intersections.

Cycling has become increasingly popular as a means of transportation. However, cyclists remain a highly vulnerable group of road users. According to accident reports, one of the most dangerous situations for cyclists are uncontrolled intersections, where cars approach from both directions. To address this issue and assist cyclists in crossing decision-making at uncontrolled intersections, we designed two visualizations that: (1) highlight occluded cars through an X-ray vision and (2) depict the remaining time the intersection is safe to cross via a Countdown. To investigate the efficiency of these visualizations, we proposed an Augmented Reality simulation as a novel evaluation method, in which the above visualizations are represented as AR, and conducted a controlled experiment with 24 participants indoors. We found that the X-ray ensures a fast selection of shorter gaps between cars, while the Countdown facilitates a feeling of safety and provides a better intersection overview.

CameraReady: Assessing the Influence of Display Types and Visualizations on Posture Guidance

Tue, 01 Jun 2021 00:00:00 +0000

Hesham Elsayed, Philipp Hoffmann, Sebastian Günther, Martin Schmitz, Martin Weigel, Max Mühlhäuser, Florian Müller
Designing Interactive Systems Conference 2021

TL;DR: We built CameraReady, a mobile and cross-platform posture guidance system that evaluates the influence of different display types and visualizations on user performance.

Computer-supported posture guidance is used in sports, dance training, expression of art with movements, and learning gestures for interaction. At present, the influence of display types and visualizations have not been investigated in the literature. These factors are important as they directly impact perception and cognitive load, and hence influence the performance of participants. In this paper, we conducted a controlled experiment with 20 participants to compare the use of five display types with different screen sizes: smartphones, tablets, desktop monitors, TVs, and large displays. On each device, we compared three common visualizations for posture guidance: skeletons, silhouettes, and 3d body models. To conduct our assessment, we developed a mobile and cross-platform system that only requires a single camera. Our results show that compared to a smartphone display, larger displays show a lower error (12%). Regarding the choice of visualization, participants rated 3D body models as significantly more usable in comparison to a skeleton visualization.

ActuBoard: An Open Rapid Prototyping Platform to Integrate Hardware Actuators in Remote Applications

Tue, 01 Jun 2021 00:00:00 +0000

Sebastian Günther, Florian Müller, Felix Hübner, Max Mühlhäuser, Andrii Matviienko
Companion of the 2021 ACM SIGCHI Symposium on Engineering Interactive Computing Systems

TL;DR: We built ActuBoard, a prototyping platform designed to minimize initial setup and facilitate the integration of hardware actuators in remote applications.

Prototyping is an essential step in developing tangible experiences and novel devices, ranging from haptic feedback to wearables. However, prototyping of actuated devices nowadays often requires repetitive and time-consuming steps, such as wiring, soldering, and programming basic communication, before HCI researchers and designers can focus on their primary interest: designing interaction. In this paper, we present ActuBoard, a prototyping platform to support 1) quick assembly, 2) less preparation work, and 3) the inclusion of non-tech-savvy users. With ActuBoard, users are not required to create complex circuitry, write a single line of firmware, or implementing communication protocols. Acknowledging existing systems, our platform combines the flexibility of low-level microcontrollers and ease-of-use of abstracted tinker platforms to control actuators from separate applications. As further contribution, we highlight the technical specifications and published the ActuBoard platform as Open Source.

VRtangibles: Assisting Children in Creating Virtual Scenes Using Tangible Objects and Touch Input

Andrii Matviienko, Marcel Langer, Florian Müller, Martin Schmitz, Max Mühlhäuser — Sat, 01 May 2021 00:00:00 +0000

Andrii Matviienko, Marcel Langer, Florian Müller, Martin Schmitz, Max Mühlhäuser
Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems

TL;DR: We built VRtangibles, a system that combines tangible objects and touch input to assist children in creating virtual scenes without needing programming knowledge.

Children are increasingly exposed to virtual reality (VR) technology as end-users. However, they miss an opportunity to become active creators due to the barrier of insufficient technical background. Creating scenes in VR requires considerable programming knowledge and excludes non-tech-savvy users, e.g., school children. In this paper, we showcase a system called VRtangibles, which combines tangible objects and touch input to create virtual scenes without programming. With VRtangibles, we aim to engage children in the active creation of virtual scenes via playful hands-on activities. From the lab study with six school children, we discovered that the majority of children were successful in creating virtual scenes using VRtangibles and found it engaging and fun to use.

Oh, Snap! A Fabrication Pipeline to Magnetically Connect Conventional and 3D-Printed Electronics

Martin Schmitz, Jan Riemann, Florian Müller, Steffen Kreis, Max Mühlhäuser — Sat, 01 May 2021 00:00:00 +0000

Martin Schmitz, Jan Riemann, Florian Müller, Steffen Kreis, Max Mühlhäuser
Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

TL;DR: We built the Oh, Snap! fabrication pipeline, which magnetically connects 3D-printed objects equipped with passive sensing structures to conventional electronics.

3D printing has revolutionized rapid prototyping by speeding up the creation of custom-shaped objects. With the rise of multi-material 3D printers, these custom-shaped objects can now be made interactive in a single pass through passive conductive structures. However, connecting conventional electronics to these conductive structures often still requires time-consuming manual assembly involving many wires, soldering or gluing. To alleviate these shortcomings, we propose : a fabrication pipeline and interfacing concept to magnetically connect a 3D-printed object equipped with passive sensing structures to conventional sensing electronics. To this end, utilizes ferromagnetic and conductive 3D-printed structures, printable in a single pass on standard printers. We further present a proof-of-concept capacitive sensing board that enables easy and robust magnetic assembly to quickly create interactive 3D-printed objects. We evaluate by assessing the robustness and quality of the connection and demonstrate its broad applicability by a series of example applications.

Let's Frets! Mastering Guitar Playing with Capacitive Sensing and Visual Guidance

Sat, 01 May 2021 00:00:00 +0000

Karola Marky, Andreas Weiß, Florian Müller, Martin Schmitz, Max Mühlhäuser, Thomas Kosch
Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems

TL;DR: We built Let’s Frets, a modular guitar support system that integrates capacitive sensing and visual guidance through an augmented fretboard and a mobile application.

Mastering the guitar requires regular exercise to develop new skills and maintain existing abilities. We present Let’s Frets - a modular guitar support system that provides visual guidance through LEDs that are integrated into a capacitive fretboard to support the practice of chords, scales, melodies, and exercises. Additional feedback is provided through a 3D-printed fretboard that senses the finger positions through capacitive sensing. We envision Let’s Frets as an integrated guitar support system that raises the awareness of guitarists about their playing styles, their training progress, the composition of new pieces, and facilitating remote collaborations between teachers as well as guitar students. This interactivity demonstrates Let’s Frets with an augmented fretboard and supporting software that runs on a mobile device.

Let's Frets! Assisting Guitar Students During Practice via Capacitive Sensing

Sat, 01 May 2021 00:00:00 +0000

Karola Marky, Andreas Weiß, Andrii Matviienko, Florian Brandherm, Sebastian Wolf, Martin Schmitz, Florian Krell, Florian Müller, Max Mühlhäuser, Thomas Kosch
Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

TL;DR: We built Let’s Frets, a modular guitar learning system that incorporates visual indicators and capacitive sensing to assist guitar students during practice.

Learning a musical instrument requires regular exercise. However, students are often on their own during their practice sessions due to the limited time with their teachers, which increases the likelihood of mislearning playing techniques. To address this issue, we present Let’s Frets - a modular guitar learning system that provides visual indicators and capturing of finger positions on a 3D-printed capacitive guitar fretboard. We based the design of Let’s Frets on requirements collected through in-depth interviews with professional guitarists and teachers. In a user study (N=24), we evaluated the feedback modules of Let’s Frets against fretboard charts. Our results show that visual indicators require the least time to realize new finger positions while a combination of visual indicators and position capturing yielded the highest playing accuracy. We conclude how Let’s Frets enables independent practice sessions that can be translated to other musical instruments.

Itsy-Bits: Fabrication and Recognition of 3D-Printed Tangibles with Small Footprints on Capacitive Touchscreens

Martin Schmitz, Florian Müller, Max Mühlhäuser, Jan Riemann, Huy Viet Viet Le — Sat, 01 May 2021 00:00:00 +0000

Martin Schmitz, Florian Müller, Max Mühlhäuser, Jan Riemann, Huy Viet Viet Le
Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

TL;DR: We built Itsy-Bits, a fabrication pipeline for 3D printing and recognizing tangibles on capacitive touchscreens with a footprint as small as a fingertip.

Tangibles on capacitive touchscreens are a promising approach to overcome the limited expressiveness of touch input. While research has suggested many approaches to detect tangibles, the corresponding tangibles are either costly or have a considerable minimal size. This makes them bulky and unattractive for many applications. At the same time, they obscure valuable display space for interaction. To address these shortcomings, we contribute Itsy-Bits: a fabrication pipeline for 3D printing and recognition of tangibles on capacitive touchscreens with a footprint as small as a fingertip. Each Itsy-Bit consists of an enclosing 3D object and a unique conductive 2D shape on its bottom. Using only raw data of commodity capacitive touchscreens, Itsy-Bits reliably identifies and locates a variety of shapes in different sizes and estimates their orientation. Through example applications and a technical evaluation, we demonstrate the feasibility and applicability of Itsy-Bits for tangibles with small footprints.

VibroMap: Understanding the Spacing of Vibrotactile Actuators across the Body

Tue, 01 Dec 2020 00:00:00 +0000

Hesham Elsayed, Martin Weigel, Florian Müller, Martin Schmitz, Karola Marky, Sebastian Günther, Jan Riemann, Max Mühlhäuser
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

TL;DR: We conducted two systematic experiments to investigate the spacing of vibrotactile actuators across different body parts to determine minimum and maximum effective distances for vibrotactile displays.

In spite of the great potential of on-body vibrotactile displays for a variety of applications, research lacks an understanding of the spacing between vibrotactile actuators. Through two experiments, we systematically investigate vibrotactile perception on the wrist, forearm, upper arm, back, torso, thigh, and leg, each in transverse and longitudinal body orientation. In the first experiment, we address the maximum distance between vibration motors that still preserves the ability to generate phantom sensations. In the second experiment, we investigate the perceptual accuracy of localizing vibrations in order to establish the minimum distance between vibration motors. Based on the results, we derive VibroMap, a spatial map of the functional range of inter-motor distances across the body. VibroMap supports hardware and interaction designers with design guidelines for constructing body-worn vibrotactile displays.

Exploring User Expectations of Proactive AI Systems

Tue, 01 Dec 2020 00:00:00 +0000

Christian Meurisch, Cristina A. Mihale-Wilson, Adrian Hawlitschek, Florian Giger, Florian Müller, Oliver Hinz, Max Mühlhäuser
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

TL;DR: We conducted an in-the-wild study to explore user expectations of proactive artificial intelligence systems across various use cases and proactivity levels with 272 participants using our mobile application ProfileMe.

Recent advances in artificial intelligence (AI) enabled digital assistants to evolve towards proactive user support. However, expectations as to when and to what extent assistants should take the initiative are still unclear; discrepancies to the actual system behavior might negatively affect user acceptance. In this paper, we present an in-the-wild study for exploring user expectations of such user-supporting AI systems in terms of different proactivity levels and use cases. We collected 3,168 in-situ responses from 272 participants through a mixed method of automated user tracking and context-triggered surveying. Using a data-driven approach, we gain insights into initial expectations and how they depend on different human factors and contexts. Our insights can help to design AI systems with varying degree of proactivity and preset to meet individual expectations.

VRSketchPen: Unconstrained Haptic Assistance for Sketching in Virtual 3D Environments

Sun, 01 Nov 2020 00:00:00 +0000

Hesham Elsayed, Mayra Donaji Barrera Machuca, Christian Schaarschmidt, Karola Marky, Florian Müller, Jan Riemann, Andrii Matviienko, Martin Schmitz, Martin Weigel, Max Mühlhäuser
26th ACM Symposium on Virtual Reality Software and Technology

TL;DR: We built VRSketchPen, a pen that uses pneumatic and vibrotactile feedback to enhance sketching accuracy in virtual three-dimensional environments without constraining user actions.

Accurate sketching in virtual 3D environments is challenging due to aspects like limited depth perception or the absence of physical support. To address this issue, we propose VRSketchPen - a pen that uses two haptic modalities to support virtual sketching without constraining user actions: (1) pneumatic force feedback to simulate the contact pressure of the pen against virtual surfaces and (2) vibrotactile feedback to mimic textures while moving the pen over virtual surfaces. To evaluate VRSketchPen, we conducted a lab experiment with 20 participants to compare (1) pneumatic, (2) vibrotactile and (3) a combination of both with (4) snapping and no assistance for flat and curved surfaces in a 3D virtual environment. Our findings show that usage of pneumatic, vibrotactile and their combination significantly improves 2D shape accuracy and leads to diminished depth errors for flat and curved surfaces. Qualitative results indicate that users find the addition of unconstraining haptic feedback to significantly improve convenience, confidence and user experience.

Therminator : Understanding the Interdependency of Visual and On-Body Thermal Feedback in Virtual Reality

Mon, 01 Jun 2020 00:00:00 +0000

Sebastian Günther, Omar Elmoghazy, Florian Müller, Max Mühlhäuser, Dominik Schön, Martin Schmitz
Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems - CHI '20

TL;DR: We built Therminator, a thermal display concept that provides localized thermal feedback using flowing liquids through a network of tubes in Virtual Reality.

Recent advances have made Virtual Reality (VR) more realistic than ever before. This improved realism is attributed to today’s ability to increasingly appeal to human sensations, such as visual, auditory or tactile. While research also examines temperature sensation as an important aspect, the interdependency of visual and thermal perception in VR is still underexplored. In this paper, we propose Therminator, a thermal display concept that provides warm and cold on-body feedback in VR through heat conduction of flowing liquids with different temperatures. Further, we systematically evaluate the interdependency of different visual and thermal stimuli on the temperature perception of arm and abdomen with 25 participants. As part of the results, we found varying temperature perception depending on the stimuli, as well as increasing involvement of users during conditions with matching stimuli.

Podoportation: Foot-Based Locomotion in Virtual Reality

Mon, 01 Jun 2020 00:00:00 +0000

Julius Von Willich, Martin Schmitz, Florian Müller, Daniel Schmitt, Max Mühlhäuser
Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems - CHI '20

TL;DR: We built and evaluated nine novel locomotion concepts for foot-based teleportation in Virtual Reality, using the 3D position and pressure of the user's feet as input modalities.

Virtual Reality (VR) allows for infinitely large environments. However, the physical traversable space is always limited by real-world boundaries. This discrepancy between physical and virtual dimensions renders traditional locomotion methods used in real world unfeasible. To alleviate these limitations, research proposed various artificial locomotion concepts such as teleportation, treadmills, and redirected walking. However, these concepts occupy the user’s hands, require complex hardware or large physical spaces. In this paper, we contribute nine VR locomotion concepts for foot-based locomotion, relying on the 3D position of the user’s feet and the pressure applied to the sole as input modalities. We evaluate our concepts and compare them to state-of-the-art point & teleport technique in a controlled experiment with 20 participants. The results confirm the viability of our approaches for foot-based and engaging locomotion. Further, based on the findings, we contribute a wireless hardware prototype implementation.

PneumoVolley: Pressure-Based Haptic Feedback on the Head through Pneumatic Actuation

Sebastian Günther, Dominik Schön, Florian Müller, Max Mühlhäuser, Martin Schmitz — Mon, 01 Jun 2020 00:00:00 +0000

Sebastian Günther, Dominik Schön, Florian Müller, Max Mühlhäuser, Martin Schmitz
Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems - CHI '20

TL;DR: We built PneumoVolley, a Virtual Reality experience that utilizes pneumatic actuation to provide pressure-based haptic feedback on the head during gameplay.

Haptic Feedback brings immersion and presence in Virtual Reality (VR) to the next level. While research proposes the usage of various tactile sensations, such as vibration or ultrasound approaches, the potential applicability of pressure feedback on the head is still under-explored. In this paper, we contribute concepts and design considerations for pressure-based feedback on the head through pneumatic actuation. As a proof-of-concept implementing our pressure-based haptics, we further present PneumoVolley: a VR experience similar to the classic Volleyball game but played with the head. In an exploratory user study with 9 participants, we evaluated our concepts and identified a significantly increased involvement compared to a no-haptics baseline along with high realism and enjoyment ratings using pressure-based feedback on the head in VR. LBW033, Page 1 CHI 2020 Late-Breaking Work

Around-Body Interaction: Leveraging Limb-Movements for Interacting in a Digitally Augmented Physical World

Florian Müller — Mon, 01 Jun 2020 00:00:00 +0000

Florian Müller
TUprints

TL;DR: We designed and developed four interaction techniques that leverage both upper and lower limb movements for interacting with Head-Mounted Displays in a digitally augmented environment.

Recent technological advances have made head-mounted displays (HMDs) smaller and untethered, fostering the vision of ubiquitous interaction with information in a digitally augmented physical world. For interacting with such devices, three main types of input - besides not very intuitive finger gestures - have emerged so far: 1) Touch input on the frame of the devices or 2) on accessories (controller) as well as 3) voice input. While these techniques have both advantages and disadvantages depending on the current situation of the user, they largely ignore the skills and dexterity that we show when interacting with the real world: Throughout our lives, we have trained extensively to use our limbs to interact with and manipulate the physical world around us. This thesis explores how the skills and dexterity of our upper and lower limbs, acquired and trained in interacting with the real world, can be transferred to the interaction with HMDs. Thereby, this thesis develops the vision of around-body interaction, in which we use the space around our body, defined by the reach of our limbs, for fast, accurate, and enjoyable interaction with information.

Around-Body Interaction: Über Die Nutzung Der Bewegungen von Gliedmaßen Zur Interaktion in Einer Digital Erweiterten Physischen Welt

Florian Müller — Fri, 01 May 2020 00:00:00 +0000

Florian Müller
Ausgezeichnete Informatikdissertationen 2019

TL;DR: We designed interaction techniques that leverage limb movements for augmented reality applications using head-mounted displays.

Durch den technischen Fortschritt sind Head-Mounted Displays (HMDs) kleiner und kabellos geworden. Sie leisten so einen Beitrag zur Vision von allgegenwärtiger Interaktion mit Informationen in einer digital erweiterten physischen Welt. Zur Interaktion mit solchen Geräten werden heute eingabeseitig - neben wenig intuitiven Fingergesten in der Luft – vor allem dreierlei Techniken verwendet: 1) Toucheingabe auf dem Gehäuse der Geräte oder 2) auf Zubehör (Controller) sowie 3) Spracheingabe. Während diese Techniken, abhängig von der aktuellen Situation des Benutzers, sowohl Vor- als auch Nachteile haben, so ignorieren sie weitgehend die Fähigkeiten und Geschicklichkeit, die wir im Umgang mit der realen Welt zeigen: Während unseres ganzen Lebens haben wir ausgiebig trainiert unsere Gliedmaßen zu benutzen, um mit der physischen Welt um uns herum zu interagieren und sie zu manipulieren. Diese Arbeit entwickelt eine Vision für eine körperlichere Interaktion mit solchen Geräten, welche die Fähigkeiten und Geschicklichkeit, die wir im Umgang mit der physischen Welt zeigen, auf die Interaktion mit HMDs überträgt.

Walk The Line: Leveraging Lateral Shifts of the Walking Path as an Input Modality for Head-Mounted Displays

Wed, 01 Apr 2020 00:00:00 +0000

Florian Müller, Martin Schmitz, Daniel Schmitt, Sebastian Günther, Markus Funk, Max Mühlhäuser
Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

TL;DR: We designed a hands-free input modality for Head-Mounted Displays by leveraging lateral shifts of the walking path, visualizing options as lanes on the ground.

Recent technological advances have made head-mounted displays (HMDs) smaller and untethered, fostering the vision of ubiquitous interaction in a digitally augmented physical world. Consequently, a major part of the interaction with such devices will happen on the go, calling for interaction techniques that allow users to interact while walking. In this paper, we explore lateral shifts of the walking path as a hands-free input modality. The available input options are visualized as lanes on the ground parallel to the user’s walking path. Users can select options by shifting the walking path sideways to the respective lane. We contribute the results of a controlled experiment with 18 participants, confirming the viability of our approach for fast, accurate, and joyful interactions. Further, based on the findings of the controlled experiment, we present three example applications.

Around-Body Interaction: Interacting While on the Go

Florian Müller, Sebastian Günther, Max Mühlhauser — Wed, 01 Apr 2020 00:00:00 +0000

Florian Müller, Sebastian Günther, Max Mühlhauser
IEEE Pervasive Computing

TL;DR: We designed a conceptual framework for around-body interaction techniques using the natural skills and dexterity of our limbs.

We provide a concise overview on how the skills and dexterity of our upper and lower limbs, acquired and trained in interacting with the physical world, can be transferred to the interaction with HMD. We present the vision of around-body interaction, in which we use the space around our body, defined by the reach of our limbs, for fast, accurate, and enjoyable interactions. In the remainder of this article, we first introduce a design space for such around-body interactions and, second, present two examples of such interaction techniques, which use the degrees of freedom of the lower limbs for on-the go interactions. Finally, we conclude with future research challenges.

Teachyverse: Collaborative E-Learning in Virtual Reality Lecture Halls

Sun, 01 Sep 2019 00:00:00 +0000

Karola Marky, Florian Müller, Markus Funk, Alexander Geiß, Sebastian Günther, Martin Schmitz, Jan Riemann, Max Mühlhäuser
Proceedings of Mensch Und Computer 2019

TL;DR: We designed and implemented Teachyverse, an immersive Virtual Reality lecture hall that blends e-learning, traditional learning, and remote collaboration.

Over the last decades, E-learning has gained a lot of popularity and enabled students to learn in front of their computers using Internet-based learning systems rather than physically attending lectures. Those E-learning systems are different from traditional learning and do not fully immerse the student in the learning environment. Thus, we propose Teachyverse, an immersive VR lecture hall that combines e-learning, traditional learning, and remote collaboration. Teachyverse immerses the student in a virtual lecture hall. A proof-of-concept study shows that students perceive lectures in Teachyverse as fun and would like to use Teachyverse as a further E-Learning option.

You Invaded My Tracking Space! Using Augmented Virtuality for Spotting Passersby in Room-Scale Virtual Reality

Sat, 01 Jun 2019 00:00:00 +0000

Julius von Willich, Markus Funk, Florian Müller, Karola Marky, Jan Riemann, Max Mühlhäuser
Proceedings of the 2019 on Designing Interactive Systems Conference - DIS '19

TL;DR: We designed and evaluated three different visualizations to represent physical passersby in a Virtual Environment using Augmented Virtuality.

With the proliferation of room-scale Virtual Reality (VR), more and more users install a VR system in their homes. When users are in VR, they are usually completely immersed in their application. However, sometimes passersby invade these tracking spaces and walk up to users that are currently immersed in VR to try and interact with them. As this either scares the user in VR or breaks the user’s immersion, research has yet to find a way to seamlessly represent physical passersby in virtual worlds. In this paper, we propose and evaluate three different ways to represent physical passersby in a Virtual Environment using Augmented Virtuality. The representations encompass showing a Pointcloud, showing a 3D-Model, and showing an Image Overlay of the passerby. Our results show that while an Image Overlay and a 3D-Model are the fastest representations to spot passersby, the 3D-Model and the Pointcloud representations were the most accurate.

VRChairRacer: Using an Office Chair Backrest as a Locomotion Technique for VR Racing Games

Sat, 01 Jun 2019 00:00:00 +0000

Julius von Willich, Dominik Schön, Sebastian Günther, Florian Müller, Max Mühlhäuser, Markus Funk
Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems - CHI '19

TL;DR: We built VRChairRacer, a novel locomotion technique that uses the backrest of an office chair to control a virtual racing cart in a multiplayer racing game.

Locomotion in Virtual Reality (VR) is an important topic as there is a mismatch between the size of a Virtual Environment and the physically available tracking space. Although many locomotion techniques have been proposed, research on VR locomotion has not concluded yet. In this demonstration, we contribute to the area of VR locomotion by introducing VRChairRacer. VRChairRacer introduces a novel mapping the velocity of a racing cart on the backrest of an office chair. Further, it maps a users’ rotation onto the steering of a virtual racing cart. VRChairRacer demonstrates this locomotion technique to the community through an immersive multiplayer racing demo.

Trends on Engineering Interactive Systems: An Overview of Works Presented in Workshops at EICS 2019

Sat, 01 Jun 2019 00:00:00 +0000

Damiano Distante, Alexandra Voit, Marco Winckler, Regina Bernhaupt, Judy Bowen, José Creissac Campos, Florian Müller, Philippe Palanque, Jan Van den Bergh, Benjamin Weyers
Proceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems - EICS '19

TL;DR: We conducted workshops at the 11th Engineering Interactive Computing Systems conference to discuss research and trends in the engineering of interactive systems.

Workshops are a great opportunity for identifying innovative topics of research that might require discussion and maturation. This paper summarizes the outcomes of the workshops track of the 11th Engineering Interactive Computing Systems conference (EICS 2019), held in Valencia (Spain) on 18-21 June 2019. The track featured three workshops, one half-day, one full-day and one two-days workshop, each focused on specific topics of the ongoing research in engineering usable and effective interactive computing systems. In particular, the list of discussed topics include novel forms of interaction and emerging themes in HCI related to new application domains, more efficient and enjoyable interaction possibilities associated to smart objects and smart environments, challenges faced in designing, developing and using interactive systems involving multiple stakeholders.

Slappyfications: Towards Ubiquitous Physical and Embodied Notifications

Sebastian Günther, Florian Müller, Markus Funk, Max Mühlhäuser — Sat, 01 Jun 2019 00:00:00 +0000

Sebastian Günther, Florian Müller, Markus Funk, Max Mühlhäuser
Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems - CHI '19

TL;DR: We built a palm-based interface for a novel notification system called Slappyfications that enables users to send three types of unmissable notifications: poke, slap, and the STEAM-HAMMER.

With emerging trends of notifying persons through ubiquitous technologies [2], such as ambient light, vibrotactile, or auditory cues, none of these technologies are truly ubiquitous and have proven to be easily missed or ignored. In this work, we propose Slappyfications, a novel way of sending unmissable embodied and ubiquitous notifications through a palm-based interface [1]. Our prototype enables the users to send three types of Slappyfications: poke, slap, and the STEAM-HAMMER. Through a Wizard-of-Oz study, we show the applicability of our system in real-world scenarios. The results reveal a promising trend, as none of the participants missed a single Slappyfication.

Proceedings of the 7th Workshop on Interacting with Smart Objects

Sat, 01 Jun 2019 00:00:00 +0000

TUprints

TL;DR: We hosted the 7th Workshop on Interacting with Smart Objects in conjunction with EICS'19.

These are the proceedings of the 7th Workshop on Interacting with Smart Objects (SmartObjects ‘19) in conjunction with EICS'19 held on June 18, 2019, in Valencia, Spain. This volume contains the ten accepted papers. Each submission was reviewed by three program committee members. Objects that we use in our everyday life are ever-expanding their interaction capabilities and provide functionalities that go far beyond their original functionality. They feature computing capabilities and are, thus, able to capture, process and store information and interact with their environments, turning them into smart objects. Their wide range was covered by the submissions to this workshop. Smart objects know something about their users and, thus, allow for natural interaction. Natural interaction, in contrast, does not imply smartness. Smartness requires interaction with users and provides help. There are already commercialized products available that expose their properties and interaction capabilities. To enrich their potential and to lower affordances, they need to communicate to each other. Making sense out of the available data in this field is still an open research question. The overall goal should be to build an interactive ecosystem that (i) seamlessly discovers, connects and talks to its environment, (ii) is ubiquitous and (iii) allows the user to be in control. The workshop examined these issues with regards to the following aspects: Smart Devices Smart Spaces Putting together SmartObjects `19 was a team effort. We would like to send out our thanks to everybody who has helped us to organize this event: The authors, who have written and submitted their papers to the workshop. The program committee and the external reviewers, for their time and effort to write substantial and constructive review reports. We hope that you will find this program interesting and thought-provoking and that the workshop will provide you with a valuable opportunity to share ideas with other researchers and practitioners from institutions around the world.

PneumAct: Pneumatic Kinesthetic Actuation of Body Joints in Virtual Reality Environments

Sat, 01 Jun 2019 00:00:00 +0000

Sebastian Günther, Mohit Makhija, Florian Müller, Dominik Schön, Max Mühlhäuser, Markus Funk
Proceedings of the 2019 on Designing Interactive Systems Conference

TL;DR: We built PneumAct, a pneumatic jacket that enables kinesthetic movements of body joints in Virtual Reality environments by using two types of actuators: Contraction Actuators to flex joints and Extension Actuators to extend them.

Virtual Reality Environments (VRE) create an immersive user experience through visual, aural, and haptic sensations. However, the latter is often limited to vibrotactile sensations that are not able to actively provide kinesthetic motion actuation. Further, such sensations do not cover natural representations of physical forces, for example, when lifting a weight. We present PneumAct, a jacket to enable pneumatically actuated kinesthetic movements of arm joints in VRE. It integrates two types of actuators inflated through compressed air: a Contraction Actuator and an Extension Actuator. We evaluate our PneumAct jacket through two user studies with a total of 32 participants: First, we perform a technical evaluation measuring the contraction and extension angles of different inflation patterns and inflation durations. Second, we evaluate PneumAct in three VRE scenarios comparing our system to traditional controller-based vibrotactile and a baseline without haptic feedback.

Mind the Tap: Assessing Foot-Taps for Interacting with Head-Mounted Displays

Sat, 01 Jun 2019 00:00:00 +0000

Florian Müller, Joshua McManus, Sebastian Günther, Martin Schmitz, Max Mühlhäuser, Markus Funk
Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI '19

TL;DR: We conducted two controlled experiments to explore foot-tapping as an input modality for interacting with Head-Mounted Displays.

From voice commands and air taps to touch gestures on frames: Various techniques for interacting with head-mounted displays (HMDs) have been proposed. While these techniques have both benefits and drawbacks dependent on the current situation of the user, research on interacting with HMDs has not concluded yet. In this paper, we add to the body of research on interacting with HMDs by exploring foot-tapping as an input modality. Through two controlled experiments with a total of 36 participants, we first explore direct interaction with interfaces that are displayed on the floor and require the user to look down to interact. Secondly, we investigate indirect interaction with interfaces that, although operated by the user’s feet, are always visible as they are floating in front of the user. Based on the results of the two experiments, we provide design recommendations for direct and indirect foot-based user interfaces.

./Trilaterate: A Fabrication Pipeline to Design and 3D Print Hover-, Touch-, and Force-Sensitive Objects

Martin Schmitz, Martin Stitz, Florian Müller, Markus Funk, Max Mühlhäuser — Sat, 01 Jun 2019 00:00:00 +0000

Martin Schmitz, Martin Stitz, Florian Müller, Markus Funk, Max Mühlhäuser
Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI '19

TL;DR: We built Trilaterate, a fabrication pipeline that enables users to 3D print custom objects capable of detecting hover, touch, and force interactions through embedded capacitive sensors.

Hover, touch, and force are promising input modalities that get increasingly integrated into screens and everyday objects. However, these interactions are often limited to fat surfaces and the integration of suitable sensors is time-consuming and costly. To alleviate these limitations, we contribute Tri-laterate: A fabrication pipeline to 3D print custom objects that detect the 3D position of a fnger hovering, touching, or forcing them by combining multiple capacitance measurements via capacitive trilateration. Trilaterate places and routes actively-shielded sensors inside the object and operates on consumer-level 3D printers. We present technical evaluations and example applications that validate and demonstrate the wide applicability of Trilaterate. CCS CONCEPTS Human-centered computing Interaction devices; Hardware Tactile and hand-based interfaces;

Assessing the Accuracy of Point & Teleport Locomotion with Orientation Indication for Virtual Reality Using Curved Trajectories

Wed, 01 May 2019 00:00:00 +0000

Markus Funk, Florian Müller, Marco Fendrich, Megan Shene, Moritz Kolvenbach, Niclas Dobbertin, Sebastian Günther, Max Mühlhäuser
Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

TL;DR: We introduced and evaluated three point & teleport locomotion techniques for Virtual Reality that allow users to specify the target orientation in addition to the target position.

Room-scale Virtual Reality (VR) systems have arrived in users’ homes where tracked environments are set up in limited physical spaces. As most Virtual Environments (VEs) are larger than the tracked physical space, locomotion techniques are used to navigate in VEs. Currently, in recent VR games, point & teleport is the most popular locomotion technique. However, it only allows users to select the position of the teleportation and not the orientation that the user is facing after the teleport. This results in users having to manually correct their orientation after teleporting and possibly getting entangled by the cable of the headset. In this paper, we introduce and evaluate three different point & teleport techniques that enable users to specify the target orientation while teleporting. The results show that, although the three teleportation techniques with orientation indication increase the average teleportation time, they lead to a decreased need for correcting the orientation after teleportation.

UPA'18: 3rd International Workshop on Ubiquitous Personal Assistance

Fri, 01 Jun 2018 00:00:00 +0000

Christian Meurisch, Max Mühlhäuser, Philipp M. Scholl, Usman Naeem, Veljko Pejović, Florian Müller, Elena Di Lascio, Pei-Yi Patricia Kuo, Sebastian Kauschke, Muhammad Awais Azam
Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers - UbiComp '18

TL;DR: We conducted the Third International Workshop on Ubiquitous Personal Assistance to discuss the latest research outcomes and interdisciplinary approaches in the development and design of digital personal assistants.

Advancements in ubiquitous technologies and artificial intelligence have paved the way for the recent rise of digital personal assistants in everyday life. The Third International Workshop on Ubiquitous Personal Assistance (UPA'18) aims to build on the success of our both previous workshops (namely SmartGuidance), organized in conjunction with UbiComp'16/17, to continue discussing the latest research outcomes of digital personal assistants. We invite the submission of papers within this emerging, interdisciplinary research field of ubiquitous personal assistance that focuses on understanding, design, and development of such digital helpers. We also welcome contributions that investigate human behaviors, underlying recognition, and prediction models; conduct field studies; as well as propose novel HCI techniques to provide personal assistance. All workshop contributions will be published in the supplemental proceedings of the UbiComp'18 conference and included in the ACM Digital Library.

TactileGlove: Assistive Spatial Guidance in 3D Space through Vibrotactile Navigation

Fri, 01 Jun 2018 00:00:00 +0000

Sebastian Günther, Florian Müller, Markus Funk, Jan Kirchner, Niloofar Dezfuli, Max Mühlhäuser
Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference

TL;DR: We built TactileGlove, a smart glove with vibrotactile actuators that provides spatial guidance in three-dimensional space.

With the recent advance in computing technology, more and more environments are becoming interactive. For interacting with these environments, traditionally 2D input and output elements are being used. However, recently interaction spaces also expanded to 3D space, which enabled new possibilities but also led to challenges in assisting users with interacting in such a 3D space. Usually, this challenge of communicating 3D positions is solved visually. This paper explores a different approach: spatial guidance through vibrotactile instructions. Therefore, we introduce TactileGlove, a smart glove equipped with vibrotactile actuators for providing spatial guidance in 3D space. We contribute a user study with 15 participants to explore how a different number of actuators and metaphors affect the user performance. As a result, we found that using a Pull metaphor for vibrotactile navigation instructions is preferred by our participants. Further, we found that using a higher number of actuators reduces the target acquisition time than when using a low number.

SmartObjects: Sixth Workshop on Interacting with Smart Objects

Fri, 01 Jun 2018 00:00:00 +0000

Florian Müller, Dirk Schnelle-Walka, Tobias Grosse-Puppendahl, Sebastian Günther, Markus Funk, Kris Luyten, Oliver Brdiczka, Niloofar Dezfuli, Max Mühlhäuser
Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems - CHI '18

TL;DR: We conducted a workshop focused on how to design interactions with smart objects to enhance user experience.

Smart objects are everyday objects that have computing capabilities and give rise to new ways of interaction with our environment. The increasing number of smart objects in our life shapes how we interact beyond the desktop. In this workshop we explore various aspects of the design, development and deployment of smart objects including how one can interact with smart objects.

Proceedings of the 6th Workshop on Interacting with Smart Objects (SmartObjects)

Fri, 01 Jun 2018 00:00:00 +0000

6th Workshop on Interacting with Smart Objects (SmartObjects)

TL;DR: We conducted the 6th Workshop on Interacting with Smart Objects to explore the interactive capabilities and functionalities of smart objects in various contexts.

These are the proceedings of the 6th Workshop on Interacting with Smart Objects (SmartObjects ‘18) in conjunction with CHI'18 held on April 21, 2018, in Montreal, Canada. This volume contains the ten accepted papers. Each submission was reviewed by three program committee members. Objects that we use in our everyday life are ever-expanding their interaction capabilities and provide functionalities that go far beyond their original functionality. They feature computing capabilities and are, thus, able to capture, process and store information and interact with their environments, turning them into smart objects. Their wide range was covered by the submissions to this workshop. Smart objects know something about their users and, thus, allow for natural interaction. Natural interaction, in contrast, does not imply smartness. Smartness requires interaction with users and provides help. There are already commercialized products available that expose their properties and interaction capabilities. To enrich their potential and to lower affordances, they need to communicate to each other. Making sense out of the available data in this field is still an open research question. The overall goal should be to build an interactive ecosystem that (i) seamlessly discovers, connects and talks to its environment, (ii) is ubiquitous and (iii) allows the user to be in control. The workshop examined these issues with regards to the following aspects: Interactive Experiences The Future of IoT Smart Home AR in the Industry Putting together SmartObjects `18 was a team effort. We would like to send out our thanks to everybody who has helped us to organize this event: The authors, who have written and submitted their papers to the workshop. The program committee and the external reviewers, for their time and effort to write substantial and constructive review reports. We hope that you will find this program interesting and thought-provoking and that the workshop will provide you with a valuable opportunity to share ideas with other researchers and practitioners from institutions around the world.

Personalized User-Carried Single Button Interfaces as Shortcuts for Interacting with Smart Devices

Fri, 01 Jun 2018 00:00:00 +0000

Florian Müller, Martin Schmitz, Markus Funk, Sebastian Günther, Niloofar Dezfuli, Max Mühlhäuser
Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems - CHI '18

TL;DR: We built a proof-of-concept prototype of a personalized user-carried single button interface for interacting with a smart coffee machine.

We are experiencing a trend of integrating computing functionality into more and more common and popular devices. While these so-called smart devices offer many possibilities for automation and personalization of everyday routines, interacting with them and customizing them requires either programming efforts or a smartphone app to control the devices. In this work, we propose and classify Personalized User-Carried Single Button Interfaces as shortcuts for interacting with smart devices. We implement a proof-of-concept of such an interface for a coffee machine. Through an in-the-wild deployment of the coffee machine for approximately three months, we report first initial experiences from 40 participants of using PUCSBIs for interacting with smart devices.

CaMea: Camera-Supported Workpiece Measurement for CNC Milling Machines

Florian Müller, Maximilian Barnikol, Markus Funk, Martin Schmitz, Max Mühlhäuser — Fri, 01 Jun 2018 00:00:00 +0000

Florian Müller, Maximilian Barnikol, Markus Funk, Martin Schmitz, Max Mühlhäuser
Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference on - PETRA '18

TL;DR: We designed and implemented CaMea, a graphical user interface that utilizes a camera to support users in measuring workpieces inside a computer numerically controlled milling machine.

We are experiencing a trend of personal fabrication that allows non-experts to produce highly individualized objects. Beyond 3D printing, this maker movement also approaches larger-scale production machines such as computer numerically controlled (CNC) milling machines that are available in local fabrication laboratories (FabLabs). While the user interfaces and interaction techniques of small-scale 3D printers for household use adapted to the new requirements of non-experts in the last years, such an overhaul of the interfaces for larger machinery is still missing. In this work, we explore the use of augmented reality methods to support novice users in the operation of CNC milling machines. As a first step towards better support for users, we provide a camera-supported graphical and easy-to-use interface for the measurement of raw workpieces inside the machine. In this paper, we contribute our concept CaMea alongside its’ prototype implementation. We further report on the findings of a first early user study.

An Analysis of Language Impact on Augmented Reality Order Picking Training

Fri, 01 Jun 2018 00:00:00 +0000

Nela Murauer, Florian Müller, Sebastian Günther, Dominik Schön, Nerina Pflanz, Markus Funk
Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference

TL;DR: We conducted a user study comparing three order picking assistance systems: textual instructions in German, textual instructions in Finnish, and an Augmented Reality system using a Microsoft HoloLens.

Order picking is a difficult and cognitively demanding task. Traditionally textual instructions are helping new workers to learn different picking routines. However, the textual instructions are sometimes not written in the workers ’ native languages. In the area of Industry 4.0, where digital functions are finding their way into manufacturing processes, language-independent instructions are possible. Through a user study with 15 participants, we compare textual feedback in the workers’ native language, textual feedback that is written in an unknown foreign language, and visual Augmented Reality (AR) feedback. We found that AR feedback is significantly faster and leads to a lower perceived cognitive load.

CheckMate: Exploring a Tangible Augmented Reality Interface for Remote Interaction

Sun, 01 Apr 2018 00:00:00 +0000

Sebastian Günther, Florian Müller, Martin Schmitz, Jan Riemann, Niloofar Dezfuli, Markus Funk, Dominik Schön, Max Mühlhäuser
Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems

TL;DR: We built CheckMate, a tangible Augmented Reality interface that integrates 3D printed objects and touch-enabled surfaces for remote interaction.

The digitalized world comes with increasing Internet capabilities, allowing to connect persons over distance easier than ever before. Video conferencing and similar online applications create great benefits bringing people who physically cannot spend as much time as they want virtually together. However, such remote experiences can also tend to lose the feeling of traditional experiences. People lack direct visual presence and no haptic feedback is available. In this paper, we tackle this problem by introducing our system called CheckMate. We combine Augmented Reality and capacitive 3D printed objects that can be sensed on an interactive surface to enable remote interaction while providing the same tangible experience as in co-located scenarios. As a proof-of-concept, we implemented a sample application based on the traditional chess game.

Cloudbits: Supporting Conversations through Augmented Zero-Query Search Visualization

Sun, 01 Oct 2017 00:00:00 +0000

Florian Müller, Sebastian Günther, Azita Hosseini Nejad, Niloofar Dezfuli, Mohammadreza Khalilbeigi, Max Mühlhäuser
Proceedings of the 5th Symposium on Spatial User Interaction

TL;DR: We built CloudBits, an augmented reality system for head-mounted displays that visualizes and retrieves information proactively during conversations.

The retrieval of additional information from public (e.g., map data) or private (e.g., e-mail) information sources using personal smart devices is a common habit in today’s co-located conversations. This behavior of users imposes challenges in two main areas: 1) cognitive focus switching and 2) information sharing. In this paper, we explore a novel approach for conversation support through augmented information bits, allowing users to see and access information right in front of their eyes. To that end, we investigate the requirements for the design of a user interface to support conversations through proactive information retrieval in an exploratory study. Based on the results, we 2) present CloudBits: A set of visualization and interaction techniques to provide mutual awareness and enhance coupling in conversations through augmented zero-query search visualization along with its prototype implementation. Finally, we 3) report the findings of a qualitative evaluation and conclude with guidelines for the design of user interfaces for conversation support.

Manufacturing Method of Sensor Using 3d Printing and 3d Printer Thereof

Thu, 01 Jun 2017 00:00:00 +0000

Woo Sug Jung, Hwa Suk Kim, Jun Ki Jeon, Seong Kyoun Jo, Hyun Woo Lee, Martin Schmitz, Florian Müller, Andreas Leister, Jan Riemann, Niloofar Dezfuli, Max Mühlhäuser, Mohammadreza Khalilbeigi
TU Darmstadt

TL;DR: We developed a manufacturing method for a sensor using three-dimensional printing techniques.

Disclosed is a manufacturing method of a sensor by using 3D printing and 3D printer therefor. According to an embodiment of the present disclosure , a manufacturing method of a sensor by using 3D printing includes : forming a first shape having an inner space by using a non-conduc tive material , and simultaneously or sequentially , forming an electrode at a preset location in the inner space by using a conductive material ; injecting conductive liquid into the inner space ; and forming a second shape on the first shape by using the non-conductive material to seal the inner space of the first shape .

SmartObjects: Fifth Workshop on Interacting with Smart Objects

Wed, 01 Mar 2017 00:00:00 +0000

Dirk Schnelle-Walka, Florian Müller, Tobias Grosse-Puppendahl, Kris Luyten, Max Mühlhäuser, Oliver Brdiczka
Proceedings of the 22nd International Conference on Intelligent User Interfaces Companion

TL;DR: We conducted a workshop focused on how to design interactions with smart objects to enhance user experience.

BYO*: Utilizing 3D Printed Tangible Tools for Interaction on Interactive Surfaces

Sebastian Günther, Martin Schmitz, Florian Müller, Jan Riemann, Max Mühlhäuser — Wed, 01 Mar 2017 00:00:00 +0000

Sebastian Günther, Martin Schmitz, Florian Müller, Jan Riemann, Max Mühlhäuser
Proceedings of the 2017 ACM Workshop on Interacting with Smart Objects

TL;DR: We designed and built two interaction concepts, Bring Your Own Information and Bring Your Own Tool, utilizing 3D printed tangible objects for enhanced interaction on touch-enabled surfaces.

Sharing and manipulating information are essential for collaborative work in meeting scenarios. Nowadays, people tend to bring their own devices as a result of increasing mobility possibilities. However, transferring data from one device to another can be cumbersome and tedious if restrictions like different platforms, form factors or environmental limitations apply. In this paper, we present two concepts to enrich interaction on and between devices through 3D printed customized tangibles: 1) Bring your own information, and 2) bring your own tools. For this, we enable interactivity for low-cost and passive tangible 3D printed objects by adding conductive material and make use of touch-enabled surfaces. Our system allows users to easily share digital contents across various devices and to manipulate them with individually designed tools without additional hardware required. Copyright © 2017 ACM.

An Evaluation of Hybrid Stacking on Interactive Tabletops

Jan Riemann, Florian Müller, Sebastian Günther, Max Mühlhäuser — Wed, 01 Mar 2017 00:00:00 +0000

Jan Riemann, Florian Müller, Sebastian Günther, Max Mühlhäuser
Proceedings of the 2017 ACM Workshop on Interacting with Smart Objects

TL;DR: We built an integrated hybrid tabletop system called StackTop that enables true hybrid stacking of digital and physical documents.

Stacking is a common practice of organizing documents in the physical world. With the recent advent of interactive tabletops, physical documents can now coexist with digital documents on the same surface. As a result, systems were developed and studied which allow piling of both types of documents with the physical documents being placed on top of the digital ones. In this paper, we study the concept of true hybrid stacking, allowing users to stack both types of documents in an arbitrary order using a hybrid tabletop system called StackTop. We discuss the results and derive implications for future hybrid tabletop systems with stacking support.

ProxiWatch: Enhancing Smartwatch Interaction through Proximity-Based Hand Input

Wed, 01 Jun 2016 00:00:00 +0000

Florian Müller, Sebastian Günther, Niloofar Dezfuli, Mohammadreza Khalilbeigi, Max Mühlhäuser
Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA '16

TL;DR: We built ProxiWatch, a one-handed proximity-based input modality for smartwatches that utilizes the elbow joint's degree of freedom for interaction.

Smartwatches allow ubiquitous and mobile interaction with digital contents. Because of the small screen sizes, tradi-tional interaction techniques are often not applicable. In this work, we show how the degree of freedom offered by the elbow joint, i.e., flexion and extension, can be leveraged as an additional one-handed input modality for smartwatches. By moving the watch towards or away from the body, the user is able to provide input to the smartwatch without a second hand. We present the results of a controlled ex-periment focusing on the human capabilities for proximity-based interaction. Based on the results, we propose guide-lines for designing proximity-based smartwatch interfaces and present ProxiWatch: a one-handed and proximity-based input modality for smartwatches alongside a proto-typical implementation.

Liquido: Embedding Liquids into 3D Printed Objects to Sense Tilting and Motion

Sun, 01 May 2016 00:00:00 +0000

Martin Schmitz, Andreas Leister, Niloofar Dezfuli, Jan Riemann, Florian Müller, Max Mühlhäuser
Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems

TL;DR: We built a method to embed liquids into 3D printed objects to sense various tilting and motion interactions using capacitive sensing.

Tilting and motion are widely used as interaction modalities in smart objects such as wearables and smart phones (e.g., to detect posture or shaking). They are often sensed with accelerometers. In this paper, we propose to embed liquids into 3D printed objects while printing to sense various tilting and motion interactions via capacitive sensing. This method reduces the assembly effort after printing and is a low-cost and easy-to-apply way of extending the input capabilities of 3D printed objects. We contribute two liquid sensing patterns and a practical printing process using a standard dual-extrusion 3D printer and commercially available materials. We validate the method by a series of evaluations and provide a set of interactive example applications. © 2016 Authors.

FreeTop: Finding Free Spots for Projective Augmentation

Sun, 01 May 2016 00:00:00 +0000

Jan Riemann, Mohammadreza Khalilbeigi, Martin Schmitz, Sebastian Doeweling, Florian Müller, Max Mühlhäuser
Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems

TL;DR: We built FreeTop, an approach that automatically identifies suitable areas for projection and touch interaction using depth and color information.

Augmenting the physical world using projection technologies or head-worn displays becomes increasingly popular in research and commercial applications. However, a common problem is interference between the physical surface’s texture and the projection. In this paper, we present FreeTop, a combined approach to finding areas suitable for projection, which considers multiple aspects influencing projection quality, like visual texture and physical surface structure. FreeTop can be used in stationary and mobile settings for locating free areas in arbitrary physical settings suitable for projective augmentation and touch interaction.

SCWT: A Joint Workshop on Smart Connected and Wearable Things

Tue, 01 Mar 2016 00:00:00 +0000

Dirk Schnelle-Walka, Lior Limonad, Tobias Grosse-Puppendahl, Joel Lanir, Florian Müller, Massimo Mecella, Kris Luyten, Tsvi Kuflik, Oliver Brdiczka, Max Mühlhäuser
Companion Publication of the 21st International Conference on Intelligent User Interfaces

TL;DR: We conducted a workshop that focused on advanced interactions with Smart Connected and Wearable Things in the context of the Internet of Things.

The increasing number of smart objects in our everyday life shapes how we interact beyond the desktop. In this workshop we discuss how advanced interactions with smart objects in the context of the Internet-of-Thingsshould be designed from various perspectives, such as HCI and AI as well as industry and academia.

A Study of On-Body User Interface: PiAM(Palm interAction Module)

Tue, 01 Dec 2015 00:00:00 +0000

Seng-Kyoun Jo, 이현우, Kim Jinsul, Florian Müller, Mohammed Khalilbeigi, Max Mühlhäuser
Journal of Knowledge Information Technology and Systems

TL;DR: We conducted an investigation into user interfaces that prioritize human interaction through intuitive design and accessibility.

User interface providing easy and efficient control environments with user is emerging as a core keyword in ICT market and smart phone industry as the influence and importance of UI has increased recently. To succeed in ICT market, however, UI has to satisfy industrial requirements including simple and easy control, intuitional access and high recognition. Thus study for providing user-friendly interface has been actively researched. In this paper, we investigate user interface based on human’s

A Study on Proximity-Based Hand Input for One-Handed Mobile Interaction

Sat, 01 Aug 2015 00:00:00 +0000

Florian Müller, Mohammadreza Khalilbeigi, Niloofar Dezfuli, Alireza Sahami Shirazi, Sebastian Günther, Max Mühlhäuser
Proceedings of the 3rd ACM Symposium on Spatial User Interaction

TL;DR: We conducted a study to investigate the efficacy of proximity-based hand input for one-handed mobile interaction using the elbow joint's flexion and extension.

On-body user interfaces utilize the human’s skin for both sensing input and displaying graphical output. In this paper, we present how the degree of freedom offered by the elbow joint, i.e., exion and extension, can be leveraged to extend the input space of projective user interfaces. The user can move his hand towards or away from himself to browse through a multi-layer information space. We conducted a controlled experiment to investigate how accurately and ef-ficiently users can interact in the space. The results revealed that the accuracy and effciency of proximity-based interactions mainly depend on the traveling distance to the target layer while neither the hand side nor the direction of interaction have a signifcant inuence. Based on our findings, we propose guidelines for designing on-body user interfaces.

Palm-Based Interaction with Head-Mounted Displays

Mon, 01 Jun 2015 00:00:00 +0000

Florian Müller, Niloofar Dezfuli, Max Mühlhäuser, Martin Schmitz, Mohammadreza Khalilbeigi
Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct - MobileHCI '15

TL;DR: We designed two interaction concepts that leverage the user's palm as a 2D interactive surface for head-mounted displays.

Head-mounted displays (HMDs) are an emerging class of wearable devices that allow users to access and alter information right in front of their eyes. However, due to their size and shape, traditional input modalities (e.g., multi-touch sensing on the device) are not practical. In this position paper, we argue that palm-based interactions have a great potential to ease the interaction with HMDs. We outline two interaction concepts and present directions for future research that can lead to more enjoyable and usable interfaces for HMDs.

PalmRC: Imaginary Palm-Based Remote Control for Eyes-Free Television Interaction

Sun, 01 Jul 2012 00:00:00 +0000

Niloofar Dezfuli, Mohammadreza Khalilbeigi, Jochen Huber, Florian Müller, Max Mühlhäuser
EuroiTV'12 - Proceedings of the 10th European Conference on Interactive TV and Video

TL;DR: We built PalmRC, an eyes-free, palm-surface-based television remote control that allows users to interact with their non-dominant hand while focusing on the television screen.

User input on television (TV) typically requires a mediator device, such as a handheld remote control. While being a well-established interaction paradigm, a handheld device has serious drawbacks: it can be easily misplaced due to its mobility and in case of a touch screen interface, it also requires additional visual attention. Emerging interaction paradigms like 3D mid-air gestures using novel depth sensors, such as Microsoft’s Kinect, aim at overcoming these limitations, but are known to be e.g. tiring. In this paper, we propose to leverage the palm as an interactive surface for TV remote control. Our contribution is three-fold: (1) we explore the conceptual design space in an exploratory study. (2) Based upon these results, we investigate the effectiveness and accuracy of such an interface in a controlled experiment. And (3), we contribute PalmRC: an eyes-free, palm-surface-based TV remote control, which in turn is evaluated in an early user feedback session. Our results show that the palm has the potential to be leveraged for device-less and eyes-free TV remote interaction without any third-party mediator device. © 2012 ACM.

CouchTV: Leveraging the Spatial Information of Viewers for Social Interactive Television Systems - Poster Presentation

Sun, 01 Jul 2012 00:00:00 +0000

Niloofar Dezfuli, Manolis Pavlakis, Florian Müller, Mohammadreza Khalilbeigi, Max Mühlhäuser
10th European Interactive TV Conference 2012

Leveraging the Palm Surface as an Eyes-Free Tv Remote Control

Tue, 01 May 2012 00:00:00 +0000

Niloofar Dezfuli, Mohammadreza Khalilbeigi, Jochen Huber, Florian Müller, Max Mühlhäuser
CHI '12 Extended Abstracts on Human Factors in Computing Systems

TL;DR: We designed a palm-based interaction system that enables device-less and eyes-free control of televisions.

User input on television typically requires a mediator device such as a handheld remote control. While being a well-established interaction paradigm, a handheld device has serious drawbacks: it can be easily misplaced due to its mobility and in case of a touch screen interface, it also requires additional visual attention. Emerging interaction paradigms like 3D mid-air gestures using novel depth sensors such as Microsoft’s Kinect aim at overcoming these limitations, but are known for instance to be tiring. In this paper, we propose to leverage the palm as an interactive surface for TV remote control. Our contribution is two-fold: (1) we have explored the conceptual design space in an exploratory study. (2) Based upon these results, we investigated the accuracy and effectiveness of such an interface in a controlled experiment. Our results show that the palm has the potential to be leveraged for device-less and eyes-free TV interactions without any third-party mediator device.