Back to Publications

CreepyCoCreator? Investigating AI Representation Modes for 3D Object Co-Creation in Virtual Reality

Julian Rasch, Julia Töws, Teresa Hirzle, Florian Müller, Martin Schmitz
CHI 2025
Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems - CHI '25
PDF DOI arXiv
TL;DR
What we did: We conducted a Wizard-of-Oz study to investigate how different representation modes of an artificial intelligence influence user perception and interaction during 3D object co-creation in Virtual Reality.
What we found: We found that an embodied AI significantly enhances user perception of its contributions, while highlighting areas for AI modification reduces user satisfaction and perceived partnership with the AI.
Takeaway: We recommend using an embodied AI for co-creation tasks to foster better collaboration, yet caution against using highlighting to avoid diminishing user enjoyment and ownership of created objects.

Abstract

Generative AI in Virtual Reality offers the potential for collaborative object-building, yet challenges remain in aligning AI contributions with user expectations. In particular, users often struggle to understand and collaborate with AI when its actions are not transparently represented. This paper thus explores the co-creative object-building process through a Wizard-of-Oz study, focusing on how AI can effectively convey its intent to users during object customization in Virtual Reality. Inspired by human-to-human collaboration, we focus on three representation modes: the presence of an embodied avatar, whether the AI’s contributions are visualized immediately or incrementally, and whether the areas modified are highlighted in advance. The findings provide insights into how these factors affect user perception and interaction with object-generating AI tools in Virtual Reality as well as satisfaction and ownership of the created objects. The results offer design implications for co-creative world-building systems, aiming to foster more effective and satisfying collaborations between humans and AI in Virtual Reality.