Study 02
Game-Mechanic-Based Interaction Design
Exploring playful interaction mechanics for engagement, exploration, and learning in museum XR.
This study investigates how playful and game-mechanic-based interaction can support engagement, exploration, and learning in head-mounted AR museum experiences.
Rather than treating games as separate entertainment layers, the research explored how playful actions could become part of the interaction itself — shaping how visitors move through space, engage with exhibits, and actively participate in the experience.
1. Making spatial interaction more playful and engaging
The first stage of the research explored how locative and spatial interaction could make HMD-based museum experiences more active and engaging.
Instead of presenting visitors with information at fixed points, the interaction encouraged them to move through the exhibition environment, discover virtual content, follow spatial cues, and engage with AR experiences distributed across physical space.
This work examined how playful movement and location-based interaction could support exploration without turning the museum experience into a conventional game. The focus was on making navigation, discovery, and participation themselves more engaging.
The resulting prototypes helped establish playful spatial interaction as a design direction for museum HMD-AR and provided a basis for later work on more explicit game mechanics.
A playful locative interaction concept in which visitors follow circular spatial cues through the exhibition environment. The prototype explored how movement, wayfinding, and discovery could become part of the interaction itself rather than serving only as transitions between informational points.
An exploratory interaction prototype in which visitors engage with animated virtual content through direct spatial action. The experience examined how playful embodied interaction could increase engagement and make museum AR experiences more active and participatory.
Related research
2. Introducing game mechanics into reusable interaction patterns
Building on the earlier exploration of playful locative interaction, the research then investigated how specific game mechanics could be integrated into reusable HMD-AR interaction patterns.
Rather than designing a complete game around the museum exhibit, game mechanics were used as interaction structures that encouraged visitors to observe, manipulate, create, and respond to exhibition content.
Three different interaction directions were explored through museum-oriented prototypes:
-
Puzzle-based interaction — visitors actively reconstruct or assemble exhibit-related content, transforming interpretation into a spatial problem-solving activity.
-
Creative drawing interaction — visitors create their own visual interpretation of exhibit content through spatial drawing and painting tools.
-
Action-based interaction — visitors respond to dynamic environmental challenges through direct spatial actions, connecting gameplay with educational or conservation-oriented content.
Together, these prototypes demonstrated how different game mechanics could be adapted for museum XR while remaining meaningfully connected to physical exhibits and learning goals.
Overview of three exhibit-oriented prototypes developed to explore how game mechanics could be integrated into HMD-AR museum experiences. These design directions connected playful interaction with exhibit interpretation, visitor participation, and learning-oriented engagement.
A puzzle-based interaction concept in which visitors reconstruct or manipulate exhibit-related virtual elements. This prototype examined how game-like problem solving could support attention, interpretation, and active engagement with museum content.
A creative drawing interaction allowing visitors to produce their own spatial visual responses to exhibit content. The prototype explored how expressive participation could become part of interpretation and learning in museum XR.
An action-based interaction prototype in which visitors respond to dynamic virtual targets through direct embodied action. The experience investigated how fast, playful interaction could be used to increase engagement while remaining linked to educational or thematic exhibit goals.
The findings were subsequently abstracted into reusable interaction patterns, contributing to the broader pattern system for playful HMD-AR experiences in museums and cultural heritage environments.
