Reusable interaction design patterns for HMD-based AR in cultural heritage

Study 06

Immersion in Museum XR

Examining how different forms of immersion shape engagement and learning in HMD-based museum experiences.

Immersion Museum Learning XR

This study examines immersion as a design dimension in HMD-based museum experiences and investigates how different immersive qualities relate to visitor engagement and learning.

The research began with the observation that playful and game-based interaction can make museum AR experiences more engaging, but that increased immersion may also shape how visitors learn from the experience. Early work therefore focused on whether HMD-based interaction could benefit from stronger spatial immersion, using the spatial affordances of head-mounted AR to strengthen visitors’ sense of being situated within an augmented exhibit experience.

As game mechanics became more central to the interaction design, however, it became clear that immersion is not a single experience quality. Playful museum XR can involve different immersive dynamics, including spatial immersion, narrative immersion, and flow-related immersion associated with gameplay and task engagement.

The later research therefore shifted from treating immersion as a general positive outcome to comparing these three forms of immersion more explicitly and examining how each may influence museum learning in different ways.

1. Exploring immersion as part of the visitor experience

Early work on playful HMD-AR interaction treated immersion as one of several important experiential qualities that could contribute to visitor engagement and museum learning.

In prototype studies using game-mechanic-based interaction, visitor experience was evaluated across multiple dimensions, including usability, cognitive load, engagement, and immersion. At this stage, particular attention was given to spatial immersion, because head-mounted AR enables visitors to perceive and interact with digital content as part of the surrounding exhibition space rather than only through a handheld screen.

To examine this dimension, the evaluations included a spatial-presence measure alongside other experience-oriented questionnaires. This helped provide an initial empirical basis for understanding whether playful HMD-AR interaction could support not only usability and engagement, but also a stronger sense of being situated within an augmented museum environment.

These early findings suggested that immersive qualities can positively contribute to the visitor experience, while also indicating that immersion should not be treated in isolation. Instead, it interacts with task demands, playful mechanics, and the broader context of museum learning.

This motivated a more focused investigation into different forms of immersion and their possible roles in learning-oriented museum XR design.

2. Comparing flow, narrative immersion, and spatial immersion for museum learning

The later study examined immersion in a more differentiated way by explicitly comparing flow immersion, narrative immersion, and spatial immersion.

To investigate these different immersive qualities, a museum-oriented HMD-AR serious game was developed in three configurations around the coral reef exhibit. Each version retained the same general topic and interaction structure while foregrounding a different design emphasis.

Flow-enhanced version of the HMD-AR museum experience

Flow-enhanced configuration, emphasizing clear goals, increasing challenge, task progression, and immediate feedback.

Narrative-enhanced version of the HMD-AR museum experience

Narrative-enhanced configuration, emphasizing story framing, character guidance, causal explanation, and the connection between visitor actions and ecological meaning.

Spatial-enhanced version of the HMD-AR museum experience

Spatial-enhanced configuration, emphasizing environmental richness, embodied exploration, and spatially distributed reef-related content.

A between-subjects museum field study compared how these three configurations influenced visitors’ perceived immersion and learning-related responses.

The results showed that immersion did not behave as a single experiential quality. The flow-enhanced variant showed the clearest alignment between design intention and perceived immersion, with the strongest flow-related tendency among the three conditions. Narrative immersion did not differ significantly between variants, but it showed the clearest association with perceived meaning and educational value.

The most unexpected result concerned spatial immersion. Although the spatial-enhanced version included richer underwater effects and more spatially distributed virtual content, it produced the lowest spatial-immersion score of the three conditions.

This result suggests a tension that is specific to HMD-AR museum design. Unlike fully virtual VR, passthrough AR continuously connects digital content to the surrounding physical exhibition. Increasing environmental richness may therefore strengthen visitors’ awareness of the relationship between real and virtual elements rather than making them feel more absorbed in the virtual scene.

For museum XR, spatial immersion should therefore not be equated with simply adding more environmental media. Spatial content needs to guide attention toward meaningful relationships between the physical exhibit, the virtual scene, and visitors’ actions.

Taken together, the findings suggest that immersion should be treated as a design lever rather than a generic goal to be maximized. Flow, narrative, and spatial immersion can support different aspects of museum experience and learning, but their effects depend on how interaction, interpretation, and the surrounding exhibition context are connected.