SpatialEventGesture Not Working to Show Hidden Menu in Immersive Panorama View - visionOS

SpatialEventGesture Not Working to Show Hidden Menu in Immersive Panorama View - visionOS

Problem Description

I'm developing a Vision Pro app that displays 360° panoramic photos in a full immersive space. I have a floating menu that auto-hides after 5 seconds, and I want users to be able to show the menu again using spatial gestures (particularly pinch gestures) when it's hidden.

However, the SpatialEventGesture implementation is not working as expected. The menu doesn't appear when users perform pinch gestures or other spatial interactions in the immersive space.

Current Implementation

Here's the relevant gesture detection code in my ImmersiveView:

import SwiftUI
import RealityKit

struct ImmersiveView: View {
    @EnvironmentObject var appModel: AppModel
    @Environment(\.openWindow) private var openWindow
    
    var body: some View {
        RealityView { content in
            // RealityView content setup with panoramic sphere...
            let rootEntity = Entity()
            content.add(rootEntity)
            // Load panoramic content here...
        }
        // Using SpatialEventGesture to handle multiple spatial gestures
        .gesture(
            SpatialEventGesture()
                .onEnded { eventCollection in
                    // Check menu visibility state
                    if !appModel.isPanoramaMenuVisible {
                        // Iterate through event collection to handle various gestures
                        for event in eventCollection {
                            switch event.kind {
                            case .touch:
                                print("Detected spatial touch gesture, showing menu")
                                showMenuWithGesture()
                                return
                            case .indirectPinch:
                                print("Detected spatial pinch gesture, showing menu") 
                                showMenuWithGesture()
                                return
                            case .pointer:
                                print("Detected spatial pointer gesture, showing menu")
                                showMenuWithGesture()
                                return
                            @unknown default:
                                print("Detected unknown spatial gesture: \(event.kind)")
                                showMenuWithGesture()
                                return
                            }
                        }
                    }
                }
        )
        // Keep long press gesture as backup
        .simultaneousGesture(
            LongPressGesture(minimumDuration: 1.5)
                .onEnded { _ in
                    if !appModel.isPanoramaMenuVisible {
                        print("Detected long press gesture, showing menu")
                        showMenuWithGesture()
                    }
                }
        )
    }
    
    private func showMenuWithGesture() {
        if !appModel.isPanoramaMenuVisible {
            appModel.showPanoramaMenu()
            if !appModel.windowExists(id: "PanoramaMenu") {
                openWindow(id: "PanoramaMenu", value: "menu")
            }
        }
    }
}

What I've Tried

1. Multiple SpatialTapGesture approaches: Originally tried using multiple .gesture() modifiers with SpatialTapGesture(count: 1) and SpatialTapGesture(count: 2), but realized they override each other.

2. SpatialEventGesture implementation: Switched to SpatialEventGesture to handle multiple event types (.touch, .indirectPinch, .pointer), but pinch gestures still don't trigger the menu.

3. Added debugging: Console logs show that the gesture callbacks are never called when performing pinch gestures in the immersive space.

4. Backup LongPressGesture: Added a simultaneous long press gesture as backup, which also doesn't work consistently.

Expected Behavior

When the panorama menu is hidden (after 5-second auto-hide), users should be able to:

• Perform a pinch gesture (indirect pinch) to show the menu
• Tap in space to show the menu
• Use other spatial gestures to show the menu

Questions

1. Is SpatialEventGesture the correct approach for detecting gestures in a full immersive RealityView?

2. Are there any special considerations for gesture detection when the RealityView contains a large panoramic sphere that might be intercepting gestures?

3. Should I be using a different gesture approach for visionOS immersive spaces?

4. Is there a way to ensure gestures work even when the RealityView content (panoramic sphere) might be blocking them?

Environment

• Xcode 16.1
• visionOS 2.5
• Testing on Vision Pro device
• App uses SwiftUI + RealityKit

Any guidance on the proper way to implement spatial gesture detection in visionOS immersive spaces would be greatly appreciated!

Additional Context

The app manages multiple windows and the gesture detection should work specifically when in the immersive panorama mode with the menu hidden.

Thank you for any help or suggestions!