CompositorServices Or RealityKit

Hello @lijiaxu,

Is there a visual effect in your app that is not achievable with RealityKit? Understanding how to use Compositor Services is not essential to creating great looking apps on visionOS, but it certainly helps to understand the underlying tech!

RealityKit makes building apps for visionOS less complex, is great for testing out ideas quickly, and contains everything you need to create a complete experience on visionOS. But RealityKit and Compositor Services are not direct replacements for the other.

It is true that both technologies cannot be used in the same immersive space. They both provide tools for rendering content, although RealityKit provides additional tools to support physics simulations, animations, rendering content alongside objects in the real world, loading USDs and content created with Reality Composer Pro, and more.

Compositor Services allow you to draw directly with Metal APIs in a way that isn't possible with RealityKit. Check out Render Metal with passthrough in visionOS for more information. You might go this route if you have a particular stylized effect you are trying to achieve, or maybe you are writing a custom game engine and need full control over every frame. Rendering directly with Metal APIs gives you more control than is possible with RealityKit, but comes at the cost of a learning curve and the lack of the those additional features like a physics simulation.

Let me know if that helps answer your question!

Hi @lijiaxu , please ask a new question in a new thread, however I'll post my response to your question here because comments on posts have a smaller character limit.

You should check out the sample for the article Interacting with virtual content blended with passthrough. This app shows how to track the users hands (using ARKit) and then render content using Metal+Compositor Services. Additionally, check out Drawing fully immersive content using Metal for a more in depth article on how to create your own render loop using Compositor Services.

HoverEffectComponent is a part of RealityKit, and there is not an equivalent API using compositor services. I strongly recommend filing an enhancement request for this feature specifically using Feedback Assistant.

ARKit and RealityKit are separate frameworks, so if you choose not to use RealityKit and render your content with Compositor Services, you can still use ARKit to track your user's hands, do scene reconstruction, etc. However anything related to RealityKit's ECS (which includes things like HoverEffectComponent, PhysicsBodyComponent, CollisionComponent , etc.) will not be available to you, and you'll need your own solutions for things like physics interactions, for example.