Pink Screen with VideoMaterial in ARKit

Hello @mr_app,

Is it a pink-striped material? That is what RealityKit renders if there is a model with no material set. Is it possible that you are removing the videoMaterial you have set in the code you have shown?

Best regards,

Greg

Same problem, video material pink on ios 18

are there any solutions?

I have the same issue: the video material is pink on iOS 18. On iOS 17, it is working fine.

When I track my AR asset, it starts playing the video. I can hear the audio, but I have this pink video. On iOS 17, it is working fine, but on iOS 18, I have this bug. Any suggestion on how to fix it?

That is my code, I still have the pink material for iOS 18. While on iOS 17 is working fine. Any advice?

// Create Video Screen
private func createVideoScreen(width: Float, height: Float, url: URL, uuid: UUID) -> ModelEntity {
    let screenMesh = MeshResource.generatePlane(width: width, height: height)
    let videoItem = createVideoItem(with: url)
    let videoMaterial = createVideoMaterial(with: videoItem, uuid: uuid)
    let videoScreenModel = ModelEntity(mesh: screenMesh, materials: [videoMaterial])
    return videoScreenModel
}

// Create Video Item
private func createVideoItem(with url: URL) -> AVPlayerItem {
    let asset = AVURLAsset(url: url)
    let videoItem = AVPlayerItem(asset: asset)
    return videoItem
}

// Create Video Material
private func createVideoMaterial(with videoItem: AVPlayerItem, uuid: UUID) -> VideoMaterial {
    let player = AVPlayer()
    player.actionAtItemEnd = .none
    let videoMaterial = VideoMaterial(avPlayer: player)
    player.replaceCurrentItem(with: videoItem)
    player.play()
    
    NotificationCenter.default.addObserver(forName: .AVPlayerItemDidPlayToEndTime, object: player.currentItem, queue: .main) { _ in
        player.seek(to: .zero)
        player.play()
    }
    videoPlayers[uuid] = player
    return videoMaterial
}

// Place Video Screen
private func placeVideoScreen(videoScreen: ModelEntity, imageAnchor: ARImageAnchor, uuid: UUID) {
    guard let arView = arView else { return }
    
    let imageAnchorEntity = AnchorEntity(anchor: imageAnchor)
    let rotationAngle = simd_quatf(angle: GLKMathDegreesToRadians(-90), axis: SIMD3(x: 1, y: 0, z: 0))
    videoScreen.setOrientation(rotationAngle, relativeTo: imageAnchorEntity)
    imageAnchorEntity.addChild(videoScreen)
    
    arView.scene.addAnchor(imageAnchorEntity)
    activeAnchors[uuid] = imageAnchorEntity
}

Same issue, iOS 18.1.1; solid pink, not striped

Same issue here! Solid pink screen!!! Works on iOS 17 but not on iOS 18+ (tested also on the latest iOS 18.2.1)

Trying to reproduce with this simple snippet:

struct ContentView : View {
    
    private let player = AVPlayer()

    var body: some View {
        RealityView { content in
            
            let model = Entity()
            let mesh = MeshResource.generatePlane(width: 1, height: 1)
            let material = VideoMaterial(avPlayer: player)
            model.components.set(ModelComponent(mesh: mesh, materials: [material]))
            model.position = [0, 1, -2]
            
            content.add(model)

            content.camera = .spatialTracking
        }
        .edgesIgnoringSafeArea(.all)
        .onAppear() {
            Task {
                let videoURL = URL(string: "http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4")
                
                let playerItem = AVPlayerItem(url: videoURL!)
//                    let videoAssetTrack = try await playerItem.asset.loadTracks(withMediaType: .video)
//                    try await videoAssetTrack.first!.load(.naturalSize)
                
                // Tell the player to load and play
                player.replaceCurrentItem(with: playerItem)
                player.play()
            }
        }
    }
}

If you argue that i'm using http instead of https, i tryed both (also with "App Transport Security Settings" -> "Allow Arbitrary Loads": yes into info.plist)

Got a lot of warnings (i suppose that some shader or some material, as VideoMaterial, are missing):

Error creating the CFMessagePort needed to communicate with PPT.
Could not locate file 'default-binaryarchive.metallib' in bundle.
Video texture allocator is not initialized.
[VideoLightSpillGenerator] [VideoLightSpillMPSCallsPrewarm] Failed to create input texture
Registering library (/System/Library/PrivateFrameworks/CoreRE.framework/default.metallib) that already exists in shader manager. Library will be overwritten.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/suFeatheringCreateMergedOcclusionMask.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arKitPassthrough.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arSegmentationComposite.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arInPlacePostProcessCombinedPermute0.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arInPlacePostProcessCombinedPermute1.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arInPlacePostProcessCombinedPermute2.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arInPlacePostProcessCombinedPermute3.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arInPlacePostProcessCombinedPermute4.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arInPlacePostProcessCombinedPermute5.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arInPlacePostProcessCombinedPermute6.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arInPlacePostProcessCombinedPermute7.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arInPlacePostProcessCombinedPermute8.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.

...

Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arInPlacePostProcessCombinedPermute11.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arInPlacePostProcessCombinedPermute12.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arInPlacePostProcessCombinedPermute13.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arInPlacePostProcessCombinedPermute14.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arInPlacePostProcessCombinedPermute15.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.

and when i play the video i got lot of these (not an encoding issue because the same video works if i use AVPlayer with plain VideoPlayer in swiftUI):

Failed to sample image from video receiver (error: -15579)

Please HEEEEEELP!!!!

Same issue on iOS 18.3.2

Same issue on iPadOS 18.3.2. Can we please escalate this issue?

And still the same issue on iOS 18.4.0.

Hello all,

Thanks for the continued feedback on this!

A couple of observations:

1. I'm unable to reproduce the behavior described. Can someone provide a focused sample project that reproduces the issue?

2. If you have not filed a bug report for this issue using Feedback Assistant, please do file one, and then post your Feedback ID on this thread so that I can track it.

--Greg

The following code works on iOS 17 and iPadOS 17.x but not on 18. The Video Plane simply appears pink instead of showing the video itself.

import SwiftUI
import RealityKit
import AVFoundation

struct ContentView : View {

    var body: some View {
        RealityView { content in

            // Create a cube model
            let model = Entity()
            let mesh = MeshResource.generateBox(size: 0.1, cornerRadius: 0.005)
            let material = SimpleMaterial(color: .gray, roughness: 0.15, isMetallic: true)
            model.components.set(ModelComponent(mesh: mesh, materials: [material]))
            model.position = [0, 0.05, 0]

            // Create horizontal plane anchor for the content
            let anchor = AnchorEntity()
            
            let player = AVPlayer(url: Bundle.main.url(forResource: "phone-video",
                                                           withExtension: "mp4")!)
            let videoMaterial = VideoMaterial(avPlayer: player)
            
            let modelEntity = ModelEntity(mesh: .generateBox(width: 2.0, height: 1.7, depth: 0.01), materials: [videoMaterial])
            
            player.play()
            
            modelEntity.position.z -= 2
            modelEntity.setParent(anchor)
            
            anchor.addChild(model)

            // Add the horizontal plane anchor to the scene
            content.add(anchor)

            content.camera = .spatialTracking

        }
        .edgesIgnoringSafeArea(.all)
    }

}

#Preview {
    ContentView()
}

Still not working on iOS 18.4 and 18.4.1. Anything we can do meanwhile?

@DTS Engineer I just submitted bug as Third Party Apps Feedback Assistant ID: FB17551022

I can confirm it works on iPadOS 18.4.1, but not on iOS 18.4.1