Navbar buttons disappear in NavigationSplitView's sidebar when changing apps

We recently migrated our app to use NavigationSplitView on iPad with a sidebar and detail setup, and we got reports that the navigation buttons on the sidebar disappear when returning to our app after using a different app. I reproduced the issue from a new empty project with the following code (issue tested on iOS 17.4 and iOS 18.3, was not able to reproduce on iOS 16.4):

import SwiftUI

@main
struct TestApp: App {
    var body: some Scene {
        WindowGroup {
            NavigationSplitView {
                Text("sidebar")
                    .toolbar {
                        ToolbarItem(placement: .topBarLeading) {
                            Button(action: {}) {
                                Image(systemName: "square.and.arrow.down")
                            }
                        }
                        ToolbarItem(placement: .topBarTrailing) {
                            Button(action: {}) {
                                Image(systemName: "square.and.arrow.up")
                            }
                        }
                    }
            } detail: {
                Text("detail")
                    .toolbar {
                        ToolbarItem(placement: .topBarLeading) {
                            Button(action: {}) {
                                Image(systemName: "eraser")
                            }
                        }
                        ToolbarItem(placement: .topBarTrailing) {
                            Button(action: {}) {
                                Image(systemName: "pencil")
                            }
                        }
                    }
            }
        }
    }
}

Please check the following GIF for the simple steps, notice how the navbar buttons in the detail view do not disappear: Here's the console output, it shows that the constraints break internally: