Crash in libswiftCore with swift::RefCounts

Question

Created 2d

Replies 5

Boosts 0

Participants 2

I'm seeing somewhat regular crash reports from my app which appear to be deep in the Swift libraries. They're happening in the same spot, so I'm apt to believe something is likely getting deallocated behind the scenes - but I don't really know how to guard against it.

Here's the specific crash thread:

0   libsystem_kernel.dylib        	0x00000001d51261dc __pthread_kill + 8 (:-1)
1   libsystem_pthread.dylib       	0x000000020eaa8b40 pthread_kill + 268 (pthread.c:1721)
2   libsystem_c.dylib             	0x000000018c5592d0 abort + 124 (abort.c:122)
3   libsystem_malloc.dylib        	0x0000000194d14cfc malloc_vreport + 892 (malloc_printf.c:251)
4   libsystem_malloc.dylib        	0x0000000194d14974 malloc_report + 64 (malloc_printf.c:290)
5   libsystem_malloc.dylib        	0x0000000194d0e8b4 ___BUG_IN_CLIENT_OF_LIBMALLOC_POINTER_BEING_FREED_WAS_NOT_ALLOCATED + 32 (malloc_common.c:227)
6   Foundation                    	0x0000000183229f40 __DataStorage.__deallocating_deinit + 104 (Data.swift:563)
7   libswiftCore.dylib            	0x0000000182f556c8 _swift_release_dealloc + 56 (HeapObject.cpp:847)
8   libswiftCore.dylib            	0x0000000182f5663c bool swift::RefCounts<swift::RefCountBitsT<(swift::RefCountInlinedness)1>>::doDecrementSlow<(swift::PerformDeinit)1>(swift::RefCountBitsT<(swift::RefCountInlinedness)1>, unsigned int) + 152 (RefCount.h:1052)
9   TAKAware                      	0x000000010240c688 StreamParser.parseXml(dataStream:) + 1028 (StreamParser.swift:0)
10  TAKAware                      	0x000000010240cdb4 StreamParser.processXml(dataStream:forceArchive:) + 16 (StreamParser.swift:85)
11  TAKAware                      	0x000000010240cdb4 StreamParser.parseCoTStream(dataStream:forceArchive:) + 360 (StreamParser.swift:108)
12  TAKAware                      	0x000000010230ac3c closure #1 in UDPMessage.connect() + 252 (UDPMessage.swift:68)
13  Network                       	0x000000018506b68c closure #1 in NWConnectionGroup.setReceiveHandler(maximumMessageSize:rejectOversizedMessages:handler:) + 200 (NWConnectionGroup.swift:458)
14  Network                       	0x000000018506b720 thunk for @escaping @callee_guaranteed (@guaranteed OS_dispatch_data?, @guaranteed OS_nw_content_context, @unowned Bool) -> () + 92 (<compiler-generated>:0)
15  Network                       	0x0000000185185df8 invocation function for block in nw_connection_group_handle_incoming_packet(NWConcrete_nw_connection_group*, NSObject<OS_nw_endpoint>*, NSObject<OS_nw_endpoint>*, NSObject<OS_nw_interface>*, NSObje... + 112 (connection_group.cpp:1075)
16  libdispatch.dylib             	0x000000018c4ad2b8 _dispatch_block_async_invoke2 + 148 (queue.c:574)
17  libdispatch.dylib             	0x000000018c4b7584 _dispatch_client_callout + 16 (client_callout.mm:85)
18  libdispatch.dylib             	0x000000018c4d325c _dispatch_queue_override_invoke.cold.3 + 32 (queue.c:5106)
19  libdispatch.dylib             	0x000000018c4a21f8 _dispatch_queue_override_invoke + 848 (queue.c:5106)
20  libdispatch.dylib             	0x000000018c4afdb0 _dispatch_root_queue_drain + 364 (queue.c:7342)
21  libdispatch.dylib             	0x000000018c4b054c _dispatch_worker_thread2 + 156 (queue.c:7410)
22  libsystem_pthread.dylib       	0x000000020eaa5624 _pthread_wqthread + 232 (pthread.c:2709)
23  libsystem_pthread.dylib       	0x000000020eaa29f8 start_wqthread + 8 (:-1)

Basically we're receiving a message via UDP that is an XML packet. We're parsing that packet using what I think it pretty straightforward code that looks like this:

func parseXml(dataStream: Data?) -> Array<String> {
    var events: [String] = []
    guard let data = dataStream else { return events }
    currentDataStream.append(data)
    var str = String(decoding: currentDataStream, as: UTF8.self)
    while str.contains(StreamParser.STREAM_DELIMTER) {
        let splitEvent = str.split(separator: StreamParser.STREAM_DELIMTER, maxSplits: 1)
        let cotEvent = splitEvent.first!
        var restOfString = ""
        if splitEvent.count > 1 {
            restOfString = String(splitEvent.last!)
        }
        events.append("\(cotEvent)\(StreamParser.STREAM_DELIMTER)")
        str = restOfString
    }
    currentDataStream = Data(str.utf8)
    return events
}

the intention is that the message may be broken across multiple packets, so we build them up here.

Is there anything I can do to guard against these crashes?

Answered by FT-cfoy in 847163022

(For posterity and web searches)

One can create a serial queue by using the default initializer for DispatchQueue:

private let serialQueue = DispatchQueue(label: "some.custom.label")

I'm then initializing the Connection Group like:

connectionGroup = NWConnectionGroup(with: multicastGroup!, using: params)

and finally starting it with:

connectionGroup.start(queue: serialQueue)

It's that last line that's super important because the default global() queue will give you a concurrent queue rather than a serial one.

Boost

Answer 1

DTS Engineer OP

Apple

2d

This is most likely a memory corruption issue. If you forced me to guess, I’d say it most likely caused by a concurrency problem. In the code snippet you posted, is there any chance that currentDataStream can be accessed by multiple threads concurrently?

Also:

In situations like this I generally recommend that you apply the standard memory debugging tools, because they can make it easier to reproduce problems like this.
Also, please post a full crash report, per the advice in Posting a Crash Report. That might reveal something else interesting.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

0

Answer 2

FT-cfoy OP

1d

2025-06-09_18-48-03.1721_-0600-43bc27c538c2f93348a83e5380b5f016cc859f71.crash

Thanks Quinn! I'm attaching one of the crash logs here. However your pointer got me to the issue - it's a race condition when the device is getting lots of messages over UDP from lots of other disparate devices and those messages are getting split over multiple packets. The completion handler is spinning up a new thread, but working off of the same instance of an underlying class which is where I think the issue is.

I was able to replicate it by flooding the device with a bunch of partial messages randomly and could easily reproduce.

0

Answer 3

DTS Engineer OP

Apple

1d

Accepted Answer

I’m glad to hear you’re making progress.

The completion handler is spinning up a new thread

I noticed that the crashing code is being called out of Network framework. One thing to watch out for there is the type of queue that you pass into Network framework when you create your NWConnection [1]. Network framework will accept either a serial queue or a concurrent queue. Internally it does its own serialisation, so it doesn’t care. However, for your own sanity it’s important to use a serial queue. If you use a concurrent queue, it becomes very hard to reason about your code.

Similarly, if you have multiple connections running simultaneously, it often makes sense to use the same serial queue for all of them.

Oh, and be wary of concurrent queues in general. See Avoid Dispatch Global Concurrent Queues.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] Well, in your case it looks like a connection group, but it’s the same story with that.

0

Answer 4

FT-cfoy OP

23h

Ah yes, great point! I think that's exactly what it was. I had fixed the issue initially by processing in a serial queue, but your comment made me go back and realize the connection group was getting started like this:

connectionGroup.start(queue: .global())

Which is, of course, a concurrent queue.

With the serial queue in place I was able to flood the device with thousands of UDP messages without issue.

Thanks so much!

0

Answer 5

FT-cfoy OP

23h

Recommended

(For posterity and web searches)

One can create a serial queue by using the default initializer for DispatchQueue:

private let serialQueue = DispatchQueue(label: "some.custom.label")

I'm then initializing the Connection Group like:

connectionGroup = NWConnectionGroup(with: multicastGroup!, using: params)

and finally starting it with:

connectionGroup.start(queue: serialQueue)

It's that last line that's super important because the default global() queue will give you a concurrent queue rather than a serial one.

0