Crashes in NEFilterPacketInterpose createChannel

Hello,

Our users are seeing random crashes in our packet filter system extension on macOS. Any help pointing me in the right direction to either avoid the issue or fix it would be greatly appreciated. Attached is the crash log.

Thank you.

Crashed Thread:        2  Dispatch queue: com.apple.network.connections

Exception Type:        EXC_BAD_ACCESS (SIGBUS)
Exception Codes:       KERN_PROTECTION_FAILURE at 0x0000000112918700
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Bus error: 10
Termination Reason:    Namespace SIGNAL, Code 0xa
Terminating Process:   exc handler [40687]

...

Thread 2 Crashed:: Dispatch queue: com.apple.network.connections
0   libsystem_kernel.dylib        	0x00007fff2089b46e os_channel_get_next_slot + 230
1   com.apple.NetworkExtension    	0x00007fff2e2e2643 __40-[NEFilterPacketInterpose createChannel]_block_invoke + 560
2   libdispatch.dylib             	0x00007fff20718806 _dispatch_client_callout + 8
3   libdispatch.dylib             	0x00007fff2071b1b0 _dispatch_continuation_pop + 423
4   libdispatch.dylib             	0x00007fff2072b564 _dispatch_source_invoke + 2061
5   libdispatch.dylib             	0x00007fff20720318 _dispatch_workloop_invoke + 1784
6   libdispatch.dylib             	0x00007fff20728c0d _dispatch_workloop_worker_thread + 811
7   libsystem_pthread.dylib       	0x00007fff208bf45d _pthread_wqthread + 314
8   libsystem_pthread.dylib       	0x00007fff208be42f start_wqthread + 15
Answered by DTS Engineer in 816174022
I've submitted logs and crash report under FB15994359.

Thanks.

Your bug report included a JSON crash report which allowed me to learn more about this crash. To start, with some internal tools I was able to find that the crashing thread’s backtrace actually looks like this:

Thread 3 Crashed:: Dispatch queue: com.apple.network.connections
0  libsystem_kernel.dylib     … os_channel_get_next_slot + 668 …
1  com.apple.NetworkExtension … receiveOneFramefromRing + 32 …
2  com.apple.NetworkExtension … receiveFramesfromRings + 248 …
3  com.apple.NetworkExtension … __40-[NEFilterPacketInterpose createChannel]_block_invoke + 392 …
4  libdispatch.dylib          … _dispatch_client_callout + 20

Frames 1 and 2 are inlined, and thus don’t show up in your backtrace.

This backtrace makes a lot more sense. Dispatch has called the NE packet filter block which is trying to get a packet from the user-space network subsystem which has trapped. This trap is within an assert that checks its internal state, so that’s got corrupted somehow. I don’t know enough about the code to offer any insight into the actual failure.

However, your bug report pointed me at another bug (r. 129563031) which shows a very similar crash in a completely different developer’s packet filter. That suggests that this is more likely to be a problem with macOS than with your code.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

That crash report was from macOS 11.6.5. Have you seen crashes on anything newer?

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

I have not seen crashes on anything newer so far.

OK.

So, yeah, that’s a wacky crash deep within NE. Some internal analysis of your crash report suggests that it was something we actually fixed, possible in macOS 12. But if you saw it on modern system, say macOS 13, that’d be definitely something new.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Hello, we have just noticed similar crash on macos 14.7.1. I've submitted logs and crash report under FB15994359.

Jakub

I've submitted logs and crash report under FB15994359.

Thanks.

Your bug report included a JSON crash report which allowed me to learn more about this crash. To start, with some internal tools I was able to find that the crashing thread’s backtrace actually looks like this:

Thread 3 Crashed:: Dispatch queue: com.apple.network.connections
0  libsystem_kernel.dylib     … os_channel_get_next_slot + 668 …
1  com.apple.NetworkExtension … receiveOneFramefromRing + 32 …
2  com.apple.NetworkExtension … receiveFramesfromRings + 248 …
3  com.apple.NetworkExtension … __40-[NEFilterPacketInterpose createChannel]_block_invoke + 392 …
4  libdispatch.dylib          … _dispatch_client_callout + 20

Frames 1 and 2 are inlined, and thus don’t show up in your backtrace.

This backtrace makes a lot more sense. Dispatch has called the NE packet filter block which is trying to get a packet from the user-space network subsystem which has trapped. This trap is within an assert that checks its internal state, so that’s got corrupted somehow. I don’t know enough about the code to offer any insight into the actual failure.

However, your bug report pointed me at another bug (r. 129563031) which shows a very similar crash in a completely different developer’s packet filter. That suggests that this is more likely to be a problem with macOS than with your code.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

@DTS Engineer We're encountering these crashes as well, including on macOS 15. Could this be related to exceptionally high internet speeds?


Thread 12 Crashed::  Dispatch queue: com.apple.network.connections
0   libsystem_kernel.dylib        	       0x19cf9fbd8 os_channel_get_next_slot + 664
1   NetworkExtension              	       0x1afc07818 __40-[NEFilterPacketInterpose createChannel]_block_invoke + 416
2   libdispatch.dylib             	       0x19ce2b85c _dispatch_client_callout + 16
3   libdispatch.dylib             	       0x19ce165e0 _dispatch_continuation_pop + 596
4   libdispatch.dylib             	       0x19ce29620 _dispatch_source_latch_and_call + 396
5   libdispatch.dylib             	       0x19ce282f8 _dispatch_source_invoke + 844
6   libdispatch.dylib             	       0x19ce1c170 _dispatch_workloop_invoke + 1612
7   libdispatch.dylib             	       0x19ce25264 _dispatch_root_queue_drain_deferred_wlh + 292
8   libdispatch.dylib             	       0x19ce24ae8 _dispatch_workloop_worker_thread + 540
9   libsystem_pthread.dylib       	       0x19cfc5e64 _pthread_wqthread + 292
10  libsystem_pthread.dylib       	       0x19cfc4b74 start_wqthread + 8

I had a quick look at the bugs mentioned upthread (FB15994359 and r. 129563031) and have no updates to share on that front.

Could this be related to exceptionally high internet speeds?

Could it be? Certainly. Is there any actual evidence that it is? Not that I’ve seen. But it’s as a good a theory as any.

Have you filed your own bug about this? If so, please post your bug number. If not, I encourage you to do so, and also post the bug number.

And if anyone has any info on how to reproduce this, please update your bug with the details and let me know by replying here. I suspect it’ll be hard to fix without a reproducible case )-:

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

@DTS Engineer In my case, user's reported such crashes on m2 pro with very high internet speed. They use ethernet.

As per user, mostly it happens when they are on video call with many other user's.

In very few time they've seen crashes during sleep. Let me check with these user's if they can collect sysdignose during issues.

Crashes in NEFilterPacketInterpose createChannel
 
 
Q