Memory Zeroing Issue After iOS 18 Update

After iOS 18, some new categories of crash exceptions appeared online, such as those related to the sqlite pcache1 module, those related to the photo album PHAsset, those related to various objc_release crashes, etc. These crash scenarios and stacks are all different, but they all share a common feature, that is, they all crash due to accessing NULL or NULL addresses with a certain offset. According to the analysis, the direct cause is that a certain pointer, which previously pointed to valid memory content, has now become pointing to 0 incorrectly and mysteriously.

We tried various methods to eliminate issues such as multi-threading problems. To determine the cause of the problem, we have a simulated malloc guard detection in production. The principle is very simple:

  1. Create some private NSString objects with random lengths, but ensure that they exceed the size of one memory physical page.
  2. Set the first page of memory for these objects to read-only (aligning the object address with the memory page).
  3. After a random period of time (3s - 10s), reset the memory of these objects to read/write and immediately release these objects. Then repeat the operation starting from step 1.

In this way, if an abnormal write operation is performed on the memory of these objects, it will trigger a read-only exception crash and report the exception stack.

Surprisingly, after the malloc guard detection was implemented, some crashes occurred online. However, the crashes were not caused by any abnormal rewriting of read-only memory. Instead, they occurred when the NSString objects were released as mentioned earlier, and the pointers pointed to contents of 0.

Therefore, we have added object memory content printing after object generation, before and after setting to read-only, and before and after reverting to read-write. The result was once again unexpected. The log showed that the isa pointer of the object became 0 after setting to read-only and before re-setting to read-write.

So why did it become 0 during read-only mode, but no crash occurred due to the read-only status?

We have revised the plan again. We have added a test group, in which after the object is created, we will mlock the memory of the object, and then munlock it again before the object is released. As a result, the test analysis showed that the test group did not experience a crash, while the crashes occurred entirely in the control group.

In this way, we can prove that the problem occurs at the system level and is related to the virtual memory function of the operating system. It is possible that inactive memory pages are compressed and then cleared to zero, and subsequent decompression fails. This results in the accidental zeroing out of the memory data.

As mentioned at the beginning, althougth this issue is a very rare occurrence, but it exists in various scenarios. definitely It appeared after iOS 18. We hope that the authorities will pay attention to this issue and fix it in future versions.

It seems like you have a memory corruption problem and have gone down the path of implementing your own memory debugging tools. That’s a hard path to go down, and it’s something I specifically recommend against if you plan to include that code in your production app. Memory debugging tools generally have to do all sorts of weird things, things that are engender significant binary compatibility liability.

Have you tried running the standard memory debugging tools against your app? That’s the first place to go with a problem like this.

It is possible that inactive memory pages are compressed and then cleared to zero, and subsequent decompression fails.

Well, anything’s possible, but that’s quite a reach. If iOS 18 had a VM bug like that, Apple would know about it. I can guarantee you that, because we have had bugs like this previously and it generated a lot of noise. Keep in mind that, when you operate at the scale of iOS, a ‘one in a million’ bug happens thousands of times a day O-:

As to the rest of your post, it’s hard for me to comment on the behaviour you’re seeing because you’re doing a bunch of gnarly low-level stuff but there isn’t a lot of details about how you’re doing it. For example, this is short on details:

2. Set the first page of memory for these objects to read-only (aligning the object address with the memory page).

How are you getting the first page of the memory for these objects? There’s no supported way to get at the backing buffer for an NSString in the general case.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Thank you for your reply. Certainly we have tried some memory debugging tools, but we didn't find any issues.

I also find it extremely hard to believe that there was a problem with the system. I have been trying to figure out what our own issue might be. "Simulated malloc guard detection" is somewhat like our final attempt.

Keep in mind that, when you operate at the scale of iOS, a ‘one in a million’ bug happens thousands of times a day O-:

We cannot reproduce this issue locally. However, with our daily user traffic reaching billions, approximately 100 reports are generated every day. (This issue has some notable characteristics. Certain models account for a large proportion, especially iPhone 14,5.)

I believe that the Apple receive an enormous number of abnormal reports every day. But how to determine whether these abnormalities are not caused by system issues or other common problems related to memory corruption? This is also why I think the feedback is necessary. If you saw some strange crashes, perhaps based on my feedback, you could consider the reasons again. Don't hastily assume that it's not a system issue but rather some common memory problem.

How are you getting the first page of the memory for these objects? There’s no supported way to get at the backing buffer for an NSString in the general case.

There is no need to find the backing buffer,we just protect memory pointed by string object pointer. Because we found that for a long NSString object we created, the object pointer points to a large block of memory. The memory starts with 8 bytes as the isa field, and then several bytes after that are all the string content. We set the first page of memory containing the isa field as read-only. (I know this is a bit tricky)

    if ((uintptr_t)string % _cachePageSize == 0 && malloc_size((__bridge void *)string) > _cachePageSize) {

We simply use string objects that meet the above conditions.

Memory Zeroing Issue After iOS 18 Update
 
 
Q