Private data is still hidden in the logs with System-wide Enable-Private-Data

I have installed the following configuration profile:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>PayloadContent</key>
    <array>
        <dict>
            <key>PayloadDisplayName</key>
            <string>Enable Private Data Logging for Unified Logging</string>
            <key>PayloadEnabled</key>
            <true/>
            <key>PayloadIdentifier</key>
            <string>com.apple.system.logging.2BFB8109-8829-4020-AEB7-BA21761AE50C</string>
            <key>PayloadType</key>
            <string>com.apple.system.logging</string>
            <key>PayloadUUID</key>
            <string>2BFB8109-8829-4020-AEB7-BA21761AE50C</string>
            <key>PayloadVersion</key>
            <integer>1</integer>
            <key>System</key>
            <dict>
                <key>Enable-Private-Data</key>
                <true/>
            </dict>
        </dict>
    </array>
    <key>PayloadDisplayName</key>
    <string>Enable Private Logging Data</string>
    <key>PayloadIdentifier</key>
    <string>Kentzo-Macbook.D000DF5D-AE7A-4D22-B1DC-8F9CD71A2DD2</string>
    <key>PayloadRemovalDisallowed</key>
    <false/>
    <key>PayloadType</key>
    <string>Configuration</string>
    <key>PayloadUUID</key>
    <string>1CF75441-D3C2-4E5B-B36A-394C397E8529</string>
    <key>PayloadVersion</key>
    <integer>1</integer>
    <key>ConsentText</key>
    <dict>
        <key>default</key>
        <string>Warning: Installing this profile will enable private data logging for all of unified logging.</string>
    </dict>
</dict>
</plist>

But both Console.app and log show values like <mask.hash: 'Z9xIxlLTn0KlWPUjmpOSkg=='> for the com.apple.mDNSResponder subsystem.

What do I need to do to reveal this information?

Answered by DTS Engineer in 834611022

Enabling private data records… well… data that’s marked as private. There are other privacy levels, and that’s what you’re seeing here. Specifically, the data is flagged as sensitive. Such data will never be recorded in the clear on production versions of macOS. mDNSResponder then opts in to mask hashes, which is why you see that hash. This is kinda cool, because it lets you correlate values across log entries without actually recording the sensitive value.

Anyway, for more on this see:

  • The OSLogPrivacy type in the Swift logging API. This is the best explanation of this stuff that I’m aware of (sadly the man pages for the C API seem to be lacking info on this topic).

  • The actual code used by mDNSResponder for this, available in the Darwin open source. See here.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Accepted Answer

Enabling private data records… well… data that’s marked as private. There are other privacy levels, and that’s what you’re seeing here. Specifically, the data is flagged as sensitive. Such data will never be recorded in the clear on production versions of macOS. mDNSResponder then opts in to mask hashes, which is why you see that hash. This is kinda cool, because it lets you correlate values across log entries without actually recording the sensitive value.

Anyway, for more on this see:

  • The OSLogPrivacy type in the Swift logging API. This is the best explanation of this stuff that I’m aware of (sadly the man pages for the C API seem to be lacking info on this topic).

  • The actual code used by mDNSResponder for this, available in the Darwin open source. See here.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Thank you.

Assuming I know what I'm looking for (I'm debugging why macOS does not display my Wire-Area DNS-SD services), can I calculate the hash separately to search the logs?

Assuming I know what I'm looking for (I'm debugging why macOS does not display my Wire-Area DNS-SD services), can I calculate the hash separately to search the logs?

No. From comments in the code Quinn linked to:

"Note that the hash performed to the data is a salted hashing transformation, and the salt is generated randomly on a per-process basis, meaning that hashes cannot be correlated across processes or devices."

Having said that, when you're working on a "active" issue as you are, what you can do is rely on intentional correlation. That is, when you're app takes a particular action, that results in specific logging from mDNSResponder. If that logging includes specific hash data, well, that's "your" data.

One thing to try here is specifically logging to the console before and after "interesting" points in your code. Those log message then create clear sections in the console log which are most likely to be tied to your app.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Thank you Kevin.

I just found that there is the mDNSResponder configuration profile provided by Apple, which has the following payload:

	<key>PayloadContent</key>
	<array>
		<dict>
			<key>PayloadDisplayName</key>
			<string>Logging Payload For mDNSResponder/srp-mdns-proxy.</string>
			<key>PayloadIdentifier</key>
			<string>com.apple.system.logging.ED3E600C-83D8-44D0-BF5B-8A7F889CDBDE</string>
			<key>PayloadType</key>
			<string>com.apple.system.logging</string>
			<key>PayloadUUID</key>
			<string>ED3E600C-83D8-44D0-BF5B-8A7F889CDBDE</string>
			<key>PayloadVersion</key>
			<integer>1</integer>
			<key>Subsystems</key>
			<dict>
				<key>com.apple.mDNSResponder</key>
				<dict>
					<key>DEFAULT-OPTIONS</key>
					<dict>
						<key>Enable-Oversize-Messages</key>
						<true/>
						<key>Level</key>
						<dict>
							<key>Enable</key>
							<string>Info</string>
							<key>Persist</key>
							<string>Info</string>
						</dict>
						<key>Privacy-Enable-Level</key>
						<string>Sensitive</string>
					</dict>
				</dict>
				<key>com.apple.mdns</key>
				<dict>
					<key>DEFAULT-OPTIONS</key>
					<dict>
						<key>Enable-Oversize-Messages</key>
						<true/>
						<key>Level</key>
						<dict>
							<key>Enable</key>
							<string>Info</string>
							<key>Persist</key>
							<string>Info</string>
						</dict>
						<key>Privacy-Enable-Level</key>
						<string>Sensitive</string>
					</dict>
				</dict>
				<key>com.apple.srp-mdns-proxy</key>
				<dict>
					<key>DEFAULT-OPTIONS</key>
					<dict>
						<key>Enable-Oversize-Messages</key>
						<true/>
						<key>Level</key>
						<dict>
							<key>Enable</key>
							<string>Debug</string>
							<key>Persist</key>
							<string>Info</string>
						</dict>
						<key>Privacy-Enable-Level</key>
						<string>Private</string>
					</dict>
				</dict>
			</dict>
		</dict>
	</array>

I wonder whether the value of <string>Sensitive</string> alongside the fact that the profile is signed by Apple allows to unmask the redacted parts in the logs.

Update

Looks like it worked! I understand it did not enable compile-time exclusions, but other than that seems to work. Or do I miss something?

Assuming I'd want to provide a similar profile for my own app, what are the signing requirements for the Unified Logging System to recognize it and reveal masked values? Is this even available to general audience?

Assuming I'd want to provide a similar profile for my own app, what are the signing requirements

That depends on your target platform:

  • macOS does not require that configuration profiles be signed.

  • iOS and its child platforms only accept profiles signed by Apple.

I talk about this more in Your Friend the System Log.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Private data is still hidden in the logs with System-wide Enable-Private-Data
 
 
Q