Thanks for being a part of WWDC25!

How did we do? We’d love to know your thoughts on this year’s conference. Take the survey here

How to avoid my local server flows in Transparent App Proxy

I have written the Transparent App Proxy and can capture the network flow and send it to my local server. I want to avoid any processing on the traffic outgoing from my server and establish a connection with a remote server, but instead of connecting to the remote server, it again gets captured and sent back to my local server.

I am not getting any clue on how to ignore these flows originating from my server.

Any pointers, API, or mechanisms that will help me?

Answered by DTS Engineer in 835370022

Your transparent proxy provider has methods like handleNewFlow(_:) where it decides whether it wants to handle the flow or not. Those methods are given the flow, that is, an object of type NEAppProxyFlow [1]. That object has a metaData property, with information about the origin of the flow. This has a few properties for identifying that origin. The right one to use is platform specific [2]. On the Mac you want sourceAppAuditToken, which you can feed into the code signing machinery to identify the main executable of the process in which the flow originated.

That last step is something I’ve discussed a bunch of times here on the forums, for example, here. That post has a warning about how to securely identify code. Heed it! To learn more about designated requirements, see TN3127 Inside Code Signing: Requirements.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] Which is actually one its subclasses.

[2] While transparent proxies are only supported on the Mac, NEAppProxyFlow is also used by app proxies, which are supported on other platforms.

Accepted Answer

Your transparent proxy provider has methods like handleNewFlow(_:) where it decides whether it wants to handle the flow or not. Those methods are given the flow, that is, an object of type NEAppProxyFlow [1]. That object has a metaData property, with information about the origin of the flow. This has a few properties for identifying that origin. The right one to use is platform specific [2]. On the Mac you want sourceAppAuditToken, which you can feed into the code signing machinery to identify the main executable of the process in which the flow originated.

That last step is something I’ve discussed a bunch of times here on the forums, for example, here. That post has a warning about how to securely identify code. Heed it! To learn more about designated requirements, see TN3127 Inside Code Signing: Requirements.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] Which is actually one its subclasses.

[2] While transparent proxies are only supported on the Mac, NEAppProxyFlow is also used by app proxies, which are supported on other platforms.

Thank you for your prompt response and the clarification. I’ve reviewed your code, which appears to be quite effective in identifying the source application. However, I’m encountering an issue where retrieving the bundle ID for certain flows results in an error code 100001. Interestingly, the code works as expected for system applications.

Could you please advise on the correct approach to reliably obtain the PID and bundle ID for any app flow? I’d appreciate it if you could let me know if I’ve misunderstood or missed something in the implementation.

func bundleIDForAuditToken(_ tokenData: Data) -> String? {
	 
	    // Get a code reference.
	 
	    var codeQ: SecCode? = nil
	    var err = SecCodeCopyGuestWithAttributes(nil, [
	        kSecGuestAttributeAudit: tokenData
	    ] as NSDictionary, [], &codeQ)
	    guard err == errSecSuccess else {
	        return nil
	    }
	    let code = codeQ!
	 
	    // Convert that to a static code.
	 
	    var staticCodeQ: SecStaticCode? = nil
	    err = SecCodeCopyStaticCode(code, [], &staticCodeQ)
	    guard err == errSecSuccess else {
	        return nil
	    }
	    let staticCode = staticCodeQ!
	 
	    // Get code signing information about that.
	 
	    var infoQ: CFDictionary? = nil
	    err = SecCodeCopySigningInformation(staticCode, [], &infoQ)
	    guard err == errSecSuccess else {
	        return nil
	    }
	    let info = infoQ! as! [String:Any]
	 
	    // Extract the bundle ID from that.
	 
	    guard
	        let plist = info[kSecCodeInfoPList as String] as? [String:Any],
	        let bundleID = plist[kCFBundleIdentifierKey as String] as? String
	    else {
	        return nil
	    }
	    return bundleID
	}
retrieving the bundle ID for certain flows results in an error code 100001.

Right. Error 100001 is the Security framework version of EPERM:

% security error 100001
Error: 0x000186A1 100001 UNIX[Operation not permitted]

It’s likely that the App Sandbox or MAC is blocking your access to the relevant app. See On File System Permissions.

If it’s the App Sandbox, you can get around this limit using a temporary exception entitlement. See App Sandbox Resources.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Yes, you were right. The App sandbox was blocking it. Now I am able to extract bundleId, but I don't understand why I can't connect with the host and load the website. It would help if you reviewed the code and guided me further if I were missing something.

import Foundation
import NetworkExtension
import os.log

class AppProxyProvider: NETransparentProxyProvider {
    
    private let log = OSLog(subsystem: "com.proxy.tcp.network.extension", category: "provider")
    
    override func startProxy(options: [String: Any]?, completionHandler: @escaping (Error?) -> Void) {
        os_log(.info, log: log, "Starting Transparent Proxy %{public}@", "")
        
        // Configure the network settings for the tunnel
        let proxySettings = NETransparentProxyNetworkSettings(tunnelRemoteAddress: "127.0.0.1")
        
        // Include both TCP and UDP rules for complete coverage
        proxySettings.includedNetworkRules = [
            NENetworkRule(
                remoteNetwork: nil,
                remotePrefix: 0,
                localNetwork: nil,
                localPrefix: 0,
                protocol: .TCP,
                direction: .outbound
            ),
            NENetworkRule(
                remoteNetwork: nil,
                remotePrefix: 0,
                localNetwork: nil,
                localPrefix: 0,
                protocol: .UDP,
                direction: .outbound
            )
        ]
        
        // Apply the settings
        setTunnelNetworkSettings(proxySettings) { error in
            if let error = error {
                os_log(.error, log: self.log, "Failed to apply tunnel settings: %{public}@", error.localizedDescription)
            } else {
                os_log(.info, log: self.log, "Tunnel settings applied successfully %{public}@", "")
            }
            completionHandler(error)
        }
    }
    
    override func stopProxy(with reason: NEProviderStopReason, completionHandler: @escaping () -> Void) {
        os_log(.info, log: log, "Stopping proxy with reason: %{public}d", reason.rawValue)
        
        // Cancel any ongoing operations
        setTunnelNetworkSettings(nil) { error in
            if let error = error {
                os_log(.error, log: self.log, "Error clearing tunnel settings: %{public}@", error.localizedDescription)
            }
            completionHandler()
        }
    }
    
    override func handleAppMessage(_ messageData: Data, completionHandler: ((Data?) -> Void)?) {
        guard let message = String(data: messageData, encoding: .utf8) else {
            os_log(.error, log: log, "Received invalid message data %{public}@", "")
            completionHandler?(nil)
            return
        }
        
        os_log(.debug, log: log, "Received message: %{public}@", message)
        
        // Handle specific message types
        if message == "get_mapping" {
            let responseMessage = "No active mappings"
            completionHandler?(responseMessage.data(using: .utf8))
        } else {
            // Default response
            completionHandler?(messageData)
        }
    }
    
    override func handleNewFlow(_ flow: NEAppProxyFlow) -> Bool {
        os_log(.debug, log: log, "Handling new flow of type: %{public}@", String(describing: type(of: flow)))
        
        // Handle TCP flows
        if let tcpFlow = flow as? NEAppProxyTCPFlow {
            guard let remoteEndpoint = tcpFlow.remoteEndpoint as? NWHostEndpoint else {
                os_log(.error, log: log, "Unable to cast remote endpoint to NWHostEndpoint %{public}@", "")
                return false
            }
            
            os_log(.info, log: log, "New TCP flow to %{public}@:%{public}@", remoteEndpoint.hostname, remoteEndpoint.port)
            
            if let token = flow.metaData.sourceAppAuditToken {
                let bundleIdFromData = self.bundleIDForAuditToken(token)
                
                //My this app bundle ID
                if let bundleID = bundleIdFromData, bundleID.contains("com.promobi.TransparentProxyMacApril") {
                    os_log(.error, log: log, "Skipping the flow as it is not from my app %{public}@", bundleID)
                    return false
                }
            }
            
            // Create a TCP connection manager for this flow
            let connectionManager = TransparentProxyManager(flow: tcpFlow, endpoint: remoteEndpoint)
            connectionManager.startExchangingData()
            
            return true
        }
        // Handle UDP flows if needed
        else if let udpFlow = flow as? NEAppProxyUDPFlow {
            os_log(.info, log: log, "UDP flow received - not handling in this implementation %{public}@", "")
            return false
        }
        
        // Unsupported flow type
        os_log(.error, log: log, "Unsupported flow type: %{public}@", String(describing: type(of: flow)))
        return false
    }
    
    func bundleIDForAuditToken(_ tokenData: Data) -> String? {
        // Create logger
        let log = OSLog(subsystem: "com.yourcompany.bundleid", category: "AuditToken")
        
        // Get a code reference.
        var codeQ: SecCode? = nil
        var err = SecCodeCopyGuestWithAttributes(nil, [
            kSecGuestAttributeAudit: tokenData
        ] as NSDictionary, [], &codeQ)
        
        guard err == errSecSuccess else {
            os_log("Failed to copy guest with attributes: %{public}d", log: log, type: .error, err)
            return nil
        }
        let code = codeQ!
        
        // Convert that to a static code.
        var staticCodeQ: SecStaticCode? = nil
        err = SecCodeCopyStaticCode(code, [], &staticCodeQ)
        guard err == errSecSuccess else {
            os_log("Failed to copy static code: %{public}d", log: log, type: .error, err)
            return nil
        }
        let staticCode = staticCodeQ!
        
        // Get code signing information about that.
        var infoQ: CFDictionary? = nil
        err = SecCodeCopySigningInformation(staticCode, [], &infoQ)
        guard err == errSecSuccess else {
            os_log("Failed to copy signing information: %{public}d", log: log, type: .error, err)
            return nil
        }
        let info = infoQ! as! [String:Any]
        
        // Extract the bundle ID from that.
        guard
            let plist = info[kSecCodeInfoPList as String] as? [String:Any],
            let bundleID = plist[kCFBundleIdentifierKey as String] as? String
        else {
            os_log("Failed to extract bundle ID from info dictionary", log: log, type: .error)
            return nil
        }
        
        os_log("Successfully extracted bundle ID: %{public}@", log: log, type: .debug, bundleID)
        return bundleID
    }
}
private func stateChangedCallback(to state: NWConnection.State) {
        switch state {
        case .ready:
            os_log(.debug, log: self.log, "TransparentProxyManager::stateChangedCallback. TCP connection is ready %{public}@", "")
            // Call the correct open method
            flow.open(withLocalEndpoint: nil) { error in
                if let error = error {
                    os_log(.error, log: self.log, "TCP flow opening failed %{public}@", error.localizedDescription)
                    self.connection.cancel()
                    return
                }
                os_log(.debug, log: self.log, "TCP flow opened successfully, starting data exchange %{public}@", "")
                self.handleOutgoingTCPData()
            }
            
        case .failed(let error):
            os_log(.error, log: self.log, "TransparentProxyManager::stateChangedCallback. TCP connection failed %{public}@", error.localizedDescription)
            self.connection.cancel()
            self.flow.closeReadWithError(error)
            self.flow.closeWriteWithError(error)
            
        case .cancelled:
            os_log(.debug, log: self.log, "TransparentProxyManager::stateChangedCallback. TCP connection is cancelled %{public}@", "")
            self.flow.closeReadWithError(nil)
            self.flow.closeWriteWithError(nil)
            
        case .preparing:
            os_log(.debug, log: self.log, "TransparentProxyManager::stateChangedCallback. TCP connection is preparing %{public}@", "")
            
        case .setup:
            os_log(.debug, log: self.log, "TransparentProxyManager::stateChangedCallback. TCP connection is in setup state %{public}@", "")
            
        case .waiting(let error):
            os_log(.error, log: self.log, "TransparentProxyManager::stateChangedCallback. TCP connection is in waiting state, %{public}@", error.localizedDescription)
            self.connection.cancel()
            self.flow.closeReadWithError(error)
            self.flow.closeWriteWithError(error)
            
        default:
            os_log(.debug, log: self.log, "TransparentProxyManager::stateChangedCallback. State is unknown %{public}@", "")
        }
    }
private func handleOutgoingTCPData() {
        os_log(.debug, log: self.log, "handleOutgoingTCPData. handling TCP flow data %{public}@", "")
        
        flow.readData { [weak self] data, error in
            guard let self = self else { return }
            
            if let error = error {
                os_log(.error, log: self.log, "handleOutgoingTCPData. Error on handling TCP flow data: %{public}@", error.localizedDescription)
                self.connection.cancel()
                self.flow.closeReadWithError(error)
                self.flow.closeWriteWithError(error)
                return
            }
 
            if let data = data, !data.isEmpty {
                // Check if this is the first packet
                if self.isFirstPacket {
                    self.isFirstPacket = false
                    
                    // Determine if this is HTTPS (port 443) or HTTP (port 80)
                    let isHttps = self.connection.endpoint.debugDescription.contains(":443")
                    let isHttp = self.connection.endpoint.debugDescription.contains(":80")
                    
                    if isHttps {
                        // Try to extract domain from TLS ClientHello (SNI)
                        if let domain = self.extractSNI(from: data) {
                            self.extractedDomain = domain
                            os_log(.debug, log: self.log, "Extracted SNI domain: %{public}@", domain)
                            
                            // Check if domain is allowed
                            self.checkDomain(domain) { allowed in
                                if !allowed {
                                    os_log(.info, log: self.log, "Blocking HTTPS connection to: %{public}@", domain)
                                    self.blockConnection = true
                                    self.connection.cancel()
                                    self.flow.closeReadWithError(nil)
                                    self.flow.closeWriteWithError(nil)
                                    return
                                }
                                
                                // Domain is allowed, continue with the connection
                                self.sendData(data)
                            }
                            return
                        } else {
                            os_log(.debug, log: self.log, "Could not extract SNI from HTTPS connection, allowing anyway %{public}@", "")
                        }
                    } else if isHttp {
                        // Try to extract Host header from HTTP request
                        if let domain = self.extractHTTPHost(from: data) {
                            self.extractedDomain = domain
                            os_log(.debug, log: self.log, "Extracted HTTP Host: %{public}@", domain)
                            
                            self.checkDomain(domain) { allowed in
                                if !allowed {
                                    os_log(.info, log: self.log, "Blocking HTTP connection to: %{public}@", domain)
                                    self.blockConnection = true
                                    self.connection.cancel()
                                    self.flow.closeReadWithError(nil)
                                    self.flow.closeWriteWithError(nil)
                                    return
                                }
                                
                                // Domain is allowed, continue with the connection
                                self.sendData(data)
                            }
                            return
                        } else {
                            os_log(.debug, log: self.log, "Could not extract Host from HTTP connection, allowing anyway %{public}@", "")
                        }
                    }
                }
                
                // If we reach here, either:
                // 1. Not the first packet, or
                // 2. Couldn't extract domain, or
                // 3. Not HTTP/HTTPS traffic
                
                if !self.blockConnection {
                    self.sendData(data)
                }
            } else if data?.isEmpty == true {
                // Empty data but not an error - continue reading
                self.handleOutgoingTCPData()
            } else {
                // No data, likely EOF
                if !self.blockConnection {
                    os_log(.debug, log: self.log, "No more outgoing data, switching to reading incoming data %{public}@", "")
                    self.handleIncomingTCPData()
                }
            }
        }
    }

@DTS Engineer

I don't understand why I can't connect with the host and load the website.

I’m not really in a position to review a giant chunk of code like that. I recommend that you break this problem down into parts:

  1. Does your provider load?

  2. Do flows actually make it to its handle-new-flow methods?

  3. If it returns false from all the handle-new-flow methods, does everything still work?

  4. If you create a small test program that starts a TCP connection, does the proxy see that flow?

  5. If you then modify it to return true from the handle-new-flow method, are you able to read data that your test program sends?

  6. And is your test program able to read data that the proxy sends?

  7. Add code to your proxy to make outbound connections. Can that connect to a test server?

  8. And can you send data that connection and receive it at the test server?

  9. And likewise for the return path?

  10. Wire up that connection to you flow code. Is your test client now able to communicate with your test server?

  11. Now open up the proxy to other flows, like those from a simple built-in programs like nc, curl, and nscurl. Can you get those working?

None of this is rocket science. It’s just implementing the code methodically and testing as you go.

I also recommend that you bring up your more complex code outside of the NE environment. See the But Should You? section of Debugging a Network Extension Provider.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

I guess I missed the next step in the path, namely WKWebView. If you create a small web view app and load the page there, does that work?

ps It’s better to reply as a reply, rather than in the comments; see Quinn’s Top Ten DevForums Tips for this and other titbits.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

How to avoid my local server flows in Transparent App Proxy
 
 
Q