[EndpointSecurity] Reliable way to detect a file has been downloaded from the internet

I have tried a few different approaches but none of them were 100% reliable.

I tried subscribing to the ES_EVENT_TYPE_NOTIFY_SETEXTATTR event to detect when a process sets the kMDItemWhereFroms attribute on a file, but the problem is that any process can set that attribute on any file. For example, I can use the xattr -w com.apple.metadata:kMDItemWhereFroms "https://example.com" SampleFile.txt command into the terminal and that would trigger an ES event.

I also tried to listen for the com.apple.quarantine attribute but for some reason my ES client never receives an event for it, also, as with any attribute, this attriubte can be set manually by any process.

Is there a recommended/reliable way to have an ES client that detects a file has been downloaded from the internet (.i.e. a remote network)?

Is there a recommended/reliable way to have an ES client that detects a file has been downloaded from the internet (.i.e. a remote network)?

No, this cannot be done, at least not "reliably". The problem here is that there's a fundamental disconnect between how we describe basic computer interactions like:

  • Creating a file.
  • Copying a file.
  • Downloading a file.

...and what is actually "happening" when any given event occurs. In terms of what actually happened, all of the events above do exactly the same thing. That is, a process creates a file on disk and writes to it. Semantically, the difference between the three operations above is the "source" of the data, but, again, the idea of "source data" is a higher-level concept we're using to describe common behavior, not something "inherent" to the specific action.

As a concrete example of this, imagine that TextEdit opens a file with the content:

"Some Content"

Then later writes a different file somewhere else with the same content:

"Some Content"

That EXACT result could have been achieved by:

  1. "Creating a file” -> The user opening up a new window, typing "Some Content", and then saving it.

  2. "Copying a file” -> Choosing "Duplicate" from the File menu, then saving the new window TextEdit opened.

...but they'll basically look "identical" as far as your ES client is concerned, because they ARE in fact identical.

You're seeing another flavor of that issue here:

I tried subscribing to the ES_EVENT_TYPE_NOTIFY_SETEXTATTR event to detect when a process sets the kMDItemWhereFroms attribute on a file, but the problem is that any process can set that attribute on any file.

Of course. That attribute exists to provide a way for apps to "label" the source of the content they generate, but by its nature, all something like this can really "mean" is that a process chose to attach that attribute to that file. That label might still be quite useful, but asking for any stronger "guarantee" is asking for something the system cannot provide.

As a side note (and another example of this), despite the name, "ES_EVENT_TYPE_AUTH_COPYFILE" does NOT in fact prevent an app from copying anything. It can prevent an app from using the "copyfile()" API, but it won't prevent the app from simply reading data from one file and writing it to another.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

[EndpointSecurity] Reliable way to detect a file has been downloaded from the internet
 
 
Q