Foursquare constantly tracking users' locations
(and how to audit your phone's application traffic yourself)
It might sound weird to accuse Foursquare of collecting location data since that is the whole point of the service, but Foursquare is overstepping its bounds by constantly keeping track of their users' every move (and more) -- even if they never open the app.
The Foursquare app contacts the service every ten minutes, providing a list of minute-by-minute locations (including timestamp and accuracy data), battery level, charging status, internet connectivity status and nearby wireless access points (complete with timestamp, MAC address and signal strength).
Also, Foursquare uses your mobile device’s ‘background location’ (formerly known as ‘Radar’) to provide the service, including to send you notifications of tips/friends/ interesting things etc. near you. If you have ‘background location’ turned on, the Foursquare app will, from time to time, tell us about your device’s location even if you are not directly interacting with the application.
They even addressed the concern that this might impact battery life on their users' phone using the data they collected, as reported by TechCrunch:
The new and improved push recommendation feature purportedly only increases battery drain by about 0.7 percent per hour — or, “the equivalent of about a 20-minute game of Angry Birds” over the course of a day.
However, the amount of data collected in this manner is, at the very least, unnecessary. Here's a sample update, lightly modified for legibility and privacy. All coordinates, access point names and MAC addresses were replaced with fictional values. I also transformed the request from
json and split a couple of semicolon-separated strings into arrays, for legibility.
Besides giving us some insights on how their tracking works (
triggers object in the response) and how it detects whether the user is on the move (
stopDetect), this updates shows just how much data is being shared with Foursquare behind the scenes.
Most of the data is self-explaining, but here's what I could gather by watching a few of these updates:
batteryStatus-- one of
batteryStrength-- amount of battery charge left (between 0 and 1)
coarseLLTimestamp-- current (coarse?) location, accuracy and timestamp
history-- set of semicolon-delimited location data points, consisting of timestamp, accuracy and coordinates
coarseLLAccin all requests I logged. Probably contains finer (GPS) data when available
wifiScan-- set of semicolon-delimited spotted access point data, consisting of timestamp, SSID (network name), BSSID (MAC address) and signal strength
One might wonder if there's a better, less invasive way to deliver the same kind of notifications. Here's a suggestion: have the app download all of the nearby location-specific notifications, then decide locally when to show them. Less creepy and more battery efficient. Win-win.
For now, there is a way to disable this behavior, buried three menus deep under
Settings > Account Settings > Privacy:
Auditing your phone's application traffic yourself
Of course, Foursquare is probably not the only app doing this, which is why you might be motivated to audit your own phone's traffic.
Observing HTTP traffic from a smartphone is simple enough -- fire up your proxy of choice, set it up as a transparent proxy or on the device and watch the logs. That's all there is to it. On the other hand, most interesting traffic occurs over HTTPS for many reasons, including privacy and security. The Foursquare requests described in this article, for example, occur over HTTPS and are not easily intercepted with a common proxy server or packet sniffer.
This section aims to illustrate how to capture HTTP and (most) HTTPS traffic originated by your device, which can be quite useful while auditing third party applications or debugging your own.
Note that not all traffic can be intercepted this way. Fiddler only supports only HTTP, HTTPS and SPDY, which means an application could communicate without showing up in Fiddler by using a different protocol (e.g. a raw or TLS socket). Some applications using certificate pinning might refuse to communicate with this interception in place (e.g. Dropbox and GMail).
Fiddler -- an excellent HTTP debugger by Eric Lawrence / Telerik.
Certificate Maker -- a Fiddler add-on that generates interception certificates compatible with iOS and Android devices. This is only required if you care about HTTPS traffic.
Setting up Fiddler
After installing Fiddler and Certificate Maker, you should be greeted by Fiddler's main window:
HTTPS decryption is off by default in Fiddler. Here are the appropriate settings for this guide (under
Tools > Fiddler Options):
You might need to restart Fiddler after changing these settings. After restarting, click the
Capturing icon on the status bar in order to ignore local traffic.
Setting up your device
This guide shows the steps for Android 4.4, but most devices have similar procedures (including iOS devices).
First, open a browser and navigate to your Fiddler instance. The default port for Fiddler is
FiddlerRoot certificate link, name your certificate and save the changes. This allows Fiddler to behave as a certification authority, generating certificates for intercepted websites on the fly.
Android 4.4 now (rightfully) warns the user when a custom root certificate is installed with a persistent notification. Kudos to Google for that. Since we are the bad guys Google is warning us about, there is no need to be concerned.
Finally, change your proxy settings to have your traffic go through Fiddler.
That's it! You should now be able to see requests originated by your phone on Fiddler. Here's an example session:
Try Snapchat if you want to learn how their API works, or any "free flashlight" app if you're feeling brave. Remember to remove all custom settings after exploring.