TL;DR: The problem that exists starting at least from iOS 8: any app, that you gave access to your photos and videos, can see ALL of them, including those in your “Hidden” album and iCloud. With the new iOS 11 machine learning features, it became super-easy for any application to steal sensitive information from the photos and videos.

I’ve written a proof-of-concept iOS app that finds all your ‘hidden’ NSFW pictures. No private APIs were used, so the app will likely pass Apple review and will be released on the AppStore. Here is the code on the Github. I’m thinking about adding a Facebook integration now.

I did this, because I wanted to demonstrate, how easy it is to abuse user’s security and privacy under current iOS permission management system.

How Photo Permissions Work

No doubts you all have seen this type of messages:

Messenger wants all your photos and videos

Most of the users just always press “OK” without ever reading it. Some of you even know that you can reclaim those permissions back in iPhone settings:

Photo access management

By the way, this is a screenshot from iOS 11 beta, where they have added a new option: “Add photos only”. By some reason, it is available only for some apps, and they just stop working if you opt it.

Now, what exactly can an app do with your photos? Anything: analyze in a background, compress, send to their servers and so on. You will never know because by pressing “OK” button you have agreed on all of those.

“Hidden” Album

In iOS 8 Apple introduced a new awesome security and privacy feature: “Hidden album.” So now you can safely store your photos in that album, right?

How to hide your photo

Hidden album in Photo app

No, you can’t. Any application that has access to your photo library also has access to the content of this folder. Even worse, by putting your sensitive photos into that album you are effectively marking them as interesting, so the malicious apps can say to the system: “Hey, give me the whole folder of those interesting photos.” And user will never know, that the app has just uploaded all his “hidden” photos to its servers. If you’re an iOS developer, you can check it yourself. That’s how what official documentation says about this feature:

What’s New in iOS 11

Ok, there was a major privacy and security hole for several years, but now in iOS 11, they have fixed this, right? Let’s see, how that had changed in the iOS 11 beta.

First of all, now I can’t reclaim permission I gave to Skype to see my photos because it’s just is not on the list of the apps, which have those privileges. I checked in Skype itself, and it was able to see all my photos. Maybe it’s just a bug in a beta version, but the alarming one: it signals that permission management system is poorly designed and is a full of bugs. Hidden album content is still visible in iOS 11 beta via Photos framework.

In iOS 11 Apple introduced a simple way to deploy neural networks and other machine learning models to your applications — CoreML framework. So now you don’t need to upload user’s photo to the cloud to do analysis there, you can do all analysis on device and users are paying electricity bills.

So, I put together a simple app that analyses the content of you ‘hidden’ album and rates all photos according to their ‘NSFW’-ness. The porn detection is done using a deep neural network. The whole app development took like an hour or two. I used a Yahoo NSFW neural network, converted it to CoreML and drag-and-dropped to the iOS project. A few lines of code and that’s it. Read the Yahoo blog post if you’re interested in how it works.

More to this, Apple provides separate APIs to get all your screenshots (what for I wonder?), all your selfies (including hidden NSFW selfies), APIs to analyze if the photo contains barcodes, QR codes, texts, faces and so on. Not speaking about GPS location which is saved in each photo’s EXIF metadata if you haven’t disabled it manually. It also provides an API to download photos from the iCloud if they are not on the device currently.

I’m not against machine learning (ML). It is great, and I used it in my apps even before the CoreML. But one should understand that it adds a lot of room for the privacy and security breaches. In fact, if you check the databases of scientific papers, you would see, there are a plenty of them about stealing some data from user’s smartphone using ML:

Stealing the PIN code

Keylogging

Voice recognition via the motion sensors

Tricking the voice assistant with human-unintelligible commands

Face and emotions recognition

and so on and so forth

Any application that has permission to your photo library (namely: Skype, Twitter, Facebook, Snapchat, Prisma, etc.) can extract info not only about you but also about anyone, who had a mischance to be in your photos.

It’s not about Porn

Researchers from Max Planck Institute for Informatics successfully used a neural network to extract the following personal information from the smartphone photo libraries:

Personal Description

Gender

Eye Color

Hair Color

Fingerprint

Signature

Face

Tattoo

Nudity

Race

(Skin) Color

Traditional Clothing

Full Name

First Name

Last Name

Place of Birth

Date of Birth

Nationality

Handwriting

Marital status

Religion

Sexual Orientation

Culture

Hobbies

Sports

Personal Occasion

General Opinion

Political Opinion

Personal Relationships

Social Circle

Professional Circle

Competitors

Spectators

Similar view

Education history

Visited Landmark

Visited Location

Home address

Date/Time of Activity

Phone no.

Username

Email address

Email content

Online conversations

Legal involvement

Vehicle Ownership

License Plate

Employment

Occupation

Work Occasion

Health

Physical disability

Medical Treatment

Medical History

Documents

National Identification

Credit Card

Passport

Drivers License

Student ID

Mail

Receipts

Tickets

Check this paper for more details:

Orekondy, T., Schiele, B., Fritz, M. (2017). Towards a Visual Privacy Advisor: Understanding and Predicting Privacy Risks in Images. arXiv

Now, next time you are giving a viral selfie-editing app access to your photos, be aware, that you just had given access to all this plenty of private information.

With all these great features in iOS 11, I don’t recommend anyone to install it until Apple introduces a way to control access to your data. Unfortunately, at the moment things look like Apple sees privacy as “just another feature,” and that’s why it always lags behind the stickers and emojis.

One More Thing…

Finally, I want to cite several people at Apple: