Facebook has submitted a patent application for technology that would predict who your family and other household members are, based on images and captions posted to Facebook, as well as your device information, like shared IP addresses. The application, titled “Predicting household demographics based on image data,” was originally filed May 10, 2017, and made public today.

Facebook did not immediately respond to a request for comment, but the patent suggests that the company is interested in exploring the technology, which is intended to help Facebook target advertising more effectively. After the story's publication, a Facebook spokesperson said, "We often seek patents for technology we never implement, and patents should not be taken as an indication of future plans.”

Facebook submitted the application before this year’s security and privacy scandals — Cambridge Analytica, a massive hack, and backlash against its recent hardware product — but the patent’s publication comes at time when the social media giant is grappling with the public’s growing distrust.

The system Facebook proposes in its patent application would use facial recognition and learning models trained to understand text to help Facebook better understand whom you live with and interact with most. The technology described in the patent looks for clues in your profile pictures on Facebook and Instagram, as well as photos of you that you or your friends post.

It would note the people identified in a photo, and how frequently the people are included in your pictures. Then, it would assess information from comments on the photos, captions, or tags (#family, #mom, #kids) — anything that indicates whether someone is a husband, daughter, cousin, etc. — to predict what your family/household actually looks like.

According to the patent application, Facebook’s prediction models would also analyze “messaging history, past tagging history, [and] web browsing history” to see if multiple people share IP addresses (a unique identifier for every internet network).

