In a past article I have taught how to create a simple bot for instagram … And today we are going to add an improvement using convolutional neural networks for the detection of content in an image.

At the time we created a bot looking at food hashtags and gave likes with a pseudo-human behavior. At the time we created a bot looking at food hashtags and gave likes with a pseudo-human behavior. Today we are going to use existing artificial intelligence models, to give appropriate comments in the correct photos.

One of the main problems of bots that post comments on instagram without taking into account the content of the image. We will detect the content of the image and comment accordingly, giving a very specific comment, masking that there really is no human behind the account

How our AI works

Our artificial intelligence is a convolutional neural network that is trained in the Imagenet dataset. We will use the VGG16 model to predict our images that can be categorized into 1000 labels.

Specifically we will comment on photos of cheeseburgers! What is a category available in the VGG16 model to predict.

But not everything is beautiful and perfect, even though in an image it contains a cheeseburger, it will not always detect it with a high percentage. That is why we will only use the predicted images as cheeseburgers with a percentage higher than 85%.

What does VGG16 think is a cheeseburger (Instagram #food pictures)

Let’s get started with code

We will make our predictions of VGG16 in python, for its simplicity when using anything related to AI. We must have Tensorflow and Keras installed.

With this simple piece of code, we can make predictions in the VGG16 model, passing a photo as a parameter of our script.

vggModel.py

It is so easy to use, how to run it on a console and separate the path of your photograph by a space. We can make console predictions this way.

python vggModel.py ./testImage.jpg

If we use the VGG model in this photograph:

cheese burger — testImage.jpg

The result we get from the prediction of the VGG16 model is 99% in this photograph. Specifically this is the terminal output:

cheeseburger-99.28165674209595

And now it’s time to start creating our bot using tools-for-instagram in javascript. I’m going to skip some parts of the explanation of how to create the configuration to use tools-for-instagram, because it’s already written in this article.

First of all we are going to write a file that allows us to make python calls from our javascript script. This task is very simple since nodejs brings the appropriate library: “child_process”

detectfood.js

And now we can start writing the code that will handle the workflow of our bot in instagram.

We will install a package that will help us download the images of a url for later processing with VGG16

npm install image-downloader --save

With this code we make the prediction of the photographs and the download.

bot.js

we will continue with the exploration of the hashtag #foodporn, lets download all images …

bot.js

These are some of the images downloaded by our bot, we can see that there are many images of cheeseburgers … Let’s start to classify them with the VGG16 model

donwloaded pictures

Now we can make a simple conditional to know when our label is “cheeseburger” and the probability is greater than 85.00%

bot.js

Let’s see the bot in action!

bot.js

These are the photos selected by the bot to comment.

100% acuraccy on VGG16

Now there is only the part of generating our comments, for which I will create a dictionary of possible comments that the bot can use.

To generate these sentences, I will rely on an article that I have written in the past, for this purpose, you can read it by clicking here. These are some of the sentences generated

burgerSentences.txt

Read the txt file, convert it to an array and choose a random sentence to comment on the post. And that’s how our final code looks …

bot.js

We are ready to run our bot. To illustrate how our bot works, I leave you with an animation of how it acts within the hashtag #foodporn

simulation

Not only can we comment on pictures of cheeseburgers, we have a wide range of classes recognized by VGG16.

Conclusions

Today the vast majority of people are able to see the difference between an automated account and an account run by a real person. A common way to detect automated accounts or bots, are comments … It is rare to see how some accounts comment “nice pic” in videos… And many other examples, which surely you could see with your own eyes

All the code is available github. Thank you very much for your time and I hope you found it useful.