TensorFlow Object Detection Model

Below, we have broken down the full Tensorflow code into blocks for ease of understanding.

A) Download the TensorFlow Model

This part of the code downloads the model that you selected (through MODEL_NAME).

B) Build TensorFlow Graph and Variables

Now we can create a class that actually builds the network by downloading the model and storing it in a Tensorflow graph, which can be used to perform predictions. We will create different class variables linked to Tensorflow variables inside the graph. This allows us to access both input and output variables of the Tensorflow model using friendly python variables. You may notice that we have also defined a function for drawing rectangles on images. This will be used later on.

C) Define the Prediction Method.

Now, we add a new method to our class to do predictions. This method takes an image, feeds it into the input of our TensorFlow model, and evaluates the output variables by creating a TensorFlow Session. The object detection model has a number of useful output variables: the model outputs bounding boxes for all the objects it believes it has found in an image, and other than that, it outputs detection classes (i.e. what is the object, indexed from 1 to 90) and even its detection score, that encodes how confident it is that a certain object is actually present.

This is important because, as you will see, many objects have a very low score, and are indeed wrong. We can set a threshold that specifies which is the score level that we trust to be correct. Our threshold is rather low, only 0.5, because we are dealing with images of people giving their back to the camera, running, and being in quite noisy ambients. For visualizing bounding boxes, we use drawing functions from the Python Imaging Library (PIL) to draw rectangles directly on our image.

One important thing that we can see is that the bounding boxes coordinates are normalized between 0 and 1 as a float. This clearly cannot be directly converted into a pixel, but we can multiply our image width and height for this number, then casting it into an integer, to actually obtain the bounding boxes expressed as pixels in our image.

D) Run Prediction on Test Images

Now, we are going to select the images that we want to process. Clearly, on a drone, it would be a real-time process, where images go from a camera directly into memory and then processed by the program. In our case, to show some realistic examples, I took some screenshots of videos depicting people jogging or simply moving in front of a drone that is following them. I loaded them into a folder and used glob library to select all the paths. After this, I create an instance of our Prediction class and use it to perform detection on the images.

NanoNets Machine Learning API

Using NanoNets API, we can do prediction in only 2 steps:

A) Get your free API Key.

Get an API Key from http://app.nanonets.com/user/api_key. Note that you will need to sign up with your email address. Now, add your newly generated API key to the current terminal session with the following command:

export NANONETS_API_KEY=replace_your_api_key

B) Prediction

Copy the following code into a file “prediction.py”.

prediction.py: import requests, os, sys

from PIL import Image, ImageDraw model_id = "ca9df6e0-37a7-4155-bb8e-5ad2ccb675bf"

api_key = os.environ.get('NANONETS_API_KEY')

image_path = sys.argv[1]

url += model_id + '/LabelFile/'

data = {'file': open(image_path, 'rb'),'modelId': ('', model_id)}

response = requests.post(url, auth=requests.auth.HTTPBasicAuth(api_key, ''), files=data) url = ' https://app.nanonets.com/api/v2/ObjectDetection/Model/' url += model_id + '/LabelFile/'data = {'file': open(image_path, 'rb'),'modelId': ('', model_id)}response = requests.post(url, auth=requests.auth.HTTPBasicAuth(api_key, ''), files=data) result = response.json()["result"][0]

im = Image.open(image_path)

draw = ImageDraw.Draw(im)

for b in result["prediction"]:

box = (b["xmin"],b["ymin"],b["xmax"],b["ymax"])

draw.rectangle(box, fill=None, outline=128)

im.show()

print(response.text)

You can run the file “prediction.py” on any test image fed as a command line argument. We have uploaded some sample images for you to test.

python prediction.py path_to_image.jpg

The printed output gives you the location of the bounding box in json format, and a confidence score between 0 and 1 which represents how sure the model is about the object’s existence. In addition, the program creates a new image with the bounding box, so you can visually assess the performance of your model.

The results that we obtain are remarkable. Here you can see how some live action scenes were processed by this network.