Step 4: Reading an image and passing it to the TFLite model

Up next, we’ll use Pathlib to iterate through a folder containing some images that we’ll be running inference on. We’ll then read each image with OpenCV, resize it to 224x224, and pass it to our model.

Once done, we’ll print the file name and the output for that file:

Upon running this code, here’s what the output might look like:

For file DSC00117, the output is [[ 75 164 17]]

For file DSC00261, the output is [[252 2 2]]

For file DSC00248, the output is [[252 2 2]]

For file DSC00116, the output is [[ 21 210 25]]

For file DSC00128, the output is [[ 9 112 136]]

For file DSC00114, the output is [[203 42 12]]

As you can see, the output contains 3 values, which add up to 256. These represent the scores for the image, which indicate whether it’s underexposed, good, or overexposed, respectively.

To get your model’s predicted output, simply fetch the index with the greatest score and map it with your label.

For example, in the first image in the output above, the highest score is the second element in the array, which maps with the label “good”; whereas, the second image is “underexposed” and the second to last is “overexposed”.

To further improve model performance, you can batch the requests as well. Let’s see how to do this.

Step 5: Batching requests for better performance

Batching with TensorFlow essentially means running a bunch of inference requests at once instead of running them one by one.

This will result in a reduced latency in our model.

Before we go ahead and batch the requests, we need to decide upon the batch size. This might vary from machine to machine, and setting a very large batch size might result in an out of memory exception.

To make things simple, we will set the batch size as 50. Here’s what the above example with batching looks like:

But upon running the code above, you might see an error, which looks as follows:

Traceback (most recent call last):

File "test.py", line 48, in <module>

interpreter.set_tensor(input_details[0]['index'], input_data)

File "/Users/harshitdwivedi/Desktop/tf_env/lib/python3.7/site-packages/tensorflow/lite/python/interpreter.py", line 175, in set_tensor

self._interpreter.SetTensor(tensor_index, value)

File "/Users/harshitdwivedi/Desktop/tf_env/lib/python3.7/site-packages/tensorflow/lite/python/interpreter_wrapper/tensorflow_wrap_interpreter_wrapper.py", line 136, in SetTensor

return _tensorflow_wrap_interpreter_wrapper.InterpreterWrapper_SetTensor(self, i, value)

ValueError: Cannot set tensor: Dimension mismatch

The reason for this is a mismatch in the shape of the input tensor vs the shape of the input that we’re providing.

The input that the model accepts is [1, 224, 224, 3], whereas the input we’re providing is [50, 224, 224, 3]. To fix this, we can simply resize our input before running inference on it.

Here’s what the code for doing so looks like:

Before running inference, we can also resize the input so that it accepts as many images as there are in the current batch of images.

On running the code above, this is what you might get as the output:

[[ 75 164 17], [252 2 2], [252 2 2], [ 21 210 25], ..., [203 42 12], [ 9 112 136]]

The length of this array would be the same as your batch size (here 50); so if you maintain another list of file names, you can reference the score for each image easily!