Business-Card-Mail-Extractor

This is where our example app (with that short, catchy name) will come in. We want to be able to take a photo of a business card and directly get the email out of it. Why, you might ask? Because we are lazy and why should we type a few letters if we can build a full-blown app for that!? So we simply snap a photo and then we want an output that will look like this:

Example for Mail extraction from an example image from: https://businesscardtemplate.info/wp-content/uploads/2018/05/3Kings-Image3.jpg

So we create our new app in our favorite IDE (which we perfectly prepared for Flutter development). Now we have to do 4 things:

take a photo

use MLKit to extract the text from the photo

search for the email in the text

1. Take a photo

We can use a wonderful little Flutter package called “camera” to get access to the camera (it is important to follow the steps described in the “Installation” part of the docs of the package). First we need to add the package in our pubspec.yaml file. Then we can integrate this into our starter app in order to use it to take a photo and save it to disk.

This (simplified) gist does all of that. We first need to import the package in our dart file with import 'package:camera/camera.dart . Then we need to get the available cameras with await availableCameras() (for simplicity we omit the check if we actually have cameras, but note that this might throw an error). The CameraController controller object handles the interaction with the camera itself and can be easily created with a single camera object and information about the resolution. Then we simply define a filePath where we want to save the image to (Note: in the real code we use the pathProvider package to get the directory for the documents of your application). Finally we ask the controller take a picture for us and save it to the specified filePath with await controller.takePicture(filePath) .

If you do not know what the await keyword means you can read more about asynchronous programming in Dart here. In short: the commands availableCameras() and takePicture() return a Future object. This sounds fancy but simply says that the execution of the function might take some time and will only return in some time in the future (hence the name). If we want to free up the CPU to do other computations while waiting for an operation to finish we can use the await keyword (Note: you also have to mark the function containing the code async ).

2. Use MLKit to extract text from the image

Another task, another package. There is one from the Flutter team themselves, that provides access to MLKit called firebase_ml_vision. Again, you can check out the link on how to configure it (especially that it runs locally).

In addition to the package you need to set up a Firebase project and configure each platform for it. You can follow the steps in this codelab. Especially steps 6 and 7 are important. After performing this you are able to use MLKit inside your Flutter app for both Android and iOS.

So let’s go through the different steps. First of all we need to get the image and transform it so that Firebase can make use of it.

First we need to create an imageFile based on the filePath we previously set up to save the taken image. Then we need to use our new package to transform our file into a FirebaseVisionImage called visionImage so that it can be processed by the MLKit platform. We use the handy factory method to get it directly from our imageFile FirebaseVisionImage.fromFile(imageFile) .

A FirebaseVisionImage is simply a consistent representation for Firebase to work with the image. It contains the raw image bytes along with some metadata information like rotation and size.

We now need to create a TextRecognizer from the vision package and simply hand the created visionImage to it. Notice that this operation returns a Future again so we need to use the await keyword once more. This is basically it. Here the magic happens and the MLKit platform will analyze the image and extract all the valuable information from it.

3. Search for mail address

Lastly we need to extract the address from all the text we received from MLKit. One cool thing of these packages is that you can access the source code of them (at least for all packages I have ever used). So we can look at the code of the VisionText type here. We see that it consists of a list of TextBlock objects and these contain lists of TextLine objects. The following image shows the structure of a VisionText object:

How a VisionText object is structured.

Perfect! With that we can finalize our heroic quest of mail address search with the following code snippet: