In the previous post we looked at how we can deal with “really big” image data as the first stage in the object detection workflow. In this post we will highlight the AI modeling stage where we create our object detection model using tools in MATLAB. We will first explain some concepts that deal with object detection then finally we will create, train, and test an object detection model that will be able to detect craters on the surface of the moon in real-time. For this example, we will use crater images obtained from Pangu to train our object detector model.
Figure 1: Deep Learning Based Detection Workflow
Deep learning is a machine learning technique that teaches computers to learn by example. Deep learning is a key technology that is behind driverless cars and voice control in consumer devices. These models make use of a large set of labeled training data and neural network architectures that contain many layers to achieve state of the art accuracy that can sometimes exceed human level performance. Deep learning is often used to accomplish 3 tasks which is classification, object detection and semantic segmentation. In the rest of this post, we will discuss object detection in greater detail.
Figure 2: Layers in a deep learning network
Object detection is a technique that is used to identify objects in an image or video with the use of a bounding box around each identified object. This is the technique that will be used to create our model that will detect craters on the surface of the moon. The types of networks that can be used for object detection are YOLOv2, YOLOv3, SSD, and many more.
What is YOLO
You Only Look Once (YOLO) is a real-time object detection network that applies a single neural network to the full image. YOLO then uses this network to divide the image up into regions and then predicts the bounding boxes and probabilities for each resulting region. The YOLO model has some advantages over other systems as it makes use of only a single network evaluation compared to the thousands of network evaluations used by the likes of R-CNN. This makes the YOLO model extremely fast compared to its counterparts, more than 1000x faster than R-CNN and 100x faster than Fast R-CNN. You can look at this paper for more details on the full YOLO system.
Even though YOLO is a good system it too has its own limitations and is not perfect. One of the biggest limitations of YOLO is its inability to predict small objects that appear in groups. This is because YOLO imposes strong spatial constraints on bounding box predictions since each grid cell only predicts two boxes and can only have one class. This spatial constraint limits the number of nearby objects that our model can predict.
Creating a YOLOv2 object detector
Now we will create our model for detecting craters on the surface of the moon. First, we will look at the live script then also see how we can create the model using MATLAB’s deep network designer app. YOLOv2 consists of two parts a feature extraction and an object detection part. For this example, we will use ResNet-50 for feature extraction than use YOLOv2 for the feature extraction part.
Live script code
Let us first look at creating a YOLOv2 object detector programmatically using live scripts. You can do this by following these easy steps.
- First, we will load the ground truth data that we made in the previous post.
2. Next, we can setup the network layers for a predesigned YOLO v2 network.
3. Now that we have loaded our data and setup our network layers, we can view a training image and its corresponding ground truth label.
The next step is to train our model but before that let us first look at another of building the detector using the deep network designer app.
Deep network designer app
Another way in which you can build a YOLOv2 model is by using MATLAB’s deep network app which is an interactive way to build the model. You can do this by following these easy steps.
- First let us open the deep network designer app from the apps ribbon in MATLAB or simply type in deepNetworkDesigner from the command window.
2. The deep network designer start page should pop up now from the start page select the ResNet-50 network. You might need to install it first if it is not yet installed.
3. Now let us select a layer from ResNet-50 that will be used for feature extraction, for this example we will use layer 140 so select all layers after 140 and delete them. In this case the 140th layer is the 13th addition layer. You can delete layers by selecting them then pressing delete.
4. Now we can add a combination of convolutional, batch normalisation and ReLu layers by just dragging them from the left pane and connecting them.
5. The next step is then to add YOLO specific layers like the YOLO transform and YOLO output layers.
6.The final step is to analyse our model and if you have no errors or warnings, you can export the model.
Now we have a complete object detection model that we can train to detect craters on the surface of the moon.
Training and testing the detector
Now that our model is built it is time to train and test it using training and testing datasets. We can train, evaluate, and test our detector by following these easy steps.
- To train the model we can first specify the training options using the trainingOptions function. Once the options have been set, we can now use the trainYOLOv2ObjectDetector function to train our model using the training data.
2.We can evaluate our trained model by running the pre-trained model using our training data. We can then plot the Precision/Recall curve to see how precise our model is. If we are not satisfied with the precision, we can change some of the options or even add and remove layers to our object detector model.
3.We can now test our detector by running it using our test images using the detect function. This is done to test how our detector performs with data that it has not seen before.
4.The result will be an image with bounding boxes around the craters with their respective probability score on top.
We have now successfully built, trained, evaluated, and tested our detector. Download the code and accompanying data and try this workflow for your own applications.
What Can I Do Next?
- Download the code.
- Request a trial.
- Find out more from the team.
- Visit our Smart Mining & Manufacturing Focus Area page to learn more.