2.1

Gra layer

In the second-order difference formula and the Pierce correlation coefficient theory, we know that there is no correlation between two independent discrete data, and the gradient features of the image edge position are more obvious. So we here stipulate that the step size l of the sliding window should be less than size of the window, ie (l<h,l<w), The schematic diagram of the Gra layer algorithm is shown in Figure 1.

Figure 1.

Schematic diagram of the Gra layer.

The feature P_m, m∈{1,2,3,…,M} intercepted by the sliding window from the feature map P is calculated by Algorithm 1 to obtain the corresponding gradient feature g_m, m∈{1,2,3,…,M}.

Two consecutive feature maps P_m and P_m+1 obtain the final gradient feature r_m through Algorithm 2, as shown in Figure 1.

2.2

GraDNet

In terms of features, the Gra layer extracts the gradient features of the image, which can ensure the spatial position invariance of edge features during the forward propagation process. In terms of structure, this paper uses the residual structure to realize the information fusion of high-dimensional resolution and low-dimensional resolution, and further realizes the precise positioning of edge features. The structure diagram of GraDNet is show in Figure 2. The network structure parameters are show in the following Table 1.

Figure 2.

The structure of GraDNet.

Table 1.

Network parameters of GraDNet.

First, the input data pass to the 11*11 convolutional layer, which can expand the range of visual field extraction. Then, that data pass to the Gra layer to obtain the gradient features. The gradient features are dot-multiplied with the input data, and finally added to the input data. In this way, the edge features can be strengthened on the one hand, and the correlation between features can be preserved on the other hand. Therefore, the point multiplication is to calibrate the spatial position of the edge features, and the purpose of addition is to strengthen the information of the edge features and preserve the correlation between the features. The GraDnet structure refers to the residual effect of Resnet, which effectively extracts edge features on the one hand, and makes the model more lightweight on the other hand.

3.1

Datasets

The Cityscapes dataset contains 5000 fine images, of which 2975 are training images, 500 validation images and 1525 testing images. In addition, the dataset contains 20k roughly annotated images. However, the performance of the algorithm is evaluated on the average precision metric of the dataset’s 8 semantic classes.

The ADE20K dataset has 25k images, of which the training set is 20k, the validation set is 2k, and the test set is 3k. This dataset covers various annotations for scenes, objects. There are a total of 150 different scenes and objects, with an average of 19.5 instances and 10.2 object classes per image class.

3.2

Comparison with the some SOTA algorithms

We first crop the images of the ADE20K and Cistyscapes datasets to a size of 500*500, and set the initial learning rate to 1*e-6 and epoch to 24000. The training platform is Ryzen 7 3800X and RTX 2070. The optimize function is the Adam optimizer. The loss function calculates the error between the true value and the predicted value using Equation (1), and evaluates the test results using mIoU. yp and yt represent the predicted and actual values, respectively.

Here we compare GraDNet with some SOTA algorithms, such as Deep snake¹², Unet, PANet¹³, FCIS¹⁴, ESE¹⁵, etc. The results for the two datasets are shown in Tables 2 and 3 below respectively.

Table 2.

Comparison results of ADE20K datasets.

Table 3.

Comparison results of Cityscapes datasets.

Judging from the accuracy results of the two datasets, GraDNet can achieve a good result. UNet and PANet can enhance the features of corresponding locations by concatenating data dimensions. Through feature splicing, the high-resolution features are enhanced by the low-resolution features, so the edge features can be accurately segmented by locating the enhanced features. However, due to the problem of feature loss in the pooling layer, the segmentation position is inaccurate. FCIS and ESE will be trained on the fully convolutional network by means of encode-decode. FCIS can effectively avoid the problem of inaccurate information caused by the loss of features in the pooling layer, but it will also reduce the calculation speed due to the excessive number of convolutional layers. Here we extract relevant feature about the edge features of the image through the Gra layer. GraDNet uses these features to enhance the edge feature and achieve feature positioning, and then achieve the effect of image segmentation, as shown in Figure 3. Compared with Deep and Unet, GraDNet’s Gra layer can obtain more gradient features, so that fewer convolutional layers used to achieve the effect of segmentation.

Figure 3.

Image is the original image. And GraDNet, Deepnet, Unet correspond to the segmentation effect image of these algorithms respectively.

From the comparison results in Figure 3, it can be directly seen that GraDNet can accurately segment the target edge. Unet and Deep snake cannot accurately locate the fine boundary contour, and have the problem of inaccurate segmentation for small volume targets. It can be seen from the comparison results of segmentation renderings and accuracy that GraDNet has certain advantages.