Image recognition and empirical application of desert plant species based on convolutional neural network

doi:10.1007/s40333-022-0086-9

Journal of Arid Land

2022, Vol. 14

Issue (12): 1440-1455 DOI: 10.1007/s40333-022-0086-9 CSTR: 32276.14.s40333-022-0086-9

Research article

Image recognition and empirical application of desert plant species based on convolutional neural network

LI Jicai¹, SUN Shiding², JIANG Haoran², TIAN Yingjie¹^,³^,⁴^,^*(

), XU Xiaoliang⁵

¹School of Economics and Management, University of Chinese Academy of Sciences, Beijing 100190, China
²School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing 100049, China
³Key Laboratory of Big Data Mining and Knowledge Management, Chinese Academy of Sciences, Beijing 100190, China
⁴Research Center on Fictitious Economy and Data Science, Chinese Academy of Sciences, Beijing 100190, China
⁵Xinjiang Institute of Ecology and Geography, Chinese Academy of Sciences, Urumqi 830011, China

Download:

HTML

PDF(2331KB)
Export: BibTeX | EndNote (RIS)

Abstract

In recent years, deep convolution neural network has exhibited excellent performance in computer vision and has a far-reaching impact. Traditional plant taxonomic identification requires high expertise, which is time-consuming. Most nature reserves have problems such as incomplete species surveys, inaccurate taxonomic identification, and untimely updating of status data. Simple and accurate recognition of plant images can be achieved by applying convolutional neural network technology to explore the best network model. Taking 24 typical desert plant species that are widely distributed in the nature reserves in Xinjiang Uygur Autonomous Region of China as the research objects, this study established an image database and select the optimal network model for the image recognition of desert plant species to provide decision support for fine management in the nature reserves in Xinjiang, such as species investigation and monitoring, by using deep learning. Since desert plant species were not included in the public dataset, the images used in this study were mainly obtained through field shooting and downloaded from the Plant Photo Bank of China (PPBC). After the sorting process and statistical analysis, a total of 2331 plant images were finally collected (2071 images from field collection and 260 images from the PPBC), including 24 plant species belonging to 14 families and 22 genera. A large number of numerical experiments were also carried out to compare a series of 37 convolutional neural network models with good performance, from different perspectives, to find the optimal network model that is most suitable for the image recognition of desert plant species in Xinjiang. The results revealed 24 models with a recognition Accuracy, of greater than 70.000%. Among which, Residual Network X_8GF (RegNetX_8GF) performs the best, with Accuracy, Precision, Recall, and F1 (which refers to the harmonic mean of the Precision and Recall values) values of 78.33%, 77.65%, 69.55%, and 71.26%, respectively. Considering the demand factors of hardware equipment and inference time, Mobile NetworkV2 achieves the best balance among the Accuracy, the number of parameters and the number of floating-point operations. The number of parameters for Mobile Network V2 (MobileNetV2) is 1/16 of RegNetX_8GF, and the number of floating-point operations is 1/24. Our findings can facilitate efficient decision-making for the management of species survey, cataloging, inspection, and monitoring in the nature reserves in Xinjiang, providing a scientific basis for the protection and utilization of natural plant resources.

Key words： desert plants image recognition deep learning convolutional neural network Residual Network X_8GF (RegNetX_8GF) Mobile Network V2 (MobileNetV2) nature reserves

Received: 01 September 2022 Published: 31 December 2022

Corresponding Authors: ^*TIAN Yingjie (E-mail: tyj@ucas.ac.cn)

	Service
	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	LI Jicai
	SUN Shiding
	JIANG Haoran
	TIAN Yingjie
	XU Xiaoliang

Cite this article:

LI Jicai, SUN Shiding, JIANG Haoran, TIAN Yingjie, XU Xiaoliang. Image recognition and empirical application of desert plant species based on convolutional neural network. Journal of Arid Land, 2022, 14(12): 1440-1455.

URL:

http://jal.xjegi.com/10.1007/s40333-022-0086-9 OR http://jal.xjegi.com/Y2022/V14/I12/1440

Fig. 1 Overview of Xinjiang and spatial distribution of nature reserves in Xinjiang. Note that the figure is based on the standard map (新S(2021)023) of the Map Service System (https://xinjiang.tianditu.gov.cn/main/bzdt.html) marked by the Xinjiang Uygur Autonomous Region Platform for Common Geospatial Information Services, and the standard map has not been modified. Satellite image source: Geospatial Data Cloud (http://www.gscloud.cn/).

Fig. 2 Images of the selected 24 desert plant species in nature reserves in Xinjiang

Species name	Family	Genera	Life form	Protection category	Field images	Total images
Ephedra intermedia (00001)	Ephedraceae	Ephedra	Shrub	Second-class national protected plant in China	100	114
Iljinia regelii (00002)	Chenopodiaceae	Iljinia	Subshrub	-	50	54
Corydalis kashgarica (00003)	Papaveraceae	Corydalis	Perennial herb	-	70	94
Zygophyllum kaschgaricum (00004)	Zygophyllaceae	Zygophyllum	Shrub	Second-class protected plant in Xinjiang, China	80	100
Ammopiptanthus nanus (00005)	Fabaceae	Ammopiptanthus	Shrub	Second-class national protected plant in China	70	73
Oxytropis bogdoschanica (00006)	Fabaceae	Oxytropis	Perennial herb	-	90	96
Caragana polourensis (00007)	Fabaceae	Caragana	Shrub	-	70	90
Glycyrrhiza inflate (00008)	Fabaceae	Glycyrrhiza	Perennial herb	Second-class national protected plant in China	50	72
Ammodendron bifolium (00009)	Fabaceae	Ammodendron	Shrub	First-class protected plant in Xinjiang, China	15	16
Eremosparton songoricum (00010)	Fabaceae	Eremosparton	Shrub	Second-class protected plant in Xinjiang, China	30	33
Lagochilus lanatonodus (00011)	Lamiaceae	Lagochilus	Perennial herb	-	30	33
Frankenia pulverulenta (00012)	Frankeniaceae	Frankenia	Annual herb	Second-class national protected plant in China	20	32
Salsola junatovii (00013)	Chenopodiaceae	Salsola	Subshrub	-	105	148
Gymnocarpos przewalskii (00014)	Caryophyllaceae	Gymnocarpos	Subshrub	First-class national protected plant in China	100	112
Helianthemum songaricum (00015)	Cistaceae	Helianthemum.	Shrub	Second-class national protected plant in China	125	125
Haloxylon persicum (00016)	Chenopodiaceae	Haloxylon	Tree	Second-class national protected plant in China	60	63
Caryopteris mongholica (00017)	Verbenaceae	Caryopteris	Shrub	-	130	149
Populus pruinosa (00018)	Salicaceae	Populus	Tree	First-class protected plant in Xinjiang, China	50	50
Tamarix taklamakanensis (00019)	Tamaricaceae	Tamarix	Shrub	Second-class national protected plant in China	151	151
Cistanche deserticola (00020)	Orobanchaceae	Cistanche	Perennial herb	Second-class national protected plant in China	100	114
Calligonum ebinuricum (00021)	Polygonaceae	Calligonum	Shrub	Second-class protected plant in Xinjiang, China	105	105
Prunus tenella (00022)	Rosaceae	Prunus	Tree	First-class protected plant in Xinjiang, China	20	31
Haloxylon ammodendron (00023)	Chenopodiaceae	Haloxylon	Tree	Second-class national protected plant in China	220	227
Populus euphratica (00024)	Salicaceae	Populus	Tree	-	230	249

Table 1 Basic information of the selected 24 desert plant species in nature reserves in Xinjiang

Fig. 3 Schematic diagram of convolutional neural network

Model	Accuracy (%)	Precision (%)	Recall (%)	F1 (%)	Params (M)	FLOPs (G)
VGG11	69.286	68.746	59.769	61.909	128.865	7.613
VGG13	67.857	64.700	58.584	58.310	129.049	11.317
VGG16	65.952	57.170	51.073	51.356	134.359	15.480
VGG19	66.667	67.341	54.269	56.148	139.669	19.643
ResNet18	72.857	75.693	61.621	64.463	11.189	1.819
ResNet34	72.857	74.432	61.448	64.091	21.297	3.671
ResNet50	73.333	69.447	63.473	65.239	23.557	4.110
ResNext50_32_4×D	72.857	71.907	61.531	63.486	23.029	4.257
ResNet101	73.095	66.146	60.646	61.639	42.549	7.832
ResNext101_32_8×D	72.857	73.280	61.886	63.927	86.792	16.475
DenseNet121	65.000	58.949	53.998	54.663	6.978	2.865
DenseNet169	65.476	55.327	54.604	53.783	12.524	3.396
DenseNet201	65.952	59.139	58.259	57.824	18.139	4.339
SqueezeNet1_0	54.286	35.847	34.203	33.751	0.748	0.739
SqueezeNet1_1	53.095	37.520	35.006	33.772	0.735	0.267
MobileNetV2	71.429	64.504	59.919	60.613	2.255	0.313
MobileNetV3-Small	61.429	56.082	51.884	52.685	1.542	0.058
MobileNetV3-Large	64.286	60.755	55.215	56.053	4.233	0.224
ShuffleNetV2_X0_5	53.095	30.013	33.321	30.101	0.366	0.042
ShuffleNetV2_X1_0	60.476	35.104	40.711	36.959	1.278	0.148
EfficientNet_B0	73.810	67.480	62.850	63.810	4.038	0.400
EfficientNet_B1	75.238	75.122	64.517	66.999	6.544	0.591
EfficientNet_B2	74.286	70.714	64.487	66.095	7.735	0.681
EfficientNet_B3	75.714	72.308	65.204	66.359	10.733	0.992
EfficientNet_B4	72.381	68.769	64.278	64.878	17.592	1.543
RegNetX_400MF	76.429	73.074	67.472	68.470	5.105	0.420
RegNetX_800MF	75.000	72.684	65.218	67.185	6.603	0.809
RegNetX_1_6GF	73.333	70.136	64.869	66.038	8.299	1.618
RegNetX_3_2GF	75.000	70.812	64.682	65.630	14.312	3.198
RegNetX_8GF	78.333	77.654	69.547	71.256	37.698	8.021
RegNetX_16GF	75.714	75.210	64.439	66.525	52.279	15.990
RegNetY_400MF	74.762	72.230	65.505	67.071	3.914	0.410
RegNetY_800MF	74.286	70.751	64.156	65.073	5.666	0.845
RegNetY_1_6GF	73.571	70.069	63.407	64.851	10.335	1.629
RegNetY_3_2GF	76.191	70.237	65.338	65.848	17.960	3.200
RegNetY_8GF	74.762	70.841	64.068	64.830	37.413	8.515
RegNetY_16GF	75.000	73.363	66.207	68.029	80.638	15.960

Table 2 Experimental results of 37 different models used in the image recognition of desert plant species

Fig. 4 Relationships of the Accuracy with the number of parameters (a) and the number of floating-point operations (b) for 37 different models used in the image recognition of desert plant species. M, megabyte, which refers to the storage space occupied by model parameters (1 M=1024 kilobytes); G, model operation speed (1 G=10⁹/s). VGG, Visual Geometry Group Network; ResNet, Residual Network; DenseNet, Dense Convolutional Network; SqueezeNet, Squeeze Network; MobileNet, Mobile Network; ShuffleNet, Shuffle Network; EfficientNet, Efficient Network; RegNet, Residual Network.

Table 3 Classification results of MobileNetV2 and RegNetX_8G in the image recognition of desert plant species

Fig. 5 Confusion matrix of MobileNetV2 (a) and RegNetX_8GF (b) in the image recognition of desert plant species. The plant species corresponding to the labels are consistent with those in Figure 2.

Fig. 6 Images of incorrectly classified samples with high similarity of external morphological features

Table 4 Performances of empirical application of MobileNetV2 and RegNetX_8GF in the image recognition of desert plant species in the Tianchi Bogda Peak Nature Reserve and Ebinur Lake Wetland National Nature Reserve


[1]	Abdullahi H S, Sheriff R E, Mahieddine F. 2017. Convolution neural network in precision agriculture for plant image recognition and classification. In: 2017 Seventh International Conference on Innovative Computing Technology (INTECH). New York: IEEE, 10: 256-272.

[2]	Barbedo J G. 2016. A review on the main challenges in automatic plant disease identification based on visible range images. Biosystems Engineering, 144: 52-60. doi: 10.1016/j.biosystemseng.2016.01.017

[3]	Bargoti S, Underwood J. 2017. Deep fruit detection in orchards. In: 2017 IEEE International Conference on Robotics and Automation (ICRA). New York: IEEE, doi: 10.48550/arXiv.1610.03677. doi: 10.48550/arXiv.1610.03677

[4]	Bekker A J, Goldberger J. 2016. Training deep neural-networks based on unreliable labels. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). New York: IEEE,2682-2686.

[5]	Cai Z. 2020. Research on deep learning model for Chinese herbal medicine planting process. MSc Thesis. Chengdu: University of Electronic Science and technology. (in Chinese)

[6]	Cao X J, Mo Y, Yan Y L. 2020. Convolutional neural network flower image recognition using transfer learning. Computer Applications and Software, 37(8): 142-148. (in Chinese)

[7]	Cao X Y, Sun W M, Zhu Y X, et al. 2018. Plant image recognition based on family priority strategy. Journal of Computer Applications, 38(11): 3241-3245. (in Chinese)

[8]	Coulibaly S, Kamsu F B, Kamissoko D, et al. 2019. Deep neural networks with transfer learning in millet crop images. Computers in Industry, 108: 115-120. doi: 10.1016/j.compind.2019.02.003

[9]	Gai R L, Cai J R, Wang S Y. 2021. Research review on image recognition based on deep learning. Journal of Chinese Computer Systems, 42(9): 1980-1984. (in Chinese)

[10]	Gao H Y, Gao X H, Feng Q S, et al. 2020. Approach to plant species identification in natural grasslands based on deep learning. Pratacultural Science, 37(9): 1931-1939. (in Chinese)

[11]	Hall D, McCool C, Dayoub F, et al. 2015. Evaluation of features for leaf classification in challenging conditions. In: 2015 IEEE Winter Conference on Applications of Computer Vision. New York: IEEE, 797-804.

[12]	He K M, Zhang X Y, Ren S Q, et al. 2016. Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE,770-778.

[13]	He M Z, Zhang J G, Wang H. 2006. Analysis of branching architecture factors of desert plants. Journal of Desert Research, (4): 625-630. (in Chinese)

[14]	Howard A G, Zhu M L, Chen B, et al. 2017. Efficient convolutional neural networks for mobile vision applications. [2022-09-01]. https://arxiv.org/abs/1704.04861v1.

[15]	Jin L T. 2020. Research on plant image recognition with complex background based on convolution neural network. MSc Thesis. Lanzhou: Lanzhou Jiaotong University. (in Chinese)

[16]	Krizhevsky A, Sutskever I, Hinton G E. 2012. ImageNet classification with deep convolutional neural networks. Advances In Neural Information Processing Systems, 25: 1097-1105.

[17]	Kussul N, Lavreniuk M, Skakun S, et al. 2017. Deep learning classification of land cover and crop types using remote sensing data. In: 2017 IEEE Geoscience and Remote Sensing Letters. New York: IEEE, 14(5): 778-782.

[18]	Lecun Y, Bengio Y. 1998. Convolutional Networks for Images, Speech, and Time Series. Cambridge, MA: MIT Press,255-258.

[19]	Li L P, Shi F P, Tian W B, et al. 2021. Wild Plant Image recognition method based on residual network and transfer learning. Radio Engineering, 51(9): 857-863. (in Chinese)

[20]	Li M M, Xia W C, Wang M, et al. 2020. Research on monitoring of Chinese nature reserves based on bibliometrics. Journal of Ecology, 40(6): 2158-2165. (in Chinese)

[21]	Li X H, Wu Z H, Liu H, et al. 2020. Species recognition of succulent plants based on convolutional neural network model. Journal of Guizhou Normal University, 36(3): 9-15. (in Chinese)

[22]	Li Y F. 2022. Research on image classification based on optimize on factors in convolutional neural network. Journal of Jinling Institute of Technology, 38(1): 26-31. (in Chinese)

[23]	Liu F Z, Du J H, Zhou Y, et al. 2018. Biodiversity monitoring technology and practice in nature reserves combining UAV and ground. Biodiversity, 26(8): 905-917. (in Chinese)

[24]	Liu H S. 2020. Panoramic plant recognition method based on CNN and GLCM fusion discrimination. MSc Thesis. Wuhan: Hubei University of Technology. (in Chinese)

[25]	Liu Y. 2018. Research on plant recognition based on deep learning. Beijing: Beijing Forestry University. (in Chinese)

[26]	Liu Y, Luo Z. 2019. Species recognition of protected area based on AutoML. Computer Systems & Applications, 28(9): 147-153. (in Chinese)

[27]	Mikolov T, Deo R A, Povey D, et al. 2011. Strategies for training large scale neural network language models. In: 2011 IEEE Workshop on Automatic Speech Recognition & Understanding. New York: IEEE, 196-201.

[28]	Ming Y. 2021. The adjusted "List of National Key Protected Wild Plants" was officially announced. Green China, (19): 74-79. (in Chinese)

[29]	Shen T M J. 2021. Video denoising based on prior information and convolutional neural network. MSc Thesis. Chengdu: University of Electronic Science and technology. (in Chinese)

[30]	Simonyan K, Zisserman A. 2014. Very deep convolutional networks for large-scale image recognition. [2022-09-01]. https://arxiv.org/pdf/1409.1556.pdf.

[31]	Su H M, Niu S M. 2016. Color Atlas of Wild Vascular Bundle Plants in Bogda Biosphere. Beijing: China Forestry Press,10-95. (in Chinese)

[32]	Tang M J. 2020. Research on fast prediction method of ship resistance performance based on convolutional neural network. MSc Thesis. Harbin: Harbin Engineering University. (in Chinese)

[33]	Wang Y W, Tang X L, Xu J P, et al. 2019. The use of big data in nature reserves. China Forestry Economy, (4): 16-20, 27. (in Chinese)

[34]	Xiao Z S. 2019. Application of infrared camera technology in wildlife inventory and assessment of natural reserves in China. Biodiversity, 27(3): 235-236. (in Chinese)

[35]	Xinjiang Flora Editorial Committee. 1992-2004. Xinjiang Flora (Volume I-Volume VI). Urumqi: Xinjiang Science and Technology Press. (in Chinese)

[36]	Yang X D, LV G H, Tian Y H, et al. 2009. Ecological grouping of plants in Lake Abby Wetland Nature Reserve in Xinjiang. Journal of Ecology, 28(12): 2489-2494. (in Chinese)

[37]	Yin L K, Pan B R, Wang Y, et al. 1991. Introduction and cultivation of rare and endangered plants in temperate desert. Arid Zone Research, (2): 1-8. (in Chinese)

[38]	Zhang S, Huai Y J. 2016. Leaf image recognition based on layered convolutions neural network deep learning. Journal of Beijing Forestry University, 38(9): 108-115. (in Chinese)

[1]	ZHOU Chongpeng, GONG Lu, WU Xue, LUO Yan. Nutrient resorption and its influencing factors of typical desert plants in different habitats on the northern margin of the Tarim Basin, China[J]. Journal of Arid Land, 2023, 15(7): 858-870.

[2]	SUN Liquan, GUO Huili, CHEN Ziyu, YIN Ziming, FENG Hao, WU Shufang, Kadambot H M SIDDIQUE. Check dam extraction from remote sensing images using deep learning and geospatial analysis: A case study in the Yanhe River Basin of the Loess Plateau, China[J]. Journal of Arid Land, 2023, 15(1): 34-51.

[3]	WANG Xiaohua, XIAO Honglang, REN Juan, CHENG Yiben, YANG Qiu. An ultrasonic humidification fluorescent tracing method for detecting unsaturated atmospheric water absorption by the aerial parts of desert plants[J]. Journal of Arid Land, 2016, 8(2): 272-283.

[4]	Sabitha SAKKIR, Junid N SHAH, Abdul Jaleel CHERUTH, Maher KABSHAWI. Phenology of desert plants from an arid gravel plain in eastern United Arab Emirates[J]. Journal of Arid Land, 2015, 7(1): 54-62.

[5]	ZhaoFeng CHANG, ShuJuan ZHU, FuGui HAN, ShengNian ZHONG. Differences in response of desert plants of different ecotypes to climate warming: a case study in Minqin, Northwest China[J]. Journal of Arid Land, 2012, 4(2): 140-150.

Viewed

Full text

Abstract

Cited

Shared

Discussed