Application of deep learning and datamining for the study of plant-pathogen interaction: the case of apple and pear scab

Fruit growing is an important niche in the structure of agriculture. Apple and pear are the most widely grown and economically significant fruit crops in the world and in Latvia, while the scab caused by Venturia inaequalis and V. pyrina are the most important diseases for these species. Considering environmental and food safety concerns, the high adaptability of pathogens and cost-effectiveness requirements, there is need to change cultivation strategies by reducing the use of pesticides, promoting their precision and purposefulness. Smart or precision horticulture is a way to ensure this and involves the close linkage between research on local resources, environmental issues and information technologies, which common work can promote the development of fruit-growing.

The aim of the study is development of an integrated decision-making system using knowledge on plant-pathogen-environment interactions in apple/V.inaequalis and pear/V.pyrina pathosystems.

The following objectives were defined to fulfil the project aim:

1) application of semantic analysis and data mining for plant-pathogen interaction data in apple/V.inaequalis and pear/V.pyrina pathosystems;

2) development and implementation of an image-based deep learning system for early identification and evaluation of apple and pear scab;

3) development of IoT-system model for apple and pear monitoring. Proposed results are knowledge of plant-pathogen interaction mechanisms, their use for disease monitoring and prognosis.

The project aim: development of an integrated decision-making system using knowledge on plant-pathogen-environment interactions in apple / V. inaequalis and pear / V. pyrina pathosystems.

The aim will be achieved by implementing following project tasks: 
1) application of semantic analysis and data mining for plant-pathogen interaction data in apple/V.inaequalis and pear/V.pyrina pathosystems;
2) development and implementation of an image-based deep learning system for early identification and evaluation of apple and pear scab;
3) development of IoT-system model for apple and pear monitoring.

Project partner: Rezekne Academy of Technologies

Project - related activities:

• Project group meeting and scientific seminar in Dobele, Institute of Horticulture, January 28, 2020
• Information about the project was presented in the first A.I. MEETUP (informal meeting related to artificial intelligence) in Latgale, which took place on March 2, 2020 at Rezekne Academy of Technology (RTA)
• Information about the project was published in the collection of theses of the Scientific-Practical Conference "Sustainable Agriculture 2020", which took place on February 20, 2020, in Jelgava (section Projects Implemented by Agricultural Scientists in 2020).
• Project participants S. Kodors, I. Zarembo, I. Apeināns and V. Žukovs participated in the 24th International Student Scientific-Practical Conference “Human. The environment. Technology. ”, which took place on April 22, 2020, RTA, Rēzekne.
• Project participant S. Kodors participated in the 19th International Scientific Conference Engineering for Rural Development, May 20-22, 2020, LLU, Jelgava with the report "Pear and apple recognition using deep learning and mobile".
• RTA students Ilmārs Apeināns, Vitālijs Žukovs and Jānis Stanķevičs involved in the project obtained a bachelor's degree by completing the 2nd level professional higher education bachelor's study program "Programming Engineer".
• Tatjana Aleško obtained a master's degree, defending a work on the topic "Apple and pear scab recognition tool for Latvian gardeners", developed within the project.
• PhD studies at Rēzekne Academy of Technology, doctoral study program of Sociotechnical Systems Modeling started project participant Toms Bartulsons (theme "Application of Deep Learning of Machines for Early Diagnosis of Plant Diseases").
• The results and experience of the project are integrated in the study course “Technical Vision” of Rēzekne Academy of Technology, where the 4th year students of the professional higher education bachelor study program “Mechatronics” acquire knowledge and skills in the field of artificial intelligence.
• Project participant S. Kodors is included in the calendar #Science for Latvia 2021, informing about the project activities: https://www.youtube.com/watch?v=FNrTAZTzp6M&feature=youtu.be.
• On January 22, 2021, an interdisciplinary online seminar was held, where the project participants interactively discussed the topic “Smart Garden - the ability to protect apples from scab”.
• Information about the research carried out in the project has been posted on the Delfi Campus website "Artificial intelligence helps to draw maps and treat trees".
• Information about the project and its results were presented in the poster presentation "Application of Deep Learning and Data Mining to Research the Interaction of Plants and Pathogens: Apple and Pear Scab Pathosystems" (authors Gunārs Lācis, Inga Moročko-Bičevska, Olga Sokolova, Sergejs Kodors) in scientific-practical conference "Sustainable agriculture 2021" ", which took place online on February 25-26, 2021.
• On March 9-11, 2021, Gunārs Lācis, the leading researcher of the DI, gave an oral report “Assessment of pear (Pyrus communis L.) genetic diversity using molecular markers linked to pear scab (Venturia pyrina Aderh.) Resistance” (authors: G. Lācis, I Kota-Dombrovskis, B. Lace) and participated in the ISHS 4th International Symposium on Horticulture in Europe (https://she-ihs-fav2020.de/), organized by the University of Hohheimheim (https: // www.uni-hohenheim.de) in cooperation with ISHS and Interplan.
• On April 30, 2021, the project participant Tom Bartulsons at the European Scientists' Night 2021 informed the participants about the activities taking place in the project, the possibilities of hyperspectral imaging. The title of the report is "Innovative methods and technological solutions in agriculture".
• On December 2, 2021, the project participant Sergejs Kodors at the international conference "The International Symposium on Digital Horticulture" organized by Bulduri Horticultural High School presented the report "Fruit-Grower Digital Tool for Apple Scab Detection".
• RTA students involved in the project Ilmārs Apeināns, Vitālijs Žukovs on February 18, 2022, obtained master's degrees in engineering within the framework of the professional master's study program "Computer Systems". The defended master's theses are related to the project implementation: Ilmārs Apeināns "Gardening expert system with elements of artificial intelligence", Vitalijs Zhukovs "Application of artificial neural networks in apple scab recognition".
• On February 24 and 25, 2022, the project participants Gunārs Lācis, Inga Moročko-Bičevska participated in the scientific-practical conference "Balanced Agriculture 2022" organized by the Latvia University of Life Sciences and Technologies, while Olga Sokolova gave an oral report, "Evaluation of apple scab and the prevalence of Venturia inaequalis genotypes in Latvia" (authors O.Sokolova, I.Moročko-Bičevska).
• On June 9 and 10, 2022, the project participant Gunārs Lācis participated in the conference "Living plant collections in the 21st century" organized by the Botanical Garden of the University of Latvia and presented a poster report "Malus and Pyrus ex situ germplasm collection as a raw material source for future breeding challenges and landfill for the development of modern technologies” (authors Gunārs Lācis, Baiba Lāce, Laila Ikase).
• Project participant Olga Sokolova on June 12-18, 2022, in Plovdiv, Bulgaria, took part in the 12th International IOBC-WPRS Workshop on Pome Fruit Diseases IOBC-WPRS Working Group "Integrated Plant Protection in Fruit Crops", Sub Group "Pome Fruit Diseases" and presented the poster titled “Evaluation of Venturia inaequalis virulence on apple immature fruits and detached leaves by an in-vitro methodology”.

Project results:

• Kodors, S., Lacis, G., Zhukov, V., Bartulsons, T. Pear and apple recognition using deep learning and mobile, Engineering for Rural Development, 2020. (dx.doi.org/10.22616/ERDev.2020.19.TF476)
• Tatjana Aleško "Apple and pear scab recognition tool for Latvian gardeners" (RTA master's thesis)
• Kodors S., Lacis G., Sokolova O., Zhukovs V., Apeinans I., Bartulsons T. 2021. Apple scab detection using CNN and Transfer Learning. Agronomy Research,19(2), 507–519 (DOI 10.15159/AR.21.045)
• Sokolova O., Moročko‐Bičevska I. 2021. Evaluation of Venturia pyrina virulence on European pear (Pyrus communis) cultivars by an in vitro methodology. Journal of Phytopathology, in press (https://doi.org/10.1111/jph.13002)
• The data sets obtained during the project were published in the open data repository: images of fruits (https://www.kaggle.com/projectlzp201910094/applescabfds) and leaves (https://www.kaggle.com/projectlzp201910094/applescablds) used in the neural network analysis
• Lācis G., Kota-Dombrovska I. and Lāce B. 2021. Assessment of pear (Pyrus communis L.) genetic diversity using molecular markers linked to pear scab (Venturia pyrina Aderh.) resistance. Acta Hortic. 1327, 57-64
  DOI: 10.17660/ActaHortic.2021.1327.7
  (https://doi.org/10.17660/ActaHortic.2021.1327.7)
• Sokolova O., Moročko-Bičevska I., Lācis G. 2022. Genetic Diversity of Venturia inaequalis in Latvia Revealed by Microsatellite Markers. Pathogens, 11, 1165. https://doi.org/10.3390/pathogens11101165
• Lācis G., Kota-Dombrovska I., Kārkliņa K., Lāce B. 2022. Genetic and relatedness of Latvian Pyrus germplasm assessment by a set of SSR markers. Proceedings of the Latvian Academy of Sciences. Section B, 76 (4/739), 438–447. DOI: 10.2478/prolas-2022-0068 (https://sciendo.com/article/10.2478/prolas-2022-0068)
• Kodors S., Lācis G., Moročko-Bičevska I., Zarembo I., Sokolova O., Bartulsons T., Apeināns I., Žukovs V. 2022. Apple scab detection in the early stage of disease using a convolutional neural network. Proceedings of the Latvian Academy of Sciences. Section B, 76 (4/739), 482–487. DOI: 10.2478/prolas-2022-0074 (https://sciendo.com/article/10.2478/prolas-2022-0074)
• Sokolova O., Moročko-Bičevska I. 2022. Evaluation of apple scab and occurrence of Venturia inaequalis races om differential Malus genotypes in Latvia. Proceedings of the Latvian Academy of Sciences. Section B, 76 (4/739), 488–494. DOI: 10.2478/prolas-2022-0075 (https://sciendo.com/article/10.2478/prolas-2022-0075)
• The set of annotated data created during the project for apple scab recognition has been published: https://www.kaggle.com/datasets/projectlzp201910094/eapplescab
• Information about the results of the project is published in the electronic publication "Professional horticulture" published by the Institute of Horticulture (https://www.darzkopibasinstituts.lv/lv/raksts/2022-11-30/profesionala-darzkopiba-nr-17)
• Zelmene K., Kārkliņa K., Ikase L., Lācis G., 2022. Inheritance of Apple (Malus × domestica (L.) Borkh) Resistance against Apple Scab (Venturia inaequalis (Cooke) Wint.) in Hybrid Breeding Material Obtained by Gene Pyramiding. Horticulturae, 8, 9, 772. 10.3390/horticulturae8090772 (https://www.mdpi.com/2311-7524/8/9/772)