Meticulous Leaf Detection from Videos under Leaf Venation and Margin using Multi-spectral CapsNet and Relational Prototypical LSTM

Meticulous Leaf Detection from Videos under Leaf Venation and Margin using Multi-Spectral CapsNet and Relational Prototypical LSTM

1 Vidyashankar, 2 Hemantha Kumar G

1 DoS in Computer Science, 2 DoS in Computer Science,

1 University of Mysore, Mysuru, INDIA

Abstract - Leaf detection is essential for understanding leaf behavior, agricultural monitoring, and ecological studies, which is necessary for species identification and understanding of environmental impacts on leaf growth. Hence a novel approach Multi-spectral CapsNet and Relational Prototypical LSTM, is proposed to overcome challenges in leaf detection from videos. Initially, videos are converted into image frames, which are then pre-processed to remove noise and boost contrast, followed by a watershed approach for segmentation. Existing algorithms struggle to extract essential data from secondary vein patterns due to their vein's hierarchical structure, which extends beyond local spatial patterns. Hence Adaptive Centricity Multi-spectral CapsNet is implemented to extract features from the leaf vein patterns. Which utilizes the Adaptive Centricity Hough Line Detection (ACHL) Algorithm to extract local features like vein spacing and branching angles. Multi-spectral Attentional CapsNet (MA-CapsNet) captures global features like contextual information by focusing on spectral channels containing relevant data for identifying vein patterns. Furthermore, in leaf detection, existing methods analyze input data sequentially without including relative nearby regions information, which limits the network's capacity to distinguish between regions with varied depths and to grasp the global spatial relations between serrations Hence, Bidirectional Relational Prototypical LSTM (Bi-RP LSTM) is introduced to capture spatial relationships between serrations and analyze under serration depth levels thereby improving leaf detection accuracy. Finally, the proposed approach is implemented in Python, making it easier and more accurate than existing models for leaf detection in terms of accuracy, recall, precision, sensitivity, and F1 score.

Keywords - Leaf detection, Vein spacing, Angle of branching, Serration depth, Medial Axis Transform