问仕威¹，葛亚豪¹，王应宽^2,3*，卫乃硕¹，周建国¹，胡广锐⁴，杨亮亮⁵，陈军^1*

(1.西北农林科技大学机械与电子工程学院，杨凌712100，陕西，中国；

2.农业农村部规划设计研究院，北京100125，中国；

3.中国农业工程学会，北京100125，中国；

4.西安工业大学设计学院，西安710021，中国；

5.北见工业大学，北海道 090-8507，日本)

摘要：为实现高效且低成本的苹果自动化采摘，该研究提出了一种基于单一传感器的多类实例分割模型SCAL（Star-CAA-LADH），仅通过RGB图像实现了对果实、挂果枝与主干的精准分割，从而获取全面的视觉信息。

该研究通过融合星形操作与上下文锚定注意力机制（Context-Anchored Attention，CAA），构建了Star-CAA模块，以增强模型的方向敏感性与多尺度特征感知能力。在Backbone与Neck网络中，通过层次化配置SCA-T/F模块，强化了高低层特征的融合效果，从而获得更连续的分割掩膜与更清晰的目标边界。在Head网络中，引入Segment_LADH模块分别对分类、边界框定位与掩膜生成进行优化，进一步提高对小目标与粘连结构的分割精度。

为适应多种不利气象条件，本研究进一步集成链式推理提示自适应增强模块（Chain-of-Thought Prompted Adaptive Enhancer, CPA），增强了模型在退化环境下的鲁棒性。试验结果表明，SCAL模型在AP_M与mAP_M上分别达到了94.9%与95.1%，较基准模型分别提升了6.6%与4.6%。在多气象条件测试集中，CPA-SCAL模型的精度指标均优于对比模型。经INT8量化后，模型大小压缩至14.5 MB，在NVIDIA Jetson AGX Xavier上达到47.2 fps的实时推理速度。在模拟果园环境中进行了实验验证，证明了SCAL模型的有效性与泛化能力，为复杂果园环境下的智能采摘提供了一种高效、全面的视觉解决方案。

关键词：苹果采摘；实例分割；多气象条件；星形操作；边缘计算设备

引用信息: Wen S W, Ge Y H, Wang Y K, Wei N S, Zhou J G, Hu G R, et al. Efficient and comprehensive visual solution for a smart apple harvesting robot in complex settings via multi-class instance segmentation. Int J Agric & Biol Eng, 2025; 18(4): 200–215.

阅读全文链接：Efficient and comprehensive visual solution for a smart apple harvesting robot in complex settings via multi-class instance segmentation

Efficient and comprehensive visual solution for a smart apple harvesting robot in complex settings via multi-class instance segmentation

Shiwei Wen¹, Yahao Ge¹, Yingkuan Wang^2,3*, Naishuo Wei¹, Jianguo Zhou¹, Guangrui Hu⁴, Liangliang Yang⁵, Jun Chen^1*

(1. College of Mechanical and Electronic Engineering, Northwest A&F University, Yangling 712100, Shaanxi, China;

2. Academy of Agricultural Planning and Engineering, Ministry of Agriculture and Rural Affairs, Beijing 100125, China;

3. Chinese Society of Agricultural Engineering, Beijing 100125, China;

4. School of Design, Xi’an Technological University, Xi’an 710021, China;

5. Laboratory of Bio-Mechatronics, Faculty of Engineering, Kitami Institute of Technology, Hokkaido 090-8507, Japan)

Abstract: To enable efficient and low-cost automated apple harvesting, this study presented a multi-class instance segmentation model, SCAL (Star-CAA-LADH), which utilizes a single RGB sensor for image acquisition. The model achieves accurate segmentation of fruits, fruit-bearing branches, and main branches using only a single RGB image, providing comprehensive visual inputs for robotic harvesting. A Star-CAA  module was proposed by integrating Star operation with a Context-Anchored Attention mechanism (CAA), enhancing directional sensitivity and multi-scale feature perception. The Backbone and Neck networks were equipped with hierarchically structured SCA-T/F modules to improve the fusion of highand low-level features, resulting in more continuous masks and sharper boundaries. In the Head network, a Segment_LADH module was employed to optimize classification, bounding box regression, and mask generation, thereby improving segmentation  accuracy for small and adherent targets. To enhance robustness in adverse weather conditions, a Chain-of-Thought Prompted Adaptive Enhancer (CPA) module was integrated, thereby increasing model resilience in degraded environments. Experimental results demonstrate that SCAL achieves 94.9% AP_M and 95.1%  mAP_M, outperforming YOLOv11s by 6.6% and 4.6%, respectively. Under multi-weather testing conditions, the CPA-SCAL variant consistently outperforms other comparison models in accuracy. After INT8 quantization, the model size was reduced to 14.5 MB, with an inference speed of 47.2 frames per second (fps) on the NVIDIA Jetson AGX Xavier. Experiments conducted in simulated orchard environments validate the effectiveness and generalization capabilities of the SCAL model, demonstrating its suitability as an efficient and comprehensive visual solution for intelligent harvesting in complex agricultural settings.

Keywords: apple harvesting, instance segmentation, multi-weather condition, star operation, edge computing device

DOI: 10.25165/j.ijabe.20251804.9619

Citation: Wen S W, Ge Y H, Wang Y K, Wei N S, Zhou J G, Hu G R, et al. Efficient and comprehensive visual solution for a smart apple harvesting robot in complex settings via multi-class instance segmentation. Int J Agric & Biol Eng, 2025; 18(4): 200–215.

支持单位：