3D Robot Perception Methods for Tree Analysis with a Real2Sim Approach
Precise characterization of tree crop structure directly impacts yield optimization, orchard management decisions, and breeding programs. Growing labor shortages in agriculture have accelerated the need to automate management tasks, yet the intelligent perception systems required for automation face two critical bottlenecks: seasonality and data scarcity. This dissertation develops robotic perception methods using 3D computer vision for tree analysis by leveraging the Real2Sim closed-loop framework that bridges real-world observations with high-fidelity simulation environments. Chapter 1 gives an overall introduction. Chapter 2 presents a comprehensive review of the Real2Sim closed-loop framework in agricultural robotics, establishing core components, analyzing existing studies, and identifying critical gaps and future directions. This review offers systematic strategies for mitigating these bottlenecks and accelerating agricultural robot development. Chapter 3 introduces AppleQSM, a geometry-based 3D processing pipeline for apple tree architectural analysis. This approach creates quantitative structural models that provide precise tree characterization while revealing fundamental limitations of classic 3D perception methods. Beyond trait extraction, AppleQSM establishes the Real2Sim direction by enabling simulation generation from real-world geometrical data. Chapters 4–6 demonstrate learning-based methods trained within the Real2Sim closed-loop framework across three perception tasks. Chapter 4 presents the first application closing the Sim2Real loop through point cloud completion for robotic pruning. A Real2Sim data generation pipeline (L-TreeGen) synthesizes 3D apple trees from real-world geometrical data without manual parameterization, training a deep learning model that jointly performs completion and skeletonization on real-world partial branches through zero-shot transfer. Chapter 5 extends the framework to hierarchical joint segmentation, integrating virtual scanners into L-TreeGen to realistically simulate sensing effects. The resulting model, trained exclusively on synthetic data, achieves superior performance with significantly fewer parameters and manual annotations. Chapter 6 addresses throughput limitations of terrestrial laser scanning by introducing DATeR, a diffusion-based reconstruction framework that recovers high-fidelity 3D tree models from sparse RGB images, achieving accuracy comparable to laser scanning while enabling orchard-scale deployment. Overall, this dissertation establishes a comprehensive 3D robot perception framework for precision tree analysis, contributing novel methodologies for characterization, simulation, and reasoning. These contributions provide scalable solutions for agricultural digital twin systems and advance both theoretical foundations and practical deployment of simulation-based approaches in agricultural robotics and tree science.