RTPSeg Challenge

Participants are required to develop a road-scene vehicle-borne LiDAR point cloud semantic segmentation algorithm based on multi-modal data from RTPSeg (LiDAR point clouds, RGB images, and thermal infrared images). The algorithm shall address the following key technical challenges: (1) **Complex Illumination Conditions**: In degraded RGB scenarios such as nighttime and backlight, how to effectively fuse thermal radiation information from thermal infrared images to achieve accurate 3D semantic segmentation of road targets. (2) **Multi-modal Heterogeneous Data Fusion**: How to handle cross-modal fusion challenges including sensor projection errors, inconsistent fields of view (FoV) among multiple sensors, mismatched spatial resolutions, and pixel‑level spatial misalignment. (3) **Modality Gap Bridging and Complementary Exploitation**: How to overcome large modality gaps and fully leverage the texture/colour information from RGB images, thermal radiation information from thermal infrared images, and 3D geometric information from LiDAR point clouds. **Semantic Categories** The algorithm focuses on 18 semantic classes in urban and rural road scenes: building, car, carriageway, high vegetation, low vegetation, truck, sidewalk, fence, pole, bus, traffic sign, motorcycle, motorcyclist, traffic barrier, pedestrian, traffic light, bicycle, cyclist. --- **Dataset Information** - **Dataset Name**: RTPSeg – A multi‑modality dataset for LiDAR point cloud semantic segmentation assisted with RGB‑thermal images in autonomous driving. - **Source**: The dataset was jointly collected and constructed by the Information Engineering University and Sun Yat‑sen University, and has been published in the *ISPRS Journal of Photogrammetry and Remote Sensing* (2026). - **Acquisition Platform**: A custom‑designed multi‑sensor integrated platform was used, equipped with: - RGB camera (ZED, 1920×1080, 30 Hz), - Thermal infrared camera (Hinet‑1280, 1280×1024, 30 Hz, thermal sensitivity Yifan Sun, Chenguang Dai, Wenke Li, Xinpu Liu, Yongqi Sun, Ye Zhang, Weijun Guan, Yongsheng Zhang, Yulan Guo, Hanyun Wang. "RTPSeg: A multi-modality dataset for LiDAR point cloud semantic segmentation assisted with RGB-thermal images in autonomous driving." *ISPRS Journal of Photogrammetry and Remote Sensing*, 233, pp. 25-38, 2026.