Pedestrian detection method in rail transit scenes based on fusion of 3D point clouds and images

HE Jia

doi:10.13890/j.issn.1000-128X.2024.03.105

您当前的位置：

首页 >

文章列表页 >

Pedestrian detection method in rail transit scenes based on fusion of 3D point clouds and images

Intelligent Technology | 更新时间：2024-08-01

- Pedestrian detection method in rail transit scenes based on fusion of 3D point clouds and images
- Electric Drive for Locomotives Issue 3, Pages: 146-155(2024)
- 作者机构：
  
  国能包神铁路集团有限责任公司，内蒙古包头 014010
- 作者简介：
- 基金信息：
- DOI：10.13890/j.issn.1000-128X.2024.03.105
  CLC： TP18;U2
- Published：10 May 2024，
  
  Received：05 May 2023，
  
  Revised：06 February 2024，
- 稿件说明：
扫描看全文
贺佳. 基于三维点云与图像融合的轨道交通场景行人检测方法[J]. 机车电传动, 2024(3): 146-155.

HE Jia. Pedestrian detection method in rail transit scenes based on fusion of 3D point clouds and images[J]. Electric drive for locomotives,2024(3): 146-155.
贺佳. 基于三维点云与图像融合的轨道交通场景行人检测方法[J]. 机车电传动, 2024(3): 146-155. DOI：10.13890/j.issn.1000-128X.2024.03.105.

HE Jia. Pedestrian detection method in rail transit scenes based on fusion of 3D point clouds and images[J]. Electric drive for locomotives,2024(3): 146-155. DOI：10.13890/j.issn.1000-128X.2024.03.105.

摘要

在轨道交通场合由于行人非法侵限造成的安全事故时有发生，严重影响列车的安全运行。利用单一传感器数据进行行人检测存在检测结果召回率低、结果缺乏类别信息或方位信息等问题，无法满足实际现场需求。针对这些问题，文章提出一种基于三维点云与图像融合的轨道交通场景行人检测方法。该方法首先利用构建的轨道交通场景行人数据集训练的深度学习模型分别对三维点云与图像进行行人检测，在此基础上根据靶标在三维点云与图像中的空间位置一致性原理求解出激光雷达与相机之间的旋转平移矩阵，最后将三维点云目标检测结果投影至图像坐标系。为解决多个相邻目标之间误匹配、检测结果之间一对多的问题，通过计算2种检测结果之间的交并比和中心点距离作为融合的约束条件，进而更精确地检测行人。在现场采集的数据集上的试验结果表明，相比于三维点云与图像检测结果，在满足时效性的同时，该方法在检测精度相差不大的情况下，召回率分别提高了4.5%和5.5%，能够有效减少由于行人漏检可能造成的安全事故，满足在列车实际运营过程中对行人检测的需求。

Abstract

Safety incidents caused by pedestrians illegally intruding onto railway tracks occur frequently in rail transit scenes

significantly affecting the safe operation of trains. Utilizing single sensor data for pedestrian detection often leads to low recall rates of detection results and lack of category or orientation information in the results

which cannot meet practical field requirements. To address these issues

this paper proposed a pedestrian detection method based on the fusion of 3D point clouds and images in rail transit scenes. This method first employed a deep learning model trained on a constructed dataset of pedestrian data in rail transit scenes to detect pedestrians separately in 3D point clouds and images. Subsequently

based on the principle of spatial position consistency of the targets in 3D point clouds and images

the rotation and translation matrix between the LiDAR and camera was solved. Finally

the 3D point cloud object detection results were projected onto the image coordinate system. To solve the issues of misalignment between multiple adjacent targets and the one-to-many relationship between detection results

the intersection over union ratio and center point distance between the two detection results were calculated as fusion constraints

enabling more accurate pedestrian detection. Experimental results using data acquired from the field demonstrate that

compared to detection results from separate data of 3D point clouds and images

while maintaining timeliness

this method improves the recall rate by 4.5% and 5.5%

respectively

effectively reducing the risk of safety accidents caused by missed pedestrian detections

meeting the demand for pedestrian detection during actual train operations.

关键词

三维点云图像检测深度学习多传感器融合目标检测

Keywords

3D point cloudsimage detectiondeep learningmulti-sensor fusionobject detection

references

于晓英, 苏宏升, 姜泽, 等. 基于YOLO的铁路侵限异物检测方法[J]. 兰州交通大学学报, 2020, 39(2): 37-42.

YU Xiaoying, SU Hongsheng, JIANG Ze, et al. Detection method of railway intruding obstacle based on YOLO algorithm[J]. Journal of Lanzhou jiaotong university, 2020, 39(2): 37-42.

刘力, 苟军年. 基于YOLO v4的铁道侵限障碍物检测方法研究[J]. 铁道科学与工程学报, 2022, 19(2): 528-536.

LIU Li, GOU Junnian. Research on detection method of railway intrusion obstacles based on YOLO v4[J]. Journal of railway science and engineering, 2022, 19(2): 528-536.

王瑞, 李霄峰, 史天运, 等. 基于视频深度学习的铁路周界入侵检测算法研究[J]. 交通运输系统工程与信息, 2020, 20(2): 61-68.

WANG Rui, LI Xiaofeng, SHI Tianyun, et al. Railway perimeter intrusion detection algorithms based on video deep learning[J]. Journal of transportation systems engineering and information technology, 2020, 20(2): 61-68.

徐岩, 陶慧青, 虎丽丽. 基于Faster R-CNN网络模型的铁路异物侵限检测算法研究[J]. 铁道学报, 2020, 42(5): 91-98.

XU Yan, TAO Huiqing, HU Lili. Railway foreign body intrusion detection based on faster R-CNN network model[J]. Journal of the China railway society, 2020, 42(5): 91-98.

SCHECHNER Y Y, NARASIMHAN S G, NAYAR S K. Instant dehazing of images using polarization[C]//IEEE. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Kauai, HI: IEEE, 2001.

SHEN Linghao, ZHAO Yongqiang, PENG Qunnie, et al. An iterative image dehazing method with polarization[J]. IEEE transactions on multimedia, 2019, 21(5): 1093-1107.

YU Long, GAO Shibin, ZHANG Dongkai, et al. A survey on automatic inspections of overhead contact lines by computer vision[J]. IEEE transactions on intelligent transportation systems, 2022, 23(8): 10104-10125.

XU Jing, ZHAO Wei, LIU Peng, et al. An improved guidance image based method to remove rain and snow in a single image[J]. Computer and information science, 2012, 5(3): 49-55.

XU Jing, ZHAO Wei, LIU Peng, et al. Removing rain and snow in a single image using guided filter[C]//IEEE. 2012 IEEE International Conference on Computer Science and Automation Engineering (CSAE). Zhangjiajie: IEEE, 2012: 304-307.

SARNES B. Inspection of contact wire position and thickness at any speed[J]. Eb-elektrische bahnen, 2001, 99(12): 490-495.

刘邦繁, 张慧源, 李晨, 等. 基于车载激光雷达的机车可通行区域检测[J]. 控制与信息技术, 2021(4): 78-82.

LIU Bangfan, ZHANG Huiyuan, LI Chen, et al. Locomotive passable area detection based on onboard LiDAR[J]. Control and information technology, 2021(4): 78-82.

李志南, 刘玮, 韩丽娟, 等. 基于二维LiDAR的轨道侵入异物检测系统与方法研究[J]. 光电子·激光, 2020, 31(3): 262-268.

LI Zhinan, LIU Wei, HAN Lijuan, et al. Research on track invasion foreign object detection system and method based on two-dimensional LiDAR[J]. Journal of optoelectronics·laser, 2020, 31(3): 262-268.

郭保青, 余祖俊, 张楠, 等. 铁路场景三维点云分割与分类识别算法[J]. 仪器仪表学报, 2017, 38(9): 2103-2111.

GUO Baoqing, YU Zujun, ZHANG Nan, et al. 3D point cloud segmentation, classification and recognition algorithm of railway scene[J]. Chinese journal of scientific instrument, 2017, 38(9): 2103-2111.

曾祥, 蒋国涛, 鲍纪宇, 等. 基于车载激光雷达的隧道内障碍物检测[J]. 控制与信息技术, 2021(1): 1-8.

ZENG Xiang, JIANG Guotao, BAO Jiyu, et al. Detection of obstacles in tunnel based on vehicle-borne LiDAR[J]. Control and information technology, 2021(1): 1-8.

ZHOU Yin, TUZEL O. VoxelNet: end-to-end learning for point cloud based 3D object detection[C]//IEEE. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 4490-4499.

YAN Yan, MAO Yuxing, LI Bo. Second: sparsely embedded convolutional detection[J]. Sensors, 2018, 18(10): 3337.

CHARLES R Q, SU Hao, KAICHUN Mo, et al. PointNet: deep learning on point sets for 3D classification and segmentation[C]//IEEE. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI: IEEE, 2017: 77-85.

何友, 彭应宁, 陆大. 多传感器数据融合模型综述[J]. 清华大学学报(自然科学版), 1996(9): 14-20.

HE You, PENG Yingning, LU Da. A review: multi-sensor data fusion model[J]. Journal of Tsinghua university: natural science edition, 1996(9): 14-20.

李梦洁. 基于激光点云与图像结合的行人检测与跟踪技术研究[D]. 长沙: 国防科技大学, 2017.

LI Mengjie. The research of pedestrian detecting and tracking technology based on the combination of laser points and image[D]. Changsha: National Defense University, 2017.

郑少武, 李巍华, 胡坚耀. 基于激光点云与图像信息融合的交通环境车辆检测[J]. 仪器仪表学报, 2019, 40(12): 143-151.

ZHENG Shaowu, LI Weihua, HU Jianyao. Vehicle detection in the traffic environment based on the fusion of laser point cloud and image information[J]. Chinese journal of scientific

instrument, 2019, 40(12): 143-151.

YIN Xuanyu, SASAKI Y, WANG Weimin, et al. 3D object detection method based on YOLO and K-Means for image and point clouds[DB/OL]. (2020-04-21) [2024-01-16]. https://arxiv.org/abs/2005.02132https://arxiv.org/abs/2005.02132.

LIANG Ming, YANG Bin, CHEN Yun, et al. Multi-task multi-sensor fusion for 3D object detection[C]//IEEE. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA: IEEE, 2019: 7345-7353.

陈毅, 张帅, 汪贵平. 基于激光雷达和摄像头信息融合的车辆检测算法[J]. 机械与电子, 2020, 38(1): 52-56.

CHEN Yi, ZHANG Shuai, WANG Guiping. Vehicle detection algorithm based on information fusion of LIDAR and camera[J]. Mechanical and electronics, 2020, 38(1): 52-56.

Views

下载量

CSCD

CNKI被引量

Alert me when the article has been cited

提交

Tools

Publicity Resources

Hard Example Mining Method for Visual Perception Based on Multi-sensor Fusion

Lightweight YOLO models for object detection based on low-rank decomposition

An end-to-end text-to-speech system for vehicle-mounted devices

Arcing detection method of metro pantograph based on improved yolov4-tiny

Related Author

HE Jia

Jun LIN

Wenbo PAN

Jun YOU

Yanghan XU

HE Kun

ZENG Yang

CHEN Qi

Related Institution

CHN ENERGY Baoshen Railway Group, ，Inner Mongolia Autonomous Region

CRRC Zhuzhou Institute Co., Ltd.

School of Computer Science and Technology, Huazhong University of Science and Technology

State Key Laboratory of Rail Transit Vehicle System, Southwest Jiaotong University

CRRC Qingdao Sifang Rolling Stock Co., Ltd.

⁰