Ultrasonic testing is widely used to detect rail internal fatigue damage in large-scale rail flaw detection vehicles. However, the existing domestic ultrasonic system architecture platform has the phenomenon of data congestion and computer crash when it is used in the detection of complex railway sections, which leads to the missing detection of sections. Moreover, the identification of the damage mainly depends on the whole manual playback. In order to improve the signal processing speed and damage identification ability, and reduce the workload of manual playback, ultrasonic flaw detection system based on new bus and intelligent damage identification technology based on convolution neural network deep learning were carried out. In order to improve the detection ability of rail surface and near surface damage, the research on rail surface image detection technology was carried out. In order to realize the synchronous data acquisition and playback of each detection system, and comprehensively determine the rail health status, a spatial synchronous positioning system was designed. After verification and comparison, on the premise that the average false alarm rate of injury was basically the same, the average artificial damage detection rate of the system using the deep learning algorithm was increased by more than 4% compared with the existing system; the synchronization error between the systems was within 1m, and the positioning accuracy of the actual review of the injury was within 3 m.