Pedestrian identification in infrared and visible images based on pose keypoints matching

Abstract

The identification of persons in multi-spectral images that are not spatially aligned is a challenging process. A correct identification can improve the pedestrian detection task for machine vision applications. In this context we propose a person identification mechanism able to correctly find the same person in infrared and visible images. The main contribution of the paper is the keypoint based matcher that uses a deep learning based solution for finding relevant human pose keypoints and a local neighbour search for extracting the best candidates for person identification. For each of the two images, infrared and color we first perform pedestrian detection. Next, on each detected instance we extract the relevant keypoints for shoulders, hands and legs. A matching algorithm between the extracted keypoints is proposed in order to perform the identification of persons in the two images. We obtain an identification accuracy of 76% for pedestrians that have a medium height with respect to the image dimensions, and have a small occlusion degree.