A New Framework for Matching Forensic Composite Sketches With Digital Images

Introduction

Face recognition is a problem that is being solved from decades by now. It is one of the most challenging tasks and plays a very significant role in forensics. Another important subtask of face recognition problem is identifying a suspect by a sketch. In crime scenarios, the suspect’s photo is not available in most cases. Only the eye-witness description can be relied on for the identification of the victim or the criminal. A sketch of the suspect’s face can be drawn by a forensic artist (Wang and Tang.,2008) or even generated by facial software from the details provided by the eye-witness (Han et al., 2012). After the sketch is generated, the next important step is to identify who that person is in the sketch. This is performed by matching the sketch to the photos of mugshots/suspects in the criminal database. The automation of this process is useful in identifying the suspects in less time (Cheragi and Lee., 2019).

Numerous challenges can occur while matching a sketch and a photo. While drawing a sketch various complementary information such as hair color, skin color, and ethnicity may not be noticeable. Another critical challenge is the phenomenological gap between a sketch and a photo, such as illumination, facial expressions, color background, brightness, etc. In addition to this, only a single sketch is available unlike face recognition. Therefore, there is a chance that can lead to the misidentification of the suspect (Peng et al., 2018). Information from other sources like a description from the multiple eye-witnesses and surveillance camera footage can be used to improve the performance of suspect identification (Best-Rowden et al., 2014).

Forensic sketches can be categorized into three types such as viewed, forensic and composite sketches. The viewed sketch is drawn by looking at a photo, and no description is provided for drawing these types of sketches. Forensic sketches are drawn by forensic artists hearing the description provided by the eye-witnesses. These sketches are used for investigation since 19th century. Composite sketches are generated using software tools such as evoFIT, FACES etc. Here different facial parts are selected to draw the composite sketch. Almost 80% of law enforcement agencies use software oriented facial suspects to identify the suspects (Klum et al., 2013). Comparing the three types of sketches, composite sketches require less time, effort and experience. So composite sketches are used in the proposed framework to identify the corresponding digital images.

In order to provide identification based on composite sketch, many artificial neural network (ANN) models are proposed. CNN (Zhang, H. et al., 2019) are also a category of ANN model which are represented in a fully connected manner and thus avoids over fitting of the data. So a new CNN framework is proposed in this work to identify matching pairs and mismatching pairs of a composite sketch and a digital image. A CNN model uses Swish as the activation function in the hidden layers. The model is tested with multiple activation functions and the Swish activation function proved to provide the best performance among all other activation functions. The input provided to the system is a pair of composite sketch and a digital image. The model trains both the images and an absolute difference is taken as final value between composite sketch F_s and digital image F_d..The final output matching score obtained from the network is compared with the threshold value and decision is made to identify whether the input pair belongs to the same class or different class.

The key contributions of this work are stated below:

• •

  A new framework is proposed to perform matching of composite sketches and digital images.

• •

  To classify whether an input pair of composite sketch and a digital image belongs to the same or different class, a new CNN framework is proposed.

• •

  A detailed analysis of the proposed framework is provided.

• •

  The result achieved from the proposed framework outperforms many state-of-the-art face sketch recognition systems.