A Parallel Framework for Multilayer Perceptron for Human Face
Recognition
Debotosh Bhattacharjee Reader,
Department of Computer Science and Engineering,
Jadavpur University,
Kolkata- 700032, India.
Mrinal Kanti Bhowmik mkb_in Lecturer,
Department of Computer Science and Engineering,
Tripura University (A Central University),
Suryamaninagar- 799130, Tripura, India.
Mita Nasipuri Professor,
Department of Computer Science and Engineering,
Jadavpur University,
exceed
Kolkata- 700032, India.
丑陋的英文>livereceptionDipak Kumar Basu Professor, AICTE Emeritus Fellow,
英语四级阅读
Department of Computer Science and Engineering,
Jadavpur University,
Kolkata- 700032, India.
Mahantapas Kundu mkundu@c.jdvu.ac.in Professor,
Department of Computer Science and Engineering,
Jadavpur University,
Kolkata- 700032, India.
Abstract
Artificial neural networks have already shown their success in face recognition and similar complex pattern recognition tasks. However, a major disadvantage of the technique is that it is extremely slow during training for larger class and hence not suitable for real-time complex problems such as pattern recognition. This is an attempt to develop a parallel framework for the training algorithm of a perceptron. In this paper, two general architectures for a Multilayer Perceptron (MLP) have been demonstrated. The first architecture is All-Class-in-One-Network (ACON) where all the class are placed in a single network and the cond one is One-Class-in-One-Network (OCON) where an individual single network is responsible for each and every class. Capabilities of the two architectures were compared and verified in solving human face recognition, which is a complex pattern recognition task where veral factors affect the recognition performance like po variations, facial expression changes, occlusions, and most importantly illumination changes. Both the structures were
implemented and tested for face recognition purpo and experimental results show that the OCON structure performs better than the generally ud ACON ones in term of training convergence speed of the network. Unlike the conventional quential approach of training the neural networks, the OC
ON technique may be implemented by training all the class of the face images simultaneously.
Keywords:Artificial Neural Network, Network architecture, All-Class-in-One-Network (ACON), One-Class-in-One-Network (OCON), PCA, Multilayer Perceptron, Face recognition.
1. INTRODUCTION
Neural networks, with their remarkable ability to derive meaning from complicated or impreci data, can be ud to extract patterns and detect trends that are too complex to be noticed by either humans or other computer techniques. A trained neural network can be thought of as an "expert" in the category of information it has been given to analyze [1]. This propod work describes the way by which an Artificial Neural Network (ANN) can be designed and implemented over a parallel or distributed environment to reduce its training time. Generally, an ANN goes through three different steps: training of the network, testing of it and final u of it. The final structure of an ANN is generally found out experimentally. This requires huge amount of computation. Moreover, the training time of an ANN is very large, when the class are linearly non-parable and overlapping in nature. Therefore, to save computation time and in order to achieve good respon time the obvious choice is either a high-end machine or a system which is collection of machines with low computational power.
In this work, we consider multilayer perceptron (MLP) for human face recognition, which has many real time applications starting from automatic daily attendance checking, allowing the authorized people to enter into highly cured area, in detecting and preventing criminals and so on. For all the cas, respon time is very critical. Face recognition has the benefit of being passive, nonintrusive system for verifying personal identity. The techniques ud in the best face recognition systems may depend on the application of the system.
Human face recognition is a very complex pattern recognition problem, altogether. There is no stability in the input pattern due to different expressions, adornments in the input images. Sometimes, distinguishing features appear similar and produce a very complex situation to take decision. Also, there are veral other that make the face recognition task complicated. Some of them are given below.
a) Background of the face image can be a complex pattern or almost same as the color of the
face.the lucky one
b) Different illumination level, at different parts of the image.
c) Direction of illumination may vary.
d) Tilting of face.
e) Rotation of face with different angle.
f) Prence/abnce of beard and/or moustache
g) Prence/Abnce of spectacle/glass.
h) Change in expressions such as disgust, sadness, happiness, fear, anger, surpri etc.
baikui) Deliberate change in color of the skin and/or hair to disgui the designed system.
From above discussion it can now be claimed that the face recognition problem along with face detection, is very complex in nature. To solve it, we require some complex neural network, which takes large amount of time to finalize its structure and also to ttle its parameters.
In this work, a different architecture has been ud to train a multilayer perceptron in faster way. Instead of placing all the class in a single network, individual networks are ud for each of the
class. Due to lesr number of samples and conflicts in the belongingness of patterns to their respective class, a later model appears to be faster in comparison to former.
2. ARTIFICIAL NEURAL NETWORK
Artificial neural networks (ANN) have been developed as generalizations of mathematical models of biological nervous systems. A first wave of interest in neural networks (also known as connectionist models or parallel distributed processing) emerged after the introduction of simplified neurons by McCulloch and Pitts (1943).The basic processing elements of neural networks are called artificial neurons, or simply neurons or nodes. In a simplified mathematical model of the neuron, the effects of the synaps are reprented by connection weights that modulate the effect of the associated input signals, and the nonlinear characteristic exhibited by neurons is reprented by a transfer function. The neuron impul is then computed as the weighted sum of the input signals, transformed by the transfer function. The learning capability of an artificial neuron is achieved by adjusting the weights in accordance to the chon learning algorithm. A neural network has to be configured such that the application of a t of inputs produces the desired t of outputs. Various methods to t the strengths of the connections exist. One way is to t the weights explicitly, using a priori knowledge. Another way is to train the neural network by feeding it teaching patterns and letting it change its weights according to some learning rule. The learning situations in neural networks may be classified into three distinct sorts. The are supervid learning, unsupervid lea
rning, and reinforcement learning. In supervid learning,an input vector is prented at the inputs together with a t of desired respons, one for each node, at the output layer. A forward pass is done, and the errors or discrepancies between the desired and actual respon for each node in the output layer are found. The are then ud to determine weight changes in the net according to the prevailing learning rule. The term supervid originates from the fact that the desired signals on individual output nodes are provided by an external teacher [3]. Feed-forward networks had already been ud successfully for human face recognition. Feed-forward means that there is no feedback to the input. Similar to the way that human beings learn from mistakes, neural networks also could learn from their mistakes by giving feedback to the input patterns. This kind of feedback would be ud to reconstruct the input patterns and make them free from error; thus increasing the performance of the neural networks. Of cour, it is very complex to construct such types of neural networks. The kinds of networks are called as auto associative neural networks. As the name implies, they u back-propagation algorithms. One of the main problems associated with back-propagation algorithms is local minima. In addition, neural networks have issues associated with learning speed, architecture lection, feature reprentation, modularity and scaling. Though there are problems and difficulties, the potential advantages of neural networks are vast. Pattern recognition can be done both in normal computers and neural networks. Computers u conventiona
l arithmetic algorithms to detect whether the given pattern matches an existing one. It is a straightforward method. It will say either yes or no. It does not tolerate noisy patterns. On the other hand, neural networks can tolerate noi and, if trained properly, will respond correctly for unknown patterns. Neural networks may not perform miracles, but if constructed with the proper architecture and trained correctly with good data, they will give amazing results, not only in pattern recognition but also in other scientific and commercial applications [4].
2A. Network Architecture
The computing world has a lot to gain from neural networks. Their ability to learn by example makes them very flexible and powerful. Once a network is trained properly there is no need to devi an algorithm in order to perform a specific task; i.e. no need to understand the internal mechanisms of that task. The architecture of any neural networks generally ud is All-Class-in-One-Network (ACON), where all the class are lumped into one super-network. Hence, the implementation of such ACON structure in parallel environment is not possible. Also, the ACON structure has some disadvantages like the super-network has the burden to simultaneously satisfy all the error constraints by which the number of nodes in the hidden layers tends to be large. The structure of the network is All-Class-in-One-Network (ACON), shown in Figure 1(a) where one single network is d
esigned to classify all the class but in One-Class-in-One-Network
(OCON), shown in Figure 1(b) a single network is dedicated to recognize one particular class. For each class, a network is created with all the training samples of that class as positive examples, called the class-one, and the negative examples for that exemplars from other class, constitute the class-two. Thus, this classification problem is a two-class partitioning problem. So far, as implementation is concerned, the structure of the network remains the same for all class and only the weights vary. As the network remains same, weights are kept in parate files and the identification of input image is made on the basis of feature vector and stored weights applied to the network one by one, for all the class.专科考研究生
(a)
(b)
Figure 1: a) All-Class-in-One-Network (ACON) b) One-Class-in-One-Network (OCON). Empirical results confirm that the convergence rate of ACON degrades drastically with respect to the network size becau the training of hidden units is influenced by (potentially conflicting) signals from different teachers. If the topology is changed to One Class in One Network (OCON) structure, where
one sub-network is designated and responsible for one class only then each sub-network specializes in distinguishing its own class from the others. So, the number of hidden units is usually small.
2B. Training of an ANN
In the training pha the main goal is to utilize the resources as much as possible and speed-up the computation process. Hence, the computation involved in training is distributed over the system to reduce respon time. The training procedure can be given as:
(1) Retrieve the topology of the neural network given by the ur,debts
(2) Initialize required parameters and weight vector necessary to train the network,
(3) Train the network as per network topology and available parameters for all exemplars of different class,
(4) Run the network with test vectors to test the classification ability,
(5) If the result found from step 4 is not satisfactory, loop back to step 2 to change the parameters like learning parameter, momentum, number of iteration or even the weight vector,
(6) If the testing results do not improve by step 5, then go back to step 1,
(7) The best possible (optimal) topology and associated parameters found in step 5 and step 6 are stored.
Although we have parallel systems already in u but some problems cannot exploit advantages of the systems becau of their inherent quential execution characteristics. Therefore, it is necessary to find an equivalent algorithm, which is executable in parallel.
In ca of OCON, different individual small networks with least amount of load, which are responsible for different class (e.g. k class), can easily be trained in k different processors and the training time must reduce drastically. To fit into this parallel framework previous training procedure can be modified as follows:
(1) Retrieve the topology of the neural network given by the ur,
(2) Initialize required parameters and weight vector necessary to train the network,
(3) Distribute all the class (say k) to available processors (possibly k) by some optimal process allocation algorithm,
(4) Ensure the retrieval the exemplar vectors of respective class by the corresponding processors,
(5) Train the networks as per network topology and available parameters for all exemplars of different class,
(6) Run the networks with test vectors to test the classification ability,
(7) If the result found from step 6 is not satisfactory, loop back to step 2 to change the parameters like learning parameter, momentum, number of iteration or even the weight vector,
(8) If the testing results do not improve by step 5, then go back to step 1,
(9) The best possible (optimal) topology and associated parameters found in step 7 and step 8 are stored,
(10) Store weights per class with identification in more than one computer [2].
During the training of two different topologies (OCON and ACON), we ud total 200 images of 10 different class and the images are with different pos and also with different illuminations. Sample images ud during training are shown Figure 2. We implemented both the topologies using
diglogMATLAB. At the time of training of our systems for both the topologies, we t maximum number of possible epochs (or iterations) to 700000. The training stops if the number of iterations exceeds this limit or performance goal is met. Here, performance goal was considered as 10-6. We have taken total 10 different training runs for 10 different class for OCON and one single training run for ACON for 10 different class. In ca of the OCON networks, performance goal was met for all the 10 different training cas, and also in lesr amount of time than ACON. After the completion of training pha of our two different topologies we tested our both the network using the images of testing class which are not ud in training.
2C. Testing Pha
During testing, the class found in the databa with minimum distance does not necessarily stop the testing procedure. Testing is complete after all the registered class are tested. During testing some points were taken into account, tho are:
>wty
