Page 197 - The-5th-MCAIT2021-eProceeding

alternative in selecting a suitable weight initialization strategy that can produce consistent results over the wide
range of datasets.

2. Related Works

The rise of research papers discussing weight initialization technique have led to various introduction of
image filters in the deep learning environment. Many initialization techniques have been proposed such as zeros
or identity initialization and random initialization (Li et al., 2020). Due to problems faced when applying the
standard weight initialization strategy, this motivates other researchers to investigate other image filters such as
PCA filter and LDA filter. LDA differs from PCA where, LDA aims to look for the components that maximize
the class separation which requires class labels while, PCA aims to extract components that maximize the
variance in the dataset (Alberti et al., 2017).
PCA initialization network has been thoroughly investigated by various works such as PCANet proposed by
(Chan et al., 2015). Other works, such as 2-dimensional PCA weight initialization with parametric equalization
normalization by (Wang et al., 2020), implementing PCA-initialized LeNet-5 model for image classification
(Ren et al., 2016) and vehicular image classification with PCA CNN proposed by (Soon et al., 2020). With the
increasing number of PCA initialization in deep learning, this shows promising future in using PCA filters in
the convolutional layer. In this work, variant of PCA filters will be introduced and applied as an initialization
strategy on LeNet-5 model and AlexNet model. The results showed that, PCA variant initialization is able to
obtain consistent results and is highly robust towards any changes in the employed image dataset.

3. PCA Variant Initialization
The generated PCA variant filters will be inserted in the first convolutional layer of LeNet-5 and AlexNet
model which will then be evaluated on numerous image datasets. Suppose that there are input training
images, { } of dimension × ℎ and the patch or sliding window size is × .
=1
3.1. Principal Component Analysis (PCA) filters

The process of generating PCA filter will utilize rectangular window, and it will be moved over the test
,
image B ( × ℎ) with a pre-defined step horizontally and then vertically to obtain sub-images, . The test
,
image B ( × ℎ) at horizontal position ∈ {0, … , − 1} and vertical position ∈ {0, … , ℎ − 1} while, the
sliding window ( × ) at position ∈ {0, … , − 1} and position ∈ {0, … , ℎ − 1}.
,
The weighting of the pixels for each sub-image, is a product of multiplying the pixel values and the
,
specific window weights, at the associated window locations. The window weights of rectangular window
,
and corresponding sub-image can be obtained based on equation (1) and equation (2) respectively,
,
,
(1)

(2)

The collected non-overlapping sub-images of the -th image, , , … , , ℎ ̂ ∈ ℝ × where each
,2
,
̂
,1

̂
denotes the -th vectorized patch in and = − ( ) , ℎ = ℎ − ( ). Each image vector is then normalized
̂

2 2
by subtracting its respective mean to ensure that the image vector is centered which results to, ̅ =
( , , … , ̅ , ℎ ̂) where each ̅ is a mean removed patch. By constructing the same matrix for all input
̅
̅
,2
,1
,
̂
images and assemble them together will results to,
(3)
The PCA filters are then expressed as follows, assuming that there are principal eigenvectors,
(4)
E- Proceedings of The 5th International Multi-Conference on Artificial Intelligence Technology (MCAIT 2021) [184]
Artificial Intelligence in the 4th Industrial Revolution