Page 199 - The-5th-MCAIT2021-eProceeding

GPCA filters slightly differs due to the usage of Gaussian window, thus the product of the standard deviations
is equals to,

(11)

4. Experiments
4.1. Descriptions of Dataset

Datasets used for the LeNet-5 model are MNIST, CIFAR-10, SVHN and GTSRB dataset while, AlexNet
model is the Covid-19 dataset (Cohen et al., 2020). The Covid-19 dataset contains 4 class category of Covid-
19, normal, pneumonia bacterial and pneumonia virus which amounts to a total of 306 images (270 for training,
36 for testing). Meanwhile, MNIST dataset contains 10 class category and total of 70k images (60k for training,
10k for testing), CIFAR-10 dataset contains 60k images with 10 classes (50k for training, 10k for testing),
SVHN dataset which comprises of 10-digit classes (73,257 for training, 26,032 for testing) and GTSRB dataset
with 43 classes for traffic signs (39,209 for training, 12,630 for testing).

4.2. Experimental Setup

In this paper, the deep learning library for Java called IntelliJ was used to implement LeNet-5 and AlexNet
model with variation of weight initialization strategy. The learning rate set for both models are 0.01 for CIFAR-
10 dataset and 0.001 for the remaining datasets. Other hyperparameter values such as weight decay is set to
−4
5 × 10 , momentum of 0.9 and epoch value is 100 for LeNet-5 model and 200 for the AlexNet model. The
gaussian parameter set in the GPCA and G-GPCA filter generation process for LeNet-5 model is 0.8 while for
the AlexNet model is 2.8. Both models are implemented on an Intel Core i7-10875H @ 2.3 - 5.1 GHz with
Nvidia RTX2070 GPU of 16GB RAM.

4.3. Preliminary Result

PCA initialization model obtains the highest accuracy for MNIST, CIFAR-10 and GTSRB dataset while,
GPCA performs the best for SVHN dataset. Based on Table 1, GPCA initialization achieves better accuracy
than G-GPCA in CIFAR-10 and SVHN dataset but only with a slight margin of (≈ 0.45%) and it achieves
lower results than G-GPCA in MNIST, GTSRB and Covid-19 dataset with a difference of (≈ 5.91%). The
slight variation in accuracy shows that applying G-GPCA can gives similar accuracy to GPCA with faster filter
generation process. PCA variant initialization shows promising results due to the consistency of achieving
similar accuracy for each weight initialization strategy and datasets used. Further comparison with other weight
initialization technique such as Xavier and Gabor will be conducted.
Table 1. Comparison of accuracy between PCA variant initialization on LeNet-5 and AlexNet model (Bolded value is the highest
accuracy).
LeNet-5 AlexNet
Filters Accuracy (%)
MNIST CIFAR-10 SVHN GTSRB Covid-19
PCA 99.13 63.97 88.81 92.77 63.89
GPCA 99.10 63.95 89.24 89.68 52.78
G-GPCA 99.12 63.23 89.07 90.72 69.44

E- Proceedings of The 5th International Multi-Conference on Artificial Intelligence Technology (MCAIT 2021) [186]
Artificial Intelligence in the 4th Industrial Revolution