The backpropagation learning algorithm that has been developed in the 1980s is still a core algorithm used in the learning of most deep learning models. However, its weight transport problem, which means that error signals are sequentially transmitted by using the weight values of upper layers to correct the weights of lower layers, leads to inefficiency of information transmission and lack of biological plausibility. As for alternative methods, the feedback alignment method has been proposed to utilize random weights that are independent of the forward weight for transmitting the reverse error signal. The direct feedback alignment method and the sign symmetry algorithm have also been developed to improve the performance of feedback alignment. Though these methods have shown their potential through basic experiments using deep learning models, the performances are not sufficiently good compared to the original backpropagation learning. In addition, the previous works are still in their early stage, and there is room for improvement.
This thesis studies the possibility of the performance improvement of feedback alignment method by combining various optimization algorithms. Through computational experiments, it is confirmed that the learning performance can be improved when the adaptive learning rate optimization algorithms such as Adam are combined with the feedback alignment method as well as sign symmetry method. Moreover, based on the previous study on the effect of weight initialization method on the learning performance, this thesis proposes a strategic selection of random weights for feedback alignment. Through experimental comparison, it is confirmed that the orthogonal initialization can improve the learning performance. Additionally, it is shown that applying kaiming uniform initialization or sparse initialization for the last layer can give positive effect on the learning performances. Experimental results also indicate that the effect of the proposed learning strategies become clear as the model deepens. As further studies, it would be necessary to conduct comprehensive experiments on large data sets and more complicated deep network structures. An in-depth analysis of the relationship among randomized weight vectors could give more insight on the effect of the proposed randomization strategies.