The speech is one of the most classic communication for humans only and also one of the most efficient interaction between a man and machine. Since user interface using speech has clear advantages that it''s intuitive and applicable in the various interaction configurations, research and its applications on Automatic Speech Recognition(ASR) have prevailed widely and actively.
In an attempt to overcome the serious deterioration of recognition rates under noise environment, Audio Visual Speech Recognition(AVSR) has been recently introduced by integrating the information from lips shape into speech information. Such an effort has reportedly shown the improvement of performance to some extent compared to the case speech information is only considered.
This thesis is one of the preceding studies for the full AVSR system and proposes lip shape recognition based on a bool matrix simplifying the shape of lip and Support Vector Machine(SVM). For the reduction of redundant image information, lip region is segmented by comparing the locations of several candidates in the image of a portrait. Based on Active Shape Model, a certain number of points located on the outer edge of lip are then chosen and the outer edge of lip are simply represented by a bool matrix explaining the coordinates of the normalized and directly connected points by linear interpolation in the segmented lip region. Finally SVM trained with 5 speakers'' lip shapes with respect to 5 pronunciations is tested on 10 speakers'' data.
As a result, this thesis shows positive feasibility in speech recognition with only lip shapes and it implies that this approach can improve the speech recognition performance under noisy environment. Based on the result, AVSR is therefore expected to be one of the best alternative to the noisy speech recognition in the end.