목차

국문초록 ···································································· ⅰ
목 차 ···································································· ⅲ
그림목차 ···································································· ⅵ
도표목차 ···································································· ⅸ
약 어 표 ···································································· ⅹ
Ⅰ. 서 론 ·················································· 1
1.1 연구 배경 및 목적 ························································ 1
1.2 연구 내용 및 범위 ························································ 2
1.3 논문의 구성 ······························································ 3
Ⅱ. 관련연구 ············································ 4
2.1 수어 인식 기술 연구 ··············································· 4
2.1.1 지문자 인식 ······················································· 4
2.1.2 수화 인식 ······················································· 4
2.2 Neural Network ············································· 5
2.3 인공신경망을 이용한 이미지 인식 ···························· 6
2.3.1 Convolution Neural Network ···························· 6
2.3.2 LeNet ························································· 7
2.3.3 CNN 신경망 구조 ················································· 9
2.4 시계열 데이터를 학습하기 위한 인공신경망 ···················· 13
2.4.1 Recurrent Neural Network ·································· 13
2.4.2 Long Short Term Memory ·································· 15
2.5 신체 핵심좌표(Keypoint)를 추정하기 위한 연구 ··················· 20
2.5.1 MediaPipe ················································· 20
2.5.2 Openpose ·················································· 24
2.5.3 YOLO v7 pose ············································ 26
2.6 자연어 ························································ 28
2.6.1 자연어에 대한 해석 ······································ 28
2.6.2 문어에 대한 해석 ········································· 29
2.6.3 구어에 대한 해석 ········································· 30
2.6.4 수어(수화)에 대한 해석 ··································· 31
Ⅲ. 수어 인식을 위한 문제정의 ··················· 34
3.1 한국 수화 언어 ··················································· 34
3.1.1 수어의 손 모양(수향) ····································· 34
3.1.2 수어의 손 움직임(수동) ····································· 35
3.1.3 수어의 손 위치(수위) ····································· 36
3.1.4 수어의 손바닥 방향(수향) ··································· 37
3.1.5 손 동작 및 모형 이외의 수어 요소(비수지기호) ·············· 38
3.2 수어 문장 인식에 대한 문제 ································ 39
3.3 수어 낱말 길이 추론 ··································· 40
3.3.1 수어 낱말 분리 ·················································· 40
3.3.2 서로 다른 길이의 수어 낱말 추론 ································· 41
3.4 신경망을 활용한 수어 인식 ··································· 42
Ⅳ. 수어 인식 시스템 설계 ························ 44
4.1 비디오 입력 모듈 ····································· 45
4.2 비디오 분할 모듈 ····································· 46
4.3 입력 데이터 전처리 모듈 ···························· 47
4.4 수어 추론 모듈 ···································· 50
4.5 시계열 데이터 학습 동작 절차 ························· 51
4.6 수어 인식(추론) 신뢰도 반복 평가 과정 ························· 54
Ⅴ. 실험 및 성능분석 ······························ 56
5.1 실험 환경 ·········································· 56
5.2 핵심좌표 데이터 활용에 따른 학습 결과 비교 ········ 57
5.2.1 사용빈도수 최상위 수어 낱말 50개 인식 결과 비교 ···· 58
5.2.2 수형 차이의 수어 낱말 인식 결과 비교 ············· 62
5.2.3 수위 차이의 수어 낱말 인식 결과 비교 ············· 65
5.2.4 수향 차이의 수어 낱말 인식 결과 비교 ············· 67
5.2.5 비수지기호를 포함하는 수어 동작 인식 결과 비교 ···· 70
Ⅵ. 결론 및 향후 연구 방향 ··························· 73
참고문헌 ··································································· 75
Abstract ··································································· 80
감사의 글(Acknowledgement) ·············································· 89
<부록 1> 수어 추론을 위한 인식 구간 조정 ······························· 83
<부록 2> MediaPipe를 활용한 모션 벡터 생성 ····························· 84
<부록 3> 모션 벡터 생성을 위한 비디오 프레임 셋 생성 ··················· 85
<부록 4> 수어 인식(추론) 및 재추론 판단 과정 ····························· 86
<부록 5> CNN과 LSTM을 활용한 비디오 인식 기계학습 ··················· 87
<부록 6> 모션 벡터를 활용한 비디오 인식 기계학습 ························ 88