지원사업
학술연구/단체지원/교육 등 연구자 활동을 지속하도록 DBpia가 지원하고 있어요.
커뮤니티
연구자들이 자신의 연구와 전문성을 널리 알리고, 새로운 협력의 기회를 만들 수 있는 네트워킹 공간이에요.
논문 기본 정보
- 자료유형
- 학술저널
- 저자정보
- 저널정보
- 한국통계학회 CSAM(Communications for Statistical Applications and Methods) CSAM(Communications for Statistical Applications and Methods) 제30권 제5호
- 발행연도
- 2023.9
- 수록면
- 501 - 516 (16page)
이용수
초록· 키워드
오류제보하기
Knowledge distillation (KD) refers to extracting knowledge from a large and complex model (teacher) and transferring it to a relatively small model (student). This can be done by training the teacher model to obtain the activation function values of the hidden or the output layers and then retraining the student model using the same training data with the obtained values. Recently, relational KD (RKD) has been proposed to extract knowledge about relative dierences in training data. This method improved the performance of the student model compared to conventional KDs. In this paper, we propose a new method for RKD by introducing a new loss function for RKD. The proposed loss function is defined using the area dierence between the teacher model and the student model in a specific hidden layer, and it is shown that the model can be successfully compressed, and the generalization performance of the model can be improved. We demonstrate that the accuracy of the model applying the method proposed in the study of model compression of audio data is up to 1.8% higher than that of the existing method. For the study of model generalization, we demonstrate that the model has up to 0.5% better performance in accuracy when introducing the RKD method to self-KD using image data.
목차
Abstract
1. Introduction
2. Related work
3. Proposed methodology
4. Experimental study
5. Conclusions
References
참고문헌 (0)
참고문헌 신청함께 읽어보면 좋을 논문
논문 유사도에 따라 DBpia 가 추천하는 논문입니다. 함께 보면 좋을 연관 논문을 확인해보세요!
-
Dickey-Fuller Test for an Extended MA Model
Quantitative Bio-Science
2019 .05
-
Fixed point meromorphic function with dierence polynomials
Proceedings of the Jangjeon Mathematical Society
2024 .01
-
Integrated Analytic Methodology Using Visual Image and Meta-Data for Product Recommendation
Quantitative Bio-Science
2022 .05
-
The alignment between contextual and model generalization: An application with PISA 2015
CSAM(Communications for Statistical Applications and Methods)
2024 .09
-
Deeper Integrative Neural Network Analysis for Multi-level Omics Data
Quantitative Bio-Science
2019 .11
-
Nobel-winning microRNA, the micromaestro of gene silencing
Molecules and Cells
2024 .01
-
Classification of Covid-19 Infection Based on Chest X-ray Pictures Using OpenCv and Convolution Neural Networks
Quantitative Bio-Science
2022 .11
-
Theoretical Discussion on Mathematical Knowledge for Teaching from Constructivists’ Perspective
수학교육연구
2015 .01
-
Tutorial and applications of convolutional neural network models in image classification
한국데이터정보과학회지
2022 .05
-
Integration of a deep-learning-based fire model into a global land surface model
한국기상학회 학술대회 논문집
2023 .11
-
Prediction of the days in hospital for next year using ensemble learning
한국데이터정보과학회지
2020 .03
-
Fuzzy regression model with monotonic response function
대한수학회논문집
2018 .01
-
Two-dimensional attention-based multi-input LSTM for time series prediction
CSAM(Communications for Statistical Applications and Methods)
2021 .01
-
A Simple Method to Obtain Maximum Likelihood Estimates for Identity Poisson Regression Models
Journal of The Korean Data Analysis Society
2015 .01
-
Certain Models of the Lie Algebra K5 and Their Connection with Special Functions
Kyungpook Mathematical Journal
2018 .01
-
북서태평양 태풍 강도 예측 컨센서스 기법
대기
2018 .09
-
A Comparison Study of the Test for Right Censored and Grouped Data
CSAM(Communications for Statistical Applications and Methods)
2015 .07
-
Grouping stocks using dynamic linear models
CSAM(Communications for Statistical Applications and Methods)
2022 .11
-
A review and comparison of convolution neural network models under a unified framework
CSAM(Communications for Statistical Applications and Methods)
2022 .03
-
평택 일부지역 대학생의 영양지식 수준과 식습관에 관한 연구
동아시아식생활학회지
2020 .06
이 논문의 저자 정보
최근 본 자료
댓글(0)
0
UCI(KEPA) : I410-151-24-02-088081425