지난 4월달에 연구단 초대로 영국 옥스포드 대학교에서 박사과정 중이며 구글 딥마인드에서 파트타임 연구원으로 재직중인 김현직 연구원이 연구주제 Interpretable Models in Probabilistic Deep Learning에 대해 세미나 발표를 해주셨습니다. 김현직 연구원은 ICML, AISTATS, NerIPS, ICLR 등 해외유수학회에 관련주제로 논문을 발표하였습니다. (https://hyunjik11.github.io/)
세미나에서는 딥러닝 모델을 이해하는 방법 등의 이론적인 부분과 모델 개발에 관한 유익한 내용을 전달해 주셨습니다.
아래 두 논문을 중심으로 발표해 주셨고, 발표자료는 저작권 이슈로 공개하지 못함을 양해 부탁드립니다.
https://arxiv.org/abs/1802.05983
Disentangling by Factorising
We define and address the problem of unsupervised learning of disentangled representations on data generated from independent factors of variation. We propose FactorVAE, a method that disentangles by encouraging the distribution of representations to be fa
arxiv.org
https://arxiv.org/abs/1901.05761
Attentive Neural Processes
Neural Processes (NPs) (Garnelo et al 2018a;b) approach regression by learning to map a context set of observed input-output pairs to a distribution over regression functions. Each function models the distribution of the output given an input, conditioned
arxiv.org
영문요약문:
As Deep Learning (DL) solutions to real-world problems are becoming increasingly common, DL researchers are striving to better understand the models that they develop. The community has been using the term ‘interpretability’ to describe models and methods that help us achieve this rather vague goal. However many claim that deep models are inherently uninterpretable due to their black-box nature, and stop paying attention to interpretability in deep models on these grounds. In this talk, we show that ‘deep’ and ‘interpretability’ are not mutually exclusive terms, hence it is both possible and necessary to devise interpretable deep models. We first clarify what is meant by the term ‘interpretability’, by listing its desiderata and properties. We then introduce examples of deep probabilistic models that enjoy various properties of interpretability: the talk will cover FactorVAE (http://proceedings.mlr.press/v80/kim18b.html), a model for learning disentangled representations, and the Attentive Neural Process (https://arxiv.org/abs/1901.05761), a model for learning stochastic processes in a data-driven fashion, focusing on their applications to image data.
Bio:
Hyunjik Kim is a Ph.D. student in Machine Learning at the University of Oxford, supervised by Prof. Yee Whye Teh in the Machine Learning group at the Department of Statistics. In parallel, he works for DeepMind at the Google London office as a research scientist. His research interests lie at the intersection of Bayesian Machine Learning and Deep Learning, especially interpretable models that arise in this intersection.
