About Me

I am currently a PhD student at KAIST, studying at the Machine Learning and Artificial Intelligence Lab under Professor Sung Ju Hwang. My research interests include:

• Large Language Models
• Efficient Transformers
• Self-Supervised Learning
• Permutation Invariant Neural Networks
• Computer Vision
• Bayesian Deep Learning

Conference Publications

[c6] SEA: Sparse Linear Attention with Estimated Attention Mask

•  Heejun Lee, Jina Kim, Jeffrey Willette, Sung Ju Hwang (*: equal contribution)
• [ICLR 2024] [Open Review] [arxiv] [bibtex]

[w1] Visulizing the Loss Landscape of Self-Supervised Vision Transformers

•  Youngwan Lee, Jeffrey Willette, Jonghee Kim, Sung Ju Hwang
• [NeurIPS 2023 - SSL Theory and Practice (workshop)] [Presentation]

[c5] Scalable Set Encoding with Universal Mini-Batch Consistency and Unbiased Full Set Gradient Approximation

•  Jeffrey Willette*, Seanie Lee*, Bruno Andreis, Kenji Kawaguchi, Juho Lee, Sung Ju Hwang (*: equal contribution)
• [ICML 2023] [Open Review] [arxiv] [bibtex]

[c4] Exploring the Role of Mean Teachers in Self-Supervised Masked Auto-Encoders

•  Youngwan Lee*, Jeffrey Willette*, Jonghee Kim, Juho Lee, Sung Ju Hwang (*:equal contribution)
• [ICLR 2023] [Open Review] [arxiv] [bibtex]

[c3] MPViT: Multi-Path Vision Transformer for Dense Prediction

•  Youngwan Lee, Jonghee Kim, Jeffrey Willette, Sung Ju Hwang
• [CVPR 2022] [arxiv] [bibtex]

[c2] Meta Learning Low Rank Covariance Factors for Energy-Based Deterministic Uncertainty

•  Jeffrey Willette, Hae Beom Lee, Juho Lee, Sung Ju Hwang
• [ICLR 2022] [Open Review] [arxiv] [bibtex]

[c1] MiniBatch Consistent Slot Set Encoder for Scalable Set Encoding

•  Bruno Andreis, Jeffrey Willette, Juho Lee, Sung Ju Hwang
• [NeurIPS 2021] [Open Review] [arxiv] [bibtex]

Preprints

Training-Free Exponential Extension of Sliding Window Context with Cascading KV Cache

•  Jeffrey Willette, Heejun Lee, Youngwan Lee, Myeongjae Jeon, Sung Ju Hwang
•  [arxiv] [bibtex]

Prior Augmented Data: Targeted Entropy Regularization for Improved Calibration

•  Jeffrey Willette, Juho Lee, Sung Ju Hwang
•  [arxiv] [bibtex]