- I’m a PhD student of Machine Learning at GaTech and a researcher at EleutherAI.
- I was a research intern at Google during the summer 2022.
- My primary interest is language models, but I’m also interested in many other areas of ML, including RL, representation learning and generative modeling in general.
- I tweet about promising newest ML papers and my thoughts on ML.
Publications & Projects
- Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints, Aran Komatsuzaki, Joan Puigcerver, James Lee-Thorp, Carlos Riquelme Ruiz, Basil Mustafa, Joshua Ainslie, Yi Tay, Mostafa Dehghani, Neil Houlsby. ICLR 2023.
- LAION-400M: Open-Source Dataset of CLIP-Filtered 400 Million Image-Text Pairs (project), Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, Aran Komatsuzaki. NeurIPS 2021 DCAI workshop.
- GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model, Ben Wang, Aran Komatsuzaki.
- Current Limitations of Language Models: What You Need is Retrieval, Aran Komatsuzaki.
- One Epoch Is All You Need, Aran Komatsuzaki.
- Extractive Summary as Discrete Latent Variables, Aran Komatsuzaki.
Expository Works & Slides & Misc.
- Proposal: Fairer Evaluation of Long-Range Language Models – April 2020
- Written Report: Improving Transformer Language Model – March 2020
- Mirror Symmetry and Mirror Conjecture– April 2017
- Weinstein Manifolds and Skeleta – November 2016 (Incomplete)
- Reshetikhin-Turaev and Turaev-Viro Models – October 2016
- Introduction to Simplicial Homotopy Theory and Higher Category – May 2016
Aran Komatsuzaki 