Tri Dao

tri_photo_2021_04.jpeg
tri [at] tridao (dot) me

Assistant Professor of Computer Science at Princeton University.
Chief Scientist at Together AI.

CV

Previously: PhD, Department of Computer Science, Stanford University

Research Interests

Machine learning and systems, with a focus on efficient training and long-range context:

  • Efficient Transformer training and inference.
  • Sequence models with long-range memory.
  • Structured sparsity for compact deep learning models.

latest posts

selected publications

  1. Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
    Tri Dao*, and Albert Gu*
    In International Conference on Machine Learning (ICML) , 2024
  2. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
    Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher Ré
    In Advances in Neural Information Processing Systems , 2022
  3. Mamba: Linear-Time Sequence Modeling with Selective State Spaces
    Albert Gu*, and Tri Dao*
    arXiv preprint arXiv:2312.00752, 2023
  4. Monarch: Expressive Structured Matrices for Efficient and Accurate Training
    Tri Dao, Beidi Chen, Nimit Sohoni, Arjun Desai, Michael Poli, Jessica Grogan, Alexander Liu, Aniruddh Rao, Atri Rudra, and Christopher Ré
    In International Conference on Machine Learning (ICML) , 2022