Songlin Yang

sonta.jpg

Songlin (松琳) is currently a second-year PhD student at MIT CSAIL, advised by Prof. Yoon Kim.

Previously, she obtained her bachelar’s degree from SUSTech in 2020 and her master’s degree from ShanghaiTech in 2023, where she was advised by Prof. Kewei Tu.

Her research is centered on the intersection of machine learning system and large language model, with a specific focus on the hardware-aware algorithm design for efficient sequence modeling.

news

Sep 25, 2024 :loudspeaker: GSA and DeltaNet have been accepted to NeurIPS’24 :fire: :fire:
Aug 20, 2024 Gave a talk at Stanford HazyResearch, “Linear Transformers for Efficient Sequence Modeling
Jun 10, 2024 :loudspeaker: New arxiv “Parallelizing Linear Transformers with the Delta Rule over Sequence Length” with a very beautiful algorithm in it :cherry_blossom:!
May 2, 2024 Gated Linear Attention Transformers (GLA) is accepted to ICML 2024 :smile: Code is available at here.
Apr 25, 2024 Gave a talk at Cornell Tech, “Gated linear Recurrence for Efficient Sequence Modeling
Jan 1, 2024 Introducing our open-source project flash-linear-attention :rocket: :rocket: :rocket:. Join Discord if you are interested in linear attention/RNN!

selected publications

  1. NeurIPS
    Gated Slot Attention for Efficient Linear-Time Sequence Modeling
    Yu Zhang*, Songlin Yang*, Ruijie Zhu, Yue Zhang, Leyang Cui, Yiqiao Wang, Bolun Wang, Freda Shi, Bailin Wang, Wei Bi, Peng Zhou, and Guohong Fu
    In , 2024
  2. NeurIPS
    Parallelizing Linear Transformers with the Delta Rule over Sequence Length
    Songlin Yang, Bailin WangYu ZhangYikang Shen, and Yoon Kim
    In , 2024
  3. COLM
    HGRN2: Gated Linear RNNs with State Expansion
    Zhen Qin*, Songlin Yang*, Weixuan Sun, Xuyang Shen, Dong Li, Weigao Sun, and Yiran Zhong
    In , 2024
  4. ICML
    Gated Linear Attention Transformers with Hardware-Efficient Training
    Songlin Yang*, Bailin Wang*Yikang ShenRameswar Panda, and Yoon Kim
    In , 2024