talks

Introduction of RoPE/Yarn and PaTH for modern position encoding in transformers.

Introduction of DeltaNet and its variants for more expressive yet scalable RNNs.

Discussion on developments in linear attention and its variants.

Introduction of linear transformers for efficient sequence modeling.

Introduction of gated linear recurrence for efficient sequence modeling.