Shiwei Liu

Principal Investigator · Group Leader

Shiwei Liu

Building more capable and efficient AI

ELLIS Institute Tübingen ↗ Max Planck Institute for Intelligent Systems ↗
sliu@tue.ellis.eu
Portrait of Shiwei Liu
About

I am a Principal Investigator at ELLIS Institute Tübingen and a group leader at the Max Planck Institute for Intelligent Systems. Previously, I was a Royal Society Newton International Fellow at the University of Oxford and a Junior Research Fellow at Somerville College. Before that, I worked with Atlas Wang as a postdoctoral fellow at UT Austin. I received my PhD from Eindhoven University of Technology, supervised by Mykola Pechenizkiy and Decebal Constantin Mocanu.

Research

Current directions

Advancing foundation models across pre-training, efficiency, post-training, architecture, and data.

03

Post-training

Supervised fine-tuning and reinforcement learning for stronger reasoning models.

Join us

We are hiring.

I am looking for self-motivated PhD students and postdocs based at ELLIS Institute Tübingen, with opportunities to connect with leading universities including ETH Zürich and the University of Oxford.

Potential collaborations: Please get in touch with the research area you would like to contribute to.

news

Jul 2026 🧑‍🤝‍🧑 NeurIPS 2026 Workshop: Our proposal, On-Device Intelligence: Foundation Models under Real-World Constraints, was accepted!
📝 COLM 2026: One paper accepted — Layer Pruning Hurts Test-Time Scaling.
May 2026 📝 6 papers got accepted in ICML 2026, GradientStabilizer, Sparsity helps curse of depth, Layerwise LR for LLMs, MoE test-time-adaptation, Diffusion Beam Search, Efficient cache for video generation.
Mar 2026 I am delighted to see that our LayerNorm Scaling has already become a winning technique in OpenAI’s Parameter Golf challenge.
I serve as the Senior Area Chair of NeurIPS 2026 DB track.
Jan 2026 📝 3 papers are accepted by ICLR 2026: Early Commit of DLM (oral), LLM Merging, Neural Sum-of-Squares.
Oct 2025 I serve as the Area Chair of ICML 2026 and ARR 2025.
Sep 2025 I will give invited talks in Tsinghua University (Shenzhen) and CUHK (Shenzhen) in September 2025.
📝 Three papers got accepted by NeurIPS 2025: Curse of Depth in LLMs, Layerwise Weight Decay, and Gated Activation Scaling.
I serve as the Program Chair for the third Conference on Parsimony and Learning (CPAL 2026) in Tübingen, Germany.
May 2025 📝 Outlier-weighed Layerwise Sampling Fine-tuning got accepted by ACL 2025.
📝3 papers got accepted by ICML 2025: Masked Next-Token Prediction, How Low-Rank Weights Emerge for LLMs, Principal Weights for LLM Fine-tuning.
Jan 2025 📝3 papers got accepted by ICLR 2025: Mix Layer Normalization, Spike-Aware Adam, LLM Composable Interventions.
Dec 2024 📝 Visual prompting for pruning got accepted by AAAI 2025.
🧑‍🤝‍🧑 We are organizing the Workshop on Sparsity in LLMs (SLLM) at ICLR 2025. Link
🧑‍🤝‍🧑 We are organizing the Scalable Optimization for Efficient and Adaptive Foundation Models (SCOPE) workshop at ICLR 2025. Link
Sep 2024 I serve as an Area Chair for ICASSP 2025.
📝 3 papers are accepted by NeurIPS 2024: Found in the Middle, Sparse 3D Medical, Alpha Pruning.
📝 2 papers are accepted by EMNLP 2024: Layer-skip LLM and Is c4 enough for LLM Pruning?
Jul 2024 I am honored to join in the organization committee of the Conference on Parsimony and Learning (CPAL). See you in Stanford.
Jun 2024 I am very happy to be offered as a Junior Research Fellow (JRF) at Somerville College, one of the first two women’s colleges at Oxford.
I will have a talk tour around Europe at the NLP group at University of Sheffield, LTL group at University of Cambridge, and BlueNN group at University of Luxembourg. :sparkles: :smile:
📝 2 papers got accepted by Interspeech 2024: Sparse Multimodal from Scratch, Dynamic Data Pruning for Speech.
May 2024 Our “Edge LLMs: Edge-Device Large Language Model Competition” competition has been accepted by NeurIPS 2024. Submission opens Link.
📝 1 paper Q-Hitter: Quantized-Sparse KV Cache got accepted by MLSys 2024.
📝 5 papers got accepted by ICML 2024: Layerwise Importance for LLMs, LLM Junk DNA, Bi-Level DST, KV Cache Merging, Saprse Cocktail.
Jan 2024 📝 3 papers got accepted by ICLR 2024: Training-Free Sparse LLM Fine-tuning, Multi-Task Vector Merging, Sparse Training with Neuron Revitalization.
📝 4 papers got accepted by NeurIPS 2023: Channel-Level DST, Essential Sparsity, Pruning Topology, Note-Path Balance.
🏆 I am highly honored to receive the Rising Star in AI from KAUST and will give a talk at Rising Stars in AI Symposium.
Nov 2023 📝 Block Sparse Training accepted by CPAL.
Oct 2023 🏆 I am highly grateful to receive the Best PhD Dissertation Runner-up Award from the Informatics Europe.
🏆 I am highly honered to receive the Rising Star Award from CPAL and will give a presentation at HKU in Jan 2024.
Sep 2023 🚀 I am grateful to receive the prestigious Newton International Fellowship from the British Academy and the Royal Society.
May 2023 📝 The work I conducted during my internship at JD Academy has been accepted by International Journal of Computer Vision (IJCV) - STU-GAN.
Apr 2023 📝 3 papers got accepted in ICML 2023, Instant Soup (Oral), Large Kernel Distillation, and Graph Ladling.
Mar 2023 📝 2 papers SNN Ten Lessons and Channel-Level DST paper has been accepted as spotlight presentations at the SNN workshop.
Jan 2023 📝 4 papers got accepted in ICLR 2023, Ramanujan Graph Pruning (oral), Sparsity May Cry Benchmark (spotlight), MoE as Dropout (spotlight), SLaK:51x51 Large Conv.
Dec 2022 📝 Our Untrained GNNs paper received the Best Paper Award from LoG 2022.11/2022*,
Nov 2022 📝 One paper Lottery-Pools got accepted in AAAI 2023.9/2022*,
Apr 2022 I got my PhD thesis abstract accepted by IDA 2022, which was also the first conference (symposium) that I attended in the first year of my PhD. PhD life is a cycle :).
Our tutorial Sparse Neural Networks Training has been accepted at ECMLPKDD 2022.
Jan 2022 📝 Two of my first-author papers are accepted by ICLR 2022: Random pruning and FreeTickets.
Sep 2021 🏆 I receive the “Outstanding Intern” honor in JD Academy Explore
📝 One paper got accepted by NeurIPs 2021: GraNet.
May 2021 📝 2 papers are accepted by ICML 2021: In-Time Over-Parameterization and Selfish RNN.