Publications

Dynamic Sparsity Is Channel-Level Sparsity Learner

Published in Conference on Neural Information Processing Systems, 2023

Lu Yin, Gen Li, Meng Fang, Li Shen, Tianjin Huang, Zhangyang Wang, Vlado Menkovski, Xiaolong Ma, Mykola Pechenizkiy, Shiwei Liu

Download here

Don’t Be So Dense: Sparse-to-Sparse GAN Training Without Sacrificing Performance

Published in International Journal of Computer Vision (IJCV), 2023

Shiwei Liu, Yuesong Tian, Tianlong Chen, Li Shen

Download here

Are Large Kernels Better Teachers than Transformers for ConvNets?

Published in International Conference on Machine Learning, ICML, 2023

Tianjin Huang, Lu Yin, Zhenyu Zhang, Li Shen, Meng Fang, Mykola Pechenizkiy, Zhangyang Wang, Shiwei Liu

Download here

Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together!

Published in The International Conference on Learning Representations, ICLR, Notable-Top-25% Spotlight, 2023

Shiwei Liu*, Tianlong Chen*, Zhenyu Zhang, Xuxi Chen, Tianjin Huang, AJAY KUMAR JAISWAL, Zhangyang Wang

Download here

More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity

Published in The International Conference on Learning Representations, ICLR, 2023

Shiwei Liu, Tianlong Chen, Xiaohan Chen, Xuxi Chen, Qiao Xiao, Boqian Wu, Mykola Pechenizkiy, Decebal Mocanu, Zhangyang Wang

Download here

Sparse MoE with Random Routing as the New Dropout: Training Bigger and Self-Scalable Models

Published in The International Conference on Learning Representations, ICLR, Notable-Top-25% Spotlight., 2023

Tianlong Chen, Zhenyu Zhang, AJAY KUMAR JAISWAL, Shiwei Liu, Zhangyang Wang

Download here

You Can Have Better Graph Neural Networks by Not Training Weights at All: Finding Untrained GNNs Tickets

Published in Proceedings of the First Learning on Graphs Conference (LoG 2022), Best Paper Award, 2022

Tianjin Huang, Tianlong Chen, Meng Fang, Vlado Menkovski, Jiaxu Zhao, Lu Yin, Yulong Pei, Decebal Constantin Mocanu, Zhangyang Wang, Mykola Pechenizkiy, Shiwei Liu

Download here

Lottery Pools: Winning More by Interpolating Tickets without Increasing Training or Inference Cost

Published in AAAI Conference on Artificial Intelligence, AAAI 2023, 2022

Lu Yin*, Shiwei Liu*, Meng Fang, Tianjin Huang, Vlado Menkovski, Mykola Pechenizkiy

Download here

Dynamic Sparse Network for Time Series Classification: Learning What to “See”

Published in 36th Annual Conference on Neural Information Processing Systems, NeurIPS2022, 2022

Qiao Xiao, Boqian Wu, Yu Zhang, Shiwei Liu, Mykola Pechenizkiy, Elena Mocanu, Decebal Constantin Mocanu

Download here

The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training

Published in The International Conference on Learning Representations, ICLR, 2022

Shiwei Liu, Tianlong Chen, Xiaohan Chen, Li Shen, Decebal Constantin Mocanu, Zhangyang Wang, Mykola Pechenizkiy

Download here

Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity

Published in The International Conference on Learning Representations, ICLR, 2022

Shiwei Liu, Tianlong Chen, Zahra Atashgahi, Xiaohan Chen, Ghada Sokar, Elena Mocanu, Mykola Pechenizkiy, Zhangyang Wang, Decebal Constantin Mocanu

Download here

Sparse Training via Boosting Pruning Plasticity with Neuroregeneration

Published in Conference on Neural Information Processing Systems (NeurIPS 2021), 2021

Shiwei Liu, Tianlong Chen, Xiaohan Chen, Zahra Atashgahi, Lu Yin, Huanyu Kou, Li Shen, Mykola Pechenizkiy, Zhangyang Wang, Decebal Constantin Mocanu

Download here

Selfish sparse rnn training

Published in Proceedings of Machine Learning Research, ICML, 2021

Shiwei Liu, Decebal Constantin Mocanu, Yulong Pei, Mykola Pechenizkiy

Download here

Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training

Published in International Conference on Machine Learning, ICML, 2021

Shiwei Liu, Lu Yin, Decebal Constantin Mocanu, Mykola Pechenizkiy

Download here

Sparse evolutionary Deep Learning with over one million artificial neurons on commodity hardware

Published in Neural Computing and Applications, 2021

Shiwei Liu, Decebal Constantin Mocanu, Amarsagar Reddy Ramapuram Matavalam, Yulong Pei, Mykola Pechenizkiy

Download here

Topological Insights into Sparse Neural Networks

Published in ECMLPKDD, 2020

Shiwei Liu, Tim Van der Lee, Anil Yaman, Zahra Atashgahi, Davide Ferraro, Ghada Sokar, Mykola Pechenizkiy, Decebal Constantin Mocanu

Download here