publications

publications by categories in reversed chronological order. generated by jekyll-scholar.

2024

  1. Preprint
    OwLore: Outlier-weighed Layerwise Sampled Low-Rank Projection for Memory-Efficient LLM Fine-tuning
    Pengxiang Li, Lu Yin, Xiaowei Gao, and 1 more author
    arXiv preprint arXiv:2405.18380, 2024
  2. Preprint
    Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients
    Zhenyu Zhang, Ajay Jaiswal, Lu Yin, and 4 more authors
    arXiv preprint arXiv:2407.08296, 2024
  3. Preprint
    From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients
    Ajay Jaiswal, Lu Yin, Zhenyu Zhang, and 4 more authors
    arXiv preprint arXiv:2407.11239, 2024
  4. MLSys2024
    Q-Hitter: A Better Token Oracle for Efficient LLM Inference via Sparse-Quantized KV Cache
    Zhenyu Zhang, Shiwei Liu, Runjin Chen, and 3 more authors
    Proceedings of Machine Learning and Systems, 2024
  5. ICML2024
    Outlier weighed layerwise sparsity (owl): A missing secret sauce for pruning llms to high sparsity
    Lu Yin, You Wu, Zhenyu Zhang, and 7 more authors
    2024
  6. ICML2024
    CaM: Cache Merging for Memory-efficient LLMs Inference
    Yuxin Zhang, Yuxuan Du, Gen Luo, and 4 more authors
    In Forty-first International Conference on Machine Learning, 2024
  7. ICML2024
    Junk DNA Hypothesis: Pruning Small Pre-Trained Weights $\backslashtextit {Irreversibly} and \backslashtextit {Monotonically} $ Impairs“Difficult" Downstream Tasks in LLMs
    Lu Yin, AJAY KUMAR JAISWAL, Shiwei Liu, and 2 more authors
    In Forty-first International Conference on Machine Learning, 2024
  8. Interspeech2024
    Dynamic Data Pruning for Automatic Speech Recognition
    Qiao Xiao, Pingchuan Ma, Adriana Fernandez-Lopez, and 7 more authors
    arXiv preprint arXiv:2406.18373, 2024
  9. Interspeech2024
    MSRS: Training Multimodal Speech Recognition Models from Scratch with Sparse Mask Optimization
    Adriana Fernandez-Lopez, Honglie Chen, Pingchuan Ma, and 5 more authors
    arXiv preprint arXiv:2406.17614, 2024
  10. ICLR2024
    Dynamic sparse no training: Training-free fine-tuning for sparse llms
    Yuxin Zhang, Lirui Zhao, Mingbao Lin, and 6 more authors
    arXiv preprint arXiv:2310.08915, 2024
  11. ICLR2024
    Adamerging: Adaptive model merging for multi-task learning
    Enneng Yang, Zhenyi Wang, Li Shen, and 4 more authors
    arXiv preprint arXiv:2310.02575, 2024

2023

  1. IJCV
    Don’t be so dense: Sparse-to-sparse gan training without sacrificing performance
    Shiwei Liu, Yuesong Tian, Tianlong Chen, and 1 more author
    International Journal of Computer Vision, 2023
  2. ICLR2023
    More convnets in the 2020s: Scaling up kernels beyond 51x51 using sparsity
    Shiwei Liu, Tianlong Chen, Xiaohan Chen, and 7 more authors
    arXiv preprint arXiv:2207.03620, 2023
  3. ICLR2023
    Revisiting pruning at initialization through the lens of ramanujan graph
    Duc NM Hoang, Shiwei Liu, Radu Marculescu, and 1 more author
    2023
  4. ICLR2023
    Sparse moe as the new dropout: Scaling dense and self-slimmable transformers
    Tianlong Chen, Zhenyu Zhang, Ajay Jaiswal, and 2 more authors
    2023
  5. ICLR2023
    Sparsity may cry: Let us fail (current) sparse neural networks together!
    Shiwei Liu, Tianlong Chen, Zhenyu Zhang, and 4 more authors
    2023

2022

  1. LoG2022
    You can have better graph neural networks by not training weights at all: Finding untrained gnns tickets
    Tianjin Huang, Tianlong Chen, Meng Fang, and 8 more authors
    2022
  2. ICLR2022
    The unreasonable effectiveness of random pruning: Return of the most naive baseline for sparse training
    Shiwei Liu, Tianlong Chen, Xiaohan Chen, and 4 more authors
    arXiv preprint arXiv:2202.02643, 2022
  3. ICLR2022
    Deep ensembling with no overhead for either training or testing: The all-round blessings of dynamic sparsity
    Shiwei Liu, Tianlong Chen, Zahra Atashgahi, and 6 more authors
    arXiv preprint arXiv:2106.14568, 2022

2021

  1. NeurIPS2021
    Sparse training via boosting pruning plasticity with neuroregeneration
    Shiwei Liu, Tianlong Chen, Xiaohan Chen, and 7 more authors
    Advances in Neural Information Processing Systems, 2021
  2. ICML2021
    Do we actually need dense over-parameterization? in-time over-parameterization in sparse training
    Shiwei Liu, Lu Yin, Decebal Constantin Mocanu, and 1 more author
    In International Conference on Machine Learning, 2021
  3. ICML2021
    Selfish sparse rnn training
    Shiwei Liu, Decebal Constantin Mocanu, Yulong Pei, and 1 more author
    In International Conference on Machine Learning, 2021