Pruning

CVPR 2024arXiv:2403.14737

#2

Isomorphic Pruning for Vision Models

Gongfan Fang, Xinyin Ma, Michael Bi Mi et al.

FedMef: Towards Memory-efficient Federated Dynamic Pruning

Hong Huang, Weiming Zhuang, Chen Chen et al.

18

dynamic pruningspeech foundation modelsmultimodal modelscomputational efficiency+4

#4

What Makes a Good Prune? Maximal Unstructured Pruning for Maximal Cosine Similarity

Gabryel Mason-Williams, Fredrik Dahlqvist

You Only Prune Once: Designing Calibration-Free Model Compression With Policy Learning

Ayan Sengupta, Siddhant Chaudhary, Tanmoy Chakraborty

Efficient Fine-Tuning and Concept Suppression for Pruned Diffusion Models

Reza Shirkavand, Peiran Yu, Shangqian Gao et al.

DARE the Extreme: Revisiting Delta-Parameter Pruning For Fine-Tuned Models

Wenlong Deng, Yize Zhao, Vala Vakilian et al.

Context-aware Dynamic Pruning for Speech Foundation Models

Masao Someki, Yifan Peng, Siddhant Arora et al.

ICLR 2025

7

NeurIPS 2025arXiv:2503.09657

#9

Týr-the-Pruner: Structural Pruning LLMs via Global Sparsity Distribution Optimization

Guanchen Li, Yixing Xu, Zeping Li et al.

structural pruninglarge language modelssparsity distribution optimizationglobal pruning+3

6

ICLR 2025arXiv:2502.17941

#10

Optimal Brain Apoptosis

Mingyuan Sun, Zheng Fang, Jiaxu Wang et al.

neural network pruningparameter importance estimationhessian matrix analysissecond-order taylor expansion+4

3

ICLR 2025arXiv:2501.12486

#11

ICP: Immediate Compensation Pruning for Mid-to-high Sparsity

Xin Luo, Fu Xueming, Zihang Jiang et al.

Flexible Group Count Enables Hassle-Free Structured Pruning

Jiamu Zhang, Shaochen Zhong, Andrew Ye et al.

The Journey Matters: Average Parameter Count over Pre-training Unifies Sparse and Dense Scaling Laws

Tian Jin, Ahmed Imtiaz Humayun, Utku Evci et al.

sparse pre-traininglarge language modelspruning schedulesscaling laws+4

1

ECCV 2024arXiv:2403.16020

#14

Probe Pruning: Accelerating LLMs through Dynamic Pruning via Model-Probing

Qi Le, Enmao Diao, Ziyan Wang et al.

PaPr: Training-Free One-Step Patch Pruning with Lightweight ConvNets for Faster Inference

Tanvir Mahmud, Burhaneddin Yaman, Chun-Hao Liu et al.

patch pruninglightweight convnetsvision transformersredundant data elimination+4

model compressiongradient reweightingpruning efficiencylarge language models+3

#16

Less is More: Unlocking Specialization of Time Series Foundation Models via Structured Pruning

Lifan Zhao, Yanyan Shen, Zhaoyang Liu et al.

Computation and Memory-Efficient Model Compression with Gradient Reweighting

Zhiwei Li, Yuesen Liao, Binrui Wu et al.

NeurIPS 2025

NeurIPS 2025arXiv:2509.23931

#18

Backdoor Mitigation via Invertible Pruning Masks

Kealan Dunnett, Reza Arablouei, Volkan Dedeoglu et al.

Each Complexity Deserves a Pruning Policy

Hanshi Wang, Yuhao Xu, Zekun Xu et al.

vision-language modelstoken pruningcomplexity-adaptive pruningvisual token redundancy+3

CVPR 2025arXiv:2503.02175

#20

DivPrune: Diversity-based Visual Token Pruning for Large Multimodal Models

Saeed Ranjbar Alvar, Gursimran Singh, Mohammad Akbari et al.

visual token pruningmultimodal modelsattention mechanisminference efficiency+4

ICCV 2025arXiv:2507.12988

#21

Straightforward Layer-wise Pruning for More Efficient Visual Adaptation

Ruizi Han, Jinglei Tang

Faster Parameter-Efficient Tuning with Token Redundancy Reduction

Kwonyoung Kim, Jungin Park, Jin Kim et al.

Variance-Based Pruning for Accelerating and Compressing Trained Networks

Uranik Berisha, Jens Mehnert, Alexandru Condurache

structured pruningmodel compressionactivation statisticsvision transformers+4

ICML 2025arXiv:2505.23807

#24

EfficientLLaVA: Generalizable Auto-Pruning for Large Vision-language Models

Yinan Liang, Ziwei Wang, Xiuwei Xu et al.

Lightweight Dataset Pruning without Full Training via Example Difficulty and Prediction Uncertainty

Yeseul Cho, Baekrok Shin, Changmin Kang et al.

Targeted Low-rank Refinement: Enhancing Sparse Language Models with Precision

Li Shen, Anke Tang, Yong Luo et al.

DLP: Dynamic Layerwise Pruning in Large Language Models

Yuli Chen, Bo Cheng, Jiale Han et al.

ICML 2024arXiv:2406.02924

#28

Let LLM Tell What to Prune and How Much to Prune

Mingzhe Yang, Sihao Lin, Changlin Li et al.

Optimal Decision Tree Pruning Revisited: Algorithms and Complexity

Juha Harviainen, Frank Sommer, Manuel Sorge et al.

Resource-Efficient Transformer Pruning for Finetuning of Large Models

Fatih Ilhan, Gong Su, Selim Tekin et al.

Pruner-Zero: Evolving Symbolic Pruning Metric From Scratch for Large Language Models

Peijie Dong, Lujun Li, Zhenheng Tang et al.

symbolic pruning metricspost-training pruninglarge language modelsgenetic programming+2

ICML 2024arXiv:2402.01089

#32

No Free Prune: Information-Theoretic Barriers to Pruning at Initialization

Tanishq Kumar, Kevin Luo, Mark Sellke

lottery ticket hypothesispruning at initializationeffective parameter countmutual information+3

ICML 2025arXiv:2502.04348

#33

BilevelPruning: Unified Dynamic and Static Channel Pruning for Convolutional Neural Networks

Shangqian Gao, Yanfu Zhang, Feihu Huang et al.

$\mathbb{D}^2$ Pruning: Message Passing for Balancing Diversity & Difficulty in Data Pruning

Adyasha Maharana, Prateek Yadav, Mohit Bansal

Prompt-based Depth Pruning of Large Language Models

Juyun Wee, Minjae Park, Jaeho Lee

ICML 2024arXiv:2401.12200

#36

APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference

Bowen Zhao, Hannaneh Hajishirzi, Qingqing Cao

parameter-efficient fine-tuningstructured pruninglanguage model compressionadaptive parameter tuning+4

CVPR 2024arXiv:2403.14729

#37

MoPE-CLIP: Structured Pruning for Efficient Vision-Language Models with Module-wise Pruning Error Metric

Haokun Lin, Haoli Bai, Zhili Liu et al.

Auto-Train-Once: Controller Network Guided Automatic Network Pruning from Scratch

Xidong Wu, Shangqian Gao, Zeyu Zhang et al.

AAAI 2024arXiv:2402.17862

#39

REPrune: Channel Pruning via Kernel Representative Selection

Mincheol Park, Dongjin Kim, Cheonjun Park et al.

CVPR 2024arXiv:2404.05621

#40

MEPSI: An MDL-Based Ensemble Pruning Approach with Structural Information

Xiao-Dong Bi, Shao-Qun Zhang, Yuan Jiang

MULTIFLOW: Shifting Towards Task-Agnostic Vision-Language Pruning

Matteo Farina, Massimiliano Mancini, Elia Cunegatti et al.

AAAI 2025arXiv:2408.14721

#42

Separate the Wheat from the Chaff: Model Deficiency Unlearning via Parameter-Efficient Module Operation

Xinshuo Hu, Dongfang Li, Zihao Zheng et al.

PAT: Pruning-Aware Tuning for Large Language Models

Yijiang Liu, Huanrui Yang, Youxin Chen et al.

llm pruningtransformer block redundancymodel compressioninference acceleration+2

#44

SLEB: Streamlining LLMs through Redundancy Verification and Elimination of Transformer Blocks

Jiwon Song, Kyungseok Oh, Taesu Kim et al.

ICML 2024

ICML 2024arXiv:2402.12479

#45

In value-based deep reinforcement learning, a pruned network is a good network

Johan Obando Ceron, Aaron Courville, Pablo Samuel Castro

deep reinforcement learningvalue-based methodssparse trainingnetwork pruning+2

model pruningnetwork sparsitycombinatorial searchoverparameterization effects+3

#46

Sparsest Models Elude Pruning: An Exposé of Pruning’s Current Capabilities

Stephen Zhang, Vardan Papyan

ICML 2024