Prune models to 50% sparsity for faster inference

Reduce LLM size via unstructured/structured pruning

Achieving 50% sparsity with minimal accuracy loss via one-shot pruning without retraining.

• · model
• · pruning method (magnitude|wanda|sparse)
• · sparsity target (0-1)

• · pruned model
• · sparsity metrics

• · torch
• · transformers

• · model loaded
• · GPU available (optional)

• · accuracy drop >threshold
• · OOM during pruning

• · Wanda method (arXiv 2306.11695)
• · SparseGPT (arXiv 2301.00774)
• · N:M sparsity support