Energy- and Memory-Efficient PEFT Methods for Personalized On-Device SLMs on Consumer GPUs
A study compares five fine-tuning methods (Full Fine-Tuning, LoRA, LoRA+, QLoRA, BitFit) on four small language models (TinyLlama-1.1B, Qwen3-1.7B, Mamba-1.4B, Mamba-2-1.3B) across GLUE and LaMP tasks, using energy-focused NetScore-E and memory-focused NetScore-M metrics. LoRA+ achieves the highest NetScore-E in 19 of 24 configurations and highest NetScore-M in 13 of 24, selected in 18 of 24.
发展脉络
- 首次出现Energy- and Memory-Efficient PEFT Methods for Personalized On-Device SLMs on Consumer GPUsarXiv cs.CL
- 当前判断The findings suggest a shift towards energy-aware model selection for edge AI, as on-device personalization becomes more feasible. The dominance of LoRA+ could influence tooling and best practices for developers targeting consumer GPUs. This may accelerate the adoption of SLMs in privacy-sensitive applications where local fine-tuning is preferred.Agent Pulse · 分析
This research evaluates parameter-efficient fine-tuning (PEFT) methods for on-device small language models, focusing on energy and memory efficiency. The study compares five methods across four models from two families (Transformer and SSM) on general and personalization benchmarks. Results show LoRA+ consistently outperforms others in energy and memory efficiency, making it a strong candidate for personalized on-device deployment. The study highlights the importance of considering energy consumption in model selection for resource-constrained environments.
LoRA+ demonstrates superior energy and memory efficiency across diverse architectures, suggesting that its optimized update rules provide better trade-offs for on-device fine-tuning. The inclusion of SSM-based models (Mamba) indicates that PEFT methods generalize beyond Transformer architectures. The use of NetScore-E and NetScore-M as composite metrics offers a practical framework for evaluating deployment constraints.
The findings suggest a shift towards energy-aware model selection for edge AI, as on-device personalization becomes more feasible. The dominance of LoRA+ could influence tooling and best practices for developers targeting consumer GPUs. This may accelerate the adoption of SLMs in privacy-sensitive applications where local fine-tuning is preferred.
For companies deploying AI on consumer devices, adopting LoRA+ can reduce energy costs and memory requirements, enabling more efficient on-device personalization. This can lower infrastructure expenses and improve user experience by enabling real-time customization without cloud dependency, offering a competitive edge in edge AI markets.
Future work may extend these comparisons to larger models and more diverse tasks, potentially establishing standardized benchmarks for energy-efficient fine-tuning. The success of LoRA+ could lead to its integration into mainstream frameworks, and further optimizations may emerge for even lower resource footprints.