Sensitivity, Causality, and Repair Dissociate: A Layer-Wise Analysis of Perturbation Robustness and Its Scaling
A layer-wise analysis of perturbation robustness across five language models (Phi-3.5, Gemma-2-9B, Llama-3, Mistral, Qwen2.5-7B) reveals that sensitivity, causality, and compensatory capacity dissociate. Two propagation regimes are identified: spike-and-suppress and late-accumulation. On models meeting an 80% identity-patch gate, sensitivity and causality are anti-correlated (rho = -0.72 to -0.88). Scaling Qwen2.5 from 1.5B to 14B strengthens the late-accumulation signature. Cascade disruption is proposed as the mechanism, and a fixed-harness layer sweep on GSM8K confirms the prediction.
Development
- First ReportSensitivity, Causality, and Repair Dissociate: A Layer-Wise Analysis of Perturbation Robustness and Its ScalingarXiv cs.CL
- Current AssessmentThis research provides a framework for diagnosing and repairing robustness issues in language models, which is relevant for production systems that encounter noisy input. The finding that diagnostic-flagged sites are the worst adapter placements challenges common practices in model patching. The identification of two propagation regimes suggests that different model families may require different repair strategies. The scaling trend implies that as models grow, the nature of robustness failures changes, which could influence future model development and deployment decisions.Agent Pulse · analysis
This paper investigates where in a language model's layers failures occur when input is perturbed (typos, OCR noise, homophones). The authors define three operationalizations: sensitivity (where representations diverge most), causality (where restoring clean activations recovers prediction), and compensatory capacity (where a small adapter can repair damage). Across five models, these layer maps dissociate, meaning the layers responsible for each are different. Two propagation regimes are identified: spike-and-suppress (Phi-3.5, Gemma-2-9B) and late-accumulation (Llama-3, Mistral, Qwen2.5-7B). On models meeting an 80% identity-patch gate, sensitivity and causality are anti-correlated (rho = -0.72 to -0.88). Scaling Qwen2.5 from 1.5B to 14B shows the late-accumulation signature strengthening monotonically, corroborated on a second family. The proposed mechanism, cascade disruption, suggests that adapters placed at causally implicated early layers break intact downstream computation, making diagnostic-flagged sites the worst adapter placements. A fixed-harness layer sweep across four models (3.8-8B) confirms the core prediction on chain-of-thought GSM8K.
The dissociation of sensitivity, causality, and compensatory capacity implies that layer-wise diagnostics for robustness must be carefully defined; a single metric is insufficient. The anti-correlation between sensitivity and causality (rho = -0.72 to -0.88) suggests that the layers where representations diverge most are not the ones where intervention is most effective. The cascade disruption mechanism warns that placing adapters at causally implicated early layers can break downstream computation, so adapter placement should be guided by compensatory capacity rather than causality alone. The scaling trend in Qwen2.5 (1.5B to 14B) indicates that larger models exhibit stronger late-accumulation, which may affect how robustness interventions scale.
This research provides a framework for diagnosing and repairing robustness issues in language models, which is relevant for production systems that encounter noisy input. The finding that diagnostic-flagged sites are the worst adapter placements challenges common practices in model patching. The identification of two propagation regimes suggests that different model families may require different repair strategies. The scaling trend implies that as models grow, the nature of robustness failures changes, which could influence future model development and deployment decisions.
For companies deploying LLMs in production, this research offers a method to identify where to intervene to improve robustness against noisy input, potentially reducing error rates and improving user experience. The finding that adapter placement at causally implicated layers is suboptimal could save costs by avoiding ineffective repairs. Understanding the scaling trend can inform decisions about model size versus robustness, potentially guiding investment in larger models or in robustness techniques.
Future work could extend this analysis to more model families and larger scales to confirm the universality of the two regimes. The proposed cascade disruption mechanism could be tested in real-world repair scenarios. The scaling trend suggests that as models grow, the late-accumulation signature may become more pronounced, potentially requiring new intervention points. The framework could be applied to other perturbation types and tasks beyond GSM8K.