ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation

Pengcheng Huang and colleagues identified mid-to-deep feed-forward networks that are disproportionately activated during unfaithful generation in retrieval-augmented LLMs. They proposed ParamMute, a framework that suppresses these unfaithfulness-associated FFNs to calibrate models toward retrieved knowledge. Evaluated on the new CoFaithfulQA benchmark and the existing ConFiQA benchmark, ParamMute significantly improved contextual faithfulness and reduced reliance on parametric memory, addressing the persistent influence of internal knowledge during generation.