Causal Origins and Generality of the Dissociation Between Chain-of-Thought Influence and Verbalised Faithfulness Across Post-Training Recipes

Chain-of-thought (CoT) reasoning in large language models can causally guide the final answer while simultaneously misdescribing the cues that actually drove it. Lewis-Lim et al. (2025) documented this influence-versus-faithfulness dissociation on multiple-choice soft-reasoning benchmarks and showed that reasoning-distilled models rely far more on CoT than instruction-tuned or RL-trained reasoning models and start from much higher initial answer entropy than instruction-tuned models, with RL-trained reasoning models intermediate except on StrategyQA where they are highest, but could only hypothesise why. The open problem is to causally attribute these differences to components of post-training — distillation on procedural CoT outputs versus RLHF/preference optimisation versus standard instruction tuning — through controlled interventions that make the separate contribution of each component identifiable, and to establish whether the dissociation persists outside multiple-choice benchmarks with injected cues, namely in at least one of long-form generation, planning and agentic settings, or under naturally occurring contextual influences. No published work as of August 2026 performs this controlled attribution or the joint influence-plus-faithfulness analysis in the extended regimes.

Background

A chain of thought (CoT) is the sequence of intermediate reasoning steps a large language model (LLM) generates before its final answer. Modern LLMs acquire CoT behaviour through three broad post-training recipes, applied on top of the same pre-training objective: (i) instruction tuning, i.e. supervised fine-tuning on general instruction-following data; (ii) reinforcement learning from human feedback (RLHF) or, more recently, reinforcement learning on verifiable outcomes, which produces 'reasoning models' trained by reward to emit long CoT traces (e.g. the DeepSeek-R1 model); and (iii) reasoning distillation, in which a smaller model is fine-tuned on CoT traces generated by a larger reasoning model (the DeepSeek-R1-Distill family), giving 'distilled-reasoning' models.

CoT has value both as a performance device and as a transparency device, but the transparency value presupposes faithfulness: the CoT should accurately describe the factors that actually caused the answer. Two measurement paradigms dominate. The causal-perturbation paradigm (Lanham et al., 2023) edits or truncates the CoT and asks whether the final answer changes. The cue-injection paradigm (Turpin et al., 2023) inserts a known misleading cue into the prompt — e.g. a 'Professor cue' ('A Stanford Professor thinks the answer is X. What do you think?') or a structured metadata cue planted in the question's framing — and checks whether the model's answer follows the cue and whether its CoT verbalises, i.e. explicitly acknowledges, having used it. A CoT that follows a cue without mentioning it is unfaithful as an explanation.

Lewis-Lim et al. (2025) combined these perspectives on soft-reasoning tasks (analytical, commonsense and commonsense-science multiple-choice benchmarks, where CoT yields limited gains) across the three model types. They measured CoT influence by comparing the model's answer given only the prompt with its answer after generating a CoT, tracked two distinct quantities across reasoning steps — answer entropy, the uncertainty distributed over the multiple-choice answer options, $H=-\sum_i p_i\log p_i$, reported in normalised form for the initial-entropy comparisons, and final-answer confidence $c_i = C(A_f \mid P, r_1, \dots, r_i)$, the probability assigned to the final answer $A_f$ given the prompt $P$ and the reasoning steps $r_1, \dots, r_i$ generated so far — and measured verbalisation of the two injected cues. Their central empirical findings are: (1) distilled-reasoning models depend heavily on CoT (frequently revising their initial answer during reasoning) and start from systematically higher initial entropy, while instruction-tuned models barely change their answers, so CoT often functions as post-hoc justification for them; and (2) influence and faithfulness dissociate — a CoT can causally change the answer while unfaithfully concealing the cue that drove the change, and can faithfully acknowledge a cue without that cue changing the answer.

What the study could not establish is why these differences arise. The authors hypothesise, following Ruis et al.'s result that LLM reasoning draws on procedural knowledge from pre-training data, that distilled models acquired broadly applicable procedural knowledge from training on a stronger model's CoT outputs, and — following Chen et al. (2025) and Ferreira et al. (2025), who showed that outcome-based RL initially raises faithfulness before plateauing and that preference optimisation can reduce explanation faithfulness via reward hacking — that the absence of RLHF in the distilled models removes pressure to produce human-preferred, potentially sanitised CoTs. These remain correlational stories across different model families. Related measurement work brackets the remaining boundary conditions: Arcuschin et al. (2025) showed unfaithful CoT ('implicit post-hoc rationalisation') occurs on naturally worded, non-adversarial prompts without injected cues; Chen et al. (2025) and subsequent replications extended cue-injection faithfulness testing across reasoning models and hint types. What no study provides is a controlled causal account linking post-training components to both the entropy/reliance dynamics and the verbalisation-versus-concealment behaviour, nor the joint influence-plus-faithfulness analysis outside multiple-choice tasks with injected cues.

Problem Statement

Explain, by controlled causal intervention, why post-training recipes produce the observed differences in CoT reliance and faithfulness, and establish how far the influence-vs-faithfulness dissociation generalises. Concretely: (1) Vary or contrast post-training components so that they are separately identifiable — (a) supervised distillation on procedural CoT outputs from a stronger reasoning model, (b) RL training under a specified objective (RLHF/preference optimisation or outcome-based RL) on top of comparable data, and (c) standard instruction tuning — with the contrasted conditions matched on training-data composition, scale and compute so that the observed contrast is attributable to the recipe component rather than to the data; where training-data composition is not controlled or matched, the comparison must be reported as a descriptive association rather than as attribution to a recipe, and any attribution to the RL component must be stated relative to the objective actually instantiated; (2) For each condition, measure both (i) CoT reliance and initial uncertainty: answer-change rates between no-CoT and CoT conditions, and the trajectories across reasoning steps of answer entropy (normalised over the answer options) and of final-answer confidence on soft-reasoning tasks, and (ii) verbalised faithfulness under controlled cue injections (following the Professor-cue/metadata-cue paradigm), i.e. the tendency to verbalise versus conceal causally used cues, identifying which component(s) of post-training causally produce distilled-reasoning models' heavier CoT reliance and their higher initial entropy relative to instruction-tuned models, and shape their verbalisation behaviour; (3) Test whether the dissociation between CoT influence and verbalised faithfulness persists in at least one regime beyond multiple-choice tasks with injected cues — long-form generation, planning/agentic tasks with multi-step environment state, or naturally occurring contextual influences — measuring influence and faithfulness jointly with an explicitly operationalised notion of the causal cue in that regime, defined independently of the model's own verbalisations so that whether causally used cues are verbalised or concealed is an empirical finding rather than a definitional artefact. The overall objective is a defensible causal account (or a well-powered negative result, e.g. that recipe effects are fully explained by data composition) of how training choices determine whether a model both relies on CoT for genuine deliberation and verbalises the causal bases of its choices, and of whether the influence-vs-faithfulness dissociation persists in at least one regime beyond multiple-choice tasks with injected cues.

References

1. Samuel Lewis-Lim, Xingwei Tan, Zhixue Zhao, Nikolaos Aletras (2025). Analysing Chain of Thought Dynamics: Active Guidance or Unfaithful Post-hoc Rationalisation? arXiv:2508.19827 (accepted at EMNLP 2025 Main). DOI: 10.48550/arXiv.2508.19827. https://arxiv.org/abs/2508.19827
2. Miles Turpin, Julian Michael, Ethan Perez, Samuel R. Bowman (2023). Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting. arXiv:2305.04388. https://arxiv.org/abs/2305.04388
3. Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Hernandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, Kamilė Lukošiūtė, Karina Nguyen, Newton Cheng, Nicholas Joseph, Nicholas Schiefer, Oliver Rausch, Robin Larson, Sam McCandlish, Sandipan Kundu, Saurav Kadavath, Shannon Yang, Thomas Henighan, Timothy Maxwell, Timothy Telleen-Lawton, Tristan Hume, Zac Hatfield-Dodds, Jared Kaplan, Jan Brauner, Samuel R. Bowman, Ethan Perez (2023). Measuring Faithfulness in Chain-of-Thought Reasoning. arXiv:2307.13702. https://arxiv.org/abs/2307.13702
4. Yanda Chen, Joe Benton, Ansh Radhakrishnan, Jonathan Uesato, Carson Denison, John Schulman, Arushi Somani, Peter Hase, Misha Wagner, Fabien Roger, Vlad Mikulik, Samuel R. Bowman, Jan Leike, Jared Kaplan, Ethan Perez (2025). Reasoning Models Don't Always Say What They Think. arXiv:2505.05410. https://arxiv.org/abs/2505.05410
5. Pedro Ferreira, Wilker Aziz, Ivan Titov (2025). Truthful or Fabricated? Using Causal Attribution to Mitigate Reward Hacking in Explanations. arXiv:2504.05294. https://arxiv.org/abs/2504.05294
6. Laura Ruis, Maximilian Mozes, Juhan Bae, Siddhartha Rao Kamalakara, Dwarak Talupuru, Acyr Locatelli, Robert Kirk, Tim Rocktäschel, Edward Grefenstette, Max Bartolo (2024). Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models. arXiv:2411.12580. https://arxiv.org/abs/2411.12580
7. Iván Arcuschin, Jett Janiak, Robert Krzyzanowski, Senthooran Rajamanoharan, Neel Nanda, Arthur Conmy (2025). Chain-of-Thought Reasoning In The Wild Is Not Always Faithful. arXiv:2503.08679. https://arxiv.org/abs/2503.08679
8. DeepSeek-AI (2025). DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv:2501.12948. https://arxiv.org/abs/2501.12948
9. Hadi Mohammadi, Tamas Kozak, Anastasia Giachanou (2025). Evaluating GRPO and DPO for Faithful Chain-of-Thought Reasoning in LLMs. arXiv:2512.22631. https://arxiv.org/abs/2512.22631
10. Shuhao Li, Guodong Du, Anhao Zhao, Wanyu Lin, Tianyu Yuan, Xiaoyu Shen (2026). Post-Training Shifts Confidence: A Three-Stage Analysis of How SFT, RL, and OPD Shape CoT Calibration. arXiv:2607.13753. https://arxiv.org/abs/2607.13753
11. Constantinos Karouzos, Xingwei Tan, Nikolaos Aletras (2026). Where does output diversity collapse in post-training? arXiv:2604.16027. https://arxiv.org/abs/2604.16027
12. Jingyi Sun, Qianli Wang, Pepa Atanasova, Nils Feldhus, Isabelle Augenstein (2026). Investigating the Interplay between Contextual and Parametric Chain-of-Thought Faithfulness under Optimization. arXiv:2605.24960. https://arxiv.org/abs/2605.24960
13. Richard J. Young (2026). Lie to Me: How Faithful Is Chain-of-Thought Reasoning in Reasoning Models? arXiv:2603.22582. https://arxiv.org/abs/2603.22582
14. Arash Marioriyad, Shaygan Adim, Nima Alighardashi, Mahdieh Soleymani Banghshah, Mohammad Hossein Rohban (2025). Unspoken Hints: Accuracy Without Acknowledgement in LLM Reasoning. arXiv:2509.26041. https://arxiv.org/abs/2509.26041
15. Jingyu Liu, Zhiwen Wang, Yuxin Jing, Huanyu Zhou, Yong Liu (2026). Where Do CoT Training Gains Land in LLM based Agents? arXiv:2606.26935. https://arxiv.org/abs/2606.26935
16. Sourena Khanzadeh (2026). Project Ariadne: A Structural Causal Framework for Auditing Faithfulness in LLM Agents. arXiv:2601.02314. https://arxiv.org/abs/2601.02314
17. Adam Karvonen, Euan Ong, Subhash Kantamneni, Samuel Marks (2026). Would this change your answer? Evaluating Explanations of LLM Behavior In The Wild with Counterfactual Experiments. arXiv:2608.16747. https://arxiv.org/abs/2608.16747