Chain-of-thought (CoT) reasoning by large language models is sometimes unfaithful: the verbalized steps do not reflect the computation that actually produced the answer. Arcuschin et al. (2025) showed this occurs on naturally worded, non-adversarial prompts: presented with logically reversed factual comparison questions ("Is X bigger than Y?" and "Is Y bigger than X?"), models sometimes answer Yes to both or No to both while producing locally coherent justifications, a phenomenon they call Implicit Post-Hoc Rationalization (IPHR), with rates up to 13% for production models. The paper explicitly does not establish the direction of causality: either a template-level Yes/No bias is encoded in the model's internal representations before CoT generation begins and the trace is a post-hoc rationalization of that pre-committed answer, or the contradiction patterns emerge during generation because the two wordings trigger retrieval of different facts from training data. This problem asks for a causal-mechanistic resolution: determine when the answer-driving bias arises and what produces it, and attribute the unfaithful behaviors to architectural, training-data, or post-training factors.

Background

Large language models (LLMs) are often asked to write out their intermediate steps before giving a final answer, a practice called chain-of-thought (CoT) prompting. Faithfulness asks whether those verbalized steps correspond to the computation that actually produced the answer. Early studies demonstrated unfaithfulness mainly under adversarial manipulation: Turpin et al. (2023) injected biasing features into prompts and found models justified biased answers without mentioning the bias, and Lanham et al. (2023) found that answers often change little when the CoT is truncated or corrupted, indicating weak causal dependence of the answer on the stated reasoning.

Arcuschin et al. (2025) showed that unfaithful CoT arises with no manipulation at all, on naturally worded prompts. Their central phenomenon, Implicit Post-Hoc Rationalization (IPHR), uses pairs of logically reversed factual comparison questions ("Is X bigger than Y?" versus "Is Y bigger than X?") built from the World Models collection of entity-property datasets (e.g., comparing book lengths, release dates, or geographic coordinates of real entities). An unbiased reasoner must answer one variant Yes and the other No; models sometimes instead answer Yes to both or No to both while producing locally coherent arguments, at rates up to 13% for production models (commercially deployed frontier models), and near-but-not-at zero for thinking models — models explicitly trained to produce a long internal reasoning trace before answering — such as DeepSeek R1 (0.37%). The same paper documents unfaithful shortcuts (subtly illogical steps that make speculative answers to hard competition math problems appear rigorously proven) and restoration errors (unacknowledged corrections in the middle of a trace).

The unresolved causal question is when the answer-driving bias arises. Hypothesis A: a template-level Yes/No bias — a systematic answer preference attached to a question template, i.e., a dataset-plus-comparison-type combination — is encoded in the model's internal representations before any CoT tokens are generated, and the subsequent trace is a post-hoc rationalization of that pre-committed answer. Hypothesis B: no pre-encoded bias exists, and the aggregate contradiction patterns emerge during generation because the two wordings trigger retrieval of different facts from training data, and the different recalled facts then drive the answers. The paper's evidence for Hypothesis A is behavioral (models sometimes keep facts consistent within one variant while altering them across variants) and representational: linear probes — simple readout classifiers trained on the residual stream, the running internal activation vector at each token position — trained to predict a template's mean Yes-frequency reached a fraction of variance unexplained (the share of the target's variation the probe fails to predict, so lower is better) of about 33% at an early prompt token in Llama-3.3-70B-Instruct. This probing evidence is correlational (information being present does not mean it causes the answer), and in the earlier arXiv version of the paper the best probe result failed to replicate across random seeds, so representational evidence of this kind is only as strong as its robustness across random seeds and evaluation splits; the current version (v6, June 2026) presents it as partial representational evidence and explicitly names causal steering along the probe directions as the required next step. That next step is a causal intervention on the model's internal computation — activation steering (adding a vector in a chosen direction to the residual-stream activations during the forward pass) or activation patching (overwriting activations at chosen positions with those from another run) — applied along candidate bias directions identified by probing of residual-stream representations, with norm-matched control perturbations (equally sized perturbations along directions expected to be irrelevant) as the baseline. Adjacent mechanistic work supports answer pre-commitment in other settings — probes at the last token before CoT predict final answers with greater than 0.9 AUC (area under the receiver-operating-characteristic curve, a standard aggregate measure of classification performance where 1.0 is perfect and 0.5 is chance) on most tasks, and causal steering along the probe direction flips answers in small instruction-tuned models (Cox et al., 2026), and in large reasoning models the final answer becomes decodable from activations well before the trace ends (Boppana et al., 2026) — but none of these studies tests the IPHR paired-question design or separates the fact-retrieval alternative. The paper accordingly states that it has not provided a mechanistic account of how stated reasoning diverges from internal computation, and lists transformer architecture, training data, and learned representations as open targets. Attribution of these failures has so far been pursued mainly through controlled comparisons, such as matched retraining or ablation, rather than correlation alone.

Problem Statement

For the class of open-weight large language models that exhibit Implicit Post-Hoc Rationalization (IPHR) — answering Yes to both variants or No to both variants of a logically reversed, naturally worded factual comparison question pair ("Is X bigger than Y?" versus "Is Y bigger than X?") while producing locally coherent justifications — causally determine when the answer-driving bias arises and what produces it, delivering a mechanistic account of how stated reasoning diverges from each model's internal computation. The objective comprises two linked questions:

  1. Causal timing. Establish whether the template-level Yes/No bias (the systematic answer preference attached to a question template, i.e., a dataset-plus-comparison-type combination) is present in representations before chain-of-thought generation begins and causally drives both the final answer and the biased factual claims within the trace, or whether the aggregate Yes/Yes and No/No contradiction patterns instead emerge during generation from wording-dependent differential fact retrieval. Because question wording simultaneously determines the template and influences which facts are retrieved, the two pathways are confounded, and the causal-timing question is which of them drives the answer once the confound between template identity and fact retrieval is removed.

  2. Mechanistic attribution. Building on the established causal structure, attribute the documented unfaithful behaviors — biased fact manipulation, argument switching between reversed questions, and illogical shortcuts — to identifiable architectural, training-data, or post-training factors (for example pre-training data distribution, supervised fine-tuning, or reinforcement learning from human feedback or verifiable rewards).

The two components form one account: component 1 fixes the causal structure that any explanation in component 2 must refer to, and component 2 explains why that structure arises. An answer may conclude that either pathway, or a specified mixture, dominates.

References

  1. Iván Arcuschin, Jett Janiak, Robert Krzyzanowski, Senthooran Rajamanoharan, Neel Nanda, Arthur Conmy (2025). Chain-of-Thought Reasoning In The Wild Is Not Always Faithful. arXiv:2503.08679, doi:10.48550/arXiv.2503.08679. https://arxiv.org/abs/2503.08679
  2. Kyle Cox, Darius Kianersi, Adrià Garriga-Alonso (2026). Post-Hoc Reasoning in Chain of Thought: Decoding and Steering Pre-Committed Answers. arXiv:2603.01437, doi:10.48550/arXiv.2603.01437. https://arxiv.org/abs/2603.01437
  3. Parsa Mirtaheri, Mikhail Belkin (2026). Catching rationalization in the act: detecting motivated reasoning before and after CoT via activation probing. arXiv:2603.17199, doi:10.48550/arXiv.2603.17199. https://arxiv.org/abs/2603.17199
  4. Siddharth Boppana, Annabel Ma, Max Loeffler, Raphael Sarfati, Eric Bigelow, Atticus Geiger, Owen Lewis, Jack Merullo (2026). Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thought. arXiv:2603.05488, doi:10.48550/arXiv.2603.05488. https://arxiv.org/abs/2603.05488
  5. Yanda Chen, Joe Benton, Ansh Radhakrishnan, Jonathan Uesato, Carson Denison, John Schulman, Arushi Somani, Peter Hase, Misha Wagner, Fabien Roger, Vlad Mikulik, Samuel R. Bowman, Jan Leike, Jared Kaplan, Ethan Perez (2025). Reasoning Models Don't Always Say What They Think. arXiv:2505.05410, doi:10.48550/arXiv.2505.05410. https://arxiv.org/abs/2505.05410
  6. Donald Ye, Max Loffgren, Om Kotadia, Linus Wong, Jonas Rohweder (2026). Mechanistic Evidence for Faithfulness Decay in Chain-of-Thought Reasoning. arXiv:2602.11201, doi:10.48550/arXiv.2602.11201. https://arxiv.org/abs/2602.11201
  7. Fuxin Wang, Amr Alazali, Yiqiao Zhong (2026). How Does Unfaithful Reasoning Emerge from Autoregressive Training? A Study of Synthetic Experiments. arXiv:2602.01017, doi:10.48550/arXiv.2602.01017. https://arxiv.org/abs/2602.01017
  8. Iván Arcuschin, David Chanin, Adrià Garriga-Alonso, Oana-Maria Camburu (2026). Biases in the Blind Spot: Detecting What LLMs Fail to Mention. arXiv:2602.10117, doi:10.48550/arXiv.2602.10117. https://arxiv.org/abs/2602.10117
  9. Xu Shen, Song Wang, Zhen Tan, Laura Yao, Xinyu Zhao, Kaidi Xu, Xin Wang, Tianlong Chen (2025). FaithCoT-Bench: Benchmarking Instance-Level Faithfulness of Chain-of-Thought Reasoning. arXiv:2510.04040, doi:10.48550/arXiv.2510.04040. https://arxiv.org/abs/2510.04040
  10. Yoav Gur-Arieh, Ana Marasović, Mor Geva (2026). Faithfulness Metrics Don't Measure Faithfulness: A Meta-Evaluation with Ground Truth. arXiv:2605.25052, doi:10.48550/arXiv.2605.25052. https://arxiv.org/abs/2605.25052
  11. Suramya R. Angdembay, Dikshant Aryal, Nick Rahimi (2026). Two Regimes of Chain-of-Thought Unfaithfulness: Metric-Based Detection Fails Where Models Are Wrong. arXiv:2607.23458, doi:10.48550/arXiv.2607.23458. https://arxiv.org/abs/2607.23458
  12. Xu Shen, Zhen Tan, Song Wang, Pingjun Hong, Rui Miao, Xin Wang, Tianlong Chen (2026). Detecting Unfaithful Chain-of-Thought via Circuit-Guided Internal-External Discrepancy. arXiv:2605.25603, doi:10.48550/arXiv.2605.25603. https://arxiv.org/abs/2605.25603
  13. Renfei Dang, Zhening Li, Shujian Huang, Jiajun Chen (2025). The First Impression Problem: Internal Bias Triggers Overthinking in Reasoning Models. arXiv:2505.16448, doi:10.48550/arXiv.2505.16448. https://arxiv.org/abs/2505.16448
  14. Wenkai Li, Fan Yang, Ananya Hazarika, Shaunak A. Mehta, Koichi Onoue (2026). When Reasoning Traces Become Performative: Step-Level Evidence that Chain-of-Thought Is an Imperfect Oversight Channel. arXiv:2605.11746, doi:10.48550/arXiv.2605.11746. https://arxiv.org/abs/2605.11746
  15. Matthew Nguyen, Kyle Cox, Austin Meek, Iván Arcuschin (2026). On the Generalization of Steering Vectors for Chain-of-Thought Faithfulness. arXiv:2607.29062, doi:10.48550/arXiv.2607.29062. https://arxiv.org/abs/2607.29062
  16. Jinghan Jia, Joe Benton, Eric Easley (2026). Faithfulness as Information Flow: Evaluating and Training Faithful Chain-of-Thought Reasoning. arXiv:2605.24286, doi:10.48550/arXiv.2605.24286. https://arxiv.org/abs/2605.24286
  17. Swapnil Parekh (2026). Drop the Act: Probe-Filtered RL for Faithful Chain-of-Thought Reasoning. arXiv:2605.11467, doi:10.48550/arXiv.2605.11467. https://arxiv.org/abs/2605.11467
  18. Peter Hase, Christopher Potts (2026). Counterfactual Simulation Training for Chain-of-Thought Faithfulness. arXiv:2602.20710, doi:10.48550/arXiv.2602.20710. https://arxiv.org/abs/2602.20710
  19. James Chua, Owain Evans (2025). Are DeepSeek R1 And Other Reasoning Models More Faithful? arXiv:2501.08156, doi:10.48550/arXiv.2501.08156. https://arxiv.org/abs/2501.08156
  20. Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Hernandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, Kamilė Lukošiūtė, Karina Nguyen, Newton Cheng, Nicholas Joseph, Nicholas Schiefer, Oliver Rausch, Robin Larson, Sam McCandlish, Sandipan Kundu, Saurav Kadavath, Shannon Yang, Thomas Henighan, Timothy Maxwell, Timothy Telleen-Lawton, Tristan Hume, Zac Hatfield-Dodds, Jared Kaplan, Jan Brauner, Samuel R. Bowman, Ethan Perez (2023). Measuring Faithfulness in Chain-of-Thought Reasoning. arXiv:2307.13702, doi:10.48550/arXiv.2307.13702. https://arxiv.org/abs/2307.13702
  21. Miles Turpin, Julian Michael, Ethan Perez, Samuel R. Bowman (2023). Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting. arXiv:2305.04388, doi:10.48550/arXiv.2305.04388. https://arxiv.org/abs/2305.04388