Contextual Redefinition in Large Language Models: Behavioural, Representational, and Causal Analysis Across Scale

Abstract

As Large Language Models (LLMs) continue to increase in scale and capability, they are increasingly expected to adapt their behaviour to information provided dynamically at inference time. This ability becomes particularly important when contextual information conflicts with knowledge acquired during pretraining, since successful adaptation requires models not only to represent the new information, but also to prioritize it over strongly established prior associations. Contextual redefinition provides a controlled setting for studying this tension by assigning nonstandard values to familiar scientific constants and units of measurement and requiring models to answer consistently under these altered definitions. In this thesis, we investigate contextual redefinition across multiple scales of the Llama 3.1, OLMo 2, and Qwen 2.5 model families, examining not only whether models follow the supplied context, but also how the competition between contextual and default knowledge develops internally. We combine behavioural evaluation with a First-Divergence Logit Lens analysis, which tracks the layer-wise preference between redefined and default targets, confidence and uncertainty analysis, and bidirectional activation patching to identify internal states that are causally relevant to the same target competition. Our results show that, across the open-weight model families evaluated here and when considering the datasets as a whole, larger models generally follow contextual redefinitions more successfully. This aggregate trend differs from previously reported inverse-scaling behaviour, which emerged mainly in specific difficulty-defined subsets and under a different model selection. At the same time, performance remains strongly affected by question difficulty, redefinition type, and response format. Internally, preference for the redefined target becomes stronger toward later layers and is strongly associated with the model’s eventual response, while lower-confidence states are more frequently associated with preference reversals and off-target outputs. Activation patching further reveals that contextual redefinition is supported by distributed causal influence across network depth rather than by a single localized mechanism, with this influence becoming more broadly distributed as model scale increases. Most importantly, causal influence emerges and reaches its strongest effect earlier than the corresponding redefinition preference becomes maximally expressed through the Logit Lens. These findings indicate that contextual adaptation in LLMs is a distributed and temporally structured process, and that the locations where information is most strongly decodable do not necessarily coincide with the locations where it first becomes functionally important.