Where are the arithmetic capabilities of a large language model located?

Language modelsareintended to process and generate text. In the extensive training process, however, theyalso develop arithmetic skills and the skills required to write programming code. In this work, we investigate whether it is possible to identifythe areas in the neurons ofthese models responsible for a specific skill. For this purpose, we consider arithmetic tasks and let a language model solve thembycompletingandextracting the activation states of the neurons via synthetically generated datasets. We then tryto reconstruct the results from individual groups of neurons using regression models to find the relevantgroups for solving the tasks. Linear regression models, regression trees, and support vector regression are used to uncover possible relationships. We identify that neuron pairs, not individual neurons, in the LLM can be identified as responsible for specific arithmetic behavior. We also find thatseveral distinct pairs of neurons in the GPT2 XLmodelare responsible for arithmetic capabilities, indicating a redundant encoding of these capabilities. In the future, this can lead to smaller models being extracted from larger ones for specific tasks.