Token-based Decision Criteria Are Suboptimal in In-context Learning

This paper investigates the limitations of token-based classification in In-Context Learning (ICL) for language models, particularly in terms of biases and under-calibration. The authors propose Hidden Calibration, which replaces token probabilities with nearest centroid classification on hidden states, improving performance by about 20% across 10 datasets. Hidden Calibration leverages the linear separability of language models' hidden representations and outperforms token-based methods, even with minimal computational cost. The study demonstrates that hidden state-based approaches, like Hidden Calibration, yield better classification criteria with lower inter-category overlap, and that the method benefits from demonstrations, enhancing linear separability and transferability across tasks with the same label space. The paper also suggests future research on automatic token selection and combining Hidden Calibration with other calibration techniques.