LMs capture many factual associations, but how do they recall them internally during inference? In a new preprint, we find that LMs build attribute-rich subject representations, from which attention heads extract the predicted attribute. (with Mor Geva @mega, Katja Filippova, And Amir Globerson) 🧵 #NLP #NLProc