Concept Acquisition

New preprint: High-resolution dissection of concept acquisition in different families of protein language models. Layer-by-layer analysis of ESM2 and AMPLIFY maps a progression from physicochemical properties and motifs to secondary structure and domain-level semantics, revealing that data and compute shape concept emergence more than model size. With Shawn Whitfield, Tom Marty, Robert Vernon, Christopher Langmead, and Dhanya Sridhar. In collaboration with Amgen.