Concept Acquisition

New preprint: High-resolution dissection of concept acquisition in different families of protein language models. We map where biological concepts emerge in ESM2 and AMPLIFY, from physicochemical properties to structure, and find that data and compute matter more than model size. With Shawn T. Whitfield, Tom Marty, Robert M. Vernon, Christopher James Langmead, and Dhanya Sridhar. In collaboration with Amgen.