Colin Klein - Transformers, Representational Structure, and the Language of Thought

Colin Klein (Australian National University)

The Center for Philosophy of Science at the University of Pittsburgh invites you to join us for our 65th Annual Lecture Series Talk. Attend in person in room 1008 in the Cathedral of Learning (10th Floor)  or visit our live stream on YouTube at https://www.youtube.com/channel/UCrRp47ZMXD7NXO3a9Gyh2sg.

The Annual Lecture Series, the Center’s oldest program, was established in 1960, the year when Adolf Grünbaum founded the Center. Each year the series consists of six to eight lectures, about three quarters of which are given by philosophers, historians, and scientists from other universities.

ALS –Colin Klein

Friday, October 11  @ 3:30 pm - 6:00 pm EDT

1008 Cathedral of Learning 

Title: Transformers, Representational Structure, and the Language of Thought

Abstract:

Transformers are an extraordinarily powerful computational architecture, applicable across a range of domains. They are, notably,  the computational foundation of contemporary Large Language Models (LLMs).  LLMs’ facility with language have led many to draw analogies between LLMs and human cognitive processing. Drawing out the consequences of what seems like an innocuous step—the need for positional encoding of the input to LLMs—I argue that transformers are broad precisely because they have so little built-in representational structure. This naturally raises questions about the need for structured representations and what (if any) advantage they might have over mere representation of structure. I develop this in particular in the context of the contemporary revival of the Language of Thought hypothesis.

YouTube at https://www.youtube.com/channel/UCrRp47ZMXD7NXO3a9Gyh2sg.