ROME: Locating and Editing Factual Associations in GPT (Paper Explained & Author Interview)
#ai #language #knowledge
Large Language Models have the ability to store vast amounts of facts about the world. But little is known, how these models actually do this. This paper aims at discovering the mechanism and location of storage and recall of factual associations in GPT models, and then proposes a mechanism for the targeted editing of such facts, in form of a simple rank-one update to a single MLP layer. This has wide implications both for how we understand such models’ inner workings, and for our ability to gain greater control over such models in the future.
OUTLINE:
0:00 - Introduction
1:40 - What are the main questions in this subfield?
6:55 - How causal tracing reveals where facts are stored
18:40 - Clever experiments show the importance of MLPs
24:30 - How do MLPs store information?
29:10 - How to edit language model knowledge with precision?
36:45 - What does it mean to know something?
39:00 - Experimental Evaluation & the CounterFact benchmark
1 view
36
13
1 month ago 01:26:11 1
Day of Anger | WESTERN | HD | Full Movie | Spaghetti Western | English
1 month ago 00:02:25 1
Orvieto, Umbria, Italy (Italia) [HD] ()
1 month ago 00:06:19 1
Trieste Is Returned To Italy (1954)
2 months ago 00:13:33 10
Rome Snowboards presents: Particle
2 months ago 00:02:54 1
Unveiling the Grandeur of Rome’s Colosseum A Journey Through Ancient Spectacle
2 months ago 00:48:05 1
Corruption in the Vatican | Scandals | The Pope | Documentary Crime & Corruption
2 months ago 00:02:56 1
Overview of the E-Cat EV Test in Latina, Italy (September 27th, 2024)
2 months ago 00:13:16 1
The City that Built the Largest Greek Temple in the World
2 months ago 00:35:47 1
Inside The Oldest City In Europe (not Rome Or Athens) 🇧🇬
2 months ago 00:47:05 1
Child Sacrifices And Vatican’s DARKEST SECRETS Exposed!
2 months ago 00:02:47 1
Victoria’s Secret Holiday 2016: A Very Private Affair
2 months ago 00:06:51 1
Metallica - Creeping Death 99( very high quality )Woodstock
2 months ago 00:04:12 1
NANOWAR OF STEEL - Winterstorm in the Night ft. Madeleine Liljestam (Official Video)| Napalm Records
2 months ago 00:03:44 1
ASTRAL EMPTINESS - AN OUTCAST’S DREAM (FT. CRISTIAN JIMÉNEZ) [OFFICIAL MUSIC VIDEO] (2024) SW EXCL
2 months ago 00:40:05 1
Experience one of the Best Croissant in Rome: A Must-Try Bakery!
3 months ago 03:18:28 1
3+ Hours Of History’s Most Mysterious Treasures
3 months ago 00:54:15 1
Rome Italy, This is Rome Right Now, Rome September 2024, Rome walking Tour. People’s Square
3 months ago 00:50:39 1
What Was The Nazi’s “Book Of Power“?
3 months ago 00:13:10 1
Italy 1940s in color, during World War II [60fps, Remastered] w/sound design added
3 months ago 00:12:52 1
Goddess Temple Project
3 months ago 00:03:28 1
ANNA — Антилюбовь (Official Music Video)
3 months ago 00:02:52 1
Dean Martin - That’s Amore - Roman Holiday - Audrey Hepburn & Gregory Peck
3 months ago 00:12:01 1
South of France 8K Ultra HD Drone Video – French Riviera and Surroundings
4 months ago 00:02:59 4
Alain Delon - Just Say I Love Him (by Timi Yuro) with lyrics