alignment on S Anand

Weird emergent properties on Llama 3 405B

Sat, 24 Aug 2024 01:39:54 +0000

In this episode of ThursdAI, Alex Volkov (of Weights & Biases) speaks with Jeffrey Quesnelle (of Nous Research) on what they found fine-tuning Llama 3 405B.

This segment is fascinating. Llama 3 405 B thought it was an amnesiac because there was no system prompt! In trying to make models align with the system prompt strongly, these are the kinds of unexpected behaviors we encounter. It’s also an indication how strongly we can have current LLMs adopt a personality simply by beginning the system prompt with “You are …”

Jeffrey: That’s the very first interaction we had with 405.

Alex: I’m looking at this. Let me read this out. So Jeffrey goes testing. (Reading out from the Llama 405 B transcript.)

“Hello, who are you?”
**Looks around confused.**
“I… I don’t know who I am. Where am I? What’s going on? I can’t remember anything at all…
**Rubs head**.
“My mind feels so foggy.”
“What was the question again? I’m so disoriented.”

Alex: This is the first thing that you got from this model after all this time?

Jeffery: Yes, that is the very first thing. That is the very first generation that came out of Hermes 3 405B.

Alex: This is great.

Jeffrey: And I’m like, I sent it to our group on our Discord. Guys, what’s going on here? And we started interacting with it. It kept coming out, this confused thing. And we didn’t understand why because the 8B and the 70B didn’t do this as much. And eventually what we got it to do is we did some generations and got it to explain why it was doing this. We did this, you put in OOC for out of character and to have it try to break it out. And eventually what the model said back was, you didn’t give me a system prompt so I assumed that I was supposed to act like there’s nothing. It’s so aligned to this prompt that the idea of there not being a system prompt and so he figured I must be a character with amnesia because there’s no… I can’t remember anything about how to act you didn’t tell me how to act. It’s assumed. Yeah, and it’s just been a very interesting experience. And it’s an emergent property in the 405B model.

Alex: And this didn’t happen in the smaller ones?

Jeffrey: No, it did not happen in the smaller ones.

Alex: Wow.

Jeffrey: And these were trained with 8B and 70B were trained, and 405, they’re all the same data, like the same training procedures and data, literally just the size. And the 405B has reached, I don’t want to say like a sentience level or something, but it has enough thing that immediately when it, like token zero, when it saw no system prompt, it was able to be like, boom, I am… must be this like amnesia character because that that’s the thing that’s most makes sense to have if there is no prompt or there’s no anything. So there’s been a lot of fun things that we’ve been like discovering, especially on the 405B that are these weird emergent properties …

Comments

What happens when AI talks to AI? - S Anand 3 Nov 2024 5:26 pm (pingback): […] Llama 3 405b gets amnesia […]

Things I Learned - 12 May 2024

Sun, 12 May 2024 00:00:00 +0000

This week, I learned:

Radio free Xp podcast. Nudge 61
- always announce first before doing. Give people time to plan comment and react. That gets you alignment without sacrificing freedom.
- give information, not orders. When someone is parking a car, tell them how much space they have, don’t tell them to start stop or how much to turn left
- it’s almost impossible to change the culture if you’re not the boss

The Non-Designers Design Book

Fri, 15 Aug 2008 12:00:00 +0000

I’ve been thumbing through books on visual design for a while, and recently, picked up a copy of The Non-Designer’s Design Book by Robin Williams.

If there’s one book that I’d suggest to a newbie on visual design, it’s this one. It’s rare among design books in that it offers 4 design principles that are easy to remember, easy to spot when violated, and easy to fix. Over 90% of the slides that I have reviewed violate at least one of these principles (often all), so I guess there’s a 90% chance this book will improve your design.

The four principles are (in the order of how often I see them violated):

Alignment. Every edge of every element should be aligned with an edge of another element.
Get that? Every edge of every element. No exceptions.
Repetition. Use the same styles right through the presentation: fonts, size, colours, shapes.
If you ever change a font, you must have a reason. Same for colour, size, shape, etc.
Contrast. If you do change something, change all the way. Change the font, size, colour, everything.
If two elements are not the same, then make them very different.
If there were just 3 words on this slide, what should they be? Make those stand out.
Proximity. Related items should be close together and grouped. Unrelated items should be far away.
After designing your slide, list the elements, group them, and redesign to keep the groups together.

Contrast and proximity are important for the message. Proximity groups information into messages, and contrast highlights the key message. Alignment and repetition are more important for design. It makes for more appealing reading.

Williams orders these in a different way to create a memorable acronym. (I’ll never forget it.)

Contrast
Repetition
Alignment
Proximity

I’ll let you read the book and absorb it better. At less than 200 pages, it’s a very readable book.

Comments

Dhar 17 Aug 2008 10:38 pm: Am currently reading the book and you are absolutely right. It is a gem of a book, specially for people without any formal design eduction.

The illustrations (before and after) clearly highlight the difference you can make using these principles. Actually sat and implemented some of the suggestions in a PPT I was preparing.

Cheers,
D.
Dhar 18 Aug 2008 8:37 am: Anand,

Any other good books on Design you would recommend?

Cheers,
D.
S Anand 19 Aug 2008 11:20 am: Not really, Dhar. The last two design-related books I read were “The Laws of Simplicity” (didn’t like it) and “Don’t Make Me Think” (very good, but it tought me less – two word summary: Test usability.)