Thu, 25 Dec 2025 02:54:41 +0000

Gemini can pass the bar exam and solve maths olympiad puzzles. But it’s music-deaf.

nitin kumar asked Gemini to rate 40 songs on joy, sadness, tension, nostalgia, etc. and compared it with human ratings. There was ZERO correlation between the two. It’s like it’s a different species.

In fact, if you just predict the average emotion for every single song, you’d still do 1.2× to 2× better than Gemini! It wasn’t adding noise to a signal. It was subtracting subtracting signal from noise! In fact, for one song, the correlation was -88%, i.e. it predicted the exact opposite emotions.

It’s not just noisy, it’s biased as well. It hears music as happier, more tender, more powerful than we do. It massively over-predicts “joyful activation” (53% of songs!) and dramatically under-estimates tension.

The emotion predictions are suspiciously correlated with each other. Power and joy are basically identical (96% correlation).

This confirms a suspicion I had: Gemini can’t actually hear the audio. It can transcribe, but beyond that it’s just guessing.

Human music taggers are safe, for now.

Story, prompts & analysis: https://sanand0.github.io/datastories/llm-music/

Experiments in sound

Tue, 12 Sep 2006 12:00:00 +0000

Wikipedia says the human voice frequency for speech is between 85 to 155 Hz for men, and 165 to 255 Hz for women. That set me thinking.

What is the limit to our hearing?
How do sounds differ?
How can we synthesise speech?

What are the limits to our hearing?

Kids can hear frequencies from 20 Hz to 20 kHz, while adults hear only up to 12-14 kHz (Frequency Range of Human Hearing).

To check the lower frequency limit, I created an MP3 with sounds from 1 Hz to 100 Hz at 1 second intervals. Just play the sound, and see when you start hearing something. (Of course, whether you can hear something also depends on the volume of your speaker, the ambient noise, etc.) I could hear nothing for the first 40 seconds: so I can’t hear frequencies lower than 40 Hz.

PS: Don’t be worried if you don’t hear anything for a while. You’re not supposed to! Keep the volume at full level, though.

To check the upper frequency limit, I created this MP3 with sounds from 1 kHz to 20 kHz in 1 second intervals. Just play the sound, and see when you stop hearing anything. I couldn’t hear anything beyond 14 seconds: so I can’t hear frequencies beyond 14 kHz.

How do sounds differ?

I took this audio file of someone reciting vowels and plotted a spectrogram (below). A spectrogram plots time on the X axis and frequency on the Y-axis.

Some observations:

All the vowels have evenly spaced bars. (In this case, they’re all multiples of something around 120 Hz.)
‘u’ has the lowest frequency mix. ‘a’ spans from low to high. ‘i’ has a bit of low and a bit of high, nothing in the middle. ‘ai’ and ‘au’ look like ‘a’ followed by ‘i’ and ‘u’ respectively.

How can we synthesise speech?

I don’t know. There are lots of speech synthesizers. They sound robotic. I’m trying to see if knowing what sounds look like improves things. I’ll let you know if I do well.

Comments

Dhar 13 Sep 2006 5:09 am: How did you go about synthesizing the MP3s with increasing frequencies? Did you use some software for that? Similarly what software did you use to plot the spectrogram?
S Anand 13 Sep 2006 6:10 am: I wrote a perl script to generate the frequencies, and Spectrogram v.0.8 to create the spectrograms.
Sumit Dhar 16 Sep 2006 9:23 am: Dude, just downloaded the files. Is the sound stereophonic or monoaural? If it is mono, do you think creating an MP3 with stereophonic sound will make a difference? Cheers, D.
Sumit Dhar 16 Sep 2006 9:25 am: Loved the experiment. Lower: 42 Hz, Higher: 15 KHz
S Anand 16 Sep 2006 9:38 am: It’s mono. I doubt if stereo would make a difference. I’d be playing the same frequency on both speakers, which is what a mono MP3 does anyway.
Dhar 17 Sep 2006 2:05 am: The reason I asked about whether the sound is Mono or Stereo is because of my experience during eye tests. Some letters that were not visible when viewed with just one eye, were clearly readable when using both eyes. Was wondering if the same may be applicable in this case?
S Anand 17 Sep 2006 7:36 am: Ah, I see what you mean. Since both speakers would be playing the sound, hopefully this should not be a factor.

Hacking using keyboard whispers

Wed, 12 May 2004 12:00:00 +0000

It’s possible to make sense out of keyboard whispers.

audio-analysis on S Anand

Experiments in sound

Comments

Hacking using keyboard whispers