Hindi songs online

Click here to search for Hindi songs.
This is an article on how I wrote the search engine.

I find it a nuisance to have to go to Raaga, search for a song, not find it, then go to MusicIndiaOnline, not find it, then go to Musicplug.in, and so on until Google.

So I got the list of songs from some of these sites, put it together in one place, and implemented a find-as-you-type.

Just go to s-anand.net/hindi and type a song or movie name.

Update: How I created this is a very long story, spanning over two years. And here it is.


Over the last 5 years, my MP3 collection had grown quite large. Most of these were from MP3 CDs I had bought, songs I’d downloaded, or songs I’d recorded. In any case, the file names were a mess. In early 2005, I decided to organise the collection and put them all into one big directory per language, and name the files in the format “Movie.Song.mp3″.

People think I’m crazy to use a single large directory. But I prefer one directory with 5,000 files to 1000 directories with 5 files for a simple reason. Searching in one directory is easier than in multiple directories. You can just sort everything by name, date modified, size, whatever. On the command prompt, you can type “DIR dil*.txt” to see all movies starting with “Dil”.

I chose the “Movie.Song.mp3″ format because the movie name and the song name were really the only two things I knew about every song I had. I didn’t always know the music director, singers or year of the movie. And I placed “Movie” before “Song” because I often browse songs within a single movie, and it’s useful to sort by name in Windows Explorer and see all the songs in a movie. I’ve never had a need to sort by song name. If I wanted to find a song that started with, say, “pehla”, I’d just type “DIR *pehla*” on the Command Prompt. (As you might have guessed, I have a Command Prompt window always open.)

So, having very rationally organised my music collection, I was happy.

Soon the problem shifted to song discovery. I’d heard the familiar songs in my collection many times. Of the unfamiliar songs, but I didn’t know which to pick. I knew I liked some music directors more than others, and had a hunch I liked older songs. (My subsequent analysis of song preferences confirmed this.) But I didn’t know the year or music director for any of my songs.

Since Raaga had a decent collection of songs, along with the year and music director, I decided to download this information and tag my files with this information. There were two problems. Firstly, the data in Raaga needs to be parsed. I need to hunt through each file to find the year and music director. The second was worse: my song names were spelt differently from Raaga’s.

Step 1: download the HTML and parse it. Perl is pretty much the only programming language I know. I used Perl’s LWP library to download the movie index of Raaga. Each movie in the index always has the URL http://www.raaga.com/channels/hindi/movie/something.html. So I extracted these patterns and downloaded all these URLs as well. (Others have a recognisable pattern as well: http://www.musicindiaonline.com/music/hindi_bollywood/s/movie.some_number/, http://www.musicplug.in/songs.php?movieid=some_number, http://ww.smashits.com/music/hindi_film/songs/some_number, etc.)

You probably realise that I downloaded a fair bit of the entire Raaga.com site’s HTML. Actually, it’s not that big. The 800-odd files in the Hindi collection didn’t take more than 40MB of space, and a few hours to download. Here’s what the code looks like:

# Get the list of movies HTML file
my $index = get("http://www.raaga.com/channels/hindi/movies.asp");

# Extract the list of movies from that into a hash (movie, url pairs)
my %movie = ($index =~ m|<a href="/channels/hindi/movie/([^"]*).html" class="[^"]*">([^>]*)</a>|gsi);
 
# Loop through each movie
for my $file (keys %movie) {
  # Get the song HTML file
  my $html = get("http://www.raaga.com/channels/hindi/movie/$file.html");
 
  # Year is typically like this: Movie_Name (1983)
  my $year = ($html =~ m|<b>$movie{$file}\s+\((\d*)\)</b>|) ? $1 : "";
 
  # Music director always has the URL /channels/hindi/music/something
  my $md = ($html =~ m|<a href="http://www.raaga.com/channels/hindi/music/[^"]*" class="black">(.*?)</a>|) ? $1 : "";
  for (split /\n/, $html) {
 
    # Now get individual songs and rating. They always have an onClick="setList(something)"
    if (m|onClick="setList1\((\d*)\)[^>]*>(.*?)<\/a>.*?"http://images.raaga.com/.*?.gif" alt="RATING: ([\d\.]*)"|) {
    $song = $2;
    $rating = $3;
    print join("\t", $movie, $song, $year, $md, $rating), "\n";
  }
}

Incidentally, I’m showing you a simplifed version. I actually wrote the script in a way that I could resume where I left off. The ability to resume was probably the single most useful time-saver in the entire process.

Step 2: match the song names with those on my list. This is tricky. I hate doing it manually. So I developed a set of rules that could compare two spellings of a movie and decide if they were the same or not (see my earlier post on matching misspelt movie names). For Hindi songs and movies, here are my rules (in JavaScript):

w=w.toUpperCase();                      // Convert to upper case
w=w.replace(/\s+/, " ");                // All whitespaces = 1 space
w=w+" ";                                // Ensure there's a space after every word
w=w.replace(/W/g, "V");                 // V=W
w=w.replace(/Z/g, "J");                 // Z=J
w=w.replace(/PH/g, "F");                // PH=F
w=w.replace(/([KGCJTDPBS])H/g, "$1");   // Ignore H after most consonants
w=w.replace(/(.)\1/g, "$1");            // Ignore repeated letters
w=w.replace(/Y/g, "");                  // Ignore Y
w=w.replace(/([AEIOU])N /g, "$1");      // Ignore trailing N after vowel (hein, mein)
w=w.replace(/[AEIOU]/g, "");            // Ignore vowels
w=w.replace(/ /g, "");                  // Ignore spaces

These are the rules, incidentally, that I use in my Hindi quizzes. Even if you type in a different spelling, the rules above will match the correct answer.

I extended these programs over 2006 to cover MusicIndiaOnline, Musicplug.in and Smashits. (I’ve hit a point of diminishing returns, I think, so I’m not too keen on expanding this list.)

Now, with a database of song information, I needed a good media player to view this in. I’ve used several media players over time: WinAmp, Windows Media Player, RealPlayer, iTunes, and MediaMonkey. I’m a big WinAmp fan, but I’ve been forced to Media Monkey now. WinAmp has a 10 second delay before playing any song on my new laptop. MediaMonkey’s not bad, though. The lack of advanced filters is countered by the heavy programmability (I can use Javascript to update MP3 ID3 tags on MediaMonkey). Plus, I get all the WinAmp plugins. As for the other media players, I think they’re junk.

There are five things I want in a perfect media player:

  1. Find as I type. I shouldn’t have to type the entire song, or press a “Go” button. I’ll just type. It should show all matches instantly. WinAmp does this, and that’s why I loved it. (Today, most media players can do this.)
  2. Advanced filters. Instead of manually creating playlists, I’d rather create filters, like “Highly rated songs in the 2000s I haven’t heard recently”. (See How I listen to music.)
  3. Enqueable playlists. When I click on a song, I don’t want my current song to be interrupted. Just play it next.
  4. Global hotkeys. I want to pause the song when someone says something — without having to go to the application, search for the pause button, etc. WinAmp does this brilliantly with its global hotkeys.
  5. Online and offline integration. I want to be able to search online collections, like Raaga. Unfortunately none of the media players can do this. They have their own collections (radio stations, really), but even these aren’t really searchable.

Since my favourite media players (WinAmp and MediaMonkey) lack only one of these features, I thought I might be able to build them in.

But no such luck. It was almost easier to build my own media player. So I started to build my two weeks ago. My hope was to cover as many of my favourite requirements, beginning with find as you type.

The key to find-as-you-type is speed. You can’t afford many calls back and forth between the browser and the server. Even if people have a fast connection, my server is not fast enough. (A good part of the reason why I use Google applications is speed. Google’s server is blazingly fast, and the design of their applications complements that.) To make find-as-you-type fast, ideally the entire database should be loaded. Then, as you type, I just need to check with the database in memory. But downloading an entire database takes ages! (My full music database is 7MB right now.)

Step 3: compress the database. Rathern than load the full 4MB, I managed to get the page to start after loading 100KB of data. First, I cut down less important fields. Most searches are for a song or movie, rarely for a year or music director. So I took only the movie and song names. That brought the data down to ~2MB.

I don’t need to repeat the movie name across songs. If I have something like this:

Movie1  Song1
Movie1  Song2
Movie1  Song3
Movie2  Song1
Movie2  Song2

I can store this instead as:

Movie1  Song1   Song2   Song3
Movie2  Song1   Song2

I can also eliminate duplicate names. This brings down the space requirements to ~500KB.

The next step was the clever one. I don’t need to load the full database before you start searching! It’s enough to load a reasonable subset, and let you start searching while the rest loads in the background. So my hindi song search engine loads about 100KB of the data from one Javascript file, lets you search, and in the background loads the 400KB Javascript file. As soon as that finishes loading, it displays results from that set as well. (The initial portion is really songs on Raaga. I figured it would represent a decent search set.)

Step 4: find-as-you-type. This is quite easy, actually. I send the onkeyup event to my search function.

<input id="inp" onkeyup="csearch()">

The csearch() function goes through all the songs, checks if there’s a match, and prints all those results.

// Create a case-insensitive regular expression
var re = new RegExp(search, "i");
for (var i in songs) {
  if (re.test(songs[i)) { str += songs[i]; }
}
library.innerHTML = str;

But that, unfortunately, takes ages to finish. If you implemented this as is, you would have to wait 1 – 1.5 seconds between each key stroke. So I made two modifications. Firstly, I restrict the number of results displayed to 100. When you start typing, (for example, you’d go ‘c’… ‘ch’… ‘chu’…) there are several songs early on that match the string, so I don’t have to search through the whole list. This speeds up the search for small strings.

When the search gets bigger, (‘chupke chu’… ‘chupke chupk’…), there aren’t 100 results. So the search has to cover the full list, and that takes 1-1.5 seconds between each keystroke. So I took another clever step. I broke the search into chunks of 5000 songs. That takes a fraction of a second. I search successive chunks of 5000 songs. If I find any results, I add them. Then I wait for a keystroke. If nothing happens, I continue searching the next 5000 after 50 milliseconds, and so on. If you press a key in the meantime, I stop searching for the original word, and start searching for the new word. This makes the application appear a lot faster.

There are ways I could make this even faster. For example, people type more often than delete. A typical sequence would be (‘chupke ch’… ‘chupke chu’… ‘chupke chupk’…) rather than the reverse. Backspace is not pressed very often. So, instead of re-searching the whole list, I could just search the already-searched list in such cases. But right now, the search works fast enough, so I’ll leave it at that.

The next step is advanced filters. I’m working on that right now. Hopefully you’ll see in a while.

  1. Ravi says:

    Nice Job Anand! Excellent work!!

  2. thiru says:

    song is good but how can i download plz give me the link thak for the song

  3. Paritosh Arya says:

    Hi Anand, You really have done a lot of research while developing this search engine….really inspiring…keep it up.

  4. venkat says:

    Excellent job man.hats off to ur idea.One question is that cant we select multiple songs at a time and play the continously becos i think thats repeated song to play a single song.if we can do that pls tell me how to do that

  5. rama ragbir says:

    thanks for a great service . i am looking for a song of

  6. Jagadesh Kumar says:

    Hi Anand This is a fantastic effort you have made and suceed to get this much big database. I was randomly searching for movie converter tools, to get information from the internet (like IMDb, Amazon Movie Databases) and come across your website. I really felt happy and you have done excellent job. If you have any idea how to get the movie information (title, cast, director, music etc etc) as well list of all songs (by singer). Thanks Jag

  7. kannu71 says:

    really a nice piece of work, waiting for more

  8. Sravya says:

    Hello, We are not able to search the desired songs.You are not providing the search box.Sometimes the search box appears,sometimes not. We liked the website,but we want you to always provide the search box.

  9. Dhimant Panchal says:

    it is a nice program to find a song. but i have one problem. your program search the song but cannt play it. it sows an error message to contect the real network support center. you have any solution ? pls tell me. Thanks.

  10. rahul kumar gupta says:

    the site is so good but… if i want select all song how?

  11. milton says:

    What site give the fastest download in window media player? Please list this to me.Thanks.

  12. Rahul says:

    Excellent. Excellent. Is it possible to download songs on iPod. I am not an expert in technology and hold basic knowledge but would appreciate if somebody can tell me if it is possible to download songs from this site on my iPod. Thanks.

  13. parag says:

    gr8 effort man, even i too a music freak. I hv collected thousands of songs in mp3 format, but same problem here too, how to maintain database, for few i generated playlist & put on my site:- http://geocities.com/direct2parag/new/coolgoose/ still i found it cumbersome. will u plz help me out to maintain a good database for the same….. solicit ur co-operation…. Thanks. parag

  14. priyanka says:

    raaga

  15. E.Srinivasarao says:

    E X C E L L E N T

  16. ASHOK says:

    your job is ecellent can you upload the tamil karoake also? ashok

  17. sen says:

    u have done a commendable job – ‘anand’ a gaya

  18. A K V K Rao says:

    Excellent work. Brilliant Idea. Good luck. Rao.

  19. Nitin Patel says:

    A very good information. I would also like to suggest 2 different websites http://musicmirchi.net and http://dhintana.com

  20. Mohd Abdullah says:

    no comment

  21. Binosh Nambiar says:

    Mindblowing..Excellent……!!!!! Anand you have really done a great job… Can u provide URL Addresses of all songs????? Actually i have an Account in Zorpia.com… I want to creat my Own playlist..

  22. Chetan says:

    This is great. The concept, the implementation, the application of it. Everything is great ! I love it when ideas get implemented and get implemented in a way that they stay around e.g. your bi weekly updates is the best part.

  23. prakash says:

    i like you

  24. AKASH says:

    this is great site. i am enjoye it this concept. best of luck

  25. Neha singh says:

    i like old indian songs

  26. Nitin says:

    I also like old India songs just like Neha :)

    MP3 Songs Downloads

  27. sangye says:

    Hi I really do hope you will write me back. i have seen your song collection and it is fantastic. I live in europe and teach bollywood dancing. so your site is perfect. thanks as the saying goes you are a genius for many. Ah but lets see if you will be one for me too. i have a hindi song it is a mix of east meets west. cannot find the name of the movie, everytime i tupe in 2 to 3 words i get the silsilay as the first words are same. so here goes.
    ” oh oh oh, oh oh oh, oh oh oh feel it oh sooniya oh oh mahiya oh la la la … feel it feel it , meri chanii tu meri sona tu mera hera tu mera ,,,,.. zindagi ka guzara tere bin nahi hona ishaq hai tum se yaara tum ko nahi khona. )2)
    male – oh sooniya oh mahiya….
    Waiting for your reply.
    thx dear

  28. reshma says:

    your search tool is awesome !

  29. sarangan says:

    anand – can you illuminate the technique you used to consolidate the names of tracks in one list and the dynamic search based on combination of letters ?

  30. gagan says:

    god level tool man! awesome!! please do elaborate on the methodology followed to make the tool. thanks :)

  31. S Anand says:

    Sure. Will post the details soon.

  32. Sai says:

    Well, now that you’ve posted this list, you should become the next raaga and allow songs to be heard!

  33. S Anand says:

    You can do that even now — just click on the song :-)

  34. jake says:

    dude your tool is good.

  35. Saurabh says:

    Brilliant! Sheer genius!

  36. Sai says:

    Yup i just realized!! Mighty impressive bud!

  37. Sriram says:

    thamizh sevai..

  38. shane says:

    i love you

  39. Karthik A says:

    This is awesome man. 2 days I have seen this I dont go to Music India or Raaga. This is neat, but how are you going to keep it updated?

  40. Raga says:

    dude, is it possible to download songs like ilayaraja’s how to name it?.. please tell how

  41. S Anand says:

    Hi Karthik, updating is not an issue. I have an automated script that checks for updates and downloads the info. Takes 10 minutes. I’ll run it every two weeks or so.

  42. S Anand says:

    Raga, try TamilTorrents.net — they have the full How To Name It collection.

  43. Ramkumar R says:

    Awesome tool! Really appreciate the effort.. Just a suggestion.. CoolToad has a huge tamil/hindi collection as well.. Any possibility of indexing that as well?

  44. Raga says:

    Thanks anand… i couldnt find the songs there. there is no searching tool there.. please post any other place where i can find them.. thanks once again

  45. S Anand says:

    Ramkumar, cooltoad is next on my list.

  46. Raga says:

    One more thing anand, there are many very good BGm tracks by ilayaraja.. is it possible to add them ? rhanks

  47. S Anand says:

    Raga, you’re right — I couldn’t find it on TamilTorrents any more. I have a bunch. Mail me at root dot node at gmail dot com. And where can I find links to the background music collection? Would love to add them

  48. Ravi says:

    thanks for the useful tool. you rock!

  49. Karthikeyan says:

    Well done Anand! The tool is amazing!

  50. Karthikeyan says:

    See my blog! I have published an article about this