I share my humorous mishaps navigating vegetarianism while traveling abroad, including accidental encounters with chicken, beef, and prawn-flavored snacks. I’ve learned that when it comes to hidden ingredients in foreign food, ignorance is often bliss.

I am a vegetarian. More out of habit than religion. (I'm not very religious.) What that means, in practice, is that I don't eat non-vegetarian food knowingly. But it has happened unknowningly. Many times. Until I was 22, I had not been out of India on my own, and things were fine. In 1998, I went to Charlotte, NC, for training. (On a KLM flight that placed me illegally in Germany, but that's another story.) I was pretty groggy after an 8-hour night flight. So when I was woken up by the sound of breakfast, I bit into the big yellow thing on the tray in front. Sleepy as I was, I did feel a little suspicious. Didn't quite taste sweet, like I was expecting it to. I saved it for the last, when Ram, my manager, walked past. "Are you eating that?" (pointing to the nibbled food in front) "Yeah... why?" "It's chicken." My nibble was small, and I was still hungry, but there was no way I could eat the rest of it. Unfortunately, that turned out to be my last meal for 24 hours. (But, like I said, that's another story.) At Charlotte, food wasn't so much of a problem. A bean burrito for dinner from Taco Bell worked just fine. But we drove to Washington DC one weekend, and that's when big trouble struck. You see, a bean burrito contains beans. That's why it's called a bean burrito. Presumably, therefore, a hamburger would contain ham. Why else would it be called a hamburger? I can't eat that. Ham is meat. But a cheeseburger is safe to eat, since it only contains cheese. So Ram, Gayathri and I (the vegetarians) ordered a cheeseburger each, while Kallol and Dev (the non-vegetarians) ordered some chicken. As usual, I was the hungriest, and took a big bite. It didn't feel right. Neither the taste, nor the look of the big red blob that I had taken a massive bite of. (mouth full) "Kallol, what's this?" (after a careful examination) "I think it's beef." Aaargh! Meat is bad, but beef is a sin. My mouth was still full of it. Panic sets in. Spit it all out. Need to wash mouth. Where do I go? Does this country use tissue for that as well? Run into men's room... After a while, Ram (who HADN'T taken a bite) calmed me down. And I was wiser. The person at the counter must have made a mistake. It was quite noisy at the Smithsonian Museum restaurants. She must have heard us say the "burger" part, and not the "cheese" part. Quite understandable. So a few days later, under much quieter circumstances, we ordered two cheeseburgers ("with extra cheese, please") each. For the night... Two years later, at London Business School, Accenture's pre-placement talk was scheduled just after lunch, and they were sponsoring lunch. Naturally, I was there. Wise as I was, I carefully picked the vegetarian sandwiches -- after verifying with two people independently (and re-verifying after one of them had taken a bite of it.) The good part about being a vegetarian is that desserts are OK. Meat doesn't go into desserts. Or biscuits. Or snacks. Or so I thought. The assortment of biscuits was good, and I helped myself to a few. One of them was a bit salty, and rather good. So I went over for a second helping. I couldn't reach it from where I stood, given the small crowd that was busily discussing work at Accenture just near the table. So I politely walked over to the other side, from where the biscuit was closer anyway... ... and read the label. "Prawn something". Shit. Panic. Nausea. This time I had actually eaten it. There was no mistaking the nausea in my stomach. I gulped three glasses of water before I calmed down a bit. Convinced myself that it's OK because it tasted quite fine until I figured it was non-vegetarian. And ended up skipping the rest of lunch and the Accenture PPT anyway. (Too queasy in the stomach.) Things haven't changed much since 2000. I've managed to avoid non-vegetarian since. And learned that Pepperoni is not a form of pepper, nor is bolognese a variant of lasagna. But on the few occassions people try to point out to the beef fat content in Mentos or fish liver oil in a sauce, my only response is: Don't tell me. I'm OK with non-vegetarian if I don't know about it. P.S: I'm OK with eggs, though. Comments Bimal 9 Jan 2007 12:00 pm: You have very well described your experience, mate. I do appreciate the Music Mission you have organised.\ \ Thanks very much.\ \ Regards,\ \ Bimal

I am a vegetarian


I share the transcript of Amitabh Bachchan’s "India Poised" speech. It captures the tension between a skeptical, traditional India and a dynamic, emerging nation ready to lead and succeed on the global stage.

India Poised : a video featuring Amitabh Bachchan. Here's the transcript. There are two Indias in this country. One India is straining at the leash, eager to spring forth and live up to all the adjectives that the world has been recently showering upon us. The other India is the leash. One India says "Give me a chance, and I'll prove myself." The other India says "Prove yourself first, and maybe then, you'll have a chance." One India lives in the optimism of our hearts. The other India lurks in the scepticism of our minds. One India wants. The other India hopes. One India leads. The other India follows. These conversions are on the rise. With each passing day, more and more people from the other India are coming over to this side. And quietly, while the world is not looking, a pulsating, dynamic India is emerging. An India whose faith in success is far greater than its fear of failure. An India that no longer boycotts foreign-made goods, but buys out the companies that makes them instead. History, they say, is a bad motorist. It rarely ever signals its intentions when it's taking a turn. This is that rarely ever moment. History is turning a page. For over half a century, our nation has sprung, stumbled, run, fallen, rolled over, got up and dusted ourself, and cantered, sometimes lurched on. But now, in our sixtieth year as a free nation, the ride has brought us to the edge of time's great precipice. And one India, a tiny little voice at the back of the head, is looking down at the bottom of the ravine, and hesitating. The other India is looking up at the sky and saying, "It's time to fly." Comments Iqbal 19 Jan 2007 6:24 am: visit www.indiapoised.com for more Bachchan, Gulzar, etc on India Poised Saurabh 20 Jan 2007 10:40 am: An initiative from TOI, Read more here at http://www.indiapoised.com/outline.htm yogesh sharma 27 Jan 2007 10:39 am: the video was fentastic, both in hindi as well as english, when i listen it first time i was shocked that this thing i was realising before 1 month ago and now appricate this video so much MJ 28 Jan 2007 4:18 pm: This was really great. I heard it on a desi radio channnel ini Toronto Canada.....I loved it and decided to search for it.... Thank you for putting it up......I am going to passthe link to many.... NJ 10 Feb 2007 1:09 am: Can someone please put the lyric for shankar mahadevan's versio.. Phalak pakad... please Titiksha 10 Feb 2007 2:14 am: Excellent lyrics, Words flowing from deep-rooted sesitivity of the poet, Masterly rendered by the born actor in his deep, resonant voice creating the precise impact. HATS OFF. Arindam Dey 10 Feb 2007 3:08 pm: Send me this shaishav 17 Feb 2007 12:06 pm: hats off to the words they r written n said so wonderfully tht they touches the inner heart n soul......... salute once again Concerned 19 Feb 2007 5:41 am: All this India Poised nonsense is pure marketing bullshit Rohan 7 Mar 2007 8:12 am: Mr.Bacchan is fantastic,and the way hes prested it ,its jus wow!!!!! subadra 18 Mar 2007 12:51 pm: it s good Deepa Sairam 22 Mar 2007 7:41 pm: Very Inspiring. harsh selarka 10 Sep 2008 9:11 am: very inspiring & fantastic job done by amitabh bachhan gaurav kumar 22 Jan 2009 5:25 pm: very inspiring..too goodddddddd Male Sosa 31 May 2011 4:16 pm: It´s not only inspiring it´s been writen with love and pride, with humble & honest words. It reaches the heart & the soul. Riya 14 Aug 2012 11:12 am: @Concerned it is only a hard heart that isnt moved by these lovely words

India Poised


I visited the Guangzhou Chen Family Temple and captured photos of incredible ivory sculptures. Some feature intricate nested spheres that seem impossible to carve, showcasing the extraordinary level of detail in traditional Chinese craftsmanship.

Ivory sculptures at the Guangzhou Chen Family Temple. The first two, especially, have spheres within spheres within spheres... which looks impossible to carve. Intricately carved ivory at the Chen Family Temple Intricately carved ivory at the Chen Family Temple Intricately carved ivory at the Chen Family Temple Intricately carved ivory at the Chen Family Temple

Ivory sculptures


I catch client-side JavaScript errors by wrapping functions in try-catch blocks and sending logs to a Perl script via dummy image requests. This gives me instant feedback on where and why my code fails across different browsers.

If something goes wrong with my site, I like to know of it. My top three problems are: 1. The site is down 2. A page is missing 3. Javascript isn't working This is the last of 3 articles on these topics. I am a bad programmer I am not a professional developer. In fact, I'm not a developer at all. I'm a management consultant. (Usually, it's myself I'm trying to convince.) Since no one pays me for what little code I write, no one shouts at me for getting it wrong. So I have a happy and sloppy coding style. I write what I feel like, and publish it. I don't test it. Worse, sometimes, I don't even run it once. I've sent little scripts off to people which wouldn't even compile. I make changes to this site at midnight, upload it, and go off to sleep without checking if the change has crashed the site or not. But no one tells me so At work, that's usually OK. On the few occasions where I've written Perl scripts or VB Macros that don't work, people call me back within a few hours, very worried that THEY'd done something wrong. (Sometimes, I don't contradict them.) It can be quite a stressful experience but good thing you can learn more here on how to cope up with it. On my site, I don't always get that kind of feedback. People just click the back button and go elsewhere. Recently, I've been doing more Javascript work on my site than writing stuff. Usually, the code works for me. (I write it for myself in the first place.) But I end up optimising for Firefox rather than IE, and for the plugins I have, etc. When I try the same app a few months later on my media PC, it doesn't work, and shockingly enough, no one's bothered telling me about it all these months. They'd just click, nothing happens, they'd vanish. But their browsers can tell me The good part about writing code in Javascript is that I can catch exceptions. Any Javascript error can be trapped. So since the end of last year, I've started wrapping almost every Javascript function I write in a try {} catch() {} block. In the catch block, I send a log message reporting the error. The code looks something like this: Any time there's an error in function abc, the log function is called. It sends the function name ("abc") and the error details (the contents of the error event) to log.pl, which stores the error, along with details like the URL, browser, time and IP address. This way, I know exactly where what error occurs. This is a fantastic for a three reasons. It tells me when I've goofed up. This is instantaneous feedback. I don't have to wait for a human. If you run my program on your machine, and it fails, I get to know immediately. (Well, as soon as I read the error log, at least.) It tells me where I've goofed up. The URL and the function name clearly indicate the point of failure. It tells me why I've goofed up. Almost. Using the browser name and the error message, I can invariably pinpoint the reason for the error. Then it's just a matter of taking the time to fix it. I'd think this sort of error reporting should be the norm for any software. At least for a web app, given how easy it is to implement. Comments CodeKitten 25 Dec 2007 12:00 pm: You should bind an onerror-triggered method to call your log method aswell :) Danko Stojanovic 10 May 2010 8:12 am: I feel your pain. I also moved away from programming to management. I have to live with customers saying "It doesn't work" and programmers "It works just fine when I test it". I so wish to know what is going on my client's computer. Thanks for this article. It helped me decide to pursue this further Allan Ebdrup 13 Nov 2011 7:23 am:The onerror event is a great way to start logging JavaScript errors, more people should use it. JavaScript errors can be just as bad as a webserver that is down, you really need to log those errors so you can do something about them.onerror, is quirky. It does not fire on iPad, iPhone, Safari and some other browseres. To catch those errors you have to insert try-catch'es arount your event handlers, xhr-requests and some other places. This will also let you get more detail about the errror (call stack in some browsers).But onerror is still much better than nothing. We've solved a lot of these issues, with a solution that you install just like Google Analytics. Currently we are in a testing phase, private beta (free). To get an invite you can got to http://muscula.com and sign up. Kiawaki 8 Jul 2011 5:45 am: A question from somebody who is even less of a pro web developer: how exactly do you use this code? Do you put it before or after the script or both? Thanks! Offbeatmammal 23 Sep 2011 12:48 am: if you want a no-brain, minimal effort then have a look at jsErrLog ... a simple utility that does this and logs it to a remote AppEngine service (all for free, in an open source project) - check out http://jsErrLog.appspot.com which links to the GitHub repository as well

Javascript error logging


I found a real-life Truman Show on Justin.tv, where Justin Kan broadcasts his life 24/7 via a wearable camera. It is a raw, continuous live stream capturing everything from daily routines to private dates in real-time.

The Truman Show is on for real, on Justin.TV. Justin wears the camera 24 hours a day, 7 days a week. Even in the bathroom. Even on a date. This is really live. Honest. Right now. Justin will wear the camera until the day he dies. By which we mean if he takes it off, we'll kill him. The show has been on for 9 days till date.

Justin TV


Explore Malcolm Gladwell's argument that limited information can improve decision-making through heuristics. A study of students shows how recognizing only one city name can lead to more accurate population estimates than having access to more data.

Malcolm Gladwell argues that knowing less can be an advantage. This is based on a study in which kids in the US were asked which was a bigger city: San Antonio or San Diego. Many didn't know. Kids in Germany were asked the same. Most knew: San Diego was bigger. Why? Because they'd heard of San Diego, but not of San Antonio. P.S: A comment mentions that the actual difference in population between these cities is only 2%. So maybe the US kids were right to be unsure... Comments texan 29 Jan 2007 4:20 am: Please read the actual pschologist's blog at http://www.dangoldstein.com/dsn/archives/2006/03/malcolm\gladwel\1.html#note to see the distinction between "city limits" & "metro area". i or any outsider ( still a US person ) would associate san diego(San Antonio) with its metro area than 'city limits'. I think the american kids were influenced by the popular saying 'Everything is big in Texas !!!' i live in Texas BTW

Knowing less is better


I built a home Media PC using a 42-inch plasma TV and an eBay Pentium III. I use it for sofa-based Skype videoconferencing, streaming Tamil movies, and managing audio wirelessly via Remote Desktop and Bose speakers.

Two weeks ago, I pulled together a Media PC. This has been a long-term ambition. I've always wanted to have a PC as the centre of all my media. To use it as a radio, TV, stereo system, CD player, DVD player, etc. I finally did it, for just under 1000 pounds. Media PC Setup At the centre of the setup is my 42" Plasma TV (LG 42PC1D). I was debating between a plasma and LCD TV. The differences, as I understand them, are: 1. Plasma TVs have a higher contrast ratio. My LG 42PC1D has a 10000:1 contrast ratio. An LG 42LC2D has a 1100:1 contrast ratio. The Plasma TV is also brighter (1500cd/m2) than the LCD TV (500cd/m2). 2. Plasma TVs are cheaper for a given size. A 42" LCD TV costs about 5-20% more. 3. LCD TVs are lighter. The only reason this matters for me is if I carry the TV back home to India. But the shipping costs are exhorbitant anyway. So I'd be better off leaving the TV behind. And the weight becomes irrelevant. 4. LCD TVs consume less power. And my power bill is quite high. But I replaced most of the bulbs in our house with energy-efficient ones. Hopefully it will balance out. 5. LCD TVs work better with computers. If you leave an image on a plasma TV for a long time, it burns on the screen. Screensavers become a must. I finally picked the Plasma TV, but it was a borderline decision. The TV is hooked to a Cyberhome DVD player with DivX and a Freeview receiver. The DVD player lets me watch DivX movies I download as torrents. The Freeview player gives me over 40 free channels for casual viewing. (I don't watch enough TV to need Sky TV.) I bought an Intel Pentium III Tower that I bought on eBay. This is my "media PC". I hooked this up to my TV (which has a PC input), a pair of Bose Mediamate speakers (a gift from my brother-in-law) with excellent sound response, and a Labtec webcam. Two other components I bought were wireless: a Microsoft wireless keyboard and mouse to control the system from my sofa, and a Linksys cordless Skype phone (with a speakerphone), so that I could hold videoconferences on Skype while on the sofa. Having set this up, I'm truly beginning to appreciate the convenience of wireless appliances. Right now, I can do any of the above things without gettin up from my sofa. My laptop, phone and wireless keyboard are always just a hand's reach away! Here are some of the things I've been doing (wirelessly): 1. Videoconferencing. I leave the computer on permanently. My parents or in-laws call me on Skype. The cordless phone rings. I can answer Skype calls directly from the phone. When I pick up the call, the webcam turns on automatically. We can sit on the sofa and speak, while they see us. I can turn on the TV and see them through their webcam. It's a full-fledged wireless video-conference setup. 2. Listening to radio. I use my laptop to connect wirelessly to my media PC using Remote Desktop, start up WinAmp, and pick a Shoutcast channel (which has a decent tamil channel list). The sound comes through the Bose speakers connected to the media PC, and I can control the volume from any room, using my laptop. 3. Listening to MP3s. Ditto, except I turn on a playlist on WinAmp. 4. Watching online videos. I turn on the TV, use my wireless keyboard, and connect to Google Video. Most of the time, I watch recent tamil movies or Google Techtalks. 5. Watching TV. (Live from BD and ShareVDO being some choices.) 6. Watching movies. I actually use "low tech" to do this. I record DivX files I download on to a DVD, and play them through my DVD player (which recognises DivX). On those occasions that I download WMV files, I play them through the computer. With this setup, it's easy to do more cool things, like a Truman-show like broadcast (which Justin.TV already does). Comments somnath 29 Mar 2007 2:15 pm: cool. complete value for money. I don't know about the other products but the cyberhome dvd player is truly vfm (bought two in US 3 years back and they still work) Watch movies 29 May 2007 1:24 am: watch movies onine all the movies can be viewed free and without any hassel so if you watch free movies online come over teh http://www.movieDesi.com Watch movies 29 May 2007 1:25 am: If you want to watch english movies online without paying any movies you should come to http://www.newShowtime.com to watch free movies online V Gopal 28 Mar 2007 12:00 pm: Nice Set up..May be you can upgrade to sonos (or something similar) to feed music to all rooms in the house rather than getting it from speakers connected to PC directly. That should make the music side of Media PC complete.!! spyder 30 Dec 2008 1:32 pm: have a similar setup, on that i am now deciding to buy a logitech wireless inovo keyboard and mouse combo, i am also using an open source software called mediaportal which is good because i manage to save all my movies on my pc and get the files updated with info from imdb.with a setup like this i have found that there is really no need for my dvd player. Trike Motorcycles · 4 Nov 2010 10:52 am: Plasma TVs still have much better contrast ratio compared to even the best LCD television "" Celebrity Gossips Forum 3 Dec 2010 12:45 pm: i think that plasma tvs are more expensive than LCD tvs and they are a bit heavier too \".

Making a Media PC


I cut my search engine's load time in half by using the script defer attribute for background loading and optimizing regular expressions. I found that processing one large string before splitting it into objects significantly improved performance across different browsers.

My music search engine takes quite a while to load (typically 40 seconds). That's an unusually long time for a page, given that most of the people that access it are on broadband connections, and are listening to music online. The reason is, firstly, that I'm loading a lot of data . Literally every single song on that you can search comes through as Javascript. All the downloadable Hindi songs, for instance, occupy 1.3 megabytes before compression. On average, this takes about 20 seconds to load. The second reason is, I'm doing a lot of processing with the data . Two things take the bulk of the time: uncompressing the data, and massaging it to allow for different spellings. On average, that takes about 20 seconds. 40 seconds is too long to wait. Actually, the perceived waiting time is 8 seconds, because I start showing the search and the results after 1/5th of the data is downloaded. But 8 seconds is still too long. I managed to cut this time by half with two things I just learned. Firstly, Javascript loads and executes sequentially . As soon as a Javascript block is loaded, it is executed. While it is being executed, the rest of the HTML file is not parsed. So any subsequent Javascript blocks (or images, CSS, any other external links) are not loaded. Bandwidth is just sitting idle while the browser is busy at work on the Javascript. Since I've split all my songs into five equal-sized Javascript files, my search engine loads the Javascript files one after another ! The problem is even worse -- the calculations I do in Javascript take up as much time as the load time. If the loading went on in parallel, by the time the first calculation is done, the second script would have loaded. This problem can be solved for Internet Explorer and Opera. The "defer" attribute loads the scripts in the background , and defers their execution. This reduces the loading time to nearly zero for all the Javascript files except for the first one or two, because by the time my calculations are done, the next script is already loaded. These Javascript files contain a list of songs as a long string, which I then split into individual songs. Then I modify each song using regular expressions so that approximate matches will still work. (For e.g., typing "aa" is the same as typing "a" on this search engine.) The performance of regular expressions is critical to me. Originally, I did this: Split the string into songs Modify each song using regular expressions Now, I changed the sequence to this: Modify the string using regular expressions Split the string into songs When I timed the speed of this change, I realised browsers differ a lot in the speed of processing regular expressions . Here is the time (in seconds) for each browser to process the Javascript before and after the change. Browser Before After Internet Explorer 6.3 5.0 Firefox 17.7 4.7 Opera 93.8 19.9 Internet Explorer wasn't affected much by this change. But Firefox and Opera were heavily impacted. I'm guessing this is because Firefox and Opera have a high setup time for matching regular expressions. Before, I was matching thousands of strings. After, I was matching only one (long) string. IE didn't care much. Firefox and Opera speeded up dramatically. (I suspect my Opera data is skewed by a few people using a rather slow machine... but then, that's probably why they should use Opera in the first place -- it works rather well old machines.) With these changes, my total load time fell from about 40 seconds to about 20 seconds. Pretty decent improvement. There are two further steps I could take from here on. Compress the songs files further Currently, I'm doing two things to compress the songs. Instead of sending the list as a (movie-song),(movie-song),... combination for every song, I send it as a (movie-song,song,song,...)(movie-song,song,song,...)... combination. So I don't repeat the movie names. Secondly, I compress them via HTTP/1.1 . (My server doesn't let me do that with Javascript, actually, because Netscape 4.x doesn't accept compressed Javascript. But since it's such an old browser and none of my viewers use it, I trick the server by renaming my Javascript files as .html, and it passes them on compressed. What I could do additionally is: Remove duplicate songs . If songs are spelt differently, I include them both. But I can knock off the duplicate ones. Compress the text . Though I'm gzipping the text before sending it, I suspect there may be ways of storing the data in a more compressed form. For example, many songs are repeated with the (male) and (female) versions. These could be clubbed (somehow...) Speed up the Javascript calculations further There are two steps I'm doing right now: Modify the string using regular expressions Split the string into songs Here's how much time each step takes (in seconds) across browsers. (Note: these were based on one day's data, sampling about 2,000 page views) Browser Regular expression Split Internet Explorer 0.88 3.04 Firefox 3.76 1.08 Safari 0.47 0.46 Opera 4.96 29.78 Firefox takes a lot of time with regular expressions. IE and Opera take a lot longer to split (which involves creating many objects). I may have to think up different optimisations for the two. The code is available in the HTML of the song search engine . It's readable. The stuff I'm talking about is in the function sl(p,s) . Any thoughts are welcome!

Making my music search engine faster


I built a custom popurls-style feed reader using Google's AJAX Feed API to manage information overload. I also explain how to use Google’s FeedFetcher as a lightweight cron alternative to schedule tasks and archive RSS history.

I have only two problems with Google Reader. The first is that it doesn't support authenticated feeds. Ideally, I'd have liked to have a single reading list that combines my e-mail with newsfeeds. GMail offers RSS feeds of your e-mail. But the feeds require authentication (obviously) and Google Reader doesn't support that right now. (So I usually don't read e-mail :-) The second is that it's tough to manage large feeds. It's a personal quirk, really. I like to read all entries. If there are 100, I read all 100. If there are 1000, I struggle but read all 1000. I'm too scared to "Mark all read" because there are some sources that I don't want to miss. The 80-20 rule is at work here. There are some prolific writers (like Scoble) who write many entries a day. There are some prolific sources (del.icio.us or digg). I can't keep up with such writers / sources. I don't particularly want to. If I missed one day of del.icio.us popular items, I'll just read the next day's. With Google Reader, that makes me uneasy. I don't like having 200 unread items. I don't like to mark them all read. In such cases, popurls' approach is useful. It shows the top 15-30 entries of the popular sites as a snapshot. Any time you're short of things to read, visit this. If you're busy, don't. Using Google's AJAX Feed API, it's quite trivial to build your own feed reader. So I cloned popurls' layout into my bookmarks page, and put in feeds that I like. You can customise my bookmarks page to use your own feeds. Save the page, open it in Notepad, and look for existing feeds. They'll look like this: The first line ("hacker news") is the title of the feed. You can call it what you want. Set entries to the number of feed entries you want to show. Set url to the RSS feed URL. Save it, and you have your own feed reader. (If you want to put it up on your site, you may want to change the Google API key.) Try it! Just save this page and edit the feeds. Here, I must point out three things about Google's AJAX Feed API that make it extremely powerful. The obvious is that is allows Javascript access to RSS in a very easy way. That makes it very easy to integrate with any web page. The second is subtler -- it includes historical entries. So even if an RSS feed had only 10 entries, I could pick up the last 100 or 1,000, as long as Google has known about the feed for long enough. This is what makes Google Reader more of a platform rather than a simple feed reader application. Google Reader is a feed archiver -- not just a feed reader. The third (I'm a bit crazy here) is that you can use it to schedule tasks. Google's FeedFetcher refreshes feeds every 3 hours or so. If you want to do something every three hours (or some multiple thereof -- say every 24 hours), you can write a program that does what you want, and subscribe to it's output as a feed. You may notice that I have a Referrers to s-anand.net on my bookmarks page. These are the sites that someone clicked on to visit my site. I have a PHP application that searches my access log for external referrers. Rather than visit that page every time, I just made an RSS feed out of it and subscribed to it. Every three hours or so, Google accesses the URL. I search my access.log and archives the latest results. So, even after my access.log is trimmed by the server, I have it all on Google Reader to catch up with later. Since Google doesn't forget to ping, I can schedule some fairly time-critical processes this way. For instance, if I wanted to download each Dilbert strip, every day as it arrives, I can create an application that downloads the file and returns a feed entry. Now, I don't need to remember to run it every day! I just subscribe to the application on Google Reader, and Google will remind the application to run every 3 hours. (I know -- I could use a crontab, but somehow, I like this.) Plus I would get the Dilbert strip on my feed reader as a bonus. Update: Google has just released PartnerBar, which is a more flexible way of viewing a snapshot of feeds.

Managing feed overload


I share Peter Norvig's talk on managing massive datasets, where he explains why having vast amounts of data can be more effective for problem-solving than complex algorithms, a core principle in modern machine learning.

Peter Norvig's brilliant talk on Managing the Data Deluge. Among other things, he talks about how having lots of data is sometimes better than having a carefully designed algorithm.

Managing the data deluge


I found a map visualizing global GDP per square kilometer, highlighting economic density rather than just national totals. It offers a unique geographic perspective on where wealth is concentrated across different continents and countries.

A map of GDP per square kilometer across the world.

Map of GDP per square kilometer


I use Site24x7 to monitor downtime and receive SMS alerts for critical issues. By analyzing uptime reports and lost ad revenue, I justified switching to a more reliable web host to ensure better sleep and site availability.

If something goes wrong with my site, I like to know of it. My top three problems are: 1. The site is down 2. A page is missing 3. Javascript isn't working I'll talk about how I manage these over 3 articles. My site used to go down a lot. Initially that was because I kept playing around with mod\rewrite and other Apache modules without quite understanding them. I'd make a change and upload it without testing. (I'm like that.) And then I'd go to sleep. Next morning, the site's down, and has been down all night. This is a bit annoying. Partly because I couldn't use my site, but mostly because of the Oh yeah, sorry -- I goofed up last night replies that I have to send out the next day. So I started using Site24x7 to track if my website was down. It's a convenient (and free) service. It pings my site every hour. If it's down, I get an SMS. If it's back up, I get an SMS. It also keeps a history of how often the site is down. Site24x7 Over time, I stopped making mistakes. But my site still kept going down, thanks to my hosting service (100WebSpace). When I goof up, it's just an annoyance, and I can fix it. But when my hosting service goes down, it's more than that. My site is where I listen to music, read comics, read RSS feeds, use custom search engines, watch movies, browse for books, etc. Not being able to do these things -- nor fix the site -- is suffocating. Worse, I couldn't sleep. I use my mobile as my alarm. It's annoying to hear an SMS from under your pillow at 3am every day -- especially if it says your site is down. So I switched to HostGator a few months ago. Nowadays, the site is down a lot less. (In times of trouble, it becomes sluggish, but doesn't actually go down.) That came at a cost, though. I was paying 100 WebSpace about $25 per annum. I'm paying Hostgator about $75 per annum. Being the kind that analyses purchases to death, the big question for me was, is this worth it. There is where my other problem with the site being down kicks in. I get a bit of ad revenue from my site, and I lose that when the site's down. (Not that it's much. Still...) According to Site24x7, my site was up 98% of the time. So I'm losing about 2% of my potential ad revenue. For the extra $50 to be worth it, my ad revenue needs to be more than $50 / 2% = $2,500 per annum. I'm nowhere near it. So the switch isn't actually a good idea economically, but it does make life convenient (which is pretty important) and I sleep better (much more so). The important thing, I've realised, is not just to track this stuff. That's useful, sure. But what really made Site24x7 useful to me is that it would alert me when there was a big problem. There are many kinds of alerting. There's a report you can view whenever you remember to view it. (It could be an RSS feed, so at least you won't have to remember the site. But you still need to read your feeds.) Then there's the more pushy alerting: sending you an e-mail. That may catch you instantly for the half of the day that you're online. Or, if you're like me, it may completely escape your attention. (I don't read e-mail.) And then there's the equivalent of shaking you by the shoulder -- getting an SMS. (At least, that's how it is for me. Incidentally, I don't reply to SMS either. Calling me gets a reply. Nothing else.) The type of alerting is clearly a function of the severity of the problem. Wake me up when my site goes down. Don't wake me up if a link is broken. Site24x7 sends me an SMS when my site is down. Fits the bill perfectly. Comments Arun 24 Dec 2007 12:00 pm: Thanks for your post on Site24x7. Glad that you liked it! Arun Site24x7 Noel 24 Dec 2007 12:00 pm: You can also try Dotcom-monitor. It smart and cheap service

Monitoring site downtime


I discovered my most bookmarked pages by analyzing web traffic with blank referrers. By combining referrer tracking with a visitor survey, I confirmed that the absence of referrer data reliably indicates bookmarks or direct URL entries.

These are the most bookmarked pages on my site: 1. My home page 2. Excel tips 3. Calvin & Hobbes quotes (I typed them all) 4. Indian torrents (I have a search engine for Indian torrents) 5. Tamil Transliterator (Lets you type Tamil in English) 6. Tamil songs quiz 7. Movie quote quiz 8. My best links 9. Top 10 lists But this post is not about these links. It's about how I found this out. Think about it... how could I know what pages have been bookmarked? The browser doesn't send any information about bookmarks. Some months ago, I moved away from Google Analytics mainly to have more control over tracking visitors. Among other things, I track referrers. When you click on any page and go to another one, the second page knows the first page you came from. That first page is the referrer. So I know every page people clicked on to get to my site. Usually it's Google. Sometimes it's someone's blog. Sometimes, it's blank. The blank referrers either indicate that the browser has blocked the referrer page, or that the person didn't visit any page before mine. The former is rare (less than 1%). So, realistically, a blank referrer has either the person typed in the URL, or bookmarked it. To make sure, I did a quick survey over the weekend. Those who came to my page without a referrer saw a survey form, asking where they came from. Almost all of them had either bookmarked or typed my page. The typists went directly to my home page. All the other links you see above are bookmarks. So all I had to do was count the number of hits for each page with blank referrers. That's the list above. Absence of information can be a powerful indicator too. Comments Chitra 16 Jan 2007 7:11 am: Oho...how many people did the list comprise of? S Anand 16 Jan 2007 12:56 pm: I had over 4,000 unique IP addresses, but I don't believe that's even representative of visitors, thanks to most ISPs using dynamic IPs. I get about 80 direct IP hits per day, so I guess the number of bookmarks is around twice that. Rupesh 17 Jan 2007 9:41 am: Would you get to know if someone has subscribed to your site feed this way? S Anand 17 Jan 2007 12:06 pm: Actually, I use Feedburner to track subscriptions to my site. But no, using this I can't always tell if it's a feed subscription. People coming through live bookmarks can't be distinguished from normal bookmarks. But people using feed readers (Google Reader / Bloglines) can be identified by their referrer. Shreyas 19 Jan 2007 5:46 am: Apologies for a completely off-topic comment but have you taken off the 'Starred Items on Google Reader' link? I kind of enjoyed reading the ones you had a couple of weeks back!

Most bookmarked pages


I explore Hans Rosling's talk on global development myths, highlighting his animated graphics that reveal hidden trends. He proves our misconceptions by showing that even experts often perform worse than random chance at identifying global statistics.

An excellent talk about the myths we hold on the developing world, supported by the most amazing graphics I've seen in a while. Among other things, the speaker (Hans Rosling) proves that chimpanzees are much smarter than the top Swedish students, and are slightly better than Swedish professors when it comes to knowing the developing world. The first one of the series that I heard was the TEDTalk by Sir Ken Robinson. May be worth hearing all the TEDTalks. Comments Anand Srini 26 Feb 2007 9:42 pm: i watched the presentaion. the animated graphics brings to life the trends hidden beneath the numbers which are otherwise so dull to look at.

Myths about the developing world


I shared a link to Oscar Torrents, a platform for downloading nominated movies and casting unofficial votes. It offers a community-driven way to catch up on Academy Award contenders through peer-to-peer sharing before the ceremony.

You can download the Oscar nominated movies at Oscar Torrents, and vote for them. (No, the decision won't be based on this -- it's completely unofficial.) via Dhar Comments zack 13 Feb 2007 12:00 pm: good downloading is enough for me

Oscar torrents


I discovered an impressive periodic table of visualization methods that organizes 100 different graphs and diagrams into six functional groups. It covers everything from metro maps and treemaps to strategic metaphors and data-driven scatterplots.

Periodic table of visualization methods. 100 visualizations (graphs, diagrams, etc.) organised into 6 groups (data, information, concept, strategy, metaphor, compound), and arranged exactly like the periodic table. (This, in itself, is a lovely visualization.) It includes diverse visuals from the Metro Map (e.g. London Underground Map) and cartoons to scatterplots and treemaps. Just browse it. It's wonderful. Comments Rajlaxmi 16 Feb 2007 6:37 pm: gr8 link n well explained. wonder how u get these links

Periodic table of visualization methods


I reduced my server's CPU load by 50% by switching from MySQL to flat files. I split blog content into alphabetized text files and used Unix grep for faster lookups, bypassing the overhead of Perl's DBI module.

I'm been using a shared hosting service with 100 WebSpace over the last 7 years. It's an ad-free account that offers 100MB of space and 3GB of bandwidth per month. Things were fine until two months ago, which was when my song search engines started attracting an audience. I had anticipated that I might run out of bandwidth, so I used a different server (that has 5GB of bandwidth per month quota) for loading the songs. But what I didn't anticipate whas that my server load would run over the allotted CPU limit. You'd think this is unusual, given how cheap computing power is, and that I'd run out of bandwidth quicker. But no. My allotted limit was 1.3% of CPU usage (whatever that meant), and about 2 months ago, I hit 1.5% a few times. I upgraded my account to one which had a 2.5% limit immediately, but the question was: why did this happen? This blog uses a lot of Perl scripts. I store all articles on a MySQL database. Every time a link is requested, I dynamically generate the HTML by pulling up the article from the MySQL database and formatting the text based on a template. Schematic of how my website displays pages I also use MySQL to store users' comments. Every time I display each page, I also pull out the comments related to that page. I can't store the files directly as HTML because I keep changing the template. Every time I change the template, I have to regenerate all the files. If I do that on my laptop and upload it, I consume a lot of bandwidth. If I do that on the server, I consume a lot of server CPU time. Anyway, since I'd upgraded my account, I thought things would be fine. Two weeks ago, I hit the 2.5% limit as well. No choice. Had to do something. If you read the O'Reilly Radar Database War Stories, you'll gather that databases are great for queries, joins and the like, while flat files are better to process large volume data as a batch. Since page requests come one by one, and I don't need to do much batch processing, I'd gone in for a MySQL design. But there's a fair bit of overhead to each databasse query, and that's the key problem. Perl takes a while to load (and I suspect my server is not using mod\perl). The DBI module takes a while to load. Connecting to MySQL takes a while. (The query itself, though, is quite fast.) So I moved to flat files instead. Instead of looking up from a database, I just look up a test file using grep. (I don't use Perl's regular expression matching because regular expression matching in UNIX is faster than in Perl.) I have a 1.6MB text file that contains all my blog entries. But looking up a 1.6MB text file takes a while. So I split the file based on the first letter of the title. So this post (Reducing the server load) would go under a file x.r.txt (for 'R') while my last post (Calvin and Hobbes animated) would go under a file x.c.txt (for 'C'). This speeds up the grep by a factor of 5-10. On average, using MySQL query used to take 0.9 seconds per query. Now, using grep, it's down to about 0.5 seconds per query. Flat files reduced the CPU load by about half. (And as a bonus, my site has no SQL code. I never did like SQL that much.) So that's why you haven't seen any posts from me the last couple of weeks. Partly because I didn't have anything to say. Partly because I was forced to revamp this site. Comments Sathya 6 Jun 2007 12:00 pm: Wow. thats quite interesting. Perhaps you could have also tried caching. keep a copy of ''hot'' (Most Recently used -MRU) stuff in memory, so you dont have to relaod it again. Given that most of the hits would be for the latest entries ( or atleast 80% of the hits are for 20% of ur posts) then caching might be an option. However, I dont know how much of RAM quota you have ;-) somnath 6 Jun 2007 12:00 pm: Do you work at your company in business consulting or technology. Having known you for the past eight years (jeez a long time) I have no doubt you manage not just both but spend quality time with your family as well. Maybe it would be a good idea to blog about your typical day when you juggle between work (consulting for a big client), passion (revaming this site or taking pictures or doing a project like building media pc etc. etc.) and family (playing with a baby daughter is wonderful). BTW got your mail but I check internet mail very infrequently. Cheers! Saurabh 6 Jun 2007 12:00 pm: This one is slightly off the topic..I also started using freehostia a few months back, however I find that 2 out of 7 days in a week, my website is not available for more than 96% of the time..(I monitor the uptime using something called as InternetSeer)..Would you have any tips here, is freehostia not a good option for free services or should I switch to some other service provider? Rocco Blegen 4 Dec 2011 4:14 am: Howdy! This post couldn’t be written any better! Reading through this post reminds me of my old room mate! He always kept talking about this. I will forward this post to him. Pretty sure he will have a good read. Thank you for sharing!

Reducing the server load


I built an XPATH user-defined function to extract live data from RSS feeds and XML into Excel. I use it to automate movie wishlist tracking and IMDb rating lookups, treating Excel as a versatile web interface.

The technique of Web lookups in Excel I described yesterday is very versatile. I will be running through some of the practical uses it can be put to over the next few days TO generalise things beyond just getting the Amazon price, I created a user-defined function called XPATH. It takes two parameters: URL of the XML feed to read\ Search XPath list string (separated by spaces) This function can be used to extract information out of any XML file on the Web and get it out as a table. For example, if you wanted to watch the Top 10 movies on the IMDb Top 250, and were looking for torrents, an RSS feed is available from mininova. The URL http://www.mininova.org/rss/movie\name/4 gives you an RSS file matching all movies with "movie\name". From this, we need to extract the and elements. That's represented by "//item title link" on my search string. Mininova RSS feed in Excel The formula becomes XPath2( "http://www.mininova.org/rss/"&A2&"/4", "//item title link"). The result is a 2-dimensional array returning individual items in rows, and the columns are title and link. Pulling it all together, you can get the sheet above. All of this could be done using a command-line program. Excel has one huge advantage though. It's one of the most powerful user-interfaces. Increasingly, I'm beginning to rely on just two user interfaces for almost any task. One is the browser, and the other is Excel. With Excel, I could have a sheet that has my movie wishlist (which changes often), and add check to see if the torrent exists. Every time I add a bunch of movies to the wishlist, it's just a matter of copying the formula down. No need to visit a torrent search engine and typing each movie in, one by one. Another example. Someone suggests 10 movies to watch. I'd rather watch the ones with a higher IMDb rating. Again, enter the Web lookup. Type in the movie names. Use a function like this to look up the rating on IMDb, and sort by the rating. The possibilities are endless. Comments Isit? 3 May 2007 12:00 pm: Is it possible to have the feeds update cumulative so the old feeds dont get replaced in excel? Alex 3 May 2007 12:00 pm: Hi, I have an Excel spreadsheet with URLs of move titles on IMDB, and I would like to retrieve the movie rating. I've read your tutorials, and this one comes closest to what I need. However I still cannot figure out what needs to be done. It looks like you have this problem solved. If so, can you possibly share your solution with me? Thank you, Alex Kishan 3 May 2007 12:00 pm: I came here searching for the other way round. That is to publish an RSS feed of what I have in Xcel! Thanks Anyways venkat 15 Oct 2010 4:09 pm: Anand Can you provide the xpath function?, I need to read a rss feed into excel, and i want only some columns of data, i would appreciate if you could send the xpath function to my email id Thanks venkat Light Sensors 2 Dec 2010 7:08 pm: movie ratings really depend on how cool the movie is, i usually rate movies depending on the story line `-; rt 27 Jun 2013 5:41 pm: Hi, nice job ! can you share with me your xpath2 (i tried your other one without succes on amazon) ?Thanks, R

RSS feeds in Excel


I discovered that cushioned running shoes can cause injuries by masking ground pressure, leading to harder landings and bone stress. Surprisingly, expensive shoes may be more harmful than cheaper ones by encouraging excessive impact force.

Running shoes may actually cause injuries. The heel has evolved to detect the pressure of hitting the ground, and to adjust the force with which to land our feet. Cushioned shoes soften the pressure. So we tend to land with greater force, creating more stress on our bones. In fact, the more expensive the shoe, the more likely the injuries! Comments Dhar 24 Feb 2007 7:03 pm: Argh... this is crazy.\ \ I have been running for the past year or so and recently shifted to Nike Pegasus because of the cushioning it provides. The whole article seemed like a rant. Will need to check the references he quotes before I can really believe this article.\ \ That said, I will nevertheless try a bit of barefoot running as the article suggests and see if it really helps. Cheers, D.

Running shoes may be harmful


I compiled a spreadsheet of major Science Fiction and Fantasy book awards to identify top-tier reads. I highlighted novels like Neuromancer and American Gods that won multiple awards, including the Hugo, Nebula, and Philip K. Dick awards.

Now that I'm well on my way to watching the Top 250 movies on IMDb, I'm slowly turned my attention to fiction. My interest is mainly in the Fantasy & Science Fiction area. Unfortunately, I don't know of any list like the IMDb Top 250, but there are a few awards that could take the place of the Oscars for books. That's probably a good place to start. The most popular awards in Science Fiction are the Hugo award and Nebula award, followed by the Philip K Dick award, John W. Campbell award, Arthur C Clarke award and other awards. I collated a list of all the awards (from LocusMag) into the spreadsheet below Very few of these books have won multiple awards. None have won more than 3 on this list. Of these, only five have 3 awards: Rendezvous with Rama by Arthur C. Clarke American Gods by Neil Gaiman Neuromancer by William Gibson Gateway by Frederik Pohl Forever Peace by Joe Haldeman Over 20 books have won two of these awards. Dune by Frank Herbert The Left Hand of Darkness by Ursula K. Le Guin The Gods Themselves by Isaac Asimov Ringworld by Larry Niven Timescape by Gregory Benford Dreamsnake by Vonda McIntyre The Child Garden by Geoff Ryman Gloriana by Michael Moorcock Fairyland by Paul J. McAuley The Forever War by Joe Haldeman Jonathan Strange & Mr Norrell by Susanna Clarke Doomsday Book by Connie Willis Boy's Life by Robert McCammon Paladin of Souls by Lois McMaster Bujold A Deepness in the Sky by Vernor Vinge The Fountains of Paradise by Arthur C. Clarke Speaker for the Dead by Orson Scott Card The Dispossessed by Ursula K. Le Guin The Time Ships by Stephen Baxter Startide Rising by David Brin Ender's Game by Orson Scott Card I haven't read most of these books. The ones I have read are: The Gods Themselves by Isaac Asimov Ender's Game by Orson Scott Card Speaker for the Dead by Orson Scott Card Rendezvous with Rama by Arthur C. Clarke Dune by Frank Herbert That's about a 50% ratio, so I guess this list doesn't quite very well work for me. Or at least, my taste doesn't match the Award critics. But maybe you will find something interesting to read... Comments Sumit Dhar 3 May 2007 1:51 am: Hey Anand, Got most of these books. Let me know which ones you are interested in. I use http://www.geocities.com/area51/cavern/6113/top100.html as a reference for what books to read. Cheers, D. Prasenjeet 3 May 2007 8:10 pm: Unfortunately, some good books disappear through the awards cracks. David Zindell's "Neverness", for example. But I \have\ read most of these and this seems like a great way for a SF-newbie to discover new fiction! D 4 May 2007 1:43 am: Hey Anand, Just realized that the email IDs are posted publicly when we comment. Would it be possible for you to do something about that? Cheers, D.

Science Fiction awards


I built a server that uses XPath to convert static web pages like the IMDb Top 250 into custom RSS feeds. I demonstrate how to target specific HTML elements to track updates when official feeds aren't provided.

If a site doesn't have an RSS feed, your simplest option is to use Page2Rss , which gives a feed of what's changed on a page. My needs, sometimes, are a bit more specific. For example, I want to track new movies on the IMDb Top 250 . They don't offer a feed. I don't want to track all the other junk on that page. Just the top 250. There's a standard called XPath . It can be used to search in an HTML document in a pretty straightforward way. Here are some examples: //a Matches all <a> links //p/b Matches all <b> bold items in a <p> para. (the <b> must be immediately under the <p>) //table//a Matches all links inside a table (the links need not be immediately inside the table -- anywhere inside the table works) You get the idea. It's like a folder structure. / matches the a tag that's immediately below. // matches a tag that's somewhere below. You can play around with XPath using the Firefox XPath Checker add-on. Try it -- it's much easier to try it than to read the documentation. The following XPath matches the IMDb Top 250 exactly. (It's a link inside the 3rd column in a table row in a table row in a table row.) Now, all I need is to get something that converts that to an RSS feed. I couldn't find anything on the Web, so I wrote my own XPath server . The URL: www.s-anand.net/xpath? url=http://www.imdb.com/chart/top& xpath=//tr//tr//tr//td[3]//a When I subscribe to this URL on Google Reader , I get to know whenever there's a new movie on the IMDb Top 250. This gives only the names of the movies, though, and I'd like the links as well. The XPath server supports this. It accepts a root XPath, and a bunch of sub-XPaths. So you can say something like: xpath=//tr//tr//tr title->./td[3]//a link->./td[3]//a/@href This says three things: //tr//tr//tr Pick all rows in a row in a row title->./td[3]//a For each row, set the title to the link text in the 3rd column link->./td[3]//a ... and the link to the link href in the 3rd column That provides a more satisfactory RSS feed -- one that I've subscribed to, in fact. Another one that I track is a list of . link->./@href">new popular movies that make it to the mininova top seeded movies category. You can whiff up more complex examples. Give it a shot. Start simple, with something that works, and move up to what you need. Use XPath Checker liberally. Let me know if you have any isses. Enjoy! Comments Mark 17 Dec 2007 12:00 pm: Have you ever thought about introducing authentication to the XPath server? I would like to parse certain fields of a page that is authenticated with cookies. Rog 28 Oct 2008 1:07 am: Any chance you could share your xpath.php code? It seems the server is no longer available. S Anand 28 Oct 2008 1:43 am: Sure Rog. I've mailed it to you S Anand 7 Mar 2009 10:35 am: Post Yahoo's introduction of Yahoo Query Language, you're much better off using that instead of my XPath utility. I've covered it in this article on client side scraping. Scraping your way to RSS Feeds! « Technosiastic! 8 Apr 2009 10:16 am (pingback): [...] another gem I figured which actually lets you run XPath query for scraping into a web page for RSS. It can be [...] Bart P 3 Mar 2012 11:48 am: It would be great if you could share this code, I really like to use this server, but want to remove session ids from the links (so my reader doesn't think all links are new every time). Is that possible? :) Ben 11 Jun 2015 12:46 am: Is that possible to share your xpath.php code? yahoo pipes is going to be shut down :(

Scraping RSS feeds using XPath


I discovered Scribd, a document-sharing platform that functions like a YouTube for books. I've used it to find technical titles such as On Lisp by Paul Graham and various software pattern guides for my reading list.

Scribd is a document sharing site. Sort of like a YouTube for documents. In other words, a book-lover's paradise. Comments Dhar 7 Mar 2007 3:52 am: Anand, I found "The God Delusion" on Scribd. What were the interesting titles you found? S Anand 7 Mar 2007 7:01 am: On Lisp by Paul Graham, Excel Time-Saving Techniques for Dummies and Patterns of Software. prakash ayer 25 Mar 2007 11:54 am: Thanks Anand. Pretty good site.

Scribd


I built an interactive quiz featuring background music from SD Burman's iconic songs. Listen to the snippets and guess the movie titles; the input box turns green when you enter a phonetically correct answer.

Here is the background music from SD Burman's songs. Can you guess which movie they are from? Don't worry about the spelling. Just spell it like it sounds, and the box will turn green. Comments Kalpesh 20 Mar 2007 1:47 pm: Dude, it throws 404 (file not found). Moreover, I think it is copy-paste problem. All tje songs point to a.mp3 ;) BTW, I like your site & have subscribed to you in bloglines. Sri 9 May 2007 8:11 pm: Cool ones.... could you give me answer to the SD Burman quiz please, I only I got 3 right :(

SD Burman songs quiz


I show how to solve multiple-choice questions faster by picking the correct answer through elimination rather than full calculation. I use shortcuts like checking the units digit, divisibility properties, and quick estimation to identify the right choice.

How would you solve this multiple-choice problem: What is 12345 x 45678? 1. 201932843 2. 563894910 3. 402394820 4. 384718349 5. 938491834 It always amazes me when people try and multiply the two numbers. In any objective-type test (multiple choice question), the aim is not to solve the problem -- it is to pick the correct answer! Most people don't seem to realise the difference. If I had to solve the problem, I'd look for shortcuts. For example, 1. All answers have 9 digits. So that's not going to help. 2. 12345 ends with a 5. All multiples of 5 end with 5. So the answer is either (2) or (3). 1. 201932843 2. 563894910 3. 402394820 4. 384718349 5. 938491834 3. Multiplying the first digits, 1 X 4 = 4. So the answer's got to start with a bit more than 4. (3) starts with 40. That's too low. C'mon, the second number STARTS with 45. So... 2. 563894910 3. 402394820 This takes me 10 seconds. Multiplying the two takes me ten times as long. Let me repeat the point: You DON'T have to get to the answer. You've already been given the answer. You just don't know which one it is. And you don't have to solve the entire problem to pick the right one. There are many ways of picking the right answer. One is what I just used: the answer must satisfy some property. In this case, the answer must be a multiple of 5. In other cases, the answer must be close to something. For example, what's 37463 x 28438? 1. 532686397 2. 1065372794 3. 2130745588 4. 4261491176 5. 8522982352 Well, let's just multiply the first two digits: 37 x 28. That's 1036. So the answer's got to be close. OK, (2) is the answer. These are not one-off techniques, nor are these applicable only to numbers. You can always pick the right choice without solving the problem. A weaker, but more general version of the above rule is: you can always eliminate choices without solving the problem. I'll blog more on this shortly. Comments anand 19 Jan 2007 9:47 am: hats off. U made me to think....... Vinodh 19 Jan 2007 11:16 am: First rule should be to eliminate the options based on the product of the last digits. So, for the second example, it's easier to eliminate all the other four options based on this rule. Of course, there are easier techniques for generic multiplication/division as well...I have an somewhat interesting technique to calculate squares/products based on simple algebraic formulae...of course they are better used with some practice, but very powerful. jake 19 Jan 2007 1:44 pm: its easier and faster to get the answer for the second one by looking at the last digit of the answers. it has to be the same as the last digit of the product of the unit place digits of the numbers being multiplied. Shreyas 20 Jan 2007 8:14 am: "All multiples of 5 end with 5" - I guess it should be "All multiples of 5 end with 5 or 0" :) Anonymous 20 Jan 2007 7:55 pm: CAN YOU PLEASE TELL ME WHERE i CAN FIND MORE TIPS ON SOLVING MCQ? I LIKED GOING THROUH THIS BUT I NEED MORE PRACTICE. THANK YOU Anonymous 22 Jan 2007 5:58 pm: There is a much better way to solve Q2. the last digit of N1 is 3 and the last digit of N2 is 8. 3\8 is 24 so N1\N2 has to end with 4. so the answer should be GV Rao 7 Mar 2007 7:42 am: I would like to multiply first 2 digits: 45 x 78, and in second case: 3 x 8, which is easy, last digits gives more guess on product. Sri Lankan Friends 10 Mar 2007 10:23 am: hmmm very interesting ali 24 May 2007 8:16 pm: good but should be more efficent via understanding vasudha 19 Jan 2007 12:00 pm: HATS OFF TO YOU... THE INFORMATION IS PRETTY GOOD.. YOU MADE ME THINK deepak 25 Feb 2011 6:20 am: phaltu trick sab hai... issss se kahi behtar tricks mere paas hai.... sirf tukka maar ke IIT me aa gaya hu YES I AM IN IIT...

Solving multiple choice questions


I'm tracking the RIAA's win as the Worst Company in America 2007. This highlights the industry's poor reputation, driven by the actions of its major members: EMI, Sony, Universal, and Warner Music Group.

RIAA wins the "Worst Company in America 2007". The RIAA is mainly EMI, Sony, Universal and Warner.

Sony Universal Warner and EMI the worst companies in 2007


I used John Resig’s sparkline library to visualize my data on HTTP download speeds. These tiny, word-sized charts effectively illustrate how performance isn't linear, bringing Edward Tufte’s dense data visualization concepts to life with simple JavaScript.

John Resig has written a Sparklines library. Here's an example. I wrote that HTTP download speeds not linear 182,315,313,319,314,459,441,445,453,525,567,552,577,587,580,581,590,663,639,658,616,705,720,695,739,750,720,741,803,800,800,818,800,856,796,816,866,841,836,828,861,893,859,905,881,885,946,944,943,984,1003,1012,994,979,977,986,1010,1017,1027,1000,1035,986,1006,1006,996,1022,1003,1053,1046,1061,1002,1064,1014,1039,1061,1023,1076,1081,1019,1064,1072,1089,1101,1069,1128,1125,1092,1155,1170,1067,1094,1082,1178,1211,1154,1169,1133,1161,1193,1167 and that they flatten out over time. A linear line would look like this: 180,190,201,211,221,232,242,252,262,273,283,293,304,314,324,335,345,355,365,376,386,396,407,417,427,438,448,458,468,479,489,499,510,520,530,541,551,561,572,582,592,602,613,623,633,644,654,664,675,685,695,705,716,726,736,747,757,767,778,788,798,808,819,829,839,850,860,870,881,891,901,912,922,932,942,953,963,973,984,994,1004,1015,1025,1035,1045,1056,1066,1076,1087,1097,1107,1118,1128,1138,1148,1159,1169,1179,1190,1200 The little red line here is a sparkline that's based on real data. John's javascript converts the data into a graph. Sparklines were introduced by Edward Tufte.

Sparklines


I share the logic and Perl regex I use to split sentences into words. I address the challenges of ambiguous punctuation like hyphens, periods in abbreviations, and commas in numbers to ensure accurate tokenization without breaking URLs.

I often need to extract words out of sentences. It's one of the things I used to build the Statistically Improbable Phrases for Calvin and Hobbes. But splitting a sentence into words isn't as easy as you think. Think about it. What is a word? Something that has spaces around it? OK, let's start with the simplest way to get words: split by spaces. Consider this piece: Splitting this by spaces (consider new lines, tabs, etc as spaces too.), we get the following: Now, some of these like "I'd" are words. But "McDonald's" isn't. I mean, there's a full-stop and a double-quotes at the end. Clearly we need to remove the punctuation as well. But, if we do that, I'd becomes Id. So we need to be careful about which punctuation to remove. Let's take a closer look. The following punctuation marks are clear word separators: spaces, the exclamation mark, the question mark, semicolon, brackets of any kind, and double-quotes (not single quotes). No word has these in the middle. If we use these as separators, our list of words is better, but we still have some words with punctuation: The issue is, these punctuation marks are ambiguous word separators: comma, hyphen, single-quote, ampersand, period and slash. These usually separate words, but there are exceptions: Comma: Not inside numbers: 3,000,000. Hyphen: Not for hyphenated words: High-fat. Single-quote: Not for possessives: McDonald's. Not for joint words: I'd. Ampersand: Not for abbreviations: R&M Period: Not for abbreviations: O.K. Not for URLs: www.s-anand.net Slash: Not for fractions: 3/4. Not for URLs: google.com/search Colon is ambiguous too. In normal English usage, it would be a separator. But URLs like http://www.s-anand.net/ use these characters, and it doesn't make sense to separate them. So here are my current rules for splitting a sentence into words. (It's a Perl regular expression. Don't worry. Cooper's Law: If you do not understand a particular word in a piece of technical writing, ignore it. The piece will make perfect sense without it.) This doesn't even scratch the surface of the issue, though. Here are some issues: Lots of files split words into two at the end of a line. How do we handle that? How do we handle incorrect punctuation? For instance, if someone types "done.Yet," without leaving a space after the full-stop, I'll think it's an abbreviation. What about other separators? Like the ± symbol or the £ symbol for instance. What about other languages?! And you thought it was easy! Comments Kasi 27 Apr 2007 10:18 pm: Hi Anand, Nice site..It was good talking to you! When u find time, pls do visit my humble less visited blog at www.raconteurkasi.blogspot.com ! Ramya 3 May 2007 7:52 am: Hi Anand, Can u guide me through the interview process of INSEAD MBA . Can you share your wisdom with me to crack the interview, and one more question .. do u sleep for less than 5 hrs a day ? Andrew 26 Apr 2007 12:00 pm: Thanks, this is very helpful.

Splitting a sentence into words


I analyzed Calvin and Hobbes to find the top 20 statistically improbable phrases like "chocolate frosted sugar bombs." I highlight the technical challenges of tokenization, such as handling contractions, and the difficulty of weighting longer, naturally rarer phrases.

My earlier list of statistically improbable phrases in Calvin and Hobbes is technically just a list of "Statistically Improbable Words". I re-did the same analysis using phrases. Here are the top 20 statistically improbable phrases (2 - 4 words only): baby sitter chocolate frosted sugar bombs comic books doing homework fearless spaceman spiff() good night hamster huey ice cream miss wormwood new year peanut butter really think slimy girls spaceman spiff stuffed tiger stupendous man sugar bombs susie derkins watch tv water balloon That is, these are the 2-4 word phrases whose frequency in Calvin and Hobbes is substantially (at least 5 times) higher than in the other books I have. While doing this, the single biggest problem that stumped me was: what is a word? Is "it's" one word or two words? Is "six-year-old" one word or three words? How do I distinguish between abbreviations (g.r.o.s.s.) and full-stops without a space ( ... homework.what's a ...)? Does a comma always split words? (It doesn't in numbers, like "3,500") The other problem is, phrases with more words are more improbable. Right now, if a phrase occurs 5 times more frequently in Calvin and Hobbes than my other books, I include it. But three-letter words rarely occur that often, and four-letter words even less so. Maybe I should have a lower cutoff for longer phrases. Anyway, this analysis is a crude first approximation. Clearly Amazon's gotten much further with their system. Comments satish 20 Feb 2007 3:56 am: Hey Stud, Satish here, your junior from IIMB. Trying to get in touch with you. Do mail me at satishkgv@hcl.in and let us get in touch. Oblio 14 Feb 2007 12:00 pm: Fantastic job man! You have unlimited patience! Reinhard Ebner 14 Feb 2007 12:00 pm: Hey, only just now came across your page, but of the hundreds, if not thousands of C&H sites and tools, this is the most useful I''ve seen! R juergwachter 14 Feb 2007 12:00 pm: hello\ nice stuff.I intend splitting a text into single words. can you please give me a hint how to do this? I guess there are simple programs doing this.\ Many thanks joe 11 Aug 2009 2:01 am: Nice. Do you have a page where I can try out v2 (phrases)? The Calvin and Hobbes search Takedown | s-anand.net 21 May 2010 11:53 am (pingback): [...] was able to do a lot of cool stuff with this, like statistically improbable phrases and many amusing [...]

Statistically improbable phrases 2


I noticed a striking similarity between a math problem scene in the movie Taare Zameen Par and Bill Watterson’s Spaceman Spiff strips. I found that Ishaan’s imaginative space collision to solve arithmetic mirrors Calvin’s iconic daydreaming style.

Watch this segment of Taare Zameen Par. Then check these Calvin & Hobbes strips . Bless them both -- Aamir and Bill . Comments Saurabh 30 Dec 2007 12:00 pm: Interesting! So we know who is the copy cat here :) BTW, I didn't know that you can just post a segment using Google Video..Is this something that can be done for Youtube videos as well. Chittaranjan 30 Dec 2007 12:00 pm: Yeah! That whole "Math Problem" and the Cartoonic representation of it Immediately reminded me of 'Spaceman Spiff'.... I did look within my C&H archives for this strip but couldn't find it. Thanks for this now... Tarun 6 Apr 2009 12:27 pm: yeah.... i realised it the moment the maths problem came in front of that fellow. JK 3 Dec 2009 5:00 am: God when will our people come up with our own ideas? Every movie and songs and albums have been a copy from some of the best in the Western countries. Sad on our education system that does not make us think out of box leading to such pathetic situation :( Transmogrifier 4 Aug 2010 8:00 pm (pingback): [...] Just found that the C & H ‘inspirations’ have been covered here and [...] BabyGodfather 15 Jan 2012 8:06 am: Maybe a little too late to respond here, but yup I did observe this too while watching the movie ! I just choose to say God Bless Bill, not Aamir for stealing / approving the stealing and making money out of it. Did the so called perfect and creative guy credit Bill Watterson there ? I highly doubt that. It's a direct theft. P.S. : Big fan of your site. Paul H 5 Sep 2012 9:36 pm: I remember a Looney Tunes cartoon about a boy that daydreams in class and has spaceman adventures during lessons. That must out date Calvin and Hobbes by at least a decade if not more. Paul H 5 Sep 2012 9:43 pm: The Looney Tunes cartoons are A to Z-Z-Z-Z (1953) and Boyhood Daze (1957). The daydreams of Calvin are very similar to those of the Ralph Phillips character. I was off by a few decades. No doubt Bill Watterson saw these cartoons as a child.

Taare Zameen Par and Calvin


I built a quiz featuring lyrics from 15 classic Tamil songs released before 1970. Test your knowledge of golden-age cinema by identifying the correct movie for each snippet; the input boxes turn green for correct answers.

Here are words from the middle of 15 old songs (before 1970). Can you guess which movie they are from? Don't worry about the spelling. Just spell it like it sounds, and the box will turn green. Comments Meera 14 Mar 2009 1:28 am: Its just recollection... but with joy Venkatramanan 6 Apr 2007 11:54 am: Nice! Enjoyed it banu 9 Apr 2007 12:24 pm: this is good Viju 10 Apr 2007 12:50 pm: your choice of songs is awesome Anil mattoo 11 Apr 2007 9:04 am: Great! ramesh lee 14 May 2007 3:30 am: Songs-Kannamma Kannamma brinda r 14 May 2007 6:11 am: interesting hameed 6 Apr 2007 12:00 pm: Could you please tell me who sang the song and in what movie- ''muttu pole manjal kottu pole- mulu nilavei nee piranthaai engal veetilei''. Thank you. sambasivam 6 Apr 2007 12:00 pm: enjoyable. vist look forward to more such quizes Neelaram 6 Apr 2007 12:00 pm: Nice jana 6 Apr 2007 12:00 pm: its a nice one. p.lakshmi 6 Apr 2007 12:00 pm: very nice floranz 6 Apr 2007 12:00 pm: golden old songs araNmaNai aRivaan ariyaNai aRivaan 6 Apr 2007 12:00 pm: golden old songs T.selvarajah 6 Apr 2007 12:00 pm: I dunno exately all songs are right?? thanks badhu 6 Apr 2007 12:00 pm: what about the answers? Vetty 6 Apr 2007 12:00 pm: It is so fantastic that people of all ages could enjoy listening to their favourite songs. So great is your service. You should continue like the service of the Sun TV for the Asatha Povathu Yaru. Hats off! INDIAN 6 Apr 2007 12:00 pm: ibotu arijavan kedubokirarkal ulaha film kalai patu atai atinsum arijamalum kedukitarkal itai mata manita jatijal mudijatu orvani tavara avantan j-aj ibadigu nalam ungal j ey indian krish 6 Apr 2007 12:00 pm: can be much better than the before.dont mistake. geetha 6 Apr 2007 12:00 pm: i have sent my asnwers priya 6 Apr 2007 12:00 pm: very nice P.Asha 6 Apr 2007 12:00 pm: This is nice santha k. nayar 6 Apr 2007 12:00 pm: iT IS VERY VERY GOOD. gOD BLESS YOU. MUNIPRAPAKAR 6 Apr 2007 12:00 pm: IT IS VERY NICE SONG IT TOUCH MY HEART HAVE TO LISTEN AT NIGHT TIME OR AT THE BEACH SIDE WITH FRESH AIR AND WAVE !!!!!! subbu CBE 6 Apr 2007 12:00 pm: its very nice and time passing.though i know 90%of songs but not able to find name of movie. you should provide ans also Jagan 30 Oct 2008 11:04 pm: Nice one.. I hum these song's but i dont know the name of Movies.\ \ Keep it up.. dilan 10 Dec 2008 1:08 am: Its coooooooooooooooooooool.Interesting. MUNIPRAPAKAR 11 Dec 2008 2:46 am: Not Bad keep it up is very Intersting and nice too\ YOU CAN BE ALL BY YOURSELF........UNGGAL NANBAN DHARMAPURI RAMIYAMPATTI S.NAVEEN 14 Dec 2008 11:03 pm: this is very useful for me. keerthivasan 19 Dec 2008 4:55 am: very Interesting kalai 25 Dec 2008 2:56 am: hi thulsi raman 29 Jan 2009 8:23 am: excellent!\ \ keep it up| sambasivam 15 Mar 2009 6:54 am: good show. keep it up. made 12/15 Jaya 22 Aug 2010 11:05 am: coolllll............... Arun 2 Nov 2010 3:06 am: Dooes anyone know the song "aadutha vettu sottha amukka nenachavan...." (Sung by Janakaraj in a movie in jail balaji 25 Nov 2010 12:10 pm: i've listioned all the songs but, i cant remember the name.. it's cool.. Shobhana 17 Oct 2011 12:32 pm: Got 15/15!!!! Sarora 30 Mar 2011 10:02 pm: ya a very nice experience...........

Tamil old song lyrics quiz


I built an interactive quiz featuring background music from hit Tamil movies released in 2006 and 2007. Test your knowledge by guessing the film titles based on these distinctive audio snippets.

Here is the background music from some hit songs from 2006 and 2007. Can you guess which movie they are from? Don't worry about the spelling. Just spell it like it sounds, and the box will turn green. (A couple of spellings are tricky. Try J instead of S and V instead of G .) Comments Anonymous 8 Jun 2007 12:00 pm: Error in plugin! natarajan 8 Jun 2007 12:00 pm: Ur quizzes are great.great idea. but in this quiz, when we download we get all the details about the clip including movie name which is the album title. Thrill of finding the name is lost. Sanketh 8 Jun 2007 12:00 pm: Awesome quiz ... couldn''t figure out the answers to 3,5,6,10 ... could you please post them some time. Or maybe I can just see the file names in the HTML :) Btw, glad someone else likes Atrai Thingal S Anand 8 Jun 2007 12:00 pm: Natarajan, you''re right. I''ve removed the clip details now. Thanks! ravishankar 8 Jun 2007 12:00 pm: all are easy clues..but where to fill in the answers??? Prin 8 Jun 2007 12:00 pm: Got 9! Couldn''t figure out number 8 - haunting me tho''. Awesome quiz and idea btw... Kumbakonam 8 Jun 2007 12:00 pm: I am the winner! I got 10! sirpi 8 Jun 2007 12:00 pm: can u plz give clues for 2 and 9 ragav 8 Jun 2007 12:00 pm: good quiz ...cant find number 6 :( shud be yuvan song :D anees khanem 8 Jun 2007 12:00 pm: awesome Senthilkumar.S 24 Mar 2009 7:18 am: Nice one... got 10 out of 10... Niranjan 18 Mar 2009 12:50 pm: cannot find no 6...... 9 out of 10 correct vidhya 5 Aug 2009 8:13 am: I got only five.It's very interesting.It's nice to work. supraja and ranjith 2 Sep 2009 11:56 am: very intresting.i got only 7 out of 10.could not find 1,6,9............................... Priya 11 Apr 2010 9:58 am: Can someone please tell me no 1? im really going crazy..... Swathi 17 Jul 2012 1:51 am: what is the answer for number 10? ramya 16 Dec 2011 5:31 pm: ans: 6 pattiyal,8 kalvanin kadhali, 1 pachaikili muthucharam and 9 sivapathikaram fowziya 24 Dec 2011 10:54 am: what a great quiz man!!!!!! awesome! next time add songs of latest 2011!!!!!!! its a great time pass............:-)xxxxxlove it Ranjani 7 Aug 2011 1:44 pm: 6/10 Kamala 25 Nov 2015 6:19 am: Great Quiz! Can you email me the answers please.

Tamil songs quiz 2006-2007


I built this interactive quiz featuring twenty iconic first interludes from Tamil cinema between the 1980s and 2000s. Challenge yourself to identify the movies from these background scores and enjoy a nostalgic musical journey.

Some background scores just stay in your mind. Here is a tribute to 20 wonderful first interludes, dating from the 1980s to the 2000s. Can you guess which movies they are from? (My intention here is not to make this tough, but rather to let you enjoy the music. So hope to see most of you score 20/20) Don't worry about the spelling. Just spell it like it sounds, and the box will turn green. Comments santhosh 27 Dec 2007 12:00 pm: Nowadays, more and more people are coming forward to post their ideas going inside their mind. It is really a very good thing, to bring out the thoughts to the world. Who knows, this may be very much useful for someone else.You make a Really Good Work . Keep Going . All the Best . If you find time kindly visit my site also regarding tamil mp3 songs for free download at http://www.movies-tv-songs.com/ Thanks in advance. Arun Kumar M K 27 Dec 2007 12:00 pm: This is an awesome collection and also a wonderful job at your end. Anand neengal anadmaga vazha en vazthukkal. I thought the 14th song was Baba (it was sounding like Baba undan vasalile pala kodi pengal kathirukka...) But somehow got it wrong. Which one is that? Congrats again on a fantastic job...I have become ur fan. Thanks, Arun Sanketh 27 Dec 2007 12:00 pm: The ID3 tags on the songs are a giveaway for those who don't have a QT plugin inside their browser. Had to forcibly ignore my music player's window title ... Nice collection, btw (7 actually has the main tune). Arun, 14 sounds very similar to the Baba track, but the title says "Unnai Ketena", so I am curious too as to what it is. SR 27 Dec 2007 12:00 pm: cant find 1 and 14.excellent entertainer. kiran 16 Feb 2009 4:44 am: 20/20. (14 and 19 were very tricky) sathish 19 Feb 2009 12:00 pm: 14 is from movie desam - swades in hindhi (its the same song of baba ) S Anand 20 Feb 2009 3:36 pm: Hint for 19: It's a Ramesh Vinayagam song Bhaavna 8 Jul 2009 1:20 am: Awesome anand!!! I cant get the first 3 though....most of the songs I cant remember the movie but I can guess the song!!!Nice one!!!! Sujatha 16 Sep 2009 6:20 am: Great work Anand ! Most of them were easy.Still working on No:7 V.Srinivasan 15 Jan 2010 5:48 am: Hi Very interesting. I got 19/20. No.2 is very familiar, but could not find out. Any clue? gurucharan 20 Feb 2010 4:40 am: No.7 is kinda easy!!!! Saku 16 Nov 2010 4:38 am: Good one.. had some of the tracks not been longer, i wouldn't have maxed this... thanks.. MICHAEL JACKSON(NIK) 27 Dec 2010 8:50 am: TO BE FRANK I SCORED JUST 5..................... I STARTED CRYING AFTER HEARING THE BGM..... Sowmya 24 Feb 2011 7:28 am: Could someone tell me the answer for the first track? It sounds very familiar. krishnan 19 May 2012 7:52 pm: Very different approach to induce interest in music. Kannan 24 Oct 2011 11:13 am: Awesom work. Got complete score... Please give me more... Anand 27 Apr 2012 9:33 am: 17/20, missed 7,12,18 :( Sridhar 26 May 2012 3:58 pm: Couldn't figure 2 & 14 :( Akila 12 Dec 2012 10:52 am: missed 2,11,12.. I'm breaking my head over 11... answers pls? Panimalar 17 Jan 2013 4:24 pm: pls answer for the first track..I got all the other songs. Keep up the gud job Anand..expecting more quizzes from you like this. Priya 16 Dec 2013 3:55 pm: 19/20.. Bt cud n't get 14.... please..? akash 28 Oct 2014 5:58 am: Nice work anand ji. Keep it up. Do tunes were simply mesmerising. But couldn't find 14th. Managed to get 19.... Kumar 26 Jan 2019 9:01 am: Good collection Revathi 16 Apr 2018 12:01 am: please tell what song is 1 and 14 xavier 7 May 2020 1:05 pm: i cant find 14th Yeskay 28 Mar 2020 3:50 am: How can we see our answers Hari 9 Mar 2018 4:39 am: All done except 18. But why my scores are not updated.

Tamil songs quiz - Enchanting first interludes


I built a Tamil spelling corrector based on Peter Norvig’s algorithm to fix transliterated song lyrics. I used Project Madurai as a dictionary source and explored probabilistic methods to map phonetic English spellings to correct Tamil words.

The Internet has a lot of tamil song lyrics in English. Finding them is not easy, though. Two problems. The lyrics are fragmented: there's no one site to search them. And Google doesn't help. It doesn't know that alaipaayudhe, alaipaayuthe and alaipayuthey are the same word. This is similar to the problem I faced with tamil audio. The solution, as before, is to make an index, and provide a search interface that is tolerant of English spellings of Tamil words. But I want to go a step further. Is it possible to display these lyrics in Tamil? My Tamil Transliterator does a lousy job of this. Though it's tolerant of mistakes, it's ignorant of spelling and grammer. So, kanda nal muthalai kathal peruguthadi ... becomes... kanda nal muthalai kathal peruguthadi ... when in fact we want... kanda naaL muthalaay kaathal peruguthadi (If you're viewing this on an RSS reader, check my post to see what I mean.) I need an automated Tamil spelling corrector. Reading Peter Norvig's "How to Write a Spelling Corrector" and actually having understood it, I gave spelling correction in Tamil a shot. Norvig's approach, in simple terms, is this: 1. Get a dictionary 2. Tweak the word you want to check (add a letter, delete one, swap 2 letters, etc.) 3. Pick all tweaks that get you to a valid word on the dictionary 4. Choose the most likely correction (most common correct word, adjusted for the probability of that mistake happening) Making a dictionary is easy. I just need lots of Tamil literature, and then I pick out words from it. For now, I'm just using the texts in Project Madurai. Tweaking the word to check is easy. Norvig's article has a working code example. Picking valid tweaks is easy. Just check against the dictionary. The tough part is choosing the likely correction. For each valid word, I need the probability of having made this particular error. Let's take an example. I've spelt kathal. A list of valid tweaks to this word include: kal, kol, kadal, kanal, and kaadhal. For each of these, I need to figure out how often the valid tweaks occur, and the probability that I typed kathal when I really meant one of these tweaks. This is what such a calculation would look like: | Tweak | Frequency | Probability of typing kathal | Product | | ------- | --------- | ---------------------------- | ------- | | kal | 1 | 0.04 | 0.04 | | kol | 4 | 0.02 | 0.08 | | kadal | 10 | 0.1 | 1.0 | | kanal | 1 | 0.01 | 0.01 | | kaadhal | 6 | 0.25 | 1.50 | Once we have this, we can see that kaadhal is the right one -- it has the maximum value (1.50) in the last column, where we multiply the frequency and the probability. (You probably realise how small my dictionary is, looking at the frequencies. Well, that's how big Project Madurai is. But increasing the size of a dictionary is a trivial problem.) Anyway, getting the frequency is easy. How do I get the probabilities, though? That's what I'm working on right now. Comments SDW 1 May 2007 9:01 am: Hi there, with regards to 'Thendral Ennai Mutham Itathu' as described on http://www.s-anand.net/blog/classical-ilayaraja/, the female voice was not SPS but B.S. Sasirekha. fyi please. ரவிசங்கர் 4 May 2007 5:33 pm: நல்ல முயற்சி, அணுகுமுறை. Ravi 11 May 2007 2:36 pm: Good article. I am very interested in text analysis, phrase extraction too (and statistical machine translation, if you will :) Is not probability just the number of occurences of each of these divided by total number of occurence? Maybe I am missing something. Did you also consider using Vikatan or Kumudam article data in addition to Project Madurai to get contemporary word counts and probabilities? It is one of my projects to create a word frequency list for Tamil. Now that I see another active soul, I might start the project :) Python or ruby I am wondering... kooliip 1 May 2007 12:00 pm: meaning for saroja sama nikalo GANESH 1 May 2007 12:00 pm: I LOVE YOU SUJA CHELLAM hajaroz@yahoo.com 1 May 2007 12:00 pm: please give me that spelling checking software....it is very good and it is new one v.arul 14 Aug 2008 6:39 am: Give me the correct tamil spelling for unnippaga shamu 10 Sep 2008 12:17 am: தமிழ்லில் எழுதுவதற்கு முய்ற்சி\ ந்ன்றி nita 29 Dec 2009 3:39 pm: hi i would like to know if there are any resources online which would help me to check if my spellings in tamil typing are right? i am not very confortable with tamil so i can use the help of spell check, i would be most greatful! kindly help, thanks:) prabhu 13 Aug 2011 8:43 am: Hi anand, Do you have any tamil spell checker tool online. If yes, please provide the link asap. Thanks in advance. thamizh selvan 19 Mar 2011 4:09 pm: i want to know the exact spelling of thamizh selvan in tamil . whether a connecting letter will come between thamizh and selvan or not .. thank you chandroo 19 Aug 2012 6:02 am: I need a Tamil spell checking software.Please tell me. saranraj 1 Jan 2013 9:00 am: Please let me know the correct spelling for the tamil word "Nadathanum". Muthu 29 Apr 2015 5:37 am: You may do easy text processing of Tamil content using open-tamil library in Python. $ pip install open-tamil To convert your data into Tamil letters (not Unicode code-points) you can type in Python, import tamil letters = tamil.utf8.get\letters( data ) Thanks, Muthu

Tamil spelling corrector


I explore Steve Yegge’s predictions for the next dominant programming language. While he hints at corporate backing for a new standard, community speculation strongly points toward JavaScript 2 as the likely candidate for widespread industry adoption.

Steve Yegge at Google talks about the features of the Next Big Language. He apparantly has inside information about the language corporates are likely to make a big push for. The comments seem to suggest Javascript 2.

The Next Big Language


I shared Kevin Smith's top ten films of 2006 from his blog. The list offers a glimpse into the director's tastes and prompted readers to discuss his own work, including Clerks, Dogma, and Chasing Amy.

Kevin Smith's top 10 films of 2006. Comments Dhar 4 Feb 2007 4:26 am: Anand, have you seen any of Kevin Smith's movies? Dogma is excellent, Jay and Silent Bob Strike Back, Chasing Amy, Clerks and Mallrats are pretty decent... Subir Chakma 2 Feb 2007 12:00 pm: I want to have a look on those 10 films. manoj pant 5 Nov 2008 7:02 am: thanku very much for giving me idea to select films. i m going to buy these films tomarrow

Top 10 films of 2006