I built a minimalist dashboard for InfyBLOGS following Tufte's design principles. I used a jQuery bookmarklet to scrape intranet data via IFRAMEs and visualized trends with Google Charts and sparklines, focusing on a high data-ink ratio.

I just finished Stephen Few's book on Information Dashboard Design. It talks about what's wrong with the dashboards most Business Intelligence vendors (Business Objects, Oracle, Informatica, Cognos, Hyperion, etc.), and brings Tuftian principles of chart design to dashboards. So I took a shot at designing a dashboard based on those principles, and made this dashboard for InfyBLOGS. You can try for yourself. Go to \ Note: This only works within the Infosys intranet. 1. Right click on the "Infyblog Dashboard" link and click "Add to Favourites..." (Non-IE users -- drag and drop it to your links bar) 2. If you get a security alert, say "Yes" to continue 3. Return to InfyBLOGS, make sure you're logged in (that's important) and click on the "Infyblog Dashboard" bookmark 4. You'll see a dashboard for your account, with comments and statistics The rest of this article discusses design principles and the technology behind the implementation. (It's long. Skim by reading just the bold headlines.) Dashboards are minimalist I'll use the design of the dashboard to highlight some of the concepts in the book. I designed the dashboard first on Powerpoint, keeping these principles in mind. 1. Fits in one screen. No scrolling. Otherwise, it isn't a dashboard. 2. Shows only what I need to see. Ideally, from this dashboard, I should receive all the information I need to act on, and no more. 3. Minimises the data-ink ratio. Don't draw a single pixel that's not required. The first was easy. I just constrained myself to one page of PowerPoint, though if you had a lot of toolbars, the viewing area of your browser may be less than mine. The second is a matter of picking what you want to see. For me, these are the things I look for when I log into InfyBLOGS: 1. Any new comments? 2. Any new posts from my friends? 3. What's new? What's hot? Then I dig deeper, occasionally, into popular entries, popular referrers, how fast the blogs are growing, etc. So I've put in what I think are the useful things. The third is reflected in the way some of this information is shown. Let me explain. Keep the charts bare Consider the graphs on the right. They look like this. Notice the wiggly line to the right. It's a graph called sparkline, and was introduced by Edward Tufte. Sparklines are great to show trends in a compact way. Forget the axes. Forget the axes labels. Forget the gridlines. The text on the left ("visitors per day") tells you what it is. The number (10475) is the current value. And the line is the trend. Clearly the number of visitors has exploded recently, from a relatively slow and flat start. The labels and axes aren't going to tell you much more. Boldly highlight what's important The most important thing here, the title, is on top. It's pure black, in large font, positioned right on top, and has a line segregating it from the rest of the page. The sections are highlighted by a bigger font, different colour, and a line, but the effect is a bit more muted. The numbers on the right are prominent only by virtue of size and position. If anything, the colour de-emphasizes them. This is to make sure that they don't overwhelm your attention. (They would if they were in red, for instance.) The number 10475 is carefully set to occupy exactly two line spaces. That alignment is very important. The small lines are at a font size of 11px, and line spacing is 1.5. So at a font size of 2 x 11px x 1.5 = 33px, the size of the large number covers exactly two rows. The labels, such as "visitors" or "sites" are in blue, bold. Nothing too out of the way, but visible enough that they stand out. The "View more..." links just use colour to stand out. They're pretty unimportant. The bulk of the text is actually made of links, unlike traditional links, they're not underlined and they're not blue. It would just add to the noise if everything where underlined. But on mouseover, they turn blue and are underlined, clearly indicating that they're links. I've used four mechanisms to highlight relative importance: position, size, colour and lines. There are many more: font, styling, boxes, underlining, indentation, etc. The purpose of good design is to draw attention to what's important, to direct the flow of the eye in a way that builds a good narrative. Don't be shy of using every tool at your disposal in doing that. While on the topic, the Non-Designer's Design Book is a fantastic and readable book on design for engineers. Always use grids to display I can't say this any better than Mark Boulton of subtraction.com, in his presentation Grids are Good. Grids are pervasive in every form of good design. It's a fantastic starting point as well. Just read the slideshow and follow it blindly. You can't go wrong. This dashboard uses 12-grid. The page is divided into two vertically. The left half has mostly text and the right half has statistics and help. There are 3 blocks within each, and they're all the same size (alignment is critical in good design). Two of the blocks on the left are subdivided into halves, while the bottom right "Links and help" section is subdivided into three. Well, it's easier to show it than explain it: Picture of grid Copy shamelessly The design for this dashboard was copied in part from Wordpress 2.7's new dashboard, part from the dashboards on Stephen Few's book, and part from the winners of the 2008 Excel dashboard competition. Most of these designs are minimalist. There's no extra graphics, jazzy logos, etc. that detract from the informational content. This is an informational dashboard, not a painting. Good design is everywhere. View this presentation on How to be a Creative Sponge to get a sense of the many sources where you can draw inspiration from. You're much much better of copying good design than inventing bad ones. You can hack the dashboard yourself I've uploaded the source for the dashboard at http://infyblog-dashboard.googlecode.com/ Please feel free to browse through it, but don't stop there! Go ahead and tweak it to suit what you think should be on an ideal dashboard. I'll give access to the project to anyone who asks (leave a comment, mail me or call me at +44 7957 440 260). Please hack at it. Besides, it's great fun learning jQuery and working on a distributed open source project. Now for some notes on how this thing works. Bookmarklets bring Greasemonkey to any browser This is implemented as a bookmarklet (a bookmark written in Javascript). It just loads the file which does all the grunt work. This is the code for the bookmarklet. This form of the bookmarklet is perhaps it's most powerful use. It lets you inject any script into any site. If you want to change it's appearance, content, anything, just write it as a bookmarklet. (Think of it as the IE and Safari equivalent of Firefox's Greasemonkey.) So if you wanted to load jQuery into any page: 1. Change the URL in line 5 above to the jQuery URL and add it as a bookmark. (Or use Karl's jQuery bookmarklet. It's better) 2. Go to any page, say the IMDB Top 250 for instance 3. Click on the bookmarklet. Now your page is jQueryified. (If you used Karl's bookmarklet, it'll tell you so.) 4. On the address bar, type "javascript:alert($('table:eq(10) tr').length)" and press enter. 5. You should see 252, the number of rows on main table of the IMDB Top 250 Notes: There are other ways of adding Greasemonkey functionality to your browser There are many bookmarklet directories where you can find bookmarklets The dashboard is better written as a server-side app, but I don't own any servers in our Intranet jQuery is a powerful Javascript library Firstly, you don't want to program Javascript without a library. You just don't. The state of browser incompatibilities and DOM usability is just to pathetic to contemplate. Stop learning DOM methods. Pick any library instead. Going by popularity, you would do well to pick jQuery. Here's a graph of searches on Google for the popular Javascript libraries, and jQuery leads the pack. https://trends.google.com/trends/explore?date=all&q=jquery,prototype,dojo,ext,yui I picked it in early 2007, and my reasons were that it: 1. Is small. At that time, it was 19KB and was the smallest of the libraries. (Today it's 18KB and has more features.) 2. Makes your code compact. It lets you chain functions, and overloads the $() function to work with DOM elements, HTML strings, arrays, objects, anything. 3. Doesn't pollute the namespace. Just 2 global variables: jQuery and $. And you can make it not use $ if you like. 4. Is very intuitive. You can learn it in an hour, and the documention is extremely helpful. 5. Is fully functional. Apart from DOM manipulation, it covers CSS, animations and AJAX, which is all I want from a library. These days, I have some additional reasons: 1. It's extensible. The plugin ecosystem is rich. 2. It's hosted at Google, and is really fast to load. Most often, it's already cached in your users' browser. 3. John Resig is a genius. After env.js and processing.js, I trust him to advance jQuery better than other libraries. So anyway, the first thing I do in the dashboard script is to load jQuery, using the same code outlined in the bookmarklet section above. The only addition is: This tells the browser to run the script "callback" once the script is loaded. My main dashboard function runs once jQuery is loaded. InfyBLOGS has most statistics the dashboard needs Thanks to posts from Utkarsh and Simon, I knew where to find most of the information for the dashboard: Comments page for your latest comments Profile page for your statistics: number of entries, interests, friends, etc. Directory page for recently updated blogs Communities page for recently updated communities Usage page for Infyblog statistics and popular referrers Meme page for recently popular links The only things I couldn't find were: Friends posts. I'd have liked to show the titles of recent posts by friends. Yeah, I know: blogs/user\name/friends has it, but thanks to user styles, it's nearly impossible to parse in a standard way across users. I'd really like an RSS feed for friends. Interests posts. Would be cool to show recent posts by users who shared your your interest. Communities posts. Again, I'd have liked to show the recent posts in communities, rather than just the names of communities. Using IFRAMEs, we can load these statistics onto any page Let's say we want the latest 10 comments. These are available on the comments page. To load data from another page, we'd normally use XMLHTTPRequest, get the data, and parse it — perhaps using regular expressions. But from a readability and maintainability perspective, regular expressions suck. A cleaner way is to use jQuery itself to parse the HTML. This works very well for simple pages, but sadly, for our statistics, this throws a stack overflow error (at least on Firefox; I didn't have the guts to try it on IE.) So on to a third approach. Why bother using Javascript to parse HTML, when we're running the whole application in a browser, the most perfect HTML parser? Using IFRAMEs, we can load the whole page within the same page, let the browser do the HTML parsing. This way, you can read any page from Javascript within the same domain. Since both the bookmarklet and statistics are on the InfyBLOGS domain, we're fine. jQuery parses these statistics rather well Now the doc variable contains the entire comments page, and we can start process it. For example, the comments are on the rows of the second table in the page. (The first table is the navigation on the left.). So gets the rows in the second table (table:eq(0) is the first table). Now, we can go through each row, extract the user, when the comment was made, links to the entry and to the comment. Google Charts API displays dynamic charts Another cool thing is that we can use Google Charts to display charts without having to create an image beforehand. For instance, the URL: http://quickchart.io/chart?cht=lc&chs=300x200&chd=t:5,10,20,40,80 shows an image with a line chart through the values (5,10,20,40,80). Google chart example The dashboard uses sparklines to plot the trends in InfyBLOG statistics. These are supported by Google Charts. We can extract the data form the usage page and create a new image using Google Charts that contains the sparkline. If you want to play around with Google Charts without mucking around with the URL structure, I have a relatively easy to use chart constructor. Always use CSS frameworks Libraries aren't restricted to Javascript. CSS frameworks exist, and as with Javascript libraries, these are a no-brainer. It's not worth hand-coding CSS from scratch. Just build on top of one of these. The state of CSS frameworks isn't as advanced as in Javascript. Yahoo has it's UI grids, Blueprint's pretty popular and I found Tripoli to be nice, but the one I'm using here is 960.gs. It lets you create a 12-grid and 16-grid on a 960-pixel wide frame, which is good enough for my purposes. At nearly 2,500 words, this is the longest post I've ever written. It took a day to design and code the dashboard, but longer to write this post. Like Paul Buchheit says, it's easier to communicate with code. I've also violated my principle of Less is More. My apologies. But to re-iterate: Please hack it Download the code and data. Comments sujays 26 Jan 2009 7:14 am: am an x-infy guy and my id(sujaychandiran\s) still works in ur "suggest friends" feature.. does this connect 2 infy DB?.. if yes, doesnt this pose a security threat ? am jus curious !\ \ me cant hack, so cant confirm :)\ \ Sujay S Anand 26 Jan 2009 9:23 am: @Sujay: nah, it's not a security risk. It's just a static dump of data, and\ is a bit out-dated in any case. If anything, it may be a privacy risk:\ recruiters might try to mine for interests and pick people that way. But in\ this economy, I guess no one's worried about that. gwatts 26 Jan 2009 6:52 pm: If you like jQuery, and you like sparklines then you can combine the two and avoid the browser roundtrips to fetch externally rendered images:\ http://omnipotent.net/jquery.sparkline/ Thejesh GN 26 Jan 2009 10:33 pm: Very cool.\ I guess we can add buttons for add/edit posts. S Anand 31 Jan 2009 5:59 am: That's a great plugin, Gareth! Look forward to using it. Thanks.

Infyblogs dashboard


I found that inline form validation can reduce completion time by 40%. Key takeaways include validating on blur rather than keypress and focusing on uncertain fields like username availability instead of common personal information.

A List Apart's article on Inline Validation is one of the most informative I've read in a while — and it's backed by solid data. Some useful lessons: Inline validation can reduce form completion time by 40% Use inline validations where the user doesn't know if they'll get it wrong (e.g. is a username available?). Don't use them if user knows the answer (e.g. their name) Validate on blur, not on keypress (it's distracting, and users can't multitask) Comments jesse 25 Sep 2009 4:15 pm: maybe u should add some inline validation on your comments form, instead of the wordpress error page?

Inline form validation


I explain how to build animated motion charts in Excel by combining bubble charts with scroll bars and a simple VBA "play" script. This method allows you to visualize four dimensions of data over time.

Creating motion charts in Excel is a simple four-step process. Get the data in a tabular format with the columns [date, item, x, y, size] Make a “today” cell, and create a lookup table for “today” Make a bubble chart with that lookup table Add a scroll bar and a play button linked to the “today” cell For the impatient, here’s a motion chart spreadsheet that you can tailor to your needs. For the patient and the puzzled, here’s a quick introduction to bubble and motion charts. What is a bubble chart? A bubble chart is a way of capturing 3 dimensions. For example, the chart below could be the birth, literacy rate and population of countries (X-axis, Y-axis and size). Or the growth, margin and market cap of companies. It lets you compare three dimensions at a glance. The size dimension is a different from the X and Y axes, though. It’s not easy to compare differences in size. And the eye tends to focus on the big objects. So usually, size is used highlight important things, and the X and Y axes used to measure the performance of these things. If I were to summarise bubble charts in a sentence, it would be: bubble charts show the performance of important things (in two dimensions) . (In contrast, Variwide charts show the same on one dimension.) Say you’re a services firm. You want to track the productivity of your most expensive groups (“the important things”). Productivity is measured by 2 parameters: utilisation and margin . The bubble chart would then have the expense of each group as the size, and its utilisation and contribution as the X and Y axes. What is a motion chart? Motion charts are animated bubble charts. They track the performance of important things over time (in two dimensions) . This is chart with 4 dimensions. But not all data with 4 dimensions can be plotted as a motion chart. One dimension has to be time, and another has to be linked to the importance of the item. Motion charts were pioneered by Hans Rosling and his TED Talk shows you the true power of motion charts. How do I create these charts? Use the Motion Chart Gadget to display any of your data on a web page. Or use Google Spreadsheets if you need to see the chart on a spreadsheet: motion charts are built in. If you or your viewer don’t have access to these, and you want to use Excel, here’s how. 1. Get the data in a tabular format Get the data in the format below. You need the X, Y and size for each thing , for each date . Date Thing X Y Size 08/02/2009 A 64% 11% 1 08/02/2009 B 14% 33% 2 08/02/2009 C 78% 55% 3 08/02/2009 D 57% 73% 4 08/02/2009 E 39% 32% 5 08/02/2009 F 40% 81% 6 09/02/2009 A 64% 12% 1 09/02/2009 B 14% 33% 2 09/02/2009 C 78% 56% 3 09/02/2009 D 57% 73% 4 09/02/2009 E 39% 32% 5 09/02/2009 F 40% 81% 6 … … … … .. To make life (and lookups) easier, add a column called “Key” which concatenates the date and the things. Typing “=A2&B2” will concatenate cells A2 and B2. ( Red cells use formulas.) Date Thing Key X Y Size 08/02/2009 A 39852A 64% 11% 1 08/02/2009 B 39852B 14% 33% 2 08/02/2009 C 39852C 78% 55% 3 08/02/2009 D 39852D 57% 73% 4 … … … … … … 2. Make a “today” cell, and create a lookup table for “today” Create a cell called “Offset” and type in 0 as its value. Add another cell called Today whose value is the start date (08/02/2009 in this case) plus the offset (0 in this case) Offset 0 (Just type 0) Today 08/02/2009 Use a formula: =STARTDATE + OFFSET Now, if you change the offset from 0 to 1, “Today” changes to 09/02/2009. By changing just this one cell, we can create a table that holds the bubble chart details for that day, like below. Thing X Y Size Formula A 44% 19% 1 X =VLOOKUP(TODAY & THING, DATA, 2, 0) Y =VLOOKUP(TODAY & THING, DATA, 3, 0) Size =VLOOKUP(TODAY & THING, DATA, 4, 0) B 6% 13% 2 C 90% 71% 3 D 41% 61% 4 E 59% 40% 5 F 16% 77% 6 Check out my motion chart spreadsheet to see how these are constructed. 3. Make a bubble chart with that lookup table This is a simple Insert – Chart. Go through the chart types and select bubble. Play around with the data selection until you get the X, Y and Size columns right. 4. Add a scroll bar and a play button linked to the “today” cell Now for the magic. Add a scroll bar below the chart. Excel 2007 users: Go to Developer – Insert and add a scroll bar. Excel 2003 users: Go to View - Toolbars - Control Toolbox and add a scroll bar Right click on the scroll bar, go to Format Control… and link the scroll bar to the “Offset” cell. Now, as you move the scroll bar, the value in the offset cell will change to reflect it. So the “today” cell will change too. So will the lookup table. And so will the chart. Next, create a button called “Play” and edit its code. Excel 2007 users: Right click the button, go to Developer – View Code. Excel 2003 users: Right click the button and select View Code. Type in the following code for the button’s click event: Now clicking on the Play button will give you this glorious motion chart in Excel: Comments Vinoth 13 Mar 2009 9:27 am: Hi Anand.Very useful article. I can use similar motion chart to showcase how each of my team member's cumulative performance is contributing at the overall level. Zoheb 21 Mar 2009 12:02 am: also could you rewrite the whole code,on how to slow it down.. i tried inserting it.but i don't think i put it in the right place. David 18 Mar 2009 2:16 pm: Thanks! That did the trick! Great Blog! David 17 Mar 2009 7:56 pm: Is there a way to slow it down? S Anand 17 Mar 2009 8:44 pm: Sure. Use the sleep function. Add this to the module: \ \ Then add a Sleep function after Application.Calculate \ Nach 17 Mar 2009 8:50 am: Thank you...this is something that I have been trying to do for a long time now...but didnt know how to write the looped macro in Excel. Zoheb 20 Mar 2009 9:52 pm: Hi I had a question, if I wanted the counter by 3, how would i do that? Thanks Zoheb S Anand 23 Mar 2009 7:39 am: I've changed the code and also revised it in the Excel file. Take a look. To increment the counter by 3, change the properties of the scroll bar to make the increment 3 instead of 1. (Right click on the scroll bar and select "Properties") Al-Hamour 23 Mar 2009 1:50 am: Great post! Thanks for sharing. Time delay code is very useful if you can upload it.. right now it goes too fast to see. Also, is there a way to make the graph trace each of the moving circles? Venkat 1 Apr 2009 3:12 pm: Thanks Anand. Very useful to represent often boring data like time series analysis. Firman 17 Apr 2009 3:53 am: Thank you so much... i hope in the near future you can give us tutorial / article about how to make it more interactive... say... after we click the bubble we can divide it into smaller / more detailed parts. Thank you Reginald Vaz 17 Apr 2009 7:08 pm: Extremely useful...thanks a lot for sharing Pradeep Jindal 2 May 2009 6:57 pm: Fabulous! I just starting to do it and find this article... everything redy and beautiful... can we add drill down to it Thank you Anand :) Mike 12 May 2009 3:43 pm: Great Tool. I am also looking for a way to pause the animation. Any ideas? Pete 4 Jun 2009 1:30 am: Is there a way to embed the animation in excel? S Anand 14 Jul 2009 5:25 pm: Don't think we can export these motion charts into Powerpoint, I'm afraid. Sunny 13 Jul 2009 9:38 pm: Is there any way I can export these motion charts into microsoft powerpoint? beth 15 Jul 2009 1:50 am: do you know how you can embed this into a powerpoint? when i try to embed, you can't do the motion. Sabnish 11 Jul 2009 11:26 am: Hi...great help...thanks a lot....but for my presentation i will need to see the evolution by years...in this example it is the date which gets incremented...how to increase the year instead...thanks...hope you can help me.... Michael Brault 8 Dec 2009 3:11 pm: I have the motion chart embed into a Dashboard with other charts and tables. The macro increments all tables. Any ida on how to limit the macro to a single table of data? Alex 22 Jan 2010 10:57 pm: Having problems creating it in Excel for mac. Any ideas? derek 14 May 2010 7:17 pm: Another kind of motion chart can be a line or bar chart showing the evolution of multiple line shapes or histograms with time. You can also add a time display to show when the events are occurring. Rosling's application cleverly displays the year as a giant watermark behind the bubbles, so the eye doesn't have to stray to a corner to see the time. I've often thought a clock face symbol might be a neat alternative to a simple text display. Jeff Koenig 5 Feb 2010 7:33 pm: Great bubble chart solution. I've been looking for a 3D bubble chart so I can move bubbles in 3D space, allowing me to track an additional dimension. Any ideas? Dirk Cornette 13 Apr 2010 10:23 pm: Great solution Anand. I am trying to customize it, showing 1 customer per buble, size = revenue, Y-axes = number of our staff working for that customer, X axes showing margin %. Solving two issues would make it more gapminder-like. 1. How to add a data label to the buble, showing the customer name ? 2. How to show the year ? Any ideas ? Mahdi Meskin 4 Jun 2010 6:24 am: Great post! thanks. Drew 11 May 2010 4:10 pm: Just found your post, and I can't wait to try it out. Thanks for the tips! NIall Tallon 18 Aug 2010 12:08 pm: HI Anand, This is great - a while since I updated and wrote macros - I am running a 64bit version of windows 7 and excel 2010 - and the macro won't run there - any ideas? Thanks, Niall Niall Tallon 18 Aug 2010 12:19 pm: HI Again, Actually just have to add "PTRsafe" in the macro near the declare statement. Pretty easy.......Email me if you want a copy with the changed macro. Best regfards and with thanks, NT Kamesh.M 15 Sep 2010 10:00 am: Great Post! This can be done without Macros also..... http://www.webanalyticsindia.com/2009-11-20/motion-chart-in-excel/ Best Regards Kamesh Pvl 27 Jun 2012 8:41 am: Hi, Anand, Thanks for good tutorial. But I'm very interested in creating such motion chart in Flash, like You posted as an example. Any suggestions how to do that? Scott 20 Dec 2010 4:31 pm: Hi, Is it possible to make this so the individual bubbles have different colors or at least unique labels? I can only seem to have all the bubbles be one color and one label 'Thing'.... Thanks, Scott James Arnott 8 Sep 2011 2:56 pm: Hi - I've come late to this but am finding it very useful. Thanks for sharing your work with us as it would have taken me ages to work it out myself. Anna 2 Nov 2011 3:52 pm: Does anyone know if it is possible to include this chart in a Share Point site? I know you can upload dashboards with the REST API but I don't know if it is possible to add motion charts. Can you shed some light on this topic. Thanks! Atisha Banjare 11 Nov 2011 11:06 am: Hi!! Great stuff and really useful. Thanks kalaisam 23 Jan 2012 11:37 am: Hai, I read Excel file from my Google account using Zend. I call Motion chart using Publish gadget script. It automatically assign the default X-axis and Y-axis .I wish to assign the X- axis and Y-axis . I choose Advanced settings option to change the Axis and then call that script string in Php code. But Default axis only displayed ! How can i fix that ? Bill 16 Apr 2012 5:41 pm: Is it possible to have bubble diameters also show thier diameter change over time? Various smoothing algorithms could be offered to choose from. Nal 23 Mar 2012 6:48 am: hi, this is fantastic! Is it possible to put this into my powerpoint presentation? thanks! Sreekar 3 Sep 2011 9:19 pm: Hi Anand - I tried using your macro and it gives a compilation error "Sub or Function not defined"; would you know how I could fix it? I am using 2010 Excel Alan 15 Sep 2011 1:01 am: Can you please post me the updated version for Excel 2010. I dont know how to update the declare statement to include "PTR Safe" Thank you Alan Chris 28 Mar 2012 11:08 pm: Wonderful! Thanks for all this work. I use Excel 2010 and it shouldn't be too hard to adapt it (I won't bother you about it). It's too bad it can't be embedded in PPT. Microsoft definitely needs to add this kind of thing to PowerPoint. Kirk 23 Jun 2012 4:57 pm: I tried this macro but got an error saying the macros in this project are disabled. Anyone have an idea on what I am doing wrong? Sub Button20\Click() Declare Sub Sleep Lib "kernel32" (ByVal dwMilliseconds As Long) Sub Button1\Click() Dim i As Integer For i = 0 To 40: Range("J1").Value = i Application.Calculate Sleep (100) Next End Sub CK 6 May 2012 10:18 am: Thank you for this useful tutorial. How did you embed the motion chart moving on your web page please? Gary 16 Jul 2012 2:37 am: Oh, sorry to mention that it stops at the Sleep (100). I even tried to add the Declare but for some odd reason it keep advancing above where I type it. I appreciate any help. Thank you Gary 16 Jul 2012 2:34 am: First let me say thank you. The code you posted did not work for me. I am using MS Excel 2010 and I keep getting the same error. Microsoft Visual Basic for Applications Compile error: Sub or Function not defined. Can you help me fix this error. This is what I have typed. Sub Button1\Click() Dim i As Integer For i = 1 to 39: Range ("C1").Value = 1 Application.Calculate Sleep (100) Next End Sub Seymon 15 Jul 2012 2:17 am: it is awesome, thanks Merry Turnip 9 Oct 2012 12:21 pm: THANK YOU SO MUCH, FOR YOUR TUTORIAL, I HAVE TRIED IT AND IT WORKS. THANK YOU SO MUCH. By the way, when I change Years to months, however, it doesn't work. Could you please teach me how to change year to months? THANK AGAIN REGARDS Mukesh 5 Sep 2012 11:24 pm: Hi Anand, I need to show the data for only 4 years. Could you please tell me how to slow down the motion further ? I tried increasing the value of the number in Sleep (), but its not making a difference maanu 19 Nov 2012 10:34 am: hi anand, i want to build a bubble chart for my applications globally with four variables, application investment strategy meaning invest, maintain or disinvest, cost, location, and functional match. i want to handle location with color as there are only 8 locations, and cost with size. i want app investment strategy as numbers (1,2 ,3) on y axis and functional match with five categories (0-20 21-40,41-60,61-80,81-100) on x axis. can we use your technique to achieve this ? Obed 18 Feb 2013 8:02 am: Thanks for sharing your wealth of knowledge with others. Great stuff!!! Michael 24 Feb 2013 5:31 am: In their infinite wisdom Google is discontinuing motion chart gadgets in 2013 so a big thank you for this alternative:-) Ganeshan Nadarajan 16 Jan 2013 12:27 pm: Motion charts are animated bubble charts. They track the performance of important things over time (in two dimensions). This is chart with 4 dimensions. But not all data with 4 dimensions can be plotted as a motion chart. One dimension has to be time, and another has to be linked to the importance of the item. isspenguin 17 Jan 2013 5:12 pm: Hello - How do I create "trails" that trace the path of individual bubbles in Excel, like they do in Google motion charts? Please feel free to email me your response at rdivya09@gmail.com. Thanks! Stefan Selby 25 Nov 2013 8:07 pm: It is interesting to see how you did your motion charts. I have created an app that does more than just bubble charts. I have done the same for most excel charts and a speedometer chart. It is shared with the world have a look at:https://googledrive.com/host/0B0Ms4sM4a2RoSW9YYmkxbVlMc2M/vbamotioncharts.html sireesh 11 Jul 2013 10:17 pm: how to transfer bubble chart to ppt? Mike 26 Jul 2013 11:02 pm: Does the code work with 2010? Nothing happens when I copy and paste it in. When I run debug the sleep (100) highlights even after I replace the data as you indicated with my own. Granted, I'm new to developer stuff so I may be making a silly mistake. Max 19 Jul 2013 6:50 pm: Thanks for the great write up. Any ideas as to how one could add a "tail" behind the moving bubble to track the past data? Diana 9 Nov 2013 4:08 am: Actually.....shows how hopeless I actually am....I am in Excel 2013, not 2010! Diana 9 Nov 2013 4:05 am: I'm also asking the same question as Mike #5. I can't get the code to work in 2010 and the sleep (100) highlights. As I'm not a programmer and really don't have a clue, I'd love to get this handled. I've come this far and have great individual graphs!. Thanks.... Loveleen 8 Apr 2014 3:57 am: I can’t get the code to work in 2010 and the sleep (100) highlights. As I’m not a programmer to understand the coding. COuld you please how it works in 2010 regards, Loveleen Ye 13 Mar 2015 6:28 pm: Thank you so much for this workaround (I usually did this with googledocs) in excel. I am trying to make this work - not based on increasing date, but on temperature. Imagine a chemistry-based excel sheet and I want to show how different components change volume, weight and surface opaqueness with rising temperature. Right now, I struggle to make the today and offset work for me since the example is (like google docs) based on a start date. Thanks for your reply in advance. Y Durga 18 Jun 2014 4:11 pm: This was very useful for me. Thank you for describing the play button logic. On the web site when I click on the pause or the stop button the chart stops at that moment but I didn't see it in the excel. Could you please tell me how to code the stop button or the pause button on excel. Nathalie 11 Jun 2014 5:06 pm: It is very interesting. However, there is one little problem that stops me from using this kind of chart: the size of the bubbles does not follow the variation of size values in time. For instance, if bubble A remains the biggest bubble over the time, it will keep the same size in the chart, even if its value has increased by a factor 3 one year. I would have expected the size to be 3 times bigger than in the previous year. Is there a way to remedy this (without having to use Google motion charts)? Stef 30 Dec 2014 12:46 pm: I have created an app that creates these and other types of motion charts. You can find it at http://developingdata.azurewebsites.net/Excel/ExcelMotionCharts Knowing excel and reading this article would help (good article) but I have tried to make the app easy to use. Des Klass 1 Jan 2015 1:01 am: Hi I am having the same problem as Mike(#5) and just can't get the motion working. Greatful for any suggestions(I am using my own data) Atom 18 Dec 2014 8:07 am: How do i change the years to seconds? Ryan 14 Aug 2015 7:42 pm: I'm also having the same Sleep (100) error in 2010. Is there a work around for this? Niall 22 Mar 2016 5:07 pm: HI There - really love the motion chart work here - I've been using it for a while - but I recently went to O365 and the graphs I made don't work anymore - any ideas? Many thanks, Niall Rochny 15 Feb 2019 12:24 pm: I can’t get the code to work in 2016 and the sleep (100) highlights. Can get help, as i am not a programmer. regards Rochny JEP 3 Apr 2017 9:26 am: The code work with 2010 if you delete "sleep (100)" :-) `

Motion charts in Excel


I released my website content and code under the WTFPL because default copyright is too restrictive. Even Creative Commons Attribution felt like a constraint; I’d rather have my work used and modified freely without requiring any credit or permission.

I don’t have any copyright declaration on this website. The problem with that is: content is copyrighted by default. As Jeff Atwood indicates, this means that people with experience in such matters won’t copy the content because they have no legal right to use it. Let me clarify: I don’t care what you do with my content. Feel free. You don’t have to ask. You don’t have to attribute it to me. You can change it. You can misquote me. Whatever. I tried to find a Creative Commons license that suits my purposes. Of their licenses, the most liberal is the Creative Commons Attribution license. This says you can do what you want as long as you attribute my content to me. But that creates a constraint. And if I had a choice, I’d rather have my content quoted than be attributed. The license that best captures this is the WTFPL, or Do What The Fuck You Want To Public License. So, in the spirit of a happy and open Internet, the contents and code in this site is released under the WTFPL. Do what you want with it. Comments Prasenjeet 4 Mar 2009 12:37 pm: +1 for Yahoo Pipes. Or write/cajole someone into writing a Python script. With a library like BeautifulSoup, screen-scraping HTML has never been easier. Rishi 4 Mar 2009 4:13 am: Though a way to do it via the PC would be to run a VB macro that imports the content and then publishes it. S Anand 4 Mar 2009 7:19 am: Use Yahoo Query Language (http://developer.yahoo.com/yql/console/) or Yahoo Pipes (http://pipes.yahoo.com/). You'll be able to create an RSS feed out of any content. Rishi 4 Mar 2009 4:07 am: Hey Anand, I know you have written "dont mail me, call me", but rather than waste your time and an ISD call on a triviality, let me take the chance of putting it as a comment. Problem: Webiste (www.orbat.com) does not have an RSS feed, and the editor is not really tech savvy enough to screw around with the code to put one. How can an external/unrelated party copy/paste the blog entries whenever it is updated (say to a blogspot account). This, independent of any code running on a PC - rather is there a way to make a hack that will do this - online? (i.e. some piece of code that will trigger a compare string (string = first 10 words of old blog post say) , and copy/ paste on blogspot till old string found, and then re-initialize the string to the new post. Something that does this check maybe once a day? Rishi IITB, IIMK S Anand 5 Mar 2009 7:46 pm: It isn't, of course. Why, did someone mention elite institutes when talking about copyright? ;-) Nah, I just have it on my front page so people can find me on Google. somnath 5 Mar 2009 11:15 pm: This is for Mihir about the elite institutes My guess you don't know Anand enough ... he happens to walk through the doors of those elite institutes at some point of time in life when we have struggled a lot :) Mihir Modi 5 Mar 2009 7:20 pm: Slightly off-topic, but why is it relevant for someone to specify which elite institutes they belong to when they're talking about copyright :P (Plain curiousity. No offense meant!) Mihir Modi 6 Mar 2009 2:30 pm: Oh no no... Anand, this is your site, you can talk about yourself as much as you want! :D I was referring to the comment by Rishi right up there! (Gosh! NOW I'm getting personal!)

No copyright


I taught networking at my old school and found that while students are online constantly, very few create content or code. I reflect on how rigid curricula stifle creativity and why we should volunteer to teach.

This vacation, I took a session each for class XI and XII at my school, Vidya Mandir . The subject was Computer Science (the only one I can teach with some confidence), and the topic was networks. It was an experiment, in two parts. The first was to understand how students of this generation interact with the Internet. (I'm twice as old as them, so I guess they qualify as the next generation.) The second was to see whether I'd leave them far behind, or they'd leave me far behind. I began the class with a series of questions. How many of you have... Expected Actual Access to a PC and the Internet (home or nearby). I was expecting 80%. Every single one of them raised their hands. Every single one. 80% 100% Chatted online. I was expecting 70%. Every single one, except for one girl, raised their hands. 70% 100% Used a bluetooth device. I was expecting around 60%. I got nearly everyone, but the remaining were wondering what that was. 60% 100% Video-chatted. I expected 50%. Got 80% 50% 80% Uploaded a photo or video. Again, far more than expected. 40% 80% Own a blog or website. This is where the surprises started. I thought that at least one in 3 would have a blog. Turns out I was wrong. There were very few. 30% 5% Written a web application. Not one soul. Some thought they had, but no... 10% 0% Contributed to an open source project. None at all. 1 or 2 0% It was an eye-opener. On the one hand, everyone has an Internet connection. (In fact, the announcements following the morning prayer began with the Principal warning about the dangers of chatting with strangers online.) On the other hand, they're doing little of the cool stuff. Some of the discussions I had after class did lessen my concern a bit. There are, as always, a few that are very interesting in hacking, and are playing around with a lot of interesting things. But still, on average... As for the other part of the experiment, I spent an hour talking about what goes on behind the scenes when they search on Google, taking them down to some of the elements of HTTP. My slides are below. I do suspect I left a fair number of them behind, but there were a handful that were with me right up to the end. Computer Networks: An Introduction View SlideShare presentation or Upload your own. (tags: http ) But I learned something that I did not expect. I spent a lot of time at the staff room, and talking with the teachers. The best way I can summarise what I learnt is through this Calvin and Hobbes strip. Somehow, I thought the bulk of the discussion at the staff room would centre around students. Or, at the very least, around education. It was eye-opening to listen to a two-hour-long argument on the political reasons behind the tea at primary school staff room being better than at high school's. I remember my first book on acting defining a modern-day magician as "an actor who plays the role of a magician". The modern-day teacher is, in similar vein, an employee assigned role of a teacher. Teaching is their profession, not passion. Not that they are disinterested, quite the opposite. But oh, it could be so much better! I read a speech by John Taylor Gatto titled " The Six-Lesson Schoolteacher ". He gave this speech on being awarded the New York State Teacher of the Year award in 1991. He teaches six lessons at school, he says. The first lesson I teach is: "Stay in the class where you belong." I don't know who decides that my kids belong there but that's not my business. The second lesson I teach kids is to turn on and off like a light switch. I demand that they become totally involved in my lessons... But when the bell rings I insist that they drop the work at once and proceed quickly to the next work station. Nothing important is ever finished in my class, nor in any other class I know of. The third lesson I teach you is to surrender your will to a predestined chain of command... As a schoolteacher I intervene in many personal decisions, issuing a Pass for those I deem legitimate, or initiating a disciplinary confrontation for behavior that threatens my control. The fourth lesson I teach is that only I determine what curriculum you will study.... Of the millions of things of value to learn, I decide what few we have time for. Curiosity has no important place in my work, only conformity. In lesson five I teach that your self-respect should depend on an observer's measure of your worth... A monthly report, impressive in its precision, is sent into students' homes to spread approval or to mark exactly -- down to a single percentage point -- how dissatisfied with their children parents should be. In lesson six I teach children that they are being watched. I keep each student under constant surveillance and so do my colleagues... Students are encouraged to tattle on each other, even to tattle on their parents. Of course I encourage parents to file their own child's waywardness, too. I smiled a bit when I read this. It had been a while since I'd been in school, and I was lucky to have been in very liberal colleges. But then I went back to school and saw it for myself. The organisation that comes closest to the school is the military... or the prison. Not exactly the best place to foster creativity. I began my class this time by saying, "Look, I might be wrong in what I tell you. Usually, it's not deliberate. Quite often, I simply may not know. Or I may mis-communicate. When in doubt, Google and Wikipedia. Let me repeat: this is the single most important thing that I can tell you. When in doubt, Google and Wikipedia. " At the end of the class, a few came over and said, "But how do we do that? Our teachers are asking us not to waste time on the Internet, and to stay away from Wikipedia!" Sir Ken Robinson gave a TED Talk on Do Schools Kill Creativity ? Do watch it. Apart from being one of the funniest 20-minute talks ever, it drives home a strong message. Schools aren't quite organised to foster creativity. When they were created, that wasn't the intent. Teaching as a profession, I imagine, does not pay as much as many others. So there's little interest for practitioners to enter the field. I can therefore understand and appreciate that it takes a long time for new knowledge to enter the curriculum. But also sad is the way the curriculum is treated. It isn't treated, as Gatto says, as choices among the million things of value to learn. It is treated as a Bible that defines knowledge. It is easy for teachers to fall into the trap. If it contradicts the curriculum, it is wrong. If it is not in the curriculum, it is irrelevant. Since I know the curriculum inside out, I know all that is required to know. It's not that I refuse to learn. Just that there is nothing more to learn that is relevant. As an institution, schools aren't going away any time soon. Nor perhaps should they. But in the interest of knowledge and creativity, I can only hope for two things. Students: keep learning what you like outside of school. It may be your only hope. Everyone else: drop by to your old school or your nearby school, and offer to teach one class any subject you have a passion for. You'd be surprised at how well you'll be received, how much you know, and how much you can learn by that interaction. Comments Kannan 13 Jan 2009 12:22 pm: I am not suprised with results at Vidya Mandir.There is big disparity between metro city(upper mid school) and rest of the state. Karthikeyan 13 Jan 2009 11:03 pm: Thanks for sharing your experience at the school. It gave an idea about how's the next generation educated? It's high time that we need to bring in the right kind of education to all irrespective of which school they belong. Anish 14 Jan 2009 9:50 pm: On slide 14 did the definitions get interchanged? Sasidhar 15 Jan 2009 5:09 am: Nice. I'd love to go back to my school and do a session like this.\ Sounds like good fun. Amandeep 15 Jan 2009 11:40 am: Really enjoyed reading this article..maybe i will try and go back to teach.... somnath 21 Jan 2009 11:43 am: Simply true though my father is an university teacher and i have seen him in class (practicals) too :)\ PS: RTed to Twitter Sachin Gopalakrishnan 23 Jan 2009 12:35 am: I can think of atleast on reason for not a single student contributing code or writing a blog. It's TABOO in India for students who concentrate on anything beyond course work!\ \ I would know, i had wasted time for atleast half a decade (during graduation)until i landed a job due to those wasted years and gained my redemption. bootcat 3 Apr 2009 7:15 am: ungal ezhuthukal ennai kavaruthu annae . naan , deepan , verizon la work panuren . naan oru made in chennai , currently in chennai . tamilku indha alavu net a pala "service" kalai ezhupeerukeenga . ungalai paaraatiyae aaganum . keep the good work going annae . neeram kadachuthunaa reply pannunga . eppadiku , bootcat aka deepan prabhu

On teaching


I successfully deployed open source in a corporate environment by demoing prototypes to business leads before IT and pitching solutions instead of technology. I learned to build apps myself using Django, jQuery, and Persevere to bypass infrastructure roadblocks.

Last month, my first application went live. I’ve been writing code for 20 years. Not one line of my code has been officially deployed in a corporate. (Loser...) It’s a happy feeling. Someone defined happiness as the intersection of pleasure and meaning. Writing code is pleasurable. Others using it is meaningful. But this post isn’t quite about that. It’s about the hoops I’ve had to jump through to make this happen. I’ve been living in a nightmare since March 2009. That was when I decided that I’d try and get corporates to use open source. March 2009 It began with a pitch to a VC firm. They were looking to build a content management system (CMS). Normally we’d pull together slides that say we’ll deliver the moon. This time, we put together demo based on WordPress’ CMS plugins. The meeting went fabulously well. We said, “Here’s a demo we’ve built for you. Do you like it?” The business lead (Stuart) was drooling and declared that that’s exactly what they wanted. The IT lead (another Stuart) was happy too, but warned the business users: “Just remember: this isn’t how we do development, so don’t get your hopes up that we can deliver stuff like this :-)” Time to make my point. I asked, “What’s your policy on open source software?” The business lead went quiet. “I don’t know,” he finally said. Fair enough. I turned to the IT lead. “Well, we don’t use it as a matter of policy... there are security concerns...” he said. “Which web server do you use?” ”Oh, OK. I see what you mean. We use Apache. So on a case to case basis, we have exceptions. But generally we have security concerns.“ ”Why? Do you believe open source software is more insecure than commercial software?“ He thought about it for a while. “Well... maybe. I don’t know.” We debated this a bit. Then we found the real issue: “It’s just that we don’t have control over the process. We don’t know enough about it to decide.” A couple of weeks later, I tried pitching to a newspaper company. This time, it was our sales team that raised the same question. “But... isn’t open source insecure?” I didn’t even bother pitching any open source stuff to them. But I’d learnt my lessons: 1. Demo the application. Don’t talk about it. 2. Show it to the business first, and then tackle IT. Aside: June 2009 In June, I got another chance. I was building the website for a large retailer. The very first thing I did was ask to see the Javascript. Total mess, and filled with browser-incompatible DOM requests. So I went over to their web development team. “Look, why don’t you guys use a Javascript library? It’ll get you cross browser compatibility and compact maintainable code at the same time.” And, to their credit, they said, “Sure. Which library?” I showed them this comparison of jQuery (blue), dojo, scriptaculous and mootools... https://trends.google.com/trends/explore?date=all&q=jquery,prototype,dojo,ext,yui ... and we agreed on jQuery. So, if nothing else, I’ve managed to get one open source library into a corporate. July 2009 I was also looking at payments, and retailer was looking to replace their chargeback application. Since I had a week off, I built a working PCI compliant prototype on Django. This time, I applied the lessons I’d learned, and demo-ed it to the business, who were thrilled. Time to tackle IT. I started with the architecture team. Matt on the architecture team was the most approachable. So I went over, demo-ed it, and said, “Matt, this took a week to put together. It’s based on some new technologies. Are you game to try these out?” He was. And quite enthused about it too. So we put together a proposal for the architecture review board, proposing a new technology stack: Django / Python and MySQL. As before, I showed the demo before I talked technology. I had prepared answers to all security related questions upfront (and practically memorised section 3 of the PCI guidelines.) The clincher, though, was the business case. To build it on Java, it would cost 1,000 person days. On Django, I’d mostly done it in 5. There was no way of justifying 1,000 person days for an application that could save, at best £100,000 a year. So they said “Go ahead, we’re fine if operations and infrastructure are fine.” It was time to find a Django developer in Infosys. I hunted for a couple of weeks but none was available. (Only 2 people knew Django in the first place.) So that effort got canned, and we were back to the 1,000 person day solution. (Which got canned too, later.) But in the process, I’d learned my third lesson. 3. If you’re trying new technologies, plan on delivering it yourself. October 2009 Another application popped up that looked like a prime candidate for introducing open source. They were using an Excel application to fraud screen orders, and wanted to make a web app out of it. I followed the same route as before. Demo it. Show it to business first, then IT. Built it myself. I skipped Architecture, since they’d already approved the technology stack, and took it straight to Infrastructure. “This application uses Apache as the web server, MySQL as the database, and uses PHP and Javascript for the application logic. Could we get a Linux server to host it?” Our entire conversation lasted 30 seconds. He said, “No. We use Windows servers” (I was fine) “... and you’ll need to chance Apache to IIS” (fine again) “... and we don’t support PHP, so it’ll have to be Java or .NET” (I don’t know .NET or Java... but fine) “... and we don’t support MySQL, it’ll have to be SQL Server” (fine, I guess) “... and we don’t have DBAs available until January, so you’ll have to wait.” (definitely not good.) So back to the drawing board on the technology stack. I needed something in Java (I know very little Java, but nothing at all in .NET) and to avoid the DBA headache, it would have to bundle in a database. I first explored key-value stores like CouchDB, Redis, etc. None of them worked on Java. The only one I found that did was Persevere, and it was a JSON data store, which fit perfectly with my plans. By this time, I’d also learn my my fourth and most important lesson. 4. Don’t try to promote open source. Just deliver the application I said, “This is a custom-built application that runs on Java. Could we get a Windows server to host it?” The answer was “Yes”, and we had it the next day. PS: December 2009 The application’s deployed and running. It has about 10,000 orders fraud screened by now. And the lessons are well learnt. So when some came over asking if there was any image resizing solution I knew off, I said: “Sure, who’s your business sponsor?” Then I went over and said, “Let me show you this open source application called ImageMagick. It handles aspect ratios correctly, and can crop too. Doesn’t this look professional?” Then I went over to IT and said, “It’s open source, so you can change it. It has Java bindings, so you can integrate it into your environment. It can handle 8 3000x2400 images a second on my puny laptop. It’s used by your competitors. And I can build it for you if you like.” I might just have my second open source entry into a corporate this year. Comments Kalpesh 15 Dec 2009 4:27 am: Great post. The feeling one gets, when the software is used by people and one hears a few praises, is priceless. I have enjoyed working with end-users & they have a lot to tell you, when they use the software. i.e. ask the viewers about the movie & not the critics ;) From your post, I think business guys are scared of the word "open source" without knowing some of the best software is open-source. I agree with your idea of "show them the software first, let them use it & be happy with it. They won't worry if its open source". Good to see a post from you after quite some time. Thejesh GN 15 Dec 2009 6:08 am: I had to pitch only one customer this year and he agreed to use JQuery (the biggest plus was, it wasn't gnu, its MIT). I think by default many customers use open source components with out actually realizing they are opensource. Like for example most of apache products, eclipse. But on the other side, I think the challenge is to convince IT Provider ( or the customer ) to go opensource. Not using but releasing some of their IP under OpenSource. How do we pitch this to them? Not undermining your efforts. I think the challenge is to convert this big users into contributors. Ashwin Iyengar 20 Jan 2010 9:55 pm: Hey anand, Got to your webpage after a long time. Good to see that you're still a true "end-to-end" consultant...we should have a chat about personal financial management solutions sometime later...-a Anand.S 20 Dec 2009 10:13 pm: Nice blog... Interesting and gives good insight in details . jyoti 21 Dec 2009 6:00 pm: nice blog,ur blog sounds great.keep wrtng. Chitra 21 Dec 2009 10:30 pm: Interesting! And congrats :-)! I won't pretend that I understood everything you penned (er...typed) here, but I do realize the inhibition of most users regarding open source technology due to 'security' issues. Perhaps you could list out the most quoted issues sometime...... Somnath 23 Dec 2009 4:04 am: Interesting - I thought your job is that of a management consultant or at least that's what your employer sells your service as. But you end up doing all the technical stuff as well. Guess with all this you are ready to setup your own company now. Happy holidays! BR, Somnath Khair 18 Dec 2009 3:58 pm: Thanks for your post, seems you have been busy with work/vacation. Frankly, quite a lot of IT Managers have no clue at all when you speak to them of OpenSource. Though 'unknowingly' they have been using either as Apache or PHP or Java. If you carefully look at how the IT Managers are recruited or groomed, you will see a culture of everything has to be 'MICROSOFT' ! Believe me for end-users/customers it makes no difference as long as the application delivers the need! Also one has to commend the aggressive marketing strategy of Microsoft. I am not against proprietery software but think corporates need to know and look for what's best for the clients and end-users. Which in turn could reduce the money spent on Infrastructure/development/deployment. With money saved they can generate more employment opportunities. Recruiting smart people | s-anand.net 14 Feb 2010 6:46 pm (pingback): [...] at least the smart people) that hang out on Stack Overflow for a given topic. Last year, when I was looking for a Django developer, I scoured the Infosys internal blogs for similar networks. (Didn’t find many, but it sure [...] Harish Kashyap 5 Mar 2010 7:56 am: Hi Anand, I could relate to the sequence of events... Fantastic post... I picked the learning to be that it is hard to change the corporate...Sometime you may have to put in more effort and time to change the corporate rather than to change the solution itself! I.e. Do not view as a constraint as it would prevent the change! Vinoth 21 May 2010 3:52 pm: Very insightful blog.. I work for Infosys too and your point about non-availability of django developers is interesting. I think this is one other prime reason why business do not prefer open source. There are simply not enough knowledgeable developers in latest open source frameworks and hence IT vendors cannot confidently pitch for open source.

Open source in corporates


I examine how companies lose the "why" behind their code when documentation vanishes and key people leave. I argue that maintaining conceptual integrity is essential to prevent organizational amnesia, especially during periods of rapid growth.

It’s amazing how much of a dependency there is on individuals writing IT systems. Reminds me of that Dilbert strip: 19940610 A few weeks ago, I was trying to figure out in what happens when there are multiple promotions. (Our client is a retailer.) I mean, if there’s a phone that costs £100 and there are 2 promotions: 10% off on phones and £10 off on phones. Do you apply the 10% off first and pay £80 or the £10 off and pay £81? Funnily enough, the organisational answer is, “I don’t know.” The person who determined the logic is no longer with the firm. The person who wrote the code was a contractor and moved on to another project. The vendor hadn’t gotten around to documenting the code. Sure, the code’s there, and you just had to read it to figure out what it does. But no human knew what it was supposed to do. Last week, there was a decision to rewrite some code that was 10 years old. A colleague who wasn’t quite involved in this work said, “I’m going to have to set aside 2-3 weeks for this. I wrote this stuff when I was a developer. The docs have vanished. The business owners have vanished. I’m the only one who has any clue on what it’s supposed to do.” This week, we were trying to figure out how their store locator system works. After fiddling around with Fiddler, and seeing that it used Microsoft Virtual Earth, I was able to figure out that it identified stores near a location using a simple JSON API. But can we get the documentation around that? Nope. Tough luck. Nobody knows how it works any longer. Personally, I don’t think this is unusual. We forget. Companies forget. But it’s usually good if what we forget is derivable. That’s how I got through my high-school physics exams: not by remembering stuff, but by being able to derive the stuff from a few principles. Organisations can do the same. But to be able to do that, you need to have commonly understood principles. As Fred Brooks put it in The Mythical Man Month, I contend that conceptual integrity is the most important consideration in system design. It is better to have a system omit certain anomalous features and improvements, but to reflect one set of design ideas, than to have one that contains many good but independent and uncoordinated ideas. One of the biggest enemies of conceptual integrity is growth. Too many people too soon, and the important decisions are taken by people who’ve never had a long chat about things. There’s another reason not to grow too fast. Comments Kalpesh 20 May 2009 2:27 am: I guess, it should help having a debug log of things as the code is executed & the decision made in the code based on the data available. e.g. the code will put the log (conditionally) gave 10% discount, thus derived value of 90 (100 - 10) reducing $10 of the calculated value above = 80 (90 - 10) Basically, having a log of the sequence of steps, the state on which the code relies, input any function takes and output thereafter (but in a user understandable manner or in business language). One step closer to this could be writing code in a language that is relevant to business I feel we still write our code in terms of technology (arrays, exceptions, serializations etc). I don't mean that we should not worry about those things. But, those things should be covered under a layer which makes easy for business people OR a new comer to come back to the code and read it like a specification. Nagarjun 13 Dec 2009 10:50 am: We faced same situation while trying to find out how the comission is calculated. In the end it so happened that the commision was being calculated based on data that was 5 years old. Nobody had realised that the new data was not comin into the system :)

Organisational amnesia


I built a random quote generator bookmarklet that uses Order-2 word-level Markov chains to mix up text on any webpage. It leverages the Readability script to identify content, often producing funny or insightful nonsense.

The Random Quotes Generator is a simple tool that creates quotes by mixing up words on a web page. The results are often funny, but sometimes surprisingly insightful. Monkey Typing Shakespeare Yes, this is the equivalent of a million monkeys typing Shakespeare, except that they’re using the works of Shakespeare as a starting point. And it doesn’t have to be Shakespeare. It could be you or your friends. To try it out, visit this page, select the link and “Add to Favorites” or drag it into your browser’s bookmark toolbar.. Then go to any web page that has a lot of text, and click the link to generate random quotes. Here’s an example of random text from TechCrunch. The net will find monetization models of theater and sporting events before them. Indeed, there has to be some way to create websites that do other than Advertising. The expected drop in internet advertising will rapidly lose its value and its impact, for reasons that can easily be understood. For the technically minded, Programming Pearls has a section on Generating Text that explains the concept. The bookmarklet uses an Order-2 word-level Markov chain. Translated into English, what that means is: I look at every pair of words in and find out what word is likely to follow that. For example, in the Generating Text page, the pair of words “we can” are followed by the words “extend”, “also”, “get” and “write” with equal probability. We pick one randomly (say “also”) and write “we can also”. Then we look at the word pair “can also”, see what word follows that, pick one at random, and so on. This is Order-2 because we pick pairs of words. And it’s word-level rather than letter-level because we use words instead of letters as the basic building blocks. When you’re trying it out, make sure that the page is large enough. If not, you may find that the page’s content is reproduced verbatim. The bookmarklet is built on top of the excellent Readability bookmarklet by Arc90, which helps identify the main content to be randomized.

Random quotes generator


I built a round button generator using the Python Image Library (PIL) after exploring various automation methods. My approach uses custom alpha masks and programmatic gradients to create transparent PNG sprites for cross-browser CSS buttons.

After much hunting, I finally settled on Hedger Wang’s simple round CSS links as the most acceptable cross-browser round button implementation. The minified CSS is about 2.5KB, and the syntax is very simple. To make an input button into a round button, just wrap it within a : … and it’s just as easy to convert a link into a rounded button: It works by using a transparent PNG / GIF that looks like this: The first button is the default button. The second appears on hover. The bottom two are for disabled buttons. Can we easily create buttons in different colours? That’s what this post is about: creating that image with round buttons and gradients. When I tried creating these rounded buttons myself (and trying to do it in an automated was as usual), I saw 3 possible approaches: 1. Create it using PowerPoint via Python and export as a PNG. So we make a curved box, put in the appropriate gradients and borders, and export it as a PNG. But the problem is I couldn’t figure out how to get transparent PNGs. 2. Create it in GIMP using script-fu plugin . The problem is, I don’t know scheme or GIMP’s API. So I gave up on this as well. 3. Create it using Python Image Library . This was inspired by Nadia’s PIL Tutorial: How to Create a Button Generator. Let me explain how this works. The first step is to create a ‘button-mask.png’ like this one: 1. Create a transparent 300 x 120 image in GIMP 2. Selecting a box from (0,0) to (300,30) 3. Shrink it by 2 pixels 4. Convert it to a rounded rectangle with a radius of 80% 5. Fill this in white 6. Copy it to 60 pixels below Now, we need code to create a gradient: Now that the gradient is created, convert that into a round button by loading button-mask.png’s alpha layer onto the gradient: There it is: a simple round button generator. You can see a sample of these buttons at my Dilbert search site. Comments helpful 27 Jun 2009 8:01 pm: The best way to do this is to use python's cairo support to fully draw the button and apply the gradient and add text. S Anand 29 Jun 2009 5:08 am: Thanks -- Cairo seems a lot more powerful than PIL. Will give it a shot.

Round buttons with Python Image Library


I share notes on focusing on employee strengths rather than weaknesses, using the Google Analytics API for tracking events and error logs, and the proliferation of sandbox environments within Google’s developer ecosystem.

I'm quite busy on a project right now, and don't get time to write long articles. So for a while, I'm going to stick to short notes on interesting stuff. 1. Peter Bregman has a very interesting piece on Why You Should Encourage Weakness. It boils down to a choice: do you focus on on improving strengths or minimising weaknesses? Conventional performance evaluations focus on the latter. I very strongly support Bregman’s view on this. The weakness isn’t why you hired the person! Unless it’s killing the organisation, just leave them to focus on their strengths. 2. Google Analytics has a fairly interesting API that I hadn’t explored until recently. Picked up Advanced Web Metrics with Google Analytics and learnt that you can track outbound clicks, page load times, Javascript events and error logs, almost anything at all using Google Analytics. You can also mirror the logging on your local server using pageTracker.\setLocalRemoteServerMode() 3. The whole concept of a Sandbox environment seems to be picking up within Google. There’s a Checkout sandbox, an AJAX API playground, an AdWords sandbox, an AdSense API sandbox, the Mapstraction API sandbox, even an event called Developer Sandbox. (After saying Sandbox 6 times, I feel a bit like Hobbes.)

Short notes


I implemented a simple URL shortening service for my WordPress blog using post IDs and the rev=canonical proposal. By adding a single PHP line and an .htaccess rewrite rule, I eliminated the need for third-party shorteners.

With all the discussion around URL shorteners, Diggbar, blocking it, and the rev=canonical proposal, I decided to implement a URL shortening service on this blog with the least effort possible. This probably won’t impact you just yet, but when tools become more popular and sophisticated, it would hopefully eliminate the need for tinyurl, bit.ly, etc. Since the blog runs on WordPress, every post has an ID. The short URL for any post will simply be http://www.s-anand.net/theID. For example, http://s-anand.net/17 is a link to post on Ubuntu on a Dell Latitude D420. At 21 characters, it’s roughly the same size as most URL shorteners could make it. The code is easy: just one line added to index.php: "> … and one line in my .htaccess: RewriteRule ^([0-9]+)$ blog/?p=$1 [L,R=301,QSA] Hopefully someone will come up with a WordPress plugin some time soon that does this. Until then, this should work for you.

Short URLs


I tracked my site's referral traffic following the launch of Microsoft’s Bing search engine. My data shows a significant initial spike followed by a rapid decline, illustrating how the search engine's early hype quickly faded.

bing.com referral statistics This graph is the number of referrals Microsoft’s new search engine, Bing, sent to my site over the last few days. Looks like the hype is dying out. Though Bing did leapfrog Yahoo briefly, that lasted just one day.

The Bing effect


I switched from Perl to Python to escape closing braces and embrace significant whitespace. I now select languages based on libraries like BeautifulSoup and PIL, using Project Euler to master Python features like generators.

I've recently switched to Python, after having programmed in Perl for many years. I'm sacrificing all my knowledge of the libraries and language quirks of Perl. The reason I moved despite that is for a somewhat trivial reason, actually. It's because Python doesn't require a closing brace. Consider this Javascript (or very nearly C or Java) code: That's 6 lines, with two lines just containing the closing brace. Or consider Perl. Again, 6 lines with 2 for the braces. The $ before the variables also drops readability just a little bit. Here's Python: On the margin, I like writing shorter programs, and it annoys me to no end to have about 20 lines in a 100-line program devoted to standalone closing braces. \ What I find is that once you've really know one language, the rest are pretty straightforward. OK, that's not true. Let me qualify. Knowing one language well out of C, C++, Java, Javascript, PHP, Perl, Python and Ruby means that you can program in any other pretty quickly -- perhaps with a day's reading and a reference manual handy. It does NOT mean that you can pick up and start coding with Lisp, Haskell, Erlang or OCaml. Occasionally, availability constrains which programming language I use. If I'm writing a web app, and my hosting provider does not offer Ruby or Python, that rules them out. If I don't have a C or Java compiler on my machine, that rules them out. But quite often, these can be overcome. Installing a compiler is trivial and switching hosting providers is not too big a deal either. Most often, it's the libraries that determine the language I pick for a task. Perl's regular expression library is why I've been using it for many years. Ruby's HPricot and Python's BeautifulSoup make them ideal for scraping, much more than any regular expression setup I could use with Perl. Python Image Library is great with graphics, though for animated GIFs, I need to go to the GIF89 library in Java. And I can't do these easily with other languages. Though each of these languages boast of vast libraries (and it's true), there are still enough things that you want done on a regular basis for which some libraries are a lot easier to use than others. So these days, I just find the library that suits my purpose, and pick the language based on that. Working with Flickr API or Facebook API? Go with their default PHP APIs. Working on AppEngine? Python. These days, I pick Python by default, unless I need something quick and dirty, or if it's heavy text processing. (Python's regular expression syntax isn't as elegant as Perl's or Javascript's, mainly because it isn't built into the language.) \ To get a better hang of Python (and out of sheer bloody-mindedness), I'm working through the problems in Project Euler in Python. For those who don't know about Project Euler, Project Euler is a series of challenging mathematical/computer programming problems that will require more than just mathematical insights to solve. Although mathematics will help you arrive at elegant and efficient methods, the use of a computer and programming skills will be required to solve most problems. Each problem has been designed according to a "one-minute rule", which means that although it may take several hours to design a successful algorithm with more difficult problems, an efficient implementation will allow a solution to be obtained on a modestly powered computer in less than one minute. It's a great way of learning a new programming language, and my knowledge of Python is pretty rudimentary. At the very least, going through this exercise has taught me the basics of generators. I've solved around 40 problems so far. Here are my solutions to Project Euler. I'm also measuring the time it takes to run each solution. My target is no more than 10 seconds per problem, rather than the one-minute, for a very practical reason: the solutions page itself is generated by a Python script that runs each script, and I can't stand waiting for more than that to refresh the page each time I solve a problem. Comments Nicola 17 Dec 2014 10:15 am: Hi guy, I think you should add a comment form on your Euler page... this is not the right page for this comment. Anyway it seems your page is one of the favourites for the students that have project Euler probems as class assignments. So I'd like to contribute... This is a better solution for the problem #18 using an easy formulation od Dijkstra algorithm. tr = ( (75, ), (95, 64, ), (17, 47, 82, ), (18, 35, 87, 10, ), (20, 4, 82, 47, 65, ), (19, 1, 23, 75, 3, 34, ), (88, 2, 77, 73, 7, 63, 67, ), (99, 65, 4, 28, 6, 16, 70, 92, ), (41, 41, 26, 56, 83, 40, 80, 70, 33, ), (41, 48, 72, 33, 47, 32, 37, 16, 94, 29, ), (53, 71, 44, 65, 25, 43, 91, 52, 97, 51, 14, ), (70, 11, 33, 28, 77, 73, 17, 78, 39, 68, 17, 57, ), (91, 71, 52, 38, 17, 14, 91, 43, 58, 50, 27, 29, 48, ), (63, 66, 4, 68, 89, 53, 67, 30, 73, 16, 69, 87, 40, 31, ), ( 4, 62, 98, 27, 23, 9, 70, 98, 73, 93, 38, 53, 60, 4, 23, ), ) sums=[0] for i in xrange(len(tr)): ns=[0]\len(tr[i]) for c,v in enumerate(tr[i]): if c>0: s0=sums[c-1]+v if s0>ns[c]: ns[c]=s0 else: pass try: s1=sums[c]+v if s1>ns[c]: ns[c]=s1 except: pass sums=ns max(sums) For each row the program stores the greater accumulator of the numbers leading to that row, so it is not an O(2^n) problem but a O(n^2). I have also a solution for problem #31: coins = (1, 2, 5, 10, 20, 50, 100, 200) count=0 def coinFlip(level, residual, pattern): global count, tests p = residual / coins[level] print level, residual, pattern, innResid while True: tests += 1 innResid = residual-p\coins[level] if level>0: coinFlip(level-1, innResid, pattern + (p,)) if p==0: break p=p-1 else: if innResid==0: count += 1 print count, pattern + (p,) break count=0 tests=0 coinFlip(7, 200, ()) It works by "changing" the coins: we start with one 2P and change it for two 1P, then we change one of the two 1P with two 50p... and so on. I'd be very glad if you like them and decide to publish my code on your site. Greetings, Nicola.

To Python from Perl


I built a replacement for the defunct twofifty.org to track my progress through the IMDb Top 250 list. It lets you mark movies as seen, track your score, and view recent additions. I also open-sourced the code.

It's been a good movie month for me, and I've managed to nudge closer to my target of watching the IMDb Top 250. But one tool I had in the past, that I sorely miss, is twofifty.org. It's a now-defunct site that kept track of the IMDb Top 250, and let you strike off the movies that you had watched. You could see which movies you hadn't seen, keep score, and discuss the movies. Since it's demise, my movie watching slowed down as well. Earlier this month, I set up a similar site at 250.s-anand.net. It has the same basic function. You can log in, strike out movies that you've seen, and keep track of what's left to see. For the more technically minded, the source-code is at two-fifty.googlecode.com. Visit 250.s-anand.net Happy movie tracking, and looking forward to your suggestions. Comments AG 20 Oct 2005 11:35 am: www.whatshouldireadnext.com is similar, but a lot more customized. For books. S Anand 4 Feb 2009 2:28 pm: Welcome, Sacha. Enjoy! SMG 5 Feb 2009 7:53 am: check out this site...\ http://www.listsofbests.com/\ You can track a lot of lists out there S Anand 5 Feb 2009 1:58 pm: Pretty neat. Looks like a site I'm likely to use quite a bit going forward.\ Thanks, SMG! Pramod Singh 6 Feb 2009 8:54 pm: Thanks dude !!! I used to track movies on twofifty.org and sorely missed it since it went down. Now I can keep track of them again and complete my mission of watching all of the top 250 movies. Great work !!! Thejesh GN 29 Jan 2009 6:43 am: Awesome.\ \ Can you just print the name and not email ids of "Top movie watchers".. S Anand 29 Jan 2009 9:39 am: That's next on my to-do list :-) Binny V A 29 Jan 2009 2:55 pm: Wow - I did not know others where as crazy as me ;-)\ \ And thanks for the app - it will come in handy. Until know, I was using a text file - I just deleted the movies I saw. S Anand 30 Jan 2009 9:48 am: Yeah, it was a bit of a shock to me as well ;-) Sacha 4 Feb 2009 1:46 am: THANK YOU! It really sucks that twofifty.org went offline, but it’s awesome that you created a replacement. Mofette 21 Feb 2009 5:08 am: Oh My God I think I love you! And they shut down just after I donated. Gits. John Baptist 8 Mar 2009 11:37 am: Cheers, I too miss twofifty.org - going to be quite a chore ticking them all off again! Does your site (like twofifty.org) also 'remember' movies that have been ticked off but temporarily fall out of the top250 list? S Anand 8 Mar 2009 4:03 pm: Yes, John, the 250.s-anand.net remembers moves that have fallen out of the Top 250 list. It also lets you know which movies have recently moved into the Top 250, and whether you'd marked them as seen. Al 24 Mar 2009 6:03 am: Awesome, thank you. Michael 17 Mar 2009 3:33 am: Thank you! I had toyed with writing something in Rails but I just didn't find the time. twofifty.org was a good site but it wasn't reliable at all. Tamlyn 28 May 2009 10:23 am: Amazing. I was crestfallen when I saw twofifty.org had disappeared but your version is even better! Thanks so much for this. And i guess as it's Google-hosted, it's here to stay. Sacha 18 Aug 2009 4:57 pm: I have to log in every single time I visit the list. Is that intended or a bug? It really sucks quite a bit. Himanshu 10 Aug 2009 9:45 am: Thanks for the site. I was planning to do something similar since the day twofifty.org is down. Brawler 17 Sep 2009 9:50 pm: http://www.icheckmovies.com seems very nice. Though is missing those codes you could use anywhere with html-code to track your count, and last 5 seen IMDb Top250 movies... Even that, I hope you'll like the site, it has more topics than Top250 ;) Limeworld.com » Blog Archive » IMDB Top 250 6 Dec 2009 1:12 am (pingback): [...] mere 95/250. My list: http://twofifty.org/user/Limegirl/ Update: twofifty.org is now defunct, but a clever fellow has launched something [...] Stijn 25 Nov 2012 2:28 pm: Just noticed the Facebook app no longer exists. So, I signed up for your alternate version. Let's see what my score is... 106/250. Still a loooong way to go. F 14 Aug 2015 2:11 pm: Hi, I really like the site but now i get a 404 when trying to access it. Is it permanently removed or just temporarily? If you don't plan to put it up again would you be interested in open sourcing the source code so i can continue to use your handy tool? :)

twofifty.org


I documented my process for installing Ubuntu 8.10 on a USB flash drive for my Dell Latitude D420. I used the built-in USB creator, fixed wireless with Broadcom drivers, and navigated issues with hibernation and NTFS disk access.

Here's the fastest way I've found to install Ubuntu on a USB flash drive, for my Dell Latitude D420. (Pendrivelinux.com is a great resource for this sort of thing.) Ingredients 1. One large USB flash drive like this one. Not less than 4GB. I'd suggest 8GB or more 2. One CD (not a DVD) 3. Ubuntu 8.10 desktop CD ISO 4. IMGBurn or any other CD burning software 5. Direct Internet via LAN cable (without proxy, without wireless) Installation 1. Burn the Ubuntu ISO file on the CD 2. Press F12 when the laptop boots up, and select CD/DVD Drive as the boot device 3. On the Ubuntu splash screen, select "Try Ubuntu without making any change to your computer" and wait 4. Insert the flash drive 5. Go to System > Administration > Create a USB startup disk and follow instructions there 6. Once done, remove the CD and reboot using the USB flash drive (pressing F12 during the boot sequence) To enable wireless, which won't work by default 1. Connect to the Internet using a LAN cable 2. Go to System > Administration > Hardware devices 3. Select the Broadcom LAN driver, and activate it That's it. It's been a fairly painless installation. I do have one big crib. I planned to use Hibernation (or suspend-to-disk on Ubuntu) to switch between Windows and Ubuntu. But there are a couple of problems: Hibernate doesn't work on Ubuntu. I need to reboot Ubuntu every time, and that takes 3 minutes When Windows is hibernating, Ubuntu can't access any files on the hard disk This means switching between Ubuntu and Windows is roughly a 6 minute shutdown-one-OS-reboot-the-other process rather than the 1-minute hibernate-one-OS-resume-the-other that I had had hoped for. Another minor problem I have is that our Exchange server doesn't seem to have an IMAP interface, at least that I know of. So I can't check mail. But like I said, it's minor. I just forward mails from my BlackBerry to GMail. Comments deeaycee 19 Jan 2009 10:50 am: Your post is how I got my wireless working. Thank you.\ I'm also using a Dell D420. I have 8.10 on a 8g Attache flash drive. I ran into a problem while updating. I'm stuck at kernel 2.6.27-7 and it should be updating to 2.6.27-11. The error it gives has something to do with running from a live-cd. After some research, I found that there is a bug that prevents solid updates from the live-cd. The work-around is to actually do a real install to the flash drive from the live-cd. This presents another problem/bug. The only way to see the flash drive during the install is to disconnect the hd, so the install prg doesn't see it. It was suggested to physically disconnect the hd, but I wonder if it could be done temporarily through the bios. I'll try this later today. I haven't even thought about the hibernation/suspend issues yet. Good luck! ActionParsnip 30 May 2010 10:22 pm: orks 100% OOTB with Ubuntu Lucid.

Ubuntu 8.10 on a Dell Latitude D420


I explain how I enabled exact WYSIWYG previews in Windows Live Writer for custom WordPress themes. By creating a view.html template and redirecting the wlwmanifest.xml link via functions.php, I forced the editor to recognize my unique layout.

One of the reasons I moved to WordPress was the ability to write posts offline, for which I use Windows Live Writer most of the time. The beauty of this is that I can preview the post exactly as it will appear on my site. Nothing else that I know is as WYSIWYG, and it’s very useful to be able to type knowing exactly where each word will be. The only hitch is: if you write your own WordPress theme, Live Writer probably won’t be able to detect your theme — unless you’re an expert theme writer. I hunted on Google to see how to get my theme to work with Live Writer. I didn’t find any tutorials. So after a bit of hit-and-miss, I’m sharing a quick primer of what worked for me. If you don't want to go through the hassle, you can always call on professionals who are adept at services like professional custom website design. Open any post on your blog (using your new theme) and save that as view.html in your theme folder. Now replace the page’s title with {post-title} and the page’s content with {post-body}. For example: This is the file Live Writer will be using as its theme. This page will be displayed exactly as it is by Live Writer, with {post-title} and {post-body} replaced with what you type. You can put in anything you want in this page — but at least make sure you include your CSS files. To let Live Writer know that view.html is what it should display, copy WordPress’ /wp-includes/wlw-manifest.xml to your theme folder and add the following lines just before . Live Writer searches for wlmanifest.xml in the tag of your home page. Since WordPress already links to its default wlwmanifest.xml, we need remove that link and add our own. So add the following code to your functions.php: That’s it. Now if you add your blog to Live Writer, it will automatically detect the theme. Comments Windows Live Writer 向 DEDECMS 发文章 | 快乐生活，快乐工作 30 Apr 2009 8:39 am (pingback): [...] WordPress themes on Live Writer [...] San Diego Web Design 22 Jun 2009 9:04 pm: Note to my first post as it doesn't show correctly: What I did is I ended up adding the part between the tags to the regular wlwmanifest.xlm file. vivek 15 Jun 2009 6:23 am: Hi, Thanks great information San Diego Web Design 22 Jun 2009 9:02 pm: Thank you, thank you, thank you! I was trying for hours to fix this before I found your post! Actually, the way you described it didn't exactly work for me (I guess I did something wrong with the paths), but I ended up just adding the WebLayout with the absolute path to the regular wlwmanifest.xlm inside the includes folder and now it works! Thanks again, your post was the only one describing this problem and helping me fix it! shawn 29 Jan 2010 7:37 pm: I was having this issue today and came across this page. I came up with a very simple fix that requires no coding. When setting up your blog in WLW, go to your admin settings in the wordpress dashboard and temporarily set your "Reading" settings to "Front page displays -> Your latest posts ". WLW will successfully download the theme, then set your Front page displays back to your custom setting. zimmi 27 Jan 2010 7:53 pm: What if it still doesn't work? Anything else I can try? S 15 Jul 2010 5:45 pm: Thank you so much!!!!!!!!!! this worked. girls information 27 Sep 2010 7:19 am: this worked for me tanks for sharing Graphene, Child Themes, and Live Writer - Eyes2Design 28 Jul 2012 4:57 am (pingback): [...] came across this page by www.s-anand.net which helped me a lot in understanding how Live Writer and WordPress get along together. So follow [...]

WordPress themes on Live Writer


I struggle with business managers who treat one-in-a-trillion risks as likely events. DHH’s point about maintaining a sense of proportion is essential for avoiding nonsense and focusing on probabilities that actually matter in daily operations.

A quote from David Heinemeier Hansson: So the problem is, a lot of business managers and especially business owners, they have no sense of probability. They can't fathom that concept. So They treat the probability of 1 to 10 trillion as the same as a 1 to a 100. And like, "We've got to deal with this 1 to a trillion probability, because, what if it happens?" No! Doesn't matter! I mean, don't care. So as soon as that sense of probability spreads, that people can treat that reasonably, I think all this nonsense just goes away. This lack of proportion, sadly, is at the heart of my every day problems. (Just watch the video!) Comments elbin 14 May 2010 5:49 pm: Money matters!

A sense of proportion


I ran an A/B test with 800 ratings to compare automated image enhancements like sharpening, saturation, and auto-levels. Sharpening was the clear winner, preferred in 86% of cases, while saturation and levels yielded more subjective results.

There are some standard enhancements that I apply to my photos consistently: auto-levels, increase saturation, increase sharpness, etc. I’d also read that Flickr sharpens uploads (at least, the resized ones) so that they look better. So last week, I took 100 of my photos and created 4 versions of each image: 1. The base image itself (example) 2. A sharpened version (example). I used a sharpening factor of 200% 3. A saturated version (example). I used a saturation factor of 125% 4. An auto-levelled version (example) I created a test asking people to compare these. The differences between these are not always noticeable when placed side-by-side, so the test flashed two images at the same place. After about 800 ratings, here are the results. (Or, see the raw data.) Sharpening clearly helps. 86% of the sharpened images were marked as better than the base images. Only 2 images (base/sharp, base/sharp) received a consistent feedback that the sharpened images were worse. (I have my doubts about those two as well.) On the whole, it seems fairly clear that sharpening helps. Saturation and levels were roughly equal, and somewhat unclear. 69% of the saturated images and 68% of auto-levelled images were marked as better than the base images. And almost an equal number of images (52%) showed saturation as being better than the auto-levelled version. For a majority of images (60%), there’s a divided opinion on whether saturation was better than levelling or the other way around. On the whole, sharpening is a clear win. When in doubt, sharpen images. For saturation and levelling, there certainly appears to be potential. 2 in 3 images are improved by either of these techniques. But it isn’t entirely obvious which (or both) to apply. Is there someone out there with some image processing experience to shed light on this? Comments Jason 14 Feb 2011 11:27 am: Depending on your settings in Photoshop (i assume that's what you're using since you refer to it as "levels"), when you adjust levels to increase contrast (when either the left or right sliders are moved toward each other), the saturation is also increased. You can click "preserve tones" or something like that, to help keep the saturation unchanged but I don't due to some reason I can't recall right now. Anyway, so when you hit Auto-Levels, it moves the sliders closer toward each other (since they start out farthest away from each other). This increases the contrast and ALSO increases the saturation. You'll also notice this effect with the dodge/burn brushes: the saturation is also effected in the applied area. Now, your test results seem similar between the 'levels' group and the 'saturation' group because both groups are seeing photos with saturation increases. Note that the converse is not true: changes made only to saturation do not yield a change in contrast. The fact that hues will now stand out more from each other sorta gives the affect of "contrast" in a sense though, even if it isn't reflected accurately in the histogram. I would still expect people to prefer the photos with Auto-Levels applied, since those photos will have greater detail and less of a hazy look in the mid-ranges, but whatever. People are hard to predict. I prefer the auto-leveled photos over the saturated photos. In fact, I almost always adjust levels, and when I do, I often decrease saturation a tiny bit depending on the subject. S Ganesh 6 Mar 2011 3:44 am: Sharpening is always better because it is non-destructive in nature and does not add any new details unlike the the saturation and levels. In a way it is illusory but it works. It basically works on the edges and emphasize them. Internally, it can be assumed as a 2 step process, the first step is to blur the image slightly and the second step is to compare the original with blur version one pixel at a time & emphasize them. If original pixel is brighter than the blurred version, then lighten; if original is darker than the blur version, then darken. The objective is to increase the contrast between each pixel and its neighbours... Sharpening works well on images that has slight blur originally, inherently due to the way in which pictures are taken... casually on the move, non-sturdy hands etc... Hence, it appears to work on most of the pictures Advanced photo editing tools provide feature called Unsharp Masking, where it is possible to control the 'radius' - amount of blur and 'Threshold' - difference in value between original and blur when it exceeds certain value sharpening is applied...

Automated image enhancement


I explore how to use Bayes' Theorem to update a probability distribution iteratively. Starting with a flat prior, I show how successive coin toss results refine the Beta distribution to better estimate unknown likelihoods.

I’ve tried understanding Bayes’ Theorem several times. I’ve always managed to get confused. Specifically, I’ve always wondered why it’s better than simply using the average estimate from the past. So here’s a little attempt to jog my memory the next time I forget. Q: A coin shows 5 heads when tossed 10 times. What’s the probability of a heads?\ A: It’s not 0.5. That’s the most likely estimate. The probability distribution is actually: dbeta(x,5,5) That’s because you don’t really know the probability with which the coin will throw a heads. It could be any number p. So lets say we have a probability distribution for it, f(p). Initially, you don’t know what this probability distribution is. So assume they’re all the same – a flat function: f(p) = 1dbeta(x,1,1) Now, given this, let’s say a heads falls on the next toss. What’s the revised probability distribution? It’s: f(p) ← f(p) \ probability(heads | x) / probability(heads) = 1 \ (x^1 \ (1-x)^0) / 1 = x dbeta(x,2,1) Let’s say the next is again a heads. Now it’s f(p) ← f(p) \ probability(heads | x) / probability(heads) = x \ (x^1 \ (1-x)^0) / 1 = x^2 dbeta(x,3,1) Now if it’s a tails, it becomes: f(p) ← f(p) \ prob(tails | x) / prob(tails) = x^2 \ (x^0 \ (1-x)^1) / 1 = x^2 \ (1-x) dbeta(x,3,2) … and so on. (This happens to be a called a Beta distribution.) Now, instead of this being the probability of heads, it could be the probability of a person having blood pressure, or a document being spam. As you get more data, the probability distribution of the probability keeps getting revised. Comments Ram 18 Jan 2012 1:59 pm: Hi Anand, f(p) ← f(p) \ probability(heads | x) / probability(heads) = 1 \ (x^1 \ (1-x)^0) / 1 = x What I understand from the above notation is f(p) stands for probability distribution function (which is used recursively), x stands for the probability of getting a head (which is unknown) and 1-x stands for the complement of x. Can you please explain why the distribution function is defined as you have done? i.e. , f(p) ← f(p) \ probability(heads | x) / probability(heads)

Bayes’ Theorem


I curated a collection of silent Calvin and Hobbes comic strips. Having spent six years manually transcribing every strip in the series, I filtered the database to highlight these wordless, visually expressive moments.

Comments Vinu 28 Apr 2010 11:25 pm: Nice ! what did you do? Manually erase the text? Knowing you, i suspect there is some automation at work here :) S Anand 29 Apr 2010 7:17 am: Oh, not quite. By 2006, I'd typed out all the Calvin and Hobbes (by hand, spending 6 years at it). This was just filtering all the strips that didn't have any text on them :-) Vinu 30 Apr 2010 5:52 pm: Ouch. That's a lot of typing. Surprised you did not crowdsource it... Perhaps for the next assignment? Asterix comics anybody?

Calvin, speechless


I built a simple command-line alarm system for Windows using a tiny Visual Basic script. It takes a duration and message as arguments, then triggers a popup reminder to help me stay on schedule while working.

When I’m in front of my laptop, I usually forget the world around. Sadly, the world around has important things that need to get done on time. Like eating medicines, turning off the washing machine or the hob, etc. The one thing I’ve been lacking on my machine was a simple alarm system. I’d like to set an alarm to remind me to do something in 5 minutes, for example. And it should be dead simple to set up. After hunting around a fair for freeware to do this, I’ve finally settled on writing this tiny piece of Visual Basic code. I’ve saved this as “alarm.vbs” somewhere in my path. When I need to set an alarm, I just type This pops up a window in 5 minutes with the alarm: An informational popup window saying Turn off the hob This turned out to be a life-saver yesterday. I had to catch a flight at the Bangalore airport, and traffic is notoriously bad. To be on the safe side, I set up the following: Turned out to be a wise thing. I ignored the first alarm. On the second, I said “OK, OK, just 1 minute…” and it really took the third alarm to get me going. Just barely made it to the flight. Comments Abhijit Sen 10 Feb 2010 12:07 am: Anand. is it required to have a VB client to run this VB script? 408wij 26 Jan 2010 10:28 pm: For us uninformed Americans, hob=stove, right? Sumit Dhar 25 Jan 2010 1:14 am: Hey Anand, Was it a flight for UK? If you are gonna be back in Bangalore, let us meet up sometime. Cheers, Sumit Abhijit Sen 12 Feb 2010 1:45 am: Hey Anand. I opened a notepad, copy pasted the above code with the specific number of minutes and message, saved the notepad as Alarm.vbs on my desktop. Nothing happened after the given number of minutes. What am I missing? S Anand 29 Jan 2010 8:37 am: But isn't an annoyance the whole point of an alarm? Anup 28 Jan 2010 12:35 pm: Nice and useful lines. Any clue for sending the annoying window to background... S Anand 10 Feb 2010 4:49 pm: No, not at all. This comes built in with Windows. Absolutely nothing else required. Ravi 28 Feb 2010 6:26 pm: You can use gadgets in Vista for. I searched and found one, free. Ravi 28 Feb 2010 6:30 pm: http://www.puttee.de/ I tried this Anand, works like a charm.. Ravi 14 Feb 2010 7:11 am: @Abhijit - You dont have to specify the minutes and message in the .vbs file. Just save it as it is in a folder. You will have to write the command from the same directory on the DOS Prompt. The command "Alarm x message" takes the minutes and the message as the input and displays them when it runs Alarm.vbs I made this change to the script since I got error in executing the original one. CHANGED "If WScript.Arguments.length

Command line alarm