If you toss Dropbox into the mix, there’s a powerful solution there. You can share files via Dropbox as Markdown, and publish them as web pages. There are already a number of solutions that let you do this. DropPages.com and Pancake.io let you share Dropbox files as web pages. Calepin.co lets you blog using Dropbox.
My needs were a bit simpler, however. I sometimes publish Markdown files on Dropbox that I want to see in a formatted way – without having to create an account. Just to test things, or share temporarily.
I’ve updated Mixamail to access Google search results via e-mail.
For those new here, Mixamail is an e-mail client for Twitter. It lets you read and update Twitter just using your e-mail (you’ll have to register once via Twitter, though).
Now, you can send an e-mail to firstname.lastname@example.org with a subject of “Google” and a body containing your query. You’ll get a reply within a few seconds (~20 seconds on my BlackBerry) with the top 8 search results along with the snippets.
It’s the snippets that contain the useful information, as far as I’m concerned. Just yesterday, I managed to find the show timings for Manmadan Ambu at the Ilford Cine World via a search on Mixamail. (Mixamail win, but the movie was a let down, given expectations.)
You don’t need to be registered to use this. So if you’re ever stuck with just e-mail access, just mail email@example.com with a subject “Google”.
Since I don’t have Internet access on my BlackBerry (because I’m in prison), I’ve had a pretty low incentive to use Twitter. Twitter’s really handy when you’re on the move, and over the last year, there were dozens of occasions where I really wanted to tweet something, but didn’t have anything except my BlackBerry on hand. Since T-Mobile doesn’t support Twitter via SMS, e-mail is my only option, and I haven’t been able to find a decent service that does what I want it to do.
I’ve kept it as simple as I could. If I send an email to firstname.lastname@example.org, it replies with the latest tweets on my Twitter home page. If I mail it again, it replies with new tweets since the last email.
I can update my status by sending a mail with “Update” as the subject. The first line of the body is the status.
I can reply to tweets. The tweets contain a “reply” link that opens a new mail and replies to it.
I can subscribe to tweets. Sending an email with “subscribe” as the subject sends the day’s tweets back to me every day at the same hour that I subscribed. (I’m keeping it down to daily for the moment, but if I use it enough, may expand it to a higher frequency.)
Soon enough, I’ll add re-tweeting and (update: added retweets on 27 Oct) a few other things. I intend keeping this free. Will release the source as well once I document it. The source code is at Github.
For the technically minded, here are a few more details. I spent last night scouting for a small, nice, domain name using nxdom. I bought it using Google Apps for $10. The application itself is written in Python and hosted on AppEngine. I use the Twitter API via OAuth and store the user state via Lilcookies. The HTML is based on html5boilerplate, and has no images.
The Application object you get here is the same Application object you’d use in Visual Basic. That’s pretty powerful. What that means is, to a good extent, you can copy and paste Visual Basic code into Python and expect it to work with minor tweaks for language syntax.
So let’s try to do something with this. First, let’s open PowerPoint and add a blank slide.
# Open PowerPoint
Application = win32com.client.Dispatch("PowerPoint.Application")# Create new presentation
Presentation = Application.Presentations.Add()# Add a blank slide
Slide = Presentation.Slides.Add(1,12)
That 12 is the code for a blank slide. In Visual Basic, you’d instead say:
Slide = Presentation.Slides.Add(1, ppLayoutBlank)
To do this in Python, run Python/Lib/site-packages/win32com/client/makepy.py and pick “Microsoft Office 12.0 Object Library” and “Microsoft PowerPoint 12.0 Object Library”. (If you have a version of Office other than 12.0, pick your version.)
This creates two Python files. I rename these files as MSO.py and MSPPT.py and do this:
import MSO, MSPPT
g =globals()for c indir(MSO.constants): g[c]=getattr(MSO.constants, c)for c indir(MSPPT.constants): g[c]=getattr(MSPPT.constants, c)
This makes constants like ppLayoutBlank, msoShapeRectangle, etc. available. So now I can create a blank slide and add a rectangle Python just like in Visual Basic:
url ='http://spreadsheets.google.com/pub?key=phNtm3LmDZEOoyu8eDzdSXw&output=csv&range=B2:C51'# Open the URL using a CSV reader
reader =csv.reader(urllib2.urlopen(url))# Convert the CSV into a list of (asset-size, bank-name) tuples
data =list((int(s.replace(',','')), b.decode('utf8'))for b, s in reader)
I created a simple Treemap class based on the squarified algorithm — you can play with the source code. This Treemap class can be fed the the data in the format we have, and a draw function. The draw function takes (x, y, width, height, data_item) as parameters, where data_item is a row in the data list that we pass to it.
def draw(x, y, w, h, n):
# Draw the box
shape = Slide.Shapes.AddShape(msoShapeRectangle, x, y, w, h)# Add text: bank name (asset size in millions)
shape.TextFrame.TextRange.Text= n + ' (' + str(int(n/1000 + 500)) + 'M)'# Reduce left and right margins
shape.TextFrame.MarginLeft= shape.TextFrame.MarginRight=0# Use 12pt font
shape.TextFrame.TextRange.Font.Size=12from Treemap import Treemap
# 720pt x 540pt is the size of the slide.
Treemap(720,540, data, draw)
Try running the source code. You should have a single slide in PowerPoint like this.
The beauty of using PowerPoint as the output format is that converting this into a cushioned Treemap with gradients like below (or changing colours, for that matter), is a simple interactive process.
The Random Quotes Generator is a simple tool that creates quotes by mixing up words on a web page. The results are often funny, but sometimes surprisingly insightful.
Yes, this is the equivalent of a million monkeys typing Shakespeare, except that they’re using the works of Shakespeare as a starting point. And it doesn’t have to be Shakespeare. It could be you or your friends.
To try it out, visit this page, select the link and “Add to Favorites” or drag it into your browser’s bookmark toolbar.. Then go to any web page that has a lot of text, and click the link to generate random quotes.
The net will find monetization models of theater and sporting events before them. Indeed, there has to be some way to create websites that do other than Advertising. The expected drop in internet advertising will rapidly lose its value and its impact, for reasons that can easily be understood.
For the technically minded, Programming Pearls has a section on Generating Text that explains the concept. The bookmarklet uses an Order-2 word-level Markov chain. Translated into English, what that means is: I look at every pair of words in and find out what word is likely to follow that.
For example, in the Generating Text page, the pair of words “we can” are followed by the words “extend”, “also”, “get” and “write” with equal probability. We pick one randomly (say “also”) and write “we can also”. Then we look at the word pair “can also”, see what word follows that, pick one at random, and so on.
This is Order-2 because we pick pairs of words. And it’s word-level rather than letter-level because we use words instead of letters as the basic building blocks.
When you’re trying it out, make sure that the page is large enough. If not, you may find that the page’s content is reproduced verbatim.
The bookmarklet is built on top of the excellent Readabilitybookmarklet by Arc90, which helps identify the main content to be randomized.
Get the data in a tabular format with the columns [date, item, x, y, size]
Make a “today” cell, and create a lookup table for “today”
Make a bubble chart with that lookup table
Add a scroll bar and a play button linked to the “today” cell
For the impatient, here’s a motion chart spreadsheet that you can tailor to your needs. For the patient and the puzzled, here’s a quick introduction to bubble and motion charts.
What is a bubble chart?
A bubble chart is a way of capturing 3 dimensions. For example, the chart below could be the birth, literacy rate and population of countries (X-axis, Y-axis and size). Or the growth, margin and market cap of companies.
It lets you compare three dimensions at a glance. The size dimension is a different from the X and Y axes, though. It’s not easy to compare differences in size. And the eye tends to focus on the big objects. So usually, size is used highlight important things, and the X and Y axes used to measure the performance of these things.
If I were to summarise bubble charts in a sentence, it would be: bubble charts show the performance of important things (in two dimensions). (In contrast, Variwide charts show the same on one dimension.)
Say you’re a services firm. You want to track the productivity of your most expensive groups (“the important things”). Productivity is measured by 2 parameters: utilisation and margin. The bubble chart would then have the expense of each group as the size, and its utilisation and contribution as the X and Y axes.
What is a motion chart?
Motion charts are animated bubble charts. They track the performance of important things over time (in two dimensions). This is chart with 4 dimensions. But not all data with 4 dimensions can be plotted as a motion chart. One dimension has to be time, and another has to be linked to the importance of the item.
Motion charts were pioneered by Hans Rosling and his TED Talk shows you the true power of motion charts.
This is a simple Insert – Chart. Go through the chart types and select bubble. Play around with the data selection until you get the X, Y and Size columns right.
4. Add a scroll bar and a play button linked to the “today” cell
Now for the magic. Add a scroll bar below the chart. Excel 2007 users: Go to Developer – Insert and add a scroll bar. Excel 2003 users: Go to View – Toolbars – Control Toolbox and add a scroll bar
Right click on the scroll bar, go to Format Control… and link the scroll bar to the “Offset” cell. Now, as you move the scroll bar, the value in the offset cell will change to reflect it. So the “today” cell will change too. So will the lookup table. And so will the chart.
Next, create a button called “Play” and edit its code. Excel 2007 users: Right click the button, go to Developer – View Code. Excel 2003 users: Right click the button and select View Code.
Type in the following code for the button’s click event:
DeclareSub Sleep Lib "kernel32" (ByVal dwMilliseconds AsLong)
Dim i AsIntegerFor i = 0 To 40: ' Replace 40 with your range
Range("J1").Value = i ' Replace J1 with your offset cell
Now clicking on the Play button will give you this glorious motion chart in Excel:
It’s been a good movie month for me, and I’ve managed to nudge closer to my target of watching the IMDb Top 250.
But one tool I had in the past, that I sorely miss, is twofifty.org. It’s a now-defunct site that kept track of the IMDb Top 250, and let you strike off the movies that you had watched. You could see which movies you hadn’t seen, keep score, and discuss the movies.
Since it’s demise, my movie watching slowed down as well.
Earlier this month, I set up a similar site at 250.s-anand.net. It has the same basic function. You can log in, strike out movies that you’ve seen, and keep track of what’s left to see. For the more technically minded, the source-code is at two-fifty.googlecode.com.
Wouldn’t it be cool to be able to search through the Dilbert archives using text?
This used to be possible at Dilbert.com some years ago, as a paid service. In late 2003, I needed to find some Dilbert strips for a client, so I’d subscribed for a year. I could then search for the quotes (I happened to be looking for "outsourcing", so you can guess the context).
But I can’t seem to find the feature any more, even as a paid service. The site looks a lot better, of course. But I can’t find strips.
This would be a much larger exercise, though. And I’m hoping to take your help. I’ve set up a site at dilbert-search.appspot.com. You can type in a comic randomly, starting from 2000. These will be made searchable on my Dilbert page. You can export the data and use it yourself, of course.
When typing in Calvin and Hobbes, I did have a few volunteers willing to pitch in, but collaboration tools weren’t easy to set up, and I ended up typing the whole thing myself. This time, I’d be delighted if even 10 people typed in just a strip each.
Yes. Google has a gadget called MotionChart that lets you do this.
Now, you could put this up on your web page, but that’s not quite useful when presenting to a client. (It is shocking, but there are many practical problems getting an Internet connection at a client site. The room doesn’t have a connection. The cable isn’t long enough. You can’t access the LAN. Their proxy requires authentication. The connection is too slow. Whatever.)
So you need this in Excel. Let me explain a variant of the technique I described earlier.
Let’s start by creating a simple bubble chart.
For each item in a bubble chart, you need 3 pieces of data: the X-axis, Y-axis and size. This graph shows three items A, B and C in one year: 2001. To animate this, you need data for more years, so let’s create that.
The first 3 rows contain the same data as before, except that I’ve added a "Year" column and a "Key" column (which is just a concatenation of the Year and the Item). The data now goes on for many more years.
Now we need to create a scroll bar that can be used to change the year. So add a scroll bar below the bubble chart…
… and right click the scroll bar and go to Format Control. Now, select the cell link to some cell ($H$1 in this case). Now, if you move the scroll bar, the cell value will change.
All you need to do is to now change the source data for the chart based on the year. From the table on the left, VLOOKUP the year + item, and put this into the table on the right. When the year in the cell H1 changes, the data updates itself. So now, as you move the scroll bar, cell H1 changes, then so does the data and hence the graph.
I read about Google AppEngine early this morning, and applied for an invite. Google’s issuing beta invites to the first 10,000 users. I was pretty convinced I wasn’t among those, but turns out I was lucky.
AppEngine lets you write web apps that Google hosts. People have been highlighting that it give you access to the Google File System and BigTable for the first time. But to me, that isn’t a big deal. (I’m not too worried about reliability, and MySQL / flat files work perfectly well for me as a data store.)
What’s more interesting unlike Amazon’s EC2 and S3, this is free up to a certain quota. And you get a fair bit of processing power and bandwidth for free. One of the reasons I’ve held back on creating some apps was simply because it would take away too much bandwidth / CPU cycles from my site. (I’ve had this problem before.) Google quota is 10 GB of bandwidth per day (which is about 30 times what my site uses). And this is on Google’s incrediblyfast servers It also offers 200 million megacycles a day. That’s like a dedicated 2.3 GHz processor (200 million megacycles = 200,000 GHz x 1 second ~ 2.3 GHz x 86,400 seconds/day) — better, because this is the average capacity, not peak capacity. The only restriction that really worries me is that only 3 apps are allowed per developer.
So I decided to give a shot at publishing some code I’d kept in reserve for a long time. You may remember my statistical analysis of Calvin & Hobbes. For this, I’d created a script in Perl that could generate Statistically Improbable Phrases (SIPs) for any text. This is based on (a somewhat limited) 23MB corpus of ebooks that I had. I’d wanted to put that up on my website, but …
AppEngine only uses Python. So the first task was to get Python, and then to learn Python. The only saving grace was that I was just cutting-and-pasting most of the time. Google wasn’t helping:
Anyway, the site is up. You can view it at sip.s-anand.net for now. Just type a URL, and it’ll tell you the improbable words in that site.
I realise that these are statistically improbable words, not phrases. I’ll get to the phrases in a while.
The logic is simple:
Get the frequency of words in a corpus. I pre-generated this file. It has over 100,000 words.
Get the URL as text. Rather than muck around with Python, I decided to use the W3 html2txt service.
Convert the text to words. Splitting text into words is tricky. For now, I’m simply assuming that any group of letters is a word, and anything that’s not a letter is a word delimiter.
Find the relative frequency (improbability) of words. This is the frequency in the URL divided by the frequency in the corpus, normalised (i.e. scale it so that the maximum value is 1.0).
Create a tag cloud. I use the word frequency as the size and the improbability as the colour. You need a bit of mathematical jugglery to get the pattern right. Right now, I’m taking the 6th root of the improbability and the logarithm of the frequency to get a reasonably smooth tag cloud.