<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/">
  <channel>
    <title>amazon-ec2 on S Anand</title>
    <link>https://www.s-anand.net/blog/tag/amazon-ec2/</link>
    <description>Recent content in amazon-ec2 on S Anand</description>
    <generator>Hugo -- 0.156.0</generator>
    <language>en-us</language>
    <lastBuildDate>Mon, 03 Jun 2013 07:59:56 +0000</lastBuildDate>
    <atom:link href="https://www.s-anand.net/blog/tag/amazon-ec2/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>The scary Internet</title>
      <link>https://www.s-anand.net/blog/the-scary-internet/</link>
      <pubDate>Mon, 03 Jun 2013 06:29:32 +0000</pubDate>
      <guid>https://www.s-anand.net/blog/the-scary-internet/</guid>
      <description>&lt;p&gt;I’m not that difficult to scare, and this log message certainly didn’t help:&lt;/p&gt;
&lt;pre tabindex=&#34;0&#34;&gt;&lt;code&gt;ip223.hichina.com [223.4.183.127] failed - POSSIBLE BREAK-IN ATTEMPT!
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;That’s the message I saw – one thousand five hundred and seventy times yesterday in /var/log/auth.log on one of my Amazon EC2 instances.
Someone, presumably from China, has been patiently trying out a variety of SSH keys to log into this system.
These were grouped as batches. There were exactly 314 attempts at 8am yesterday, then 314 at 12noon, then 314 at 4pm, then 314 at 8pm, then 232 at 3am today. (All times are in UTC – that is, UK time without daylight saving). Every burst took 9 minutes to run through all 314 attempts.
The worst part was, when I tried using SSH this morning, I wasn’t able to log in. (It turned out that I had made a configuration error, but this is the sort of thing that gets me quite worried.)
Perhaps I shouldn’t be complaining. I’ve written enough scrapers to make most webmasters cringe at their logs. I remember a few years ago, when I was working on a project at Tesco, and was scraping bestsellers lists from most sites. (Here’s a &lt;a href=&#34;https://www.s-anand.net/blog/dear-tesco-your-books-are-expensive/&#34;&gt;blog post&lt;/a&gt; about it.) We were putting together a prototype to see how real-time competitive pricing could help.
The scraper was a pretty mild one. It would visit a hundred links, roughly at the pace of one a second. No images were loaded, of course, just the HTML.
One fine day, a few weeks after this had started, I got a call from Andy.
“Hi Anand, are you running any scrapers on our books website?”
“Yes, why?”
“Oh! The site’s very slow. Could you shut it down immediately?”
Turns out that not a single page on the site loaded, and it had almost crawled to a halt. Now, obviously, my little 100-page script could hardly cause damage, but it’s easy to understand their reactions. &lt;strong&gt;No unauthorised scraping!&lt;/strong&gt; After a few days of trying to figure out what the problem was, they increased the memory and things went back to normal. Not a bad solution, actually – throw hardware at the problem, and if it vanishes, it’s probably the cheapest solution.
But anyway, I’m sure it’s some nice chap who’s just curious to know what I’ve got on my servers. I’d be happy to share some of it. And even if it’s not so nice a chap, there’s little that I can do, is there?
&lt;strong&gt;Update (1pm India, 3rd June)&lt;/strong&gt;: Actually, I now realise that this has been happening ever four hours since May 29th, as regular as a clockwork. Wish I knew enough UNIX programming to pull a prank&amp;hellip;&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&#34;comments&#34;&gt;Comments&lt;/h2&gt;
&lt;!-- wp-comments-start --&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;&lt;a href=&#34;http://www.allassignmenthelp.com&#34;&gt;Rajeev&lt;/a&gt;&lt;/strong&gt; &lt;em&gt;23 Dec 2013 2:54 pm&lt;/em&gt;:
The unwanted trial to login and break in very common on the blogs and the websites that provides a login account. I had many such problems initially and I block such IP&amp;rsquo;s every time once I notice there is something going on which is not expected. I loved your blog.
Thanks
Rajeev&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;a href=&#34;http://amit.chakradeo.net/&#34;&gt;Amit Chakradeo&lt;/a&gt;&lt;/strong&gt; &lt;em&gt;3 Jun 2013 8:38 pm&lt;/em&gt;:
That is not necessarily an actual break-in attempt. SSH prints that if the reverse name lookup on the ip address does not resolve back to the same IP address.
Some pointers to address these things:
&lt;ol&gt;
&lt;li&gt;Run ssh on non-standard port. (this kills 90% of people trying dictionary passwords out for root).&lt;/li&gt;
&lt;li&gt;Prevent root login for ssh and disable password authentication (use keys only)&lt;/li&gt;
&lt;li&gt;Run denyhosts which denies known bad IP&amp;rsquo;s and hosts trying to exploit your box.&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Gopal&lt;/strong&gt; &lt;em&gt;3 Jun 2013 8:06 am&lt;/em&gt;:
Now you know why US is getting angry with china! :-)
Anyways as you know it might be to put in some malware for DDOS bot attacks?&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;a href=&#34;http://thejeshgn.com&#34;&gt;Thejesh GN&lt;/a&gt;&lt;/strong&gt; &lt;em&gt;4 Jun 2013 10:23 am&lt;/em&gt;:
I use DenyHosts (&lt;a href=&#34;http://denyhosts.sourceforge.net/&#34;&gt;http://denyhosts.sourceforge.net/&lt;/a&gt;) on all my servers. It blacklists ip addresses trying to brute force. Its easy to setup and run.
I can see that the script blacklists at least one IP everyday.&lt;/li&gt;
&lt;/ul&gt;
&lt;!-- wp-comments-end --&gt;
</description>
    </item>
    <item>
      <title>Hosting options</title>
      <link>https://www.s-anand.net/blog/hosting-options/</link>
      <pubDate>Sat, 01 Jun 2013 05:48:55 +0000</pubDate>
      <guid>https://www.s-anand.net/blog/hosting-options/</guid>
      <description>&lt;p&gt;I&#39;ve been trying out a number of options for hosting recently, and have settled on &lt;a href=&#34;http://aws.amazon.com/ec2/spot-instances/&#34;&gt;Amazon spot instances&lt;/a&gt;.&lt;/p&gt; &lt;p&gt;Here were my options:&lt;/p&gt; &lt;ul&gt; &lt;li&gt;Application hosting, like &lt;a href=&#34;http://appengine.google.com/&#34;&gt;Google AppEngine&lt;/a&gt;. I used this a lot until 2 years ago. Then they changed their pricing, and I realised what “lock-in” means. I can’t just take that code and move it to another server. Besides, I’m a bit wary of &lt;a href=&#34;https://www.s-anand.net/blog/goodbye-google/&#34;&gt;Google pulling the plug&lt;/a&gt;. &lt;a href=&#34;https://www.heroku.com/&#34;&gt;Heroku&lt;/a&gt;? Same problem. I just want to take the code elsewhere and run it.  &lt;/li&gt;&lt;li&gt;Shared hosting, like &lt;a href=&#34;http://www.hostgator.com/&#34;&gt;Hostgator&lt;/a&gt;. &lt;a href=&#34;http://www.s-anand.net/&#34;&gt;This blog&lt;/a&gt; is run on Hostgator and I’m extremely happy with them. But the trouble is, with shared hosting, I don’t get to run long-running processes on any ports I like.  &lt;/li&gt;&lt;li&gt;Run you own servers. The problem here is quite simple: power cuts in India.  &lt;/li&gt;&lt;li&gt;Dedicated hosting, like &lt;a href=&#34;http://aws.amazon.com/ec2/&#34;&gt;Amazon EC2&lt;/a&gt;, &lt;a href=&#34;http://www.windowsazure.com/en-us/&#34;&gt;Azure&lt;/a&gt;, &lt;a href=&#34;https://cloud.google.com/products/compute-engine&#34;&gt;GCE&lt;/a&gt;, etc. This remains as pretty much the main hosting option&lt;/li&gt;&lt;/ul&gt; &lt;p&gt;I’m a price optimisation freak. So I ran the numbers for a year’s worth of usage. I was looking at the CPU cost of a large machine with 7-8GB RAM. Bandwidth and storage are negligible. The cost per hour worked out to:&lt;/p&gt; &lt;ul&gt; &lt;li&gt;Amazon: $0.32 / hr in Singapore, $0.24 in Virginia  &lt;/li&gt;&lt;li&gt;Google: $0.29 / hr in Europe  &lt;/li&gt;&lt;li&gt;Microsoft: $0.32 / hr in US&lt;/li&gt;&lt;/ul&gt; &lt;p&gt;The price is not all that different, but I need &lt;a href=&#34;http://cloudharmony.com/speedtest&#34;&gt;low latency&lt;/a&gt;, so Singapore it what it’ll have to be.&lt;/p&gt; &lt;table class=&#34;numbers lines&#34; style=&#34;color: #444&#34;&gt; &lt;thead&gt; &lt;tr&gt; &lt;th&gt;EC2 location&lt;/th&gt; &lt;th&gt;Latency (ms)&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt; &lt;tbody&gt; &lt;tr&gt; &lt;td&gt;Singapore&lt;/td&gt; &lt;td&gt;139&lt;/td&gt;&lt;/tr&gt; &lt;tr&gt; &lt;td&gt;Oregon, US&lt;/td&gt; &lt;td&gt;334&lt;/td&gt;&lt;/tr&gt; &lt;tr&gt; &lt;td&gt;Japan&lt;/td&gt; &lt;td&gt;517&lt;/td&gt;&lt;/tr&gt; &lt;tr&gt; &lt;td&gt;Ireland&lt;/td&gt; &lt;td&gt;618&lt;/td&gt;&lt;/tr&gt; &lt;tr&gt; &lt;td&gt;Australia&lt;/td&gt; &lt;td&gt;620&lt;/td&gt;&lt;/tr&gt; &lt;tr&gt; &lt;td&gt;California, US&lt;/td&gt; &lt;td&gt;677&lt;/td&gt;&lt;/tr&gt; &lt;tr&gt; &lt;td&gt;Virginia, US&lt;/td&gt; &lt;td&gt;710&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt; &lt;p&gt;Now comes the choice of the right model. At $0.32 per hour, that’s $230 a month.&lt;/p&gt; &lt;p&gt;Amazon offers some ways of getting this down. Instead of &lt;a href=&#34;http://aws.amazon.com/ec2/pricing/#on-demand&#34;&gt;on-demand instances&lt;/a&gt;, I could go for &lt;a href=&#34;http://aws.amazon.com/ec2/pricing/#reserved&#34;&gt;reserved instances&lt;/a&gt;. For a year of usage, that’d get the price down to about $131 a month, nearly halving it. ($739 upfront for a heavy utilisation large reserved instance, with $0.095 * 24 * 365.25 for the year.)&lt;/p&gt; &lt;p&gt;In this case, I know I’ll need the servers for a year. Probably more, but then, I might want to switch later. So this isn’t a bad move. But we can do better. Amazon also offers &lt;a href=&#34;http://aws.amazon.com/ec2/pricing/#spot&#34;&gt;spot instances&lt;/a&gt;. Spot instances might get shut down any time – but in reality, so can on-demand instances. I need to plan for it anyway. I’m not going to host anything that’s so sensitive that if it’s down for a few hours, I’ll have a problem.&lt;/p&gt; &lt;p&gt;But what’s attractive is the pricing. Typically, it’s $0.04 per hour, making it about $29 per month. Even if it shoots up to twice that, at $58, it’s less than a fourth of the on-demand price and less than half the reserved instance price.&lt;/p&gt; &lt;p&gt;I’ve managed to script the entire setup up sequence as shell scripts, and it takes less than an hour to get a new server up and running the software I need. I need to work out a decent backup mechanism. Plus, I could use more reliable storage like like &lt;a href=&#34;http://aws.amazon.com/ebs/&#34;&gt;Amazon’s EBS&lt;/a&gt; to preserve the data. But on the whole, the pricing is far too attractive and makes the risks worthwhile.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&#34;comments&#34;&gt;Comments&lt;/h2&gt;
&lt;!-- wp-comments-start --&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;&lt;a href=&#34;http://anandology.com/&#34;&gt;Anand Chitipothu&lt;/a&gt;&lt;/strong&gt; &lt;em&gt;2 Jun 2013 3:36 pm&lt;/em&gt;:
Did you look at Linode? I&amp;rsquo;ve been using Linode for couple of years and pretty happy with it. The 8GB RAM model costs about $160 per month. Sadly, they don&amp;rsquo;t have a data center in Singapore. You can test the download speeds and latency of their data centers from &lt;a href=&#34;https://www.linode.com/speedtest/&#34;&gt;https://www.linode.com/speedtest/&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;a href=&#34;http://www.bestdoorlocksets.com&#34;&gt;jouko&lt;/a&gt;&lt;/strong&gt; &lt;em&gt;7 Jul 2013 4:21 pm&lt;/em&gt;:
I am using Inmotion hosting for my blog. May be i should also shift to amazon cloud.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;a href=&#34;http://www.s-anand.net/&#34;&gt;S Anand&lt;/a&gt;&lt;/strong&gt; &lt;em&gt;3 Jun 2013 7:30 am&lt;/em&gt;:
I did try Linode early on, and it&amp;rsquo;s a fairly decent option. It&amp;rsquo;s just that for my needs, the $29/month that I can get with Amazon is too attractive.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Amit Chakradeo&lt;/strong&gt; &lt;em&gt;3 Jun 2013 8:33 pm&lt;/em&gt;:
If you don&amp;rsquo;t mind occasional downtimes and occasional ummm complete loss of data :-), you can consider low end VPS boxes. (&lt;a href=&#34;http://lowendbox.com/&#34;&gt;http://lowendbox.com/&lt;/a&gt;) You can find some decent deals if you check out the posts there&amp;hellip;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Sriram&lt;/strong&gt; &lt;em&gt;15 Jan 2014 4:18 pm&lt;/em&gt;:
have u tried openshift ?&lt;/li&gt;
&lt;/ul&gt;
&lt;!-- wp-comments-end --&gt;
</description>
    </item>
    <item>
      <title>Faster data crunching</title>
      <link>https://www.s-anand.net/blog/faster-data-crunching/</link>
      <pubDate>Fri, 23 Sep 2011 18:20:10 +0000</pubDate>
      <guid>https://www.s-anand.net/blog/faster-data-crunching/</guid>
      <description>&lt;p&gt;I’ve been playing with big data lately.&lt;/p&gt; &lt;p&gt;The good part is, it’s easy to get interesting results. The data is so unwieldy that even average value calculations provoke a “Amazing! I didn’t know that,” response (No exaggeration. I heard this from two separate ~ $1bn businesses this month.)&lt;/p&gt; &lt;p&gt;The bad part is that calculating even that simple average is slow.&lt;/p&gt; &lt;p&gt;For example, take this &lt;a href=&#34;https://files.s-anand.net/blog/a/school_10.rpt.bz2&#34;&gt;40MB file&lt;/a&gt; (380MB unzipped) and extract the first column.&lt;/p&gt; &lt;p&gt;The simplest Python script to get the first column looks like this:&lt;/p&gt;
```python
for row in csv.reader(fileinput.input(), delimiter=&#39;\t&#39;):
    if len(row) &gt; 0: print row[0]
```
&lt;p&gt;That took a good 3 minutes to execute on my laptop.&lt;/p&gt;
&lt;p&gt;Since I’m used to &lt;a href=&#34;http://en.wikibooks.org/wiki/Ad_Hoc_Data_Analysis_From_The_Unix_Command_Line&#34;&gt;UNIX data processing&lt;/a&gt;, I tried &lt;code&gt;cut -f1&lt;/code&gt;. Weirdly, that’s worse. 5 minutes. Paradoxically, &lt;/code&gt;awk &#39;{print $1}&#39;&lt;/code&gt; only takes 17 seconds. That&#39;s about 12 times faster. Clearly the tool makes a big difference. And we always knew &lt;a href=&#34;http://swtch.com/~rsc/regexp/regexp1.html&#34;&gt;UNIX&lt;/a&gt; was &lt;a href=&#34;http://lists.freebsd.org/pipermail/freebsd-current/2010-August/019310.html&#34;&gt;fast&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;But I also ran these on an &lt;a href=&#34;http://aws.amazon.com/ec2/&#34;&gt;Amazon EC2&lt;/a&gt; server, and a &lt;a href=&#34;http://www.hostgator.com/&#34;&gt;Hostgator&lt;/a&gt; server. Here’re the results.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&amp;nbsp;&lt;/th&gt;
&lt;th&gt;&lt;strong&gt;python&lt;/strong&gt;&lt;/th&gt;
&lt;th&gt;&lt;strong&gt;cut&lt;/strong&gt;&lt;/th&gt;
&lt;th&gt;&lt;strong&gt;awk&lt;/strong&gt;&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;a href=&#34;http://www.dell.com/us/business/p/latitude-e5400/pd&#34;&gt;My Dell E5400&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;3:04 (&lt;b&gt;1x&lt;/b&gt;)&lt;/td&gt;
&lt;td&gt;5:42 (&lt;b&gt;0.5x&lt;/b&gt;)&lt;/td&gt;
&lt;td&gt;0:17 (&lt;b&gt;11x&lt;/b&gt;)&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;a href=&#34;http://aws.amazon.com/ec2/#instance&#34;&gt;EC2 standard&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;0:33 (&lt;b&gt;6x&lt;/b&gt;)&lt;/td&gt;
&lt;td&gt;0:5.6 (&lt;b&gt;33x&lt;/b&gt;)&lt;/td&gt;
&lt;td&gt;0:16 (&lt;b&gt;11x&lt;/b&gt;)&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;a href=&#34;http://support.hostgator.com/articles/hosting-plans/server-specifications-specs&#34;&gt;Hostgator&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;0:19 (&lt;b&gt;10x&lt;/b&gt;)&lt;/td&gt;
&lt;td&gt;0:2.5 (&lt;b&gt;74x&lt;/b&gt;)&lt;/td&gt;
&lt;td&gt;0:0.7 (&lt;b&gt;265x&lt;/b&gt;)&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;What took 3 minutes with Python my Dell E5400 took &lt;em&gt;less than a second&lt;/em&gt; on Hostgator’s server with awk. Over &lt;em&gt;250 times&lt;/em&gt; faster. (Not 250%. 250 &lt;em&gt;times&lt;/em&gt;). &lt;/p&gt;
&lt;p&gt;And it’s not just hardware. A good tool (awk) made things 11x faster on my machine. Good hardware (hostgator) made the same program 10x faster. But choosing the right combination can make things go faster than 11 x 10 = 110 times. Much faster.&lt;/p&gt;
&lt;p&gt;There are a few of things I’m taking away from this.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Good hardware can speed you up much as (or more than) choosing the right tool.&lt;/li&gt;
&lt;li&gt;Good hardware can be rented. From many places. Cheaply.&lt;/li&gt;
&lt;li&gt;Always test what’s fast. awk’s fastest on my machine and Hostgator, but not on EC2.&lt;/li&gt;&lt;/ol&gt;
</description>
    </item>
    <item>
      <title>SSH Tunneling through web filters</title>
      <link>https://www.s-anand.net/blog/ssh-tunneling-through-web-filters/</link>
      <pubDate>Sat, 09 Jan 2010 19:46:39 +0000</pubDate>
      <guid>https://www.s-anand.net/blog/ssh-tunneling-through-web-filters/</guid>
      <description>&lt;p&gt;You can defeat most &lt;a href=&#34;http://www.google.co.in/search?q=web+filter&#34;&gt;web filters&lt;/a&gt; by spending &lt;del&gt;&lt;a href=&#34;http://aws.amazon.com/ec2/#pricing&#34;&gt;around 8 cents/hr&lt;/a&gt;&lt;/del&gt; &lt;a href=&#34;http://aws.amazon.com/free/&#34;&gt;0 cents/hr&lt;/a&gt; on Amazon EC2. (It’s usually worth the money. It’s a fraction of the cost a phone call or a sandwich. And I usually end up wasting that money anyway on calling someone or eating my way out of the misery of corporate proxies.)
Most web filters and proxies block all ports except the &lt;a href=&#34;http://en.wikipedia.org/wiki/Http&#34;&gt;HTTP port (80)&lt;/a&gt; and the &lt;a href=&#34;http://en.wikipedia.org/wiki/Https&#34;&gt;HTTPS port (443)&lt;/a&gt;. But it’s used to carry encrypted traffic, and, &lt;a href=&#34;http://proxytunnel.sourceforge.net/paper.php&#34;&gt;as Mark explains&lt;/a&gt;:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;since all the traffic that passed through the tunnel is supposed to be SSL encrypted (so as to form an unhindered SSL session between the browser and the HTTPS server), there are little or no access controls possible on such a tunnel&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;That means web filters can’t really block HTTPS traffic. So we can redirect web traffic to a local HTTPS server, and set up a server outside the firewall that redirects them back to the regular servers.
&lt;a href=&#34;http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html&#34;&gt;Putty&lt;/a&gt; will be our local HTTPS server. &lt;a href=&#34;http://aws.amazon.com/ec2/&#34;&gt;Amazon EC2&lt;/a&gt; gives us a server outside the firewall.
So here’s a 16-step recipe to bypass your web filter. (This is the simplest I could make it.)
In Steps 1-7, we’ll launch a server on Amazon EC2 with 2 tweaks. Step 1 enables Port 443, and step 6 re-configures SSH to run on Port 443 instead of on Port 22. (Remember: most proxies block all ports other than 80 and 443). &lt;a href=&#34;http://alestic.com/&#34;&gt;Alestic&lt;/a&gt;’s article on how to &lt;a href=&#34;http://alestic.com/2009/06/ec2-user-data-scripts&#34;&gt;Automate EC2 Instance Setup with user-data Scripts&lt;/a&gt; and this thread on &lt;a href=&#34;http://developer.amazonwebservices.com/connect/thread.jspa?messageID=126265&amp;amp;#126265&#34;&gt;running SSH on port 443&lt;/a&gt; are invaluable.
In Steps 8-13, we’ll set up Putty as our local HTTPS server. Read how to set up &lt;a href=&#34;http://digitalpbk.blogspot.com/2009/05/ssh-proxy-windows-linux-orkut-bypass.html&#34;&gt;Putty as a SOCKS server&lt;/a&gt; and how to use &lt;a href=&#34;http://meinit.nl/using-putty-and-an-http-proxy-to-ssh-anywhere-through-firewalls&#34;&gt;Putty with a HTTP proxy&lt;/a&gt;. All I did was to combine the two.
In steps 14-16, we’ll configure the browser to use the Putty as the SOCKS server.
&lt;strong&gt;Ingredients&lt;/strong&gt;&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;a href=&#34;http://aws.amazon.com/&#34;&gt;Amazon AWS account&lt;/a&gt; (sign up for free – you won’t be charged until you use it)&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html&#34;&gt;Putty&lt;/a&gt; (which may be available on your Intranet, if you’re lucky)&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;strong&gt;Directions&lt;/strong&gt;&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;On the &lt;a href=&#34;https://console.aws.amazon.com/ec2/&#34;&gt;AWS EC2 Console&lt;/a&gt;, click on &lt;strong&gt;Security Groups&lt;/strong&gt; and select the &lt;strong&gt;default&lt;/strong&gt; security group. At the bottom, select &lt;strong&gt;HTTPS&lt;/strong&gt; as the connection method, and save it.
&lt;a href=&#34;https://www.s-anand.net/blog/assets/ec2security.webp&#34;&gt;&lt;img loading=&#34;lazy&#34; src=&#34;https://www.s-anand.net/blog/assets/ec2security.webp&#34;&gt;&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Click on &lt;strong&gt;Key Pairs&lt;/strong&gt;, select &lt;strong&gt;Create Key Pair&lt;/strong&gt; and type in some name. Click on the &lt;strong&gt;Create&lt;/strong&gt; button and you’ll be asked to download a key file. Save it somewhere safe.&lt;a href=&#34;https://www.s-anand.net/blog/assets/ec2keypair.webp&#34;&gt;&lt;img loading=&#34;lazy&#34; src=&#34;https://www.s-anand.net/blog/assets/ec2keypair.webp&#34;&gt;&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Run PuttyGen (it comes with Putty), click &lt;strong&gt;Load&lt;/strong&gt; and select the key file you just saved. Now click on &lt;strong&gt;Save private key&lt;/strong&gt; and save it as &lt;strong&gt;privatekey.ppk&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;Back on the &lt;a href=&#34;https://console.aws.amazon.com/ec2/&#34;&gt;AWS EC2 Console&lt;/a&gt;, click on &lt;strong&gt;Launch Instance&lt;/strong&gt;.
&lt;a href=&#34;https://www.s-anand.net/blog/assets/ec20.webp&#34;&gt;&lt;img loading=&#34;lazy&#34; src=&#34;https://www.s-anand.net/blog/assets/ec20.webp&#34;&gt;&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Select &lt;strong&gt;Community AMIs&lt;/strong&gt; and find &lt;strong&gt;ami-ccf615a5&lt;/strong&gt;. It’s a Ubunty Jaunty 9.04 instance that&amp;rsquo;s been customised to run scripts passed as user-data. You may pick any other alestic instance. (The screenshot below picks a different instance. Ignore that.)
&lt;a href=&#34;https://www.s-anand.net/blog/assets/ec2launch.webp&#34;&gt;&lt;img loading=&#34;lazy&#34; src=&#34;https://www.s-anand.net/blog/assets/ec2launch.webp&#34;&gt;&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Continue until you get to &lt;strong&gt;Advanced Instance Options&lt;/strong&gt;. Here, copy and paste the following under &lt;strong&gt;User Data&lt;/strong&gt;. &lt;strong&gt;Do not make a mistake here!&lt;/strong&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;#!/bin/bash
mv /etc/ssh/sshd_config /etc/ssh/x
sed &amp;ldquo;s/^#?Port.*/Port 443/&amp;rdquo; /etc/ssh/x &amp;gt; /etc/ssh/sshd_config
/etc/init.d/ssh restart&lt;/p&gt;
&lt;pre tabindex=&#34;0&#34;&gt;&lt;code&gt;   [![](/blog/assets/ec2userdata.webp)](/blog/assets/ec2userdata.webp)
7. Keep pressing **Continue** and **Launch** the instance. Once launched, click on “Instances” on the left, and keep refreshing the page until the status turns green (running). Now, copy the **Public DNS** of the instance.
   [![](/blog/assets/ec2instancerunning.webp)](/blog/assets/ec2instancerunning.webp)
8. Run Putty. Type in **root@****&amp;lt;the-public-DNS-you-just-copied&amp;gt;** as the host name, and **443** as the port
   [![](/blog/assets/putty1.webp)](/blog/assets/putty1.webp)
9. Under **Connection &amp;gt; Proxy**, set **HTTP** as the proxy type. Type in the **Proxy hostname** and **Port** you normally use to access the Internet. Select **Yes** for **Do DNS name lookup at proxy end**. Type in your Windows login ID and password.
   [![](/blog/assets/putty2.webp)](/blog/assets/putty2.webp)
10. Under **Connection &amp;gt; SSH**, select **Enable Compression**.
    [![](/blog/assets/putty5.webp)](/blog/assets/putty5.webp)
11. Under **Connection &amp;gt; SSH &amp;gt; Auth**, click **Browse** and select the **privatekey.ppk** file you’d saved earlier.
    [![](/blog/assets/putty3.webp)](/blog/assets/putty3.webp)
12. Under **Connection &amp;gt; SSH &amp;gt; Tunnels**, type **9090** as the **Source port**, **Dynamic** as the **Destination**, and click **Add**.
    [![](/blog/assets/putty4.webp)](/blog/assets/putty4.webp)
13. Now click **Open**. You should get a terminal into your Amazon EC2 instance.
14. Open your Browser, and set the SOCKS server to localhost:9090. For Internet Explorer, go to **Tools – Options – Connections – LAN Settings**, select **Use a proxy ...**, click on **Advanced**, and type **localhost**:**9090** as the **Socks** server. Leave all other fields blank.
    [![](/blog/assets/ieconfig.webp)](/blog/assets/ieconfig.webp)
15. For Firefox, go to **Tools – Options – Advanced – Network – Settings** and select **Manual proxy configuration**. Set the Socks Host to **localhost**:**9090** and leave all other fields blank.
    [![](/blog/assets/ffconfig.webp)](/blog/assets/ffconfig.webp)
16. Also, go to URL **about:config**, and make sure that **network.proxy.socks\_remote\_dns** is set to **true**.

That’s it. You should now be able to check [most blocked sites](http://www.moon-blog.com/2009/02/top-ten-most-blocked-websites.html) like [Facebook](http://www.facebook.com/) and [YouTube](http://www.youtube.com/).
Those who favour the command line may want to automate Steps 1-7 by downloading Amazon’s [EC2 API tools](http://developer.amazonwebservices.com/connect/entry.jspa?externalID=351). [EC2 API tools work from behind a proxy too](http://developer.amazonwebservices.com/connect/message.jspa?messageID=52178). The commands you’ll need to use to setup are:
`set EC2_HOME=your-ec2-home-directory
set EC2_CERT=your-ec2-certificate
set EC2_PRIVATE_KEY=your-ec2-private-key
ec2-add-keypair mykeypair
ec2-authorize default -p 443
set EC2_JVM_ARGS=-DproxySet=true -DproxyHost=yourproxy \
-DproxyPort=yourport -Dhttps.proxySet=true \
-Dhttps.proxyHost=yourproxy -Dhttps.proxyPort=yourport \
-Dhttp.proxyUser=yourusername -Dhttps.proxyUser=yourusername \
-Dhttp.proxyPass=yourpassword -Dhttps.proxyPass=yourpassword
ec2-run-instances ami-ccf615a5 --key mykeypair --user-data-file your-startup-file-containing-lines-in-step-6`
You can go further and use any software (such as Skype) if you install [FreeCap](http://www.freecap.ru/eng/). More details are in this article on [Secure Firefox and IM with Putty](http://thinkhole.org/wp/2006/05/10/howto-secure-firefox-and-im-with-putty/).
Linux users may want to check out [ProxyTunnel](http://proxytunnel.sourceforge.net/) and this article on [Tunneling SSH over HTTP(S)](http://dag.wieers.com/howto/ssh-http-tunneling/).
**Update**: Follow-ups on [hacker news comments](http://news.ycombinator.com/item?id=1043413), [twitter](http://tweetmeme.com/story/427812352/ssh-tunneling-through-web-filters-s-anandnet), [delicious](http://delicious.com/url/0a6b39e211f481515ae02cab92cec1e7) and [digg](http://digg.com/security/SSH_Tunneling_through_web_filters_s_anand_net).

---

## Comments

&amp;lt;!-- wp-comments-start --&amp;gt;
- **[SSH Tunneling via Rackspacecloud | s-anand.net](http://www.s-anand.net/blog/ssh-tunneling-via-rackspacecloud/)** _11 Feb 2010 5:42 pm_ _(pingback)_:
  [...] Tunneling via Rackspacecloud February 11th, 2010 How I do things S Anand I wrote about SSH Tunneling through web filters using Amazon’s EC2 at 8 cents/hr. With Rackspacecloud, you can get that down to 1.5 cents/hr. [...]
- **[Janitha](http://www.janitha.com)** _10 Jan 2010 7:36 pm_:
  Good writeup, ssh tunnels are something I can&amp;#39;t live without...
  Step 9 can be skipped completely if no proxy is needed to be configured.
  Also don&amp;#39;t forget, doing all of this still sends the DNS requests in the clear to the usual/old dns server and not through EC2. If the DNS server is also meant to filter and redirect, this can be a issue. To go around that, in firefox you can go to about:config and set network.proxy.socks\_remote\_dns = true
- **[Janitha](http://www.janitha.com)** _10 Jan 2010 7:38 pm_:
  And for linux folks, you can simply run the ssh command with the -D switch.
- **[uberVU - social comments](http://www.ubervu.com/conversations/www.s-anand.net/blog/ssh-tunneling-through-web-filters/)** _10 Jan 2010 8:42 pm_ _(trackback)_:
  **Social comments and analytics for this post...**
  This post was mentioned on Twitter by proactivedefend: News Update: SSH Tunneling through web filters http://ow.ly/16iWFp...
- **[SSH Tunneling through web filters | s-anand.net &amp;amp;laquo; Netcrema &amp;amp;#8211; creme de la social news via digg + delicious + stumpleupon + reddit](http://www.netcrema.com/?p=24319)** _10 Jan 2010 10:16 pm_ _(pingback)_:
  [...] SSH Tunneling through web filters | s-anand.nets-anand.net [...]
- **[=== popurls.com === popular today](http://popurls.com/pop)** _10 Jan 2010 10:40 pm_ _(trackback)_:
  **=== popurls.com === popular today...**
  yeah! this story has entered the popular today section on popurls.com...
- **Kiril** _10 Jan 2010 10:52 pm_:
  Thanks for great tutorial. I have set it all up and it works!
  The only thing is that step 6 commands didn&amp;#39;t execute after I started new instance. I had to log in and execute each line from the terminal.
- **kyle** _11 Jan 2010 1:27 am_:
  If you have access to a webserver that has sshd running you can do this as well. You use the &amp;#39;D&amp;#39; flag with ssh for dynamic port forwarding. Its the same thing you are doing here just without aws. I use:
  ssh -D 1080 username@site.com and set firefox to use a socks proxy localhost 1080
- **[SSH Tunneling through web filters | s-anand.net : Popular Links : eConsultant](http://popular.econsultant.com/ssh-tunneling-through-web-filters-s-anand-net/)** _11 Jan 2010 2:43 am_ _(pingback)_:
  [...] rest is here: SSH Tunneling through web filters | s-anand.net 10 January 2010 | Uncategorized | Trackback | del.icio.us | Stumble it! | View Count : 0 Next [...]
- **dataminer** _11 Jan 2010 5:29 pm_:
  Or you can use a cheap linode / slicehost / prgmr virtual server and openvpn
- **spif** _11 Jan 2010 8:09 pm_:
  Good endpoint security can prevent this sort of thing, and even more sophisticated methods. But many organizations don&amp;#39;t implement endpoint security at all, and others don&amp;#39;t block or even monitor for this sort of thing.
- **Andrew** _10 Jan 2010 6:50 pm_:
  You should make sure in Firefox to set &amp;#34;network.proxy.socks\_remote\_dns&amp;#34; to TRUE under about:config. Otherwise, DNS requests will not be passed through your proxy and it will still be pretty obvious which sites you&amp;#39;re browsing.
- **[S Anand](http://www.s-anand.net/)** _29 Jan 2010 8:34 am_:
  Thanks Coop, I fixed it. (Actually, I changed it to &amp;#34;/etc/init.d/ssh restart&amp;#34; which works for me. Haven&amp;#39;t tried reload.)
- **Coop** _28 Jan 2010 4:04 pm_:
  FYI. There&amp;#39;s a typo in the instructions on step 6 for those of you trying to get this to work.
  The final line of the command reads:
  /etc/init.d/sshd reload
  It should read:
  /etc/init.d/ssh reload
  Change that and it&amp;#39;ll work. Pretty speedy too. The only downside to this is the need to constantly stop and start instances because you&amp;#39;re cheap like me. Wish I could somehow trigger a new instance and terminate it after a set amount of time whilst updating the public DNS entry in putty. Un-likely.
- **[TTA](http://www.tarunactivity.com)** _18 Jan 2010 3:49 pm_:
  Good .. except for the fact that ~most if not all offices would forbid tunnelling for this very reason :P ...
- **[&amp;amp;raquo; How to get through your office&amp;amp;#8230; Thej Live](http://thej.in/?p=4583)** _18 Jan 2010 4:02 pm_ _(pingback)_:
  [...] to get through your office proxy using AWS http://www.s-anand.net/blog/ssh-tunneling-through-web-filters/ [...]
- **[S Anand](http://www.s-anand.net/)** _3 Feb 2010 5:26 pm_:
  Incidentally, with Rackspace Cloud&amp;#39;s 1.5 cents per hour pricing, this becomes significantly more attractive.
- **anderbill** _19 Jan 2010 11:05 am_:
  I use ssh tunnel easy, get it from http://www.networktunnel.net
  Ssh Tunnel Easy is an innovative ssh tunneling software, it can make an encrypted ssh tunnel between your machine and ssh server host, then tunnel your program TCP connection automatically through this encrypted tunnel to data forwarded. It help you unblock and surf securely in the internet.
  Main features
  All in One
  A simple all in one solution, you do not need a complex sockscap/firefox + autoproxy + myentunnel + putty combination, and only a few mouse click, all config completed. Auto reconnect, support https proxy and NTLM authentication.
  Multi channel load balancing
  Under normal circumstances, because the SSH server is limiting the number of simultaneous connections, if you have too much TCP concurrent connections in one tunnel, may be cause your SSH tunnel freeze. Therefore, Ssh Tunnel Easy use similar to IE&amp;#39;s LCIE (Loosely-Coupled IE) multi-channel load-balancing technology to improve it. In other words, Ssh Tunnel Easy will automatically create multiple ssh tunnel, then your browser&amp;#39;s tcp connections will automatically be distributed to each ssh channel average, so that each channel ssh connections will not be too much, solve this problem perfect, and it can also significantly speed up your browsing speed.
- **Morbius** _14 Feb 2010 12:28 am_:
  I have read with interest the articles related to SSH Tunneling (through Web Filters and RackSpaceCloud) published on this web site.
  The problem I am trying to solve is slightly different, but I believe that someone might be able to guide me towards a solution.
  DESCRIPTION OF MY PROBLEM
  I would like to connect to several multi-media web sites located in several countries (namely France, Netherlands and Germany) and access some of the contents present on those sites. However, I am located in another country (I am based in Asia), and the problem is that those websites filter the access to their contents based on location: for instance, the French web site only allows access to its contents to users located in France. The filtering seems to be done via the IP address.
  Are their any tools that I could use and which could allow me to bypass this filtering, by making me appear like someone being based in the said country (for instance, in the example given above, France) ? I imagine that I should be able to select the country I want to appear being in.
  Could someone kindly help me and guide me towards a possible solution to my problem ?
  Thanks for reading, and I would just like to add that I use a client platform based on Windows 7.
- **[S Anand](http://www.s-anand.net/)** _14 Feb 2010 4:56 pm_:
  @Morbius: Not sure... you&amp;#39;d need a server in each of those countries. EC2 has a Europe instance that&amp;#39;s based in Dublin. Not sure which cloud vendors have servers in those places, though.
- **[S Anand](http://www.s-anand.net/)** _5 Mar 2010 2:24 pm_:
  http://www.mtu.net/~engstrom/ssh-proxy.php explains how to use corkscrew on Linux machines to use SSH via a proxy.
- **Morbius** _18 Feb 2010 1:11 pm_:
  Thank you for your answer, S.Anand.
  Do you think that using the Tor Network (I am afraid it might be a little slow, given that the data I need to access are video files...) or XeroBank (heard of it, but never used it...) could help?
- **[Ravi Atluri](http://blog.raviatluri.in)** _12 Nov 2010 5:19 pm_:
  I am having some problems with proxy authentication while SSH tunneling.
  Did you face any issue with ISA authentication ?
- **[Using sshuttle to remain Safe on Insecure Wi-Fi | Thejesh GN](http://thejeshgn.com/2012/06/28/using-sshuttle-to-remain-safe-on-insecure-free-wifi/)** _28 Jun 2012 12:57 pm_ _(pingback)_:
  [...] If you are on Windows try Anand’s method. [...]
- **[It&amp;amp;#8217;s Raining min()!: Running SEASR&amp;amp;#8217;s MEANDRE software as an Amazon Cloud Instance | devolution](http://www.devingriffiths.com/archive/2011/06/its-raining-min-running-seasrs-meandre-software-as-an-amazon-cloud-instance/)** _29 Jun 2011 6:07 pm_ _(pingback)_:
  [...] path viable.  But in the short term, all I needed to do was bind those ports on my local PC to an ssh tunnel, and configure my Firefox browser to use the linux box as a proxy using the ssh tunnel. [...]
- **[How to remain anonymous online | Thejesh GN](http://thejeshgn.com/2012/04/10/howto-remain-anonymous-online/)** _10 Apr 2012 9:23 am_ _(pingback)_:
  [...] is similar to the one explained in this blog title Tor on SSH. Its not very difficult to build an SSH tunnel yourself. [g] SSHing over Tor is very [...]
- **[Nikolay Kolev](http://nikolay.com)** _4 Dec 2011 8:21 pm_:
  I&amp;#39;ve been using Hamachi for the same purposes (now part of LogMeIn) and, just in case, I SSH over Hamachi&amp;#39;s connection: [LogMeIn Hamachi](https://secure.logmein.com/products/hamachi/).
- **[Software for my new laptop 2 | s-anand.net](http://www.s-anand.net/blog/software-for-my-new-laptop-2/)** _27 Sep 2011 7:16 pm_ _(pingback)_:
  [...] Putty [new]: SSH for Windows, but can also act as an SSH tunnel [...]
- **[S Anand](http://www.s-anand.net/)** _17 Jul 2011 8:15 am_:
  @Lemming, as long as you have PuTTY running on your machine, others can use YOUR machine as a Socks proxy. Or you could create a non-root account for them on the EC2 server to log into.
- **Lemming** _15 Jul 2011 1:49 am_:
  Hi S Anand, I followed your instructions and the SOCKS server/proxy is working great on my machine. Thanks for the detailed instructions!
  I&amp;#39;m wondering if I could share the SSH server with some friends. A number of them face restrictive proxies like you described, while others have to travel to China for work. How would I share my SSH server, without giving away my private key and root access to the EC2 instance? My friends would also be using Windows/PuTTY.
- **Lemming** _20 Jul 2011 3:46 pm_:
  Hi S Anand, thanks for the info. I&amp;#39;ll look into a non-root account solution.
- **Mohamed Infaz** _11 Jun 2012 3:40 am_:
  Sir, lets say i have successfully done this!! can i use this sock5 proxy to by pass torrent block in my university network?! We are using a proxy sever to connect to the internet in our university and otherwise we are not able to connect to the network! I have successfully used ultrasurf software to good effect to bypass download limits! so i am sure that i would be able to pent 9996 port. Can i use that port, as the source port for incoming connection in utorrent?! Please help! :)
&amp;lt;!-- wp-comments-end --&amp;gt;
&lt;/code&gt;&lt;/pre&gt;</description>
    </item>
  </channel>
</rss>
