europython 2010: TinyHTTPProxy and troublesome wireless
I would like to write some lines about a small (but really useful) script Antonio Cuni showed me yesterday. It is not a script he had written by himself, and probably you could find it in the Internet (UPDATE: In fact it is called TinyHTTPProxy ;D).
While having some beers at the Walkabout last night, we tried to connect to the Internet using my laptop and the Walkabout free wifi, but (after many tries) we found out that it didn't play well with the https protocol. Each time a website (like twitter or gmail) was redirecting us to a secure https connection the connection just got stucked.
My first solution was quite a bit dirty. I opened an SSH connection to one of the machines I've at home, setting up a tunnel like this:
ssh -L 443:twitter.com:443 user@myhost
That will cause that each request sent to https://localhost on my laptop will be forwarded through the tunnel to my box at home, and then it will be forwarded to twitter.com. Then I added a line like this to the /etc/hosts file (yes, that works in OSx too ;D):
127.0.0.1 twitter.com
It mostly works (the Walkabout wifi was a little bit troublesome anyway). From time to time there were timeouts and it was not so usable.
The solution Antonio provided was, indeed, by far more elegant. He showed me this python script which is a fully-functional http proxy in only 123 lines of code!. Amazing, isn't it?
All you need is download it and execute it, for example, in your home box (or any other box out there you can reach):
python proxy.py
It will listen on port 8000 on that box:
Any clients will be served... Serving HTTP on 0.0.0.0 port 8000 ...
Then you only have to modify your browser to use a proxy connection with http/https/etc requests using the hostname/ipaddress of that box as the proxy address. We tried that and it worked just perfectly fine, but with the limitations of the flaky wifi inside the pub.
It is even more useful, as it will send to stdout a lot of useful messages about the connections managed by the proxy:
78.41.210.130 - - [19/Jul/2010 22:42:08] "CONNECT mail.google.com:443 HTTP/1.1" 200 -
connect to ocsp.thawte.com:80
78.41.210.130 - - [19/Jul/2010 22:42:08] "POST http://ocsp.thawte.com/ HTTP/1.1" - -
You can even notice when there are connection problems between you and the proxy (even if this is a little bit ugly and should be handled properly through a try...except catch):
----------------------------------------
Exception happened during processing of request from ('78.41.210.130', 49507)
Traceback (most recent call last):
File "/usr/lib/python2.6/SocketServer.py", line 558, in process_request_thread
self.finish_request(request, client_address)
File "/usr/lib/python2.6/SocketServer.py", line 320, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib/python2.6/SocketServer.py", line 615, in __init__
self.handle()
File "./proxy.py", line 20, in handle
self.__base_handle()
File "/usr/lib/python2.6/BaseHTTPServer.py", line 329, in handle
self.handle_one_request()
File "/usr/lib/python2.6/BaseHTTPServer.py", line 323, in handle_one_request
method()
File "./proxy.py", line 46, in do_CONNECT
self._read_write(soc, 300)
File "./proxy.py", line 93, in _read_write
out.send(data)
error: [Errno 32] Broken pipe
----------------------------------------
So, the final solution could be a mix of both approaches. I'll try to do this tonight if we go again to the Walkabout, because I think the problem was that the wifi was causing some kind of trouble with http/https requests. So, what if you use the proxy python script to set up a proxy at home (just like we did before) but instead of setting up the proxy like in the previous example, you do open an SSH connection home, creating a tunnel to forward requests to the proxy?, just like this:
ssh -L 8000:localhost:8000 user@myhost
Then you do not need to modify your /etc/hosts file, you don't need to open port 8000 at home and you do send all the http requests through the ssh tunnel, which in the end is an already established connection (it shouldn't close) while http requests from the browser are going to be multiple requests sent through the wifi link.
Imho, this would be an almost-perfect approach, but I'll try tonight and check if there is any difference.