europython 2010: TinyHTTPProxy and troublesome wireless
November 2017
Sun Mon Tue Wed Thu Fri Sat
      1 2 3 4
5 6 7 8 9 10 11
12 13 14 15 16 17 18
19 20 21 22 23 24 25
26 27 28 29 30    
About
This site is an effort to share some of the base knowledge I have gathered through all this years working with Linux, FreeBSD, OpenBSD, Python or Zope, among others. So, take a look around and I hope you will find the contents useful.
Recent Entries
Recent Comments
Recent Trackbacks
Categories
OpenBSD (9 items)
BSD (0 items)
FreeBSD (19 items)
Linux (3 items)
Security (3 items)
Python (22 items)
Zope (13 items)
Daily (144 items)
e-shell (9 items)
Hacks (14 items)
PostgreSQL (3 items)
OSX (8 items)
Nintendo DS (0 items)
enlightenment (0 items)
Apache (3 items)
Nintendo Wii (1 items)
Django (24 items)
Music (12 items)
Plone (7 items)
Varnish (0 items)
Lugo (2 items)
Sendmail (0 items)
europython (7 items)
Cherokee (1 items)
self (1 items)
Nature (1 items)
Hiking (0 items)
uwsgi (0 items)
nginx (0 items)
cycling (9 items)
Networking (1 items)
DNS (0 items)
Archives

Syndicate this site (XML)

RSS/RDF 0.91

20 julio
2010

europython 2010: TinyHTTPProxy and troublesome wireless

There are always those good things you learn or discover while being in this kind of events

Europython 2010

I would like to write some lines about a small (but really useful) script Antonio Cuni showed me yesterday. It is not a script he had written by himself, and probably you could find it in the Internet (UPDATE: In fact it is called TinyHTTPProxy ;D).

While having some beers at the Walkabout last night, we tried to connect to the Internet using my laptop and the Walkabout free wifi, but (after many tries) we found out that it didn't play well with the https protocol. Each time a website (like twitter or gmail) was redirecting us to a secure https connection the connection just got stucked.

My first solution was quite a bit dirty. I opened an SSH connection to one of the machines I've at home, setting up a tunnel like this:

ssh -L 443:twitter.com:443 user@myhost

That will cause that each request sent to https://localhost on my laptop will be forwarded through the tunnel to my box at home, and then it will be forwarded to twitter.com. Then I added a line like this to the /etc/hosts file (yes, that works in OSx too ;D):

127.0.0.1 twitter.com

It mostly works (the Walkabout wifi was a little bit troublesome anyway). From time to time there were timeouts and it was not so usable.

The solution Antonio provided was, indeed, by far more elegant. He showed me this python script which is a fully-functional http proxy in only 123 lines of code!. Amazing, isn't it?

All you need is download it and execute it, for example, in your home box (or any other box out there you can reach):

python proxy.py

It will listen on port 8000 on that box:

Any clients will be served...
Serving HTTP on 0.0.0.0 port 8000 ...

Then you only have to modify your browser to use a proxy connection with http/https/etc requests using the hostname/ipaddress of that box as the proxy address. We tried that and it worked just perfectly fine, but with the limitations of the flaky wifi inside the pub.

It is even more useful, as it will send to stdout a lot of useful messages about the connections managed by the proxy:

78.41.210.130 - - [19/Jul/2010 22:42:08] "CONNECT mail.google.com:443 HTTP/1.1" 200 -
        connect to ocsp.thawte.com:80
78.41.210.130 - - [19/Jul/2010 22:42:08] "POST http://ocsp.thawte.com/ HTTP/1.1" - -

You can even notice when there are connection problems between you and the proxy (even if this is a little bit ugly and should be handled properly through a try...except catch):

----------------------------------------
Exception happened during processing of request from ('78.41.210.130', 49507)
Traceback (most recent call last):
  File "/usr/lib/python2.6/SocketServer.py", line 558, in process_request_thread
    self.finish_request(request, client_address)
  File "/usr/lib/python2.6/SocketServer.py", line 320, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python2.6/SocketServer.py", line 615, in __init__
    self.handle()
  File "./proxy.py", line 20, in handle
    self.__base_handle()
  File "/usr/lib/python2.6/BaseHTTPServer.py", line 329, in handle
    self.handle_one_request()
  File "/usr/lib/python2.6/BaseHTTPServer.py", line 323, in handle_one_request
    method()
  File "./proxy.py", line 46, in do_CONNECT
    self._read_write(soc, 300)
  File "./proxy.py", line 93, in _read_write
    out.send(data)
error: [Errno 32] Broken pipe
----------------------------------------

So, the final solution could be a mix of both approaches. I'll try to do this tonight if we go again to the Walkabout, because I think the problem was that the wifi was causing some kind of trouble with http/https requests. So, what if you use the proxy python script to set up a proxy at home (just like we did before) but instead of setting up the proxy like in the previous example, you do open an SSH connection home, creating a tunnel to forward requests to the proxy?, just like this:

ssh -L 8000:localhost:8000 user@myhost

Then you do not need to modify your /etc/hosts file, you don't need to open port 8000 at home and you do send all the http requests through the ssh tunnel, which in the end is an already established connection (it shouldn't close) while http requests from the browser are going to be multiple requests sent through the wifi link.

Imho, this would be an almost-perfect approach, but I'll try tonight and check if there is any difference.

Posted by wu at 17:20 | Comments (2) | Trackbacks (0)
<< europython 2010: First conference day | Main | Europython 2010: Second conference day >>
Comments
Re: europython 2010: TinyHTTPProxy and troublesome wireless

You can use ssh -D to run a SOCKS proxy.

ssh -D 8080 -N user@your.host

Not all applications support SOCKS, but most interesting do. Easy and out-of-the-box :D

Posted by: Juanjo at julio 20,2010 18:07
Re: europython 2010: TinyHTTPProxy and troublesome wireless

Damn, Juanjo beat me to posting the solution!

Posted by: betabug at julio 21,2010 09:40
Re: europython 2010: TinyHTTPProxy and troublesome wireless

Thnx for the comments guys.

@Juanjo: Didn't know about that, but it does not involve python at all! c'mon! ;D

@betabug: You are loosing your skills... ;)

Posted by: Wu at julio 21,2010 11:07
Trackbacks
Please send trackback to:http://blog.e-shell.org/255/tbping
There are no trackbacks.
Post a comment