Few days ago I introduced EvServer. In this post I’ll present a simple EvServer example.
EvServer is a normal WSGI server, but with one additional feature. Instead of blocking in your WSGI application you yield a file descriptor to the server. On descriptor activity the server will continue your WSGI app till it yields again.
I’ll show how to wait for AMQP messages inside the WSGI application and how to push them up to the browser. If you can’t wait till the end of the post, please feel free to view
the online demo(outdated) of the code described below.
Consuming (from) the Rabbit
To consume AMQP messages we need a python amqp library (version 0.6). Barry Pederson rewrote it lately and I must admit I really like the code now. Great work Barry! Usually, the code that consumes messages (a subscriber) looks like this:
import amqplib.client_0_8 as amqp
def callback(msg): print msg.body
def main(): conn = amqp.Connection('localhost', userid='guest', password='guest') ch = conn.channel() ch.access_request('/data', active=True, read=True) ch.exchange_declare('myfan', 'fanout', auto_delete=True) qname, _, _ = ch.queue_declare() ch.queue_bind(qname, 'myfan') ch.basic_consume(qname, callback=callback) while ch.callbacks: ch.wait() # blocking here! ch.close() conn.close()
The problem is that we need to use it in an Asynchronous WSGI application which is non blocking. The conversion is not very difficult. First, we need to identify the socket descriptor on which the wait() blocks, then we need to make it non-blocking. This solution is quite hackish, but it’s all we can do. Maybe this hack should be put in the Hall of Fame of Dirty Hacks…
sd = conn.transport.sock
At this point, a bad thing happens when ch.wait() tries to block on a non-blocking descriptor. We’d expect the socket to raise socket.error: (11, ‘Resource temporarily unavailable’). Actually, it fails with quite a strange exception, but fortunately it doesn’t break anything inside py-amqplib:
Traceback (most recent call last): [...] File "/home/majek/amqplib-0.6/amqplib/client_0_8/connection.py", line 201, in _wait_method
self.method_reader.read_method() TypeError: 'NoneType' object is not iterable
We use this exception to identify when the library wants to block. Having this knowledge, the main loop of a non-blocking consumer is now easy to write:
conn.transport.sock.setblocking(False) while True: try: while True: # until the exception ch.wait() except TypeError: pass <block till activity on conn.transport.sock>
At this point modifying the code to became a valid AWSGI application is straightforward. Full code can be found in the EvServer examples directory. Here’s the simplified version:
def wsgi_subscribe(environ, start_response): start_response("200 OK", [('Content-type','text/plain')]) msgs =  def callback(msg): msgs.append(msg.body) msg.channel.basic_ack(msg.delivery_tag) <setup connection, channel, queue> conn.transport.sock.setblocking(False) try: while ch.callbacks: try: while True: # until exception ch.wait() except (TypeError,), e: pass yield 'got messages: %rn' % (msgs,) while msgs: msgs.pop() # empty the queue # block! yield environ['x-wsgiorg.fdevent.readable'](conn.transport.sock) except GeneratorExit: pass
Introducing the Comet library
There are many different encapsulation types for Comet messages and the format of the messages emitted from the server depends sensitively on the browser type. Fortunately the WSGI application can remain quite simple by using the evserver.transports wrapper:
import evserver.transports as transports
def simplest_comet_application(environ, start_response): t = transports.get_transport(<comet transport family>) start_response('200 OK', t.get_headers()) yield t.start() yield t.write('fist message!') yield t.write('second message!')
Please, don’t build a chat
I think we’re all sick of yet-another-chat examples. I propose to build something similar, but instead of broadcasting chat messages let’s broadcast user-agent and referrer http fields of people who view the site.
The html site is going to be dead simple.
On the server side we will serve comet.js file and the main HTML file. While serving the HTML, we’ll send an AMQP message. We also need to create a Comet channel URL. Let’s use Django – in total we need three Django views, one of which can be generic.
sudo apt-get install erlang-nox setuptools subversion
sudo dpkg -i rabbitmq-server_1.5.1-1_all.deb
sudo easy_install evserver-*.egg
tar xvzf amqplib-0.6.tgz
sudo python setup.py install
tar xzvf Django-1.0.2-final.tar.gz
sudo python setup.py install
svn checkout http://evserver.googlecode.com/svn/trunk/evserver/examples/django_agentpush django_agentpush
After these steps, to run the code just type:
PYTHONPATH=".." DJANGO_SETTINGS_MODULE=django_agentpush.settings evserver --framework=django
LShift is recruiting! (thanks to all of you that already have contacted us, we’re still reviewing your CVs)