Friday, September 18, 2009

Zero MQ + Lisp

Last year I got interested in AMQP.

The most advanced implementation was/is Qpid, because it's a flagship product of such famous companies like JPMorgan, Goldman Sachs, Cisco, RedHat and many others. That means the standard was bloated a lot, you can see it, for example, by looking at specs 0-9 and 0-10: first is relatively simple, later is not. I'm afraid, it will ends up like Corba did. Unfortunately, RedHat and other guys are not interested in Lisp, but I thought it might be useful to have such library in Lisp, so thick and rich customers can use "official AI language" in their projects, and Lisp can benefits from it.

After a two weeks of dancing around Lisp client implementation, I gave up. It was too huge work for me, beginner... A year later I've returned back to that theme and found ready to use Qpid client, fully implemented in Python. Python is closer to Lisp, I thought, so it's possible to use Python codes as a guideline. Heh... Sources are more than 350k in tarball, and it's only a client... It's easy to imagine, how much time will it sweep away from my life doing the same way in Lisp. A lot of complicated code to write, a lot of bugs to fix. No, I didn't really want it. I've decided to wrap Lisp calls, minimally required by standard, into Qpidc calls via CFFI. However, Qpidc is written in scary C++ with a lot of Boost (scary for bindings), so it required me to to write a wrapper for calling C++ from C first.

Next idea, which came to my head due to laziness, is to google if there're any good AMQP realizations with ready to use bindings in C. It can be less powerful and less bleeding edge, but I just needed simple, convenient messaging. I found ØMQ.

Zero MQ is not AMQP messaging library, but can use AMQP as an underlying wire-level transport. Feature list said it also has support of OFED, like Qpidc has. But the most impression I got by looking at API. It darn pretty simple! And it is even more simpler in version 2.0, which is in alpha stage at the moment. It is so simple, that I was able to CFFI'ize the only required zmq.h in one hour, and it worked for me. Impressing!

Latency measurement using server and client both written in Lisp showed me quite good results:

message size: 1024 [B]
roundtrip count: 1000
average latency: 48.7915 [us]


It's even slightly better than in C (with asserts commented out) :)

Current bindings are in the early stage and have ugly CFFI interface, but I'm gonna make it a bit nicer and easy to use.

Sources are here: http://repo.or.cz/w/cl-zmq.git

No comments: