Secure communication in the Cloud

David Ireland wrote “The internet does reliability end to end. That is, when a node A sends a message to node B, the message travels through many other nodes. These intervening nodes make a best effort at delivery, but if they don’t succeed, they just forget about it. B must confirm receipt, and A must try again if…”

SpringSource / VMWare Acquire Rabbit Technologies

Mike Rowlands wrote “SpringSource, a division of VMware, Inc. today announced the acquisition by VMware of Rabbit Technologies, Ltd, a company set up by LShift and partners Monadic and CohesiveFT. Read the full story”

On the limits of concurrency: Worker Pools in Erlang

Matthew Sackman wrote “A worker pool is a very common pattern, and they exist in the standard libraries for many languages. The idea is simple: submit some sort of closure to a service which commits to running the closure in the future in some thread. Normally the work is shared out among many different threads and in the…”

The fine art of holding a file descriptor

Matthew Sackman wrote “People tend to like certain software packages to be scalable. This can have a number of different meanings but mostly it means that as you throw more work at the program, it may require some more resources, in terms of memory or CPU, but it nevertheless just keeps on working. Strangely enough, it’s fairly difficult…”

By Tangopaso (Self-photographed) [Public domain], via Wikimedia Commons

Memory matters – even in Erlang

Marek Majkowski wrote “Some time ago we got an interesting bug report for RabbitMQ. Surprisingly, unlike other complex bugs, this one is easy to describe:  At some point basic.get suddenly starts being very slow – about 9 times slower!”

RabbitMQ-shovel: Message Relocation Equipment

Matthew Sackman wrote “In several applications, it’s very useful to be able to take messages out of one RabbitMQ broker, and insert them into another. Many people on our mailing list have being asking for such a shovel, and we’ve recently been able to devote some time to writing one. This takes the form of a plugin for…”

Plugin exchange types for RabbitMQ

Michael Bridgen wrote “An obvious extension point for an AMQP broker is the addition of new types of exchange. An exchange type essentially represents an algorithm for dispatching messages to queues, usually based on the message's routing key, given how the queues are bound to the exchange -- it's a routing algorithm. RabbitMQ now supports plugin exchange types, currently on an experimental branch.”

Merry Christmas: Toke — Tokyo Cabinet driver for Erlang

Matthew Sackman wrote “Tokyo Cabinet is a rather excellent key-value store, with the ability to write to disk in a sane way (i.e. not just repeatedly dumping the same data over and over again), operate in bounded memory, and go really fast. I like it a lot, and there’s a likelihood that there’ll be a RabbitMQ plugin fairly…”

RabbitMQ at the Skills Matter Functional Programming Exchange

Matthew Sackman wrote “Today I was lucky enough to give a talk at the Skills Matter Functional Programming Exchange. I talked about resource management in RabbitMQ and how we’re improving this in upcoming versions of RabbitMQ. All the sessions were videotaped and it would seem that a podcast will be going up shortly. In the mean time you…”

Garbage Collection in Erlang

Matthew Sackman wrote “The new persister that is being developed for RabbitMQ is nearing completion and is currently working its way through code review and QA. It’s being pretty thoroughly tested and generally stressed to see what could go wrong. One of the issues that we’ve come across in the past has to do with Erlang’s garbage collector:…”

HTTP Routing with RabbitMQ and Trapeze

Paul Jones wrote “After building Hookout a little while back, I’ve been considering other things you could do to funnel clients through a server without them necessarily being reachable, or having an entire address space of their own. Hookout was working within the constraints of the reverse http protocol, where clients could speak only http. I wanted to…”

Grouping and collapsing in WireIt

James Uther wrote “I have recently been modifying the WireIt code to allow collapsing of multiple containers down into 1 composite container. A quick summary of WireIt (from their site) WireIt is an open-source javascript library to create web wirable interfaces for dataflow applications, visual programming languages, graphical modeling, or graph editors. I got started on this when…”

Rabbit on the treadmill: Run Rabbit, Run!

Matthew Sackman wrote “For the last couple of months I’ve been working on rewriting RabbitMQ’s persister so that it will scale to volumes of data that won’t fit in RAM, and will perform consistently across a wide variety of use cases. This work is coming to a conclusion now, and although the code is not yet released, nor…”

Achieving Scale with Messaging and the Cloud

Tony Garnock-Jones wrote “On the 9th, last Thursday, I spoke at the Online Gaming High Scalability SIG at Skills Matter. The talk covered – an introduction to Messaging (what it’s for, why you might like to use it), – a couple of pointers in the directions of examples of Messaging being used at scale in the cloud, and…”

PubSub-over-Webhooks with RabbitHub

Tony Garnock-Jones wrote “RabbitHub is our implementation of PubSubHubBub, a straightforward pubsub layer on top of plain old HTTP POST — pubsub over Webhooks. It’s not well documented yet (understatement), but that will change. It gives every AMQP exchange and queue hosted by a RabbitMQ broker a couple of URLs: one to use for delivering messages to the…”

Cranial Surgery: Giving Rabbit more Memory

Matthew Sackman wrote “Many users of Rabbit have been asking us about how Rabbit copes with many large messages in queues, to the extent that the total size of these messages exhausts the available physical memory (RAM). As things stand at the moment, the answer is not very well. Although we have a persistence mechanism, that is not…”