Cloud Foundry™ is the industry’s first open platform as a service. Providing a choice of developer frameworks, application services and deployment clouds, Cloud Foundry simplifies application development and makes it faster and easier to build, test, deploy and scale applications. Cloud Foundry already supports multiple application services including MySQL, MongoDB and Redis. And we are working to add more.
Today, we are pleased to announce an important milestone: the RabbitMQ messaging service, available today as a free public beta. You can get started right away: on CloudFoundry.com, you will find sample applications and guides. If you create a cool application, please let us know and we’ll link to it. And if you get stuck, tell us what we need to get right.
Our goal is to make it as easy as possible for you to use messaging to create and connect to cloud applications. This radically simpler portability of applications and integration of services is the essence of Cloud Foundry. To understand what this means for messaging as a service, let’s look at how people use RabbitMQ in the cloud and then see why we think offering RabbitMQ as a service makes sense.
Messaging for the cloud
In the cloud, communication is the design center.
Messaging is fundamentally a communication technology and therefore a cloud enabler. People talk about “event streams” and “task queues” but the core pattern is “moving a lot of data reliably”. Cloud applications that involve multiple processes or VMs can benefit from a communication centric design. Take a look at this blog post from Union Station for example, and notice that their architecture is not a ‘stack’ but a ‘network’. Or have a look at Google’s design for their Rocksteady monitoring and analytics system.
You can see this in any development meeting. Increasingly people who build applications ask “how do these components wire together?” and “what’s my event model?” where they used to ask “what’s my database schema and what’s my language platform”. This was always true for “enterprise integration” but now these patterns are needed even for new applications. The applications that are leading the way are also some of the biggest, e.g. near real-time data analytics and activity streams on global scale social and mobile enabled apps. The shift to these design patterns is driving adoption of streaming data services and event based technologies like messaging.
Messaging is essential for successful cloud applications for two reasons.
First, messaging provides stable communication patterns that scale across multiple network topologies. In the cloud it is critical for applications to cope when networks change or grow. This means that the constituent components of cloud applications cannot be coupled directly. Instead, asynchronous decoupling and indirection based on message brokers is used. These two patterns enable work to be addressed, routed and delivered between components. Those components may be running in one cloud, or several, or on multiple devices e.g. phones. RabbitMQ has a proven track record of moving a lot of data reliably in these scenarios at scale.
Second, because those components are likely to be written in multiple programming languages, it is necessary that communication be “polyglot” and therefore “data centric”. Because of this, people look for a scalable and stable data fabric or communication backbone that can cope with a wide range of client languages, broadcast and load patterns. Messaging systems are designed to be robust with respect to arbitrary traffic conditions, e.g. reliably paging data to disk when consumers go offline. Again, RabbitMQ has the required breadth to deliver on this requirement.
Here are some benefits for cloud applications:
- Maintainability: messaging lets you more easily maintain your application without impacting uptime. Decoupling components reduces complexity and simplifies integration. Moreover logging, alerts, reporting and management benefit from a messaging based design.
- Scalability: you can improve responsiveness in applications. For instance, messaging lets you separate factor time-consuming processing tasks out of web-facing applications and into a separate tier of worker instances. In addition you can add more of these workers and back end services to achieve scale.
- Efficiency: messaging pushes data to users, devices and applications, which is much more efficient than using a database to share information between applications and continually querying for changes. In the cloud, polling is aggravated by variable latency. Pushing data when it changes is more effective.
- Robustness: in the cloud, not all components are connected or online at all times. Messaging lets you queue up data for subsequent consumption without fear of losing your data.
What’s next? Simple, Effective Cloud Messaging
Arguably, the success of RabbitMQ thus far has been due to providing a convenient and trustworthy way to get the benefits above, across lots of languages, and with a simple open source license. And certainly this is visible ‘in the cloud’ where RabbitMQ is a top choice both on the main clouds and running some of them under the hood too. One example is NASA’s Nebula cloud, but there are many more.
But there’s one problem – in the age of ‘IT as a service’, not everyone wants to install and look after their own RabbitMQ server. Wouldn’t it be great if management, maintenance, upgrades, rollbacks and all the other potential hindrances to scaling a business… just were taken care of by someone else? In 2011 all this seems obvious and logical to people across all kinds of infrastructure – including customers’ own hardware, behind their own security perimeter.
The self-service experience is quite natural for messaging, and requests like “I just want a dev queue” or “spin me up ten instances of rabbit” should be part of the cloud. And that means part of the PaaS. So the first step is to provide RabbitMQ as a service for CloudFoundry.com.
Get Started
Cloud Foundry.com makes it easy to begin using RabbitMQ. If you don’t have a CloudFoundry.com account yet, signup for free today.
In Cloud Foundry, the RabbitMQ server is a service on par with MySQL, MongoDB, and Redis. For instance, you use standard Cloud Foundry vmc commands to create, bind, unbind, clone, and delete RabbitMQ services.
To learn more about using RabbitMQ on Cloud Foundry, you can use the following resources:
- Ruby on Rails tutorial, sample application, and suitable RabbitMQ Ruby client (ruby gem)
- Java Spring tutorial 1, tutorial 2, sample application, and AMQP client for Spring
- Ruby Sinatra sample application and suitable RabbitMQ Ruby client (ruby gem)
- Node.js tutorial, sample application and AMQP client
- Grails tutorial 1, tutorial 2, tutorial 3
- Additional sample applications
- RabbitMQ service FAQ
These applications should get you started but you can implement other patterns too. For example, you can use the RabbitMQ service to integrate distinct applications running on Cloud Foundry, even applications built using different languages and frameworks. Or perhaps you could wire RabbitMQ up with another service such as MongoDB. We want people to test the limit of what is possible so that we can stretch it.
The RabbitMQ service lets you implement a full set of essential cloud messaging patterns:
- Work queues that distribute time-consuming tasks among multiple workers, in order to avoid doing a resource-intensive task immediately and having to wait for it to complete. Work queues let you schedule the task to be done later. This pattern is especially useful in web applications where it’s impossible to handle a complex task during a short HTTP request window. See for example the Celery project.
- Publish/Subscribe to distribute messages to multiple consumers. This greatly simplifies integration and improves maintainability, since message producers and message consumers don’t need to know about each other. Publishers simply send their messages to an exchange (part of RabbitMQ server), which then looks at properties of the messages to distribute them to the appropriate queues, which are then read by one or more consumers.
- Routing, to enables message consumers to subscribe only to a subset of the messages.
- Topics, to do routing based on multiple criteria.
- Remote Procedure Call (RPC), enabling a software application to run a function on a remote computer and wait for the result.
These core patterns are described in our online tutorials for RabbitMQ and in the Manning book “RabbitMQ in Action”. For a short introduction to the RabbitMQ AMQP messaging model see here.
If you don’t already have a CloudFoundry.com account, signup here (it’s free).
Background, Pricing and Availability
The RabbitMQ service is now available on www.CloudFoundry.com as a free public beta. It is currently implemented using RabbitMQ server 2.4.1, and can be used by all languages and frameworks that are supported by Cloud Foundry and that have an AMQP 0-8 or 0-9-1 client available, including Ruby on Rails, Ruby Sinatra, Spring Java, and node.js.
RabbitMQ on Cloud Foundry is a cloud-based message broker. This is a type of server software that is used to integrate distinct software systems, by sending and receiving data (in the form of messages) between those systems. These systems can be components of the same application, or part of distinct applications, and can reside on the same machine (physical or virtual), on different machines in the same datacenter, or can be physically remote.
RabbitMQ stands out among message brokers for several reasons. RabbitMQ scales and is extremely easy for developers to use. As the leading implementation of AMQP, an open-standard messaging protocol, RabbitMQ works with dozens of languages and frameworks that support the Advanced Message Queuing Protocol (AMQP), enabling developers to integrate applications built with completely different technologies, such as a Ruby application that needs to connect to a Java app. These capabilities – performance, usability, and ease of integration — have helped RabbitMQ become a leading choice, growing its popularity to the point where it is included as a standard message broker for major Linux distributions including Ubuntu, Fedora and Debian.
Please try the RabbitMQ service on Cloud Foundry and let us know how we can improve the experience by asking us questions or posting feature requests in our community forums.
