How Cloud Foundry Works When a New Application is Deployed

Previously in this blog post, we covered how the client side does a vmc push to deploy an application onto the Cloud Foundry.com service.  In this post we are going to look at how things work from Cloud Foundry’s perspective.  This post is a detailed walk through of the inner workings of deploying an application on Cloud Foundry.

Step 1 : Cloud Foundry is targeted by the vmc client.  The request is done by passing a GET request to http://api.cloudfoundry.com/info

Step 2 : The request passes through one of the Load Balancers where it is passed to the Routers

Step 3 : The request passes from a specific Router to the Cloud Controller, the Cloud Controller then returns the following JSON document {“name”:”vcap”,”build”:2222,”support”:”[email protected]”,”version”:”0.999”,”description”:”VMware’s Cloud Application Platform”} .

This document passes back through the Router and Load Balancer

Step 4 : The JSON document (the response) is returned to the vmc client, after validating the JSON document, it returns “Successfully targeted to [ http://api.cloudfoundry.com]”

After targeting Cloud Foundry, the client needs to authenticate before it can issue instructions to the Cloud Controller.

Step 1 : vmc login api.cloudfoundry.com on the client side sends an authentication request which passes through the Load Balancers

Step 2 : The request gets passed through the appropriate Routers on its way to the Cloud Controller

Step 3 : The Cloud Controller takes the login request and looks at the credentials (email and password) and if the validate, it creates a Security Token.  The Security Token has a configurable lease which defaults ~2 Weeks (this can be shortened/lengthened).  Once the Security Token is Generated it is sent in the response, which is passed back through the Router and then Load Balancer.

Step 4 : The Security Token is then passed from the Load Balancer to the vmc client where it is cached and sent as an authorization header with each new request.  Once the Security Token is received, this is when the response “Successfully logged into [ http://api.cloudfoundry.com]” is displayed.

The previous two vmc actions dealt with Targeting and Authenticating to Cloud Foundry, which are basic commands which have been in use by many systems for a long time.  The vmc push command is one of the most powerful and complicated commands to handle on the server side of Cloud Foundry.  This is due to the number of components and messages that have to be brought together to successfully deploy an application.

Metadata included in the below steps encompasses the name, url, instances, services, and more, however no application bits are sent.

Step 1 : vmc push sends the Security Token along with the metadata (that was described in the vmc client side blog post) to the Load Balancers

Step 2 : The Load Balancers move the request (the metadata and the token) through to the Routers

Step 3 : The Routers pass the request down to the Cloud Controller which authenticates the token before proceeding

This diagram is a follow-on to the previous (which is why the numbers start at 4).

Step 4 : After the Cloud Controller validates the token it opens the Manifest and compares the Fingerprints to the Fingerprint Index (Which is embedded into a directory structure).

Step 5 : Once all Fingerprints have been compared, any Fingerprints (these are the SHA-1 Crypto Hashes against the files that the vmc client wants to send) that are not matched are put into a Differences Manifest.

Step 6 : The Differences (Diff’d) Manifest is then sent back through the Router

Step 7 : The Router passes the Diff’d Manifest to the Load Balancer

Step 8 : The Load Balancer passes the Diff’d Manifest back to the vmc client

Once the vmc client receives the Diff’d Manifest it then takes the files that were not found in the Fingerprint Index and creates a compressed (zip) file with them. Once the compressed file has been created it is called the Application Package.

In this phase the Application Package will become a Gold App Package

Step 1 : The Diff’d Manifest (the same as was received from the Cloud Controller) and Application Package are sent along with the Security Token (All of which are in JSON format, either JSON Docs or Blobs) to the Load Balancer

Step 2 : The Load Balancer passes the Manifest, Package, and Token to the Router

Step 3 : The Router passes the Manifest, Package, and Token to the Cloud Controller

Step 4 : The Cloud Controller validates the Security Token and Examines the Manifest

Step 5 : The Cloud Controller uncompresses the Package and begins to add the missing files that it already had Fingerprints for.

Step 6 : Once the Gold App Package has been created, it is re-compressed, time-stamped, fingerprinted via SHA1, and saved to a shared directory

Step 7 : Once the Gold App Package has been successfully stored in the shared directory it is “finished”.  The Cloud Controller now has all of the information from the client it needs regarding metadata describing the application and the application bits.

Step 8 : The Cloud Controller releases the vmc client call, and uploading is considered complete.

Step 9 : The Router having received the message from the Cloud Controller sends it to the Load Balancer

Step 10 : The Load Balancer sends the message to the vmc client where it updates the terminal with “Push Status: OK” (Assuming that it was successful)

Once the Push Phase has completed, the vmc client will ask the Cloud Controller to start the application, by changing its state, and then begins a polling process to find out the current status of the Application.  The state transitions from Stopped to Started and waits for the staging to be completed.

It is important to understand that this process is independent of what is happening inside Cloud Foundry.  The only thing that happens between the vmc client and Cloud Foundry is whether the Cloud Controller responds with Staging Successful, Application Started, an error, or still working.  This polling process waiting for the application status to be considered Started occurs asynchronously from the vmc client to the Cloud Controller.

Step 1 : The vmc client requests a status on if the Application has been Staged, that request is sent to a Load Balancer

Step 2 : The Load Balancer passes the request to a Router

Step 3 : The Router passes the request to a Cloud Controller, which checks the status of the Application to see if it has completed Staging.  In this case the Application has not completed Staging, but it also has not errored out.

Step 4 : The Cloud Controller sends a response that it is waiting, which is passed to a Router

Step 5 : The Router passes the response to a Load Balancer

Step 6 : The Load Balancer passes the response back the the vmc client, where it waits (~1 second before polling again for the status)

Now that vmc client polling has been described, we can look at what is occurring for Staging to complete successfully.

Step 1 : The Cloud Controller tells the Stager that it has a Gold App Package that needs to be Staged

Step 2 : The Framework specific Stager makes a copy of the Gold App Package and uncompresses it.  It then looks at the Metadata for the Gold App Package and decides what the necessary frameworks, runtimes, and services are needed.  The decision is based on what the Application Startup Requirements, Application Shutdown Requirements, and Application Service Requirements are.  For example, if the application is a Ruby on Rails Application it will need the Ruby runtime, the Rails Framework, and most likely MySQL.  Once all of the Requirements are known, it gathers all of the necessary files and creates the Staged Application Package.

Step 3 : The Staged Application Package is first compressed, then written into an indexed and shared directory, where it is versioned.

Step 4 : The Stager sends a notice that the Application has been Staged (because the Staged Application Package has been successfully created)

Step 5 : The Cloud Controller updates the status that this Application is now Staged, this will be picked up by the vmc client the next time it performs a poll.

If the vmc push contained a —no-start, then these next few steps would be skipped under the assumption that other things (such as bringing online other services of some type) need to be done before the Application can be Started.  Otherwise, the default behavior is to attempt to Start the application which is what follows.

Step 1 : The Cloud Controller broadcasts that it has a Staged App Package that it needs started.

Step 2 : The Cloud Controller broadcast goes out to the DEA pool, where the DEAs ALL receive the request.  Internally each DEA looks to see if it meets the necessary requirements to respond to the Cloud Controller that it can Start that type of Application.  Some of the requirements include: Runtime, Memory Needed, Current Memory Available, CPU Available.

Step 3 : Assuming a DEA can fulfill all of these requirements, it will respond that it can Start this type of Application.  The first DEA to respond that it can Start this type of Application will be the DEA that gets assigned the Application.  DEAs may pause in responding based on memory pressure, or whether they are already running the application, providing for better placement of new applications.

Step 4 : The Cloud Controller sends a message containing the Staged App Package information, including the Fingerprint and a URI pointing to where to get the Staged  App Package if it doesn’t already have a copy locally.

Step 5 : If the DEA doesn’t have a copy already, it will look several places and if all else fails, it will use the URI to retrieve it.

Following the preceding Diagram, the DEA has gotten a copy of the Staged App Package.

Step 6 : The DEA extracts the Staged App Package and brings up the Application Environment, along with the necessary Services required.  The DEA only understands the Start and Stop requirements, all of the work to make the application behave correctly has been done by the Cloud Controllers, Stagers, and is contained within the Staged App Package.  Once the DEA believes that the application has fully started, it will broadcast a message to this effect.

Step 7 : Until now we haven’t mentioned the PubSub component, this is an intermediary component that sits in between all of the components.  It is a lightweight message bus allowing components to publish/send/broadcast messages, while allowing other components to subscribe/listen/receive messages.  This is being called out because in the case of Step 6, there are 2 subscribers to the message that the Application has started.

Step 8 : The Router receives the message that the Application has started and updates its routing tables to begin routing traffic to the Application when requests are sent to it.

Step 9 : The Health Manager (which subscribes to most messages being passed around the system) receives the message and adds this information to it current database of activities that it has seen.

Step 10 : When the next vmc poll event occurs, the Cloud Controller asks the DEA pool for the status of the instances, and it asks the Health Manager for its health report on the application including all instances. The Cloud Controller sends the response that it has started to the vmc client. If the DEA had reported that the application had crashed or was starting and crashing (flapping) then the vmc client would pull certain files (crashlogs) from the crashed instance and would send the diagnostic output to the console.

Once the vmc client has gotten the poll response from the Cloud Controller that the Application has successfully started, it returns “Starting Application: OK”.

Now that there is a running Application on Cloud Foundry, we can now see what happens when a user goes to access the application (via. web browser).

Step 1 : The browser makes a request to http://hello.cloudfoundry.com which is routed by DNS to the Load Balancers in Cloud Foundry.

Step 2 : The Load Balancers pass the request to the Routers.

Step 3 : A Router directs the request to the Application running on a DEA.

Step 4 : The Application receives and processes the request to create a response.

Step 5 : While receiving requests, DEA is also sending out heartbeats to the Health Manager letting it know that the Application is still working correctly.

Step 6 : Once the response has generated by the Application it is ready to be sent.

Step 7 : The response is the sent to a Router

Step 8 : The Router sends the response to a Load Balancer

Step 9 : The Load Balancer sends the response back to the client (web browser) that made the request.

In Summary

Understanding the flow from both the client side and the service side of Cloud Foundry is important as application bits aren’t simply being “pushed” into a black box.  Cloud Foundry operates in a manner that is designed to horizontally scale at all layers, while making it easy for the developer to deploy and update their code to the platform. When application bits are pushed they are done in the most efficient manner possible, while maintaining easy and fast portability through each step of the process.  The vmc client operates asynchronously and polls the Cloud Controller, it does not block Cloud Foundry in any way.  Once an application has been started, it is continuously monitored by the Health Manager.  The vmc polls the Cloud Controller, finds that the application has be started and updates the console.  The developer can now access their application from the url they created when they pushed their application.