Now You Can Tunnel into Any Cloud Foundry Data Service

Cloud Foundry now enables developers to open a tunnel to any Cloud Foundry data service via a local port. Now you can use familiar client applications to directly analyze, manipulate, or port the data contained in the data services that are bound to your Cloud Foundry applications. This solution is ideal for importing and exporting data when moving application between different clouds running Cloud Foundry (e.g. CloudFoundry.com and Micro Cloud Foundry), for debugging during development, as well as ad-hoc queries and modification to data in a deployed application.

This new feature is made available with a preview release of Cloud Foundry command line tool (‘VMC’). Underlying it is a simple Ruby application, named Caldecott (after the Caldecott Tunnel in California), that gets deployed to your CloudFoundry.com account to connect to services in your account. Caldecott is currently being provided as a preview, as the team is working on making it more robust and performant in the coming weeks. Caldecott provides an HTTP endpoint that facilitate a port forward on your local box. VMC contains built in scripts to run some of the popular client software in the local environment and automatically connect to specified services, or simply create the connection to the service for you to connect using any client software of your choice. Let’s take a look at how this works.

First, install or update your Cloud Foundry command line tool (‘VMC’) to the latest preview version using the following command:

gem install vmc --pre

You can verify that you got the right version using:

vmc -v

which should show the version to be 0.3.14.beta.4 or higher.

You also need the caldecott gem, which you can install using:

gem install caldecott

Note that currently the caldecott gem requires the ‘eventmachine’ gem, which in turn requires native compilation. We are working to simplify this.

The first step is to make sure that you are targeting the Cloud Foundry instance that you want to connect to and log in.  See “Getting Started Guide – Command Line (VMC) users” if you are unfamiliar with VMC.

Next, review all of the services that you have:

vmc services

Under the Provisioned Services section, you will see all of the services that you have created.  For example, here’s an output of my services:

=========== Provisioned Services ============

    +------------------+------------+
    | Name             | Service    |
    +------------------+------------+
    | mongodb-12345    | mongodb    |
    | mysql-12345      | mysql      |
    | postgresql-12345 | postgresql |
    | redis-12345      | redis      |
    +------------------+------------+

Now, you can connect to any service listed through a simple command:

vmc tunnel <servicename> [--port]

The port is an optional parameter that allows you to define a specific port number that you want. By default, VMC will automatically select an available port in the 10000 range. The service name parameter is also optional, you can interactively select the service from a list to establish the connection by using the command without any parameters.

vmc tunnel

As an example, let’s connect to the PostgreSQL service in the list above by typing the following command:

 vmc tunnel postgresql-12345

VMC will first determine if an existing Caldecott application instance has been deployed, if not, it will deploy a new one.

Trying to deploy Application: 'caldecott'.
    Create a password: 

Whenever you create a tunnel, you will be prompted for a password. You will need to furnish a new password initially for the first time the application is deployed, so that VMC can secure future connections to the Caldecott application instance. For additional service connections, you will have to supply the same password that was originally created for security. Following the password creation, you should see the following progress output.

Uploading Application:
 Checking for available resources: OK
 Processing resources: OK
 Packing application: OK
 Uploading (1K): OK
Push Status: OK
Binding Service [postgresql-12345]: OK
Staging Application: OK
Starting Application: OK
Getting tunnel connection info: .OK

You can see that VMC is uploading and starting the Caldecott application and binding the specified service to that application instance. In successive VMC tunnel command calls, it will not be necessary to create a new application instance. Instead, the existing instance will be stopped, the specified service will be bound to the application, and then the same Caldecott application will get restarted.

The next section of the output will display the credentials necessary to connect to the service. These credentials were originally automatically generated by the Cloud Controller when provisioning the service for you. These are the credentials that Cloud Foundry’s staging code will automatically insert into your application’s configurations to connect to the service.

Service connection info:
 username : ua76003139485469989d57b22f9cb83xx
 password : p8214ebd2a7254698913e01c86a4432xx
 name : da281fa2162f947028876d0500ae31cxx

In the case of a database service, the username and password are displayed, along with the database name. Lastly, when the tunnel is established, VMC will offer to start a known client that exists in your execution path or allow you to connect to the local port with your choice of client application.

For the Postgres example, the following prompt is displayed:

Starting tunnel to postgresql-12345 on port 10000.
1: none
2: psql
What client would you like to start?: 

Enter 1 to connect with your own client application and the previously displayed credentials, or 2 to automatically start the client application and pass in the credentials. If you choose the latter, VMC will start the client automatically (in this case psql).

What client would you like to start?: 2
Launching 'PGPASSWORD='p8214ebd2a7254698913e01c86a4432xx' psql -h localhost -p 10001 -d da281fa2162f947028876d0500ae31cxx -U ua76003139485469989d57b22f9cb83xx -w'

psql (9.0.4)
Type "help" for help.
da281fa2162f947028876d0500ae31cxx=> d
                     List of relations
 Schema | Name  | Type  |               Owner
--------+-------+-------+-----------------------------------
 public | hotel | table | ua76003139485469989d57b22f9cb83xx

da281fa2162f947028876d0500ae31cxx> select * from hotel;
  1 | 3555 S. Ocean Drive                    | Hollywood           | USA           | Westin Diplomat                | 199.00 | FL        | 33019
  2 | 890 Palm Bay Rd NE                     | Palm Bay            | USA           | Jameson Inn                    |  60.00 | FL        | 32905
  3 | The Cottage, Southampton Business Park | Southampton         | UK            | Chilworth Manor                | 199.00 | Hants     | SO16 7JF
...

Alternatively, you can supply the desired client to execute at the invocation of the tunnel command to bypass the prompt:

vmc tunnel postgresql-12345 psql

That’s it, at this point, you can choose to view, modify, or export the data from your service.

Caldecott is currently a preview software, and we are working hard to improve it. Please feel free to send us any feedback via the Cloud Foundry Support Forums

– The Cloud Foundry Team

Don’t have a Cloud Foundry account yet? Signup for free today