Container Clustering with Rancher Server (Part 4) – Deploying and Managing Containerised GoCD Continuous Delivery

This post was written by Rich Bosomworth.

This is the fourth post in an on-going series exploring Rancher Server deployment, configuration and extended use. In the last post I detailed how to add an extra level of HA by configuring an AWS EFS volume for shared storage and mounting it using the Rancher-NFS service. In this post we focus on GoCD. GoCD is an open source continuous delivery server developed by ThoughtworksWe adapt the previous configuration with AWS EBS volumes to enable single node HA for GoCD server. We will also be deploying auto-scaled self registering GoCD agents.

Experience with AWS is assumed, along with an understanding of all Rancher related systems from previous posts. Content is best viewed from a desktop or laptop.

Series Links

Container Clustering with Rancher Server (Part 1) – Local Server Installation on Linux using Vagrant Host Nodes

Container Clustering with Rancher Server (Part 2) – Single Node Resilience in AWS

Container Clustering with Rancher Server (Part 3) – AWS EFS mounts using Rancher-NFS

Container Clustering with Rancher Server (Part 4) – Deploying and Maintaining Containerised GoCD Continuous Delivery

Container Clustering with Rancher Server (Part 5) – Automating the deployment of AWS infrastructure and Rancher with Terraform

Container Clustering with Rancher Server (Part 6) – Creating and deploying custom catalog items for GoCD

Container Clustering with Rancher Server (Part 7) – Stack and service build out to create a custom catalog item for Splunk


Build Architecture

Fig:1 represents the underlying infrastructure. It is based on the architecture used in Part 3, but with the following modifications:

  • Single Availability Zone for Rancher Hosts
  • Rancher AWS EBS Plugin

Single Node HA Rancher Server with AZ Failover and S-AZ GoCD Hosts

Fig:1

Fig:2 represents the GoCD deployment layer. This layer sits above the underlying infrastructure. As the post continues we shall examine and explain the configuration and components used.

GoCD Deployment on Rancher S-AZ EC2 Hosts

Fig:2

GoCD and EBS

GoCD server installs its files in the following locations on your file system (i.e. within the Docker container):

/var/lib/go-server #contains the binaries and database
/etc/go #contains the pipeline configuration files
/var/log/go-server #contains the server logs
/usr/share/go-server #contains the start script
/etc/default/go-server #contains all the environment variables with default values.

In absence of an external DB connector (i.e. for AWS RDS) and to facilitate HA, a method of replicating the contents of these locations is required, specifically for /var/lib/go-server and /etc/go.

rancher-ebsTests with the Rancher NFS plugin proved ineffective for different mount points to the same instance. For this reason the Rancher EBS plugin was chosen as it provides options for multi-volume mounts.

As EBS volumes are zone specific the GoCD server host must therefore be single-AZ, in order to facilitate EBS volume re-attach in the event of a container or host failure.

*NOTE* – The method for configuring the Rancher EBS plugin and mounting volumes is different to the procedure for Rancher EFS.

EBS Plugin Configuration

With the EBS plugin installed, within Rancher (under infrastructure/storage) declare x2 storage volumes (Fig:3). We will name our volumes eb1 and ebs2. It is important to specify both the volume type and the volume size.

vols create

Fig:3

GoCD Server Service

The GoCD server and agent Rancher catalog items are developed and maintained by Raúl Sánchez. We have forked the catalog and modified the GoCD service to accommodate the latest Docker builds. Our forked repository can be added to your own Rancher build as a Custom Catalog.

*NOTE* – At time of writing GoCD v17+ Docker agents do not auto-register with the GoCD server when deployed within Rancher. The methods in this post are based on v16.12.0.

As shown from Fig:1 and Fig:2, we use separate Rancher hosts for both the GoCD server and the GoCD agents. This is to facilitate single-AZ HA for the GoCD server via EBS volumes. Both sets of hosts are labelled accordingly when deployed (i.e. gosrvhst and goagthst). This enables fixed deployment of each service to the relevant host (Fig:4).

gosrvhst-label

Fig:4

Adding the EBS volumes and mount points for GoCD server is done so via Volumes/Add Volume (Fig:5).

vols-add-vols

Fig:5

With the GoCD service upgraded and running on gosrvhst, successful EBS volume mounts will be shown as active under Storage Drivers (Fig:6). You can also view the EBS volumes as created in-situ from the EC2 console (Fig:7).

ebs-storage-mounted

Fig:6

ebs-vols-ec2

Fig:7

GoCD Agents

GoCD agents are launched via the same route as the GoCD server service, selecting the host label via scheduling (i.e. goagthst) and ‘upgrading’ the service. As the GoCD agents need to link to the GoCD server service it is important to make sure the stack service is referenced in the configuration options (Fig:8)

gocd-agent-stack-service

Fig:8

When launched, the GoCD agents will auto-register and show as enabled with the GoCD server (Fig:9)

gocd-agents

Route53 DNS Plugin

To enable consistency of access in the event of a host or container failure, we use the Rancher Route 53 plugin. The plugin is installed via the Rancher catalog and creates dynamic R53 DNS entries for public facing services. Our method is to create a fixed CNAME record for the dynamic GoCD server service A record (Fig:10)

r53-plugin-cname

Fig:10

In the next post we will look at automating deployment of the AWS infrastructure and both the Rancher server and Rancher hosts using Terraform.

If you would like further information on any of the methods and systems detailed in this post, please feel free to get in touch via the comments section, or through our contact page.


The reference material contained within this post expands on the presentation delivered by Matthew Skelton at Continuous Delivery Amsterdam.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: