Setting up CI&CD for VPS-hosted apps with Shipit.js and Codeship

In this article, we’ll create a simple Express.js app, then set up a full-blown CI & CD solution using Codeship and Shipit.js

With the rise of Docker's popularity, developers have now access to tons of great deployment and hosting services - ECS, Heroku and others. All offer the benefits of containerized cloud hosted apps. However, with great features comes greater price, making those solutions less affordable for some cases - especially simple apps, or apps in the stage of development.

One of the most affordable options to host apps is still VPS. Services like DigitalOcean provide great value for the price - starting at 5$/month. On such server, we could attempt to organise a container based solution (like Dokku), but it's important to remember: setting up and maintaining VPS instance relies fully on the owner, so we might not want to overcomplicate it.

What do we need

  • code repository
  • VPS server
  • CI&CD service
  • script that will handle deployment

While providers of first two can be chosen freely, we'll be focusing on Codeship and Shipit.js for the former two.

Creating demo app

For the purposes of this article, we'll create a very basic "Hello world" Express.js app with a simple unit test (using jest and supertest).

src/app.js

const express = require('express')
const app = express()

app.get('/', (req, res) => {
  res.status(200).send('Hello World!')
})

module.exports = app

index.js

const app = require('./src/app')

app.listen(8000, () => {
  console.log('Example app listening on port 8000!');
})

src/app.js

const request = require('supertest')
const app = require('../src/app')

describe('Test root path of the app', () => {
  test('It should respond 200 OK', async () => {
    const response = await request(app).get('/');
    expect(response.statusCode).toBe(200);
  });
})

Full source code up until this point

Codeship setup

Our CI&CD service of choice is Codeship Pro. It provides an easy and flexible set of tools and is based on Docker. Since Codeship Basic is basically a simple pre-defined docker image with ability to run a custom script, you should be able to use it as well - however, some modifications will be required.

After initializing Codeship Pro project, create ci directory and following files:

ci/shipit.dockerfile - defines Codeship’s docker image. We’ll base it on the official node image, update it, install ssh client and install necessary project dependencies.

FROM node:8.12.0

# Update & install SSH Client
RUN apt-get update && apt-get install -y ssh

# Create workdir in the image and move to it
RUN mkdir /usr/src/codeship
WORKDIR /usr/src/codeship

# ADD app files
ADD . .

# Copy files to upload into separate directory
COPY . ./files

# Install project dependencies
RUN npm install

# Install npx for easy execution of local shipit-cli
RUN npm install -g npx

# Add execution permissions to main deployment script
RUN chmod +x ci/deploy.sh

# Add execution permissions to test script
RUN chmod +x ci/run-tests.sh

ci/run-tests.sh - it will execute all necessary tests.

#!/bin/bash

npm test

ci/deploy.sh - should execute all steps related to deployment

#!/bin/bash

# Shipit.js
npx shipit vps deploy --shipitfile ci/shipitfile.js

We'll get back to the shipit deploy command later!

Lastly, configuration files required by codeship need to be defined in project's root directory:

codeship-services.yml - we describe our deployer service and provide path to its dockerfile

deployer:
  build:
    image: deployer-image
    dockerfile: ./ci/shipit.dockerfile

codeship-steps.yml - we define series of deployment steps: first the app needs to be tested - if all tests pass then it is deployed.

- name: Perform tests
  tag: master
  service: deployer
  command: ./ci/run-tests.sh

- name: Deploy the app
  tag: master
  service: deployer
  command: ./ci/deploy.sh

Full source code up until this point

Create and configure VPS

The exact procedure to create VPS instance depends on hosting service you have chosen. Choose server location, performance and OS that suits the needs of your application. Moving forward we'll be using DigitalOcean and Ubuntu 16.04.

Log in to your VPS instance via SSH with credentials provided by your service provider, then perform rudimentary update:

apt-get update && apt-get upgrade

Now it is a good time to install and config system-wide project dependencies, such as Postgres, Python etc. For the purposes of our simple app, the only thing we need is Node.js

curl -sL https://deb.nodesource.com/setup_8.x -o nodesource_setup.sh
sudo bash nodesource_setup.sh
sudo apt-get install -y nodejs

Create user

Next step is to create a new unix user that will be handling deploying, as well as running the app. Using root user for such purposes is ill-advised as it gives potential attacker full access to the system.

Create user deploy and his home directory:

useradd -s /bin/bash -m -d /home/deploy -c “app deployer” deploy

Grant safe access to sudo level commands:

usermod -aG sudo deploy

Set secure password for the user:

passwd deploy

We can now switch to our new user with: su - deploy

App persistence

To keep our app persistent between reboots we’ll use pm2

npm i pm2 -g

Then run pm2 startup (as deploy user) command and follow instructions provided by it.

Authorizing Codeship to access the server

In order to access our VPS instance, Codeship deployer service needs to be able to log in via SSH. The best and most secure way to authorize it - is to generate SSH key. We should not, however, include raw, unencrypted keys in our source code, therefore alternative solution must be devised.

Download project’s AES key file from project settings page at codeship and place it in the root folder as codeship.aes
Follow steps provided by first part of the Codeship article, otherwise visible below:

# generate codeship_deploy_key and codeship_deploy_key.pub, configured to not require passphrase
docker run -it --rm -v $(pwd):/keys/ codeship/ssh-helper generate "<YOUR_EMAIL>" && \
# store codeship_deploy_key as one liner entry into codeship.env file under `PRIVATE_SSH_KEY`
docker run -it --rm -v $(pwd):/keys/ codeship/ssh-helper prepare && \
# remove original private key file
rm codeship_deploy_key && \
# encrypt file
jet encrypt codeship.env codeship.env.encrypted && \
# ensure that `.gitignore` includes all sensitive files/directories
docker run -it --rm -v $(pwd):/app -w /app ubuntu:16.04 \
/bin/bash -c 'echo -e "codeship.aes\ncodeship_deploy_key\ncodeship_deploy_key.pub\ncodeship.env\n.ssh" >> .gitignore'

It will generate new SSH key and encrypt it using AES file into coneship.env.encrypted - that way the key won’t be stored unsafely in the repository and yet Codeship can decrypt it.

Add Codeship key to authorized_keys

Now, we need to let our server know that generated key is trusted. As a deploy user run following commands:

mkdir -p ~/.ssh
touch ~/.ssh/authorized_keys
chmod -R go-rwx ~/.ssh/

Open newly created authorized_keys file and paste-in contents of the codeship_deploy_key.pub file - generated public key.

nano ~/.ssh/authorized_keys

Update codeship config

The next step is to update codeship config so that it utilizes generated key.

codeship-services.yml - in addition to providing filename of generated encrypted file, we also mount .ssh folder as a volume and provide SERVER_IP variable.

deployer:
  build:
    image: deployer-image
    dockerfile: ./ci/shipit.dockerfile
  encrypted_env_file: codeship.env.encrypted
  volumes:
    - ./.ssh:/root/.ssh
  environment:
    SERVER_IP: <YOUR-VPS-IP>

Codeship will now decrypt the codeship.env.encrypted file and inject its variables into deployer image, resulting in generated key being available as PRIVATE_SSH_KEY. In order for ssh client to use it, it has to be printed into a file located in the .ssh folder. Additionally, server IP has to be added as known host for butter smooth deployment process. In order to do all that, we update deploy.sh:

ci/deploy.sh

#!/bin/bash

# SSH key config
echo -e $PRIVATE_SSH_KEY >> /root/.ssh/id_rsa
chmod 600 /root/.ssh/id_rsa
ssh-keyscan -H $SERVER_IP >> /root/.ssh/known_hosts

# Shipit.js
npx shipit vps deploy --shipitfile ci/shipitfile.js

Full source code up until this point

Shipit.js configuration

With codeship and server configured, it is time to take care of the next step - actual deployment. shipit-deploy will handle upload, version and symlink management so our concern is reduced to basic installation flow:

  1. Install dependencies
  2. Stop the previous version of the app
  3. Start the new version of the app

With PM2, we are also capable of performing zero deploy deployments but we’ll not cover it in this article.

shipit-deploy

shipit-deploy is a set of predefined shipit.js tasks that allow for creating efficient and modular capistrano-style deployment configurations.

├── current -> /home/deploy/app/releases/20180120111520/
├── releases
│   ├── 20180120111520
│   ├── 20180315114505
│   └── 20180120113504

Library provides two main tasks - deploy and rollback, both of linear workflow separated into subtasks.

There’s, however, one catch - shipit-deploy is usually utilized for local git-based deployments. By analyzing the source code of the first subtask of the deploy task - deploy:fetch - we can see that by default it will attempt to fetch files from configured branch and repository. Since codeship already provides proper files, of which shipfile.js is a part of, there is no need for such operation. By analyzing further, we can also infer that the only side effect of the fetch task, beyond getting the files, is to set shipit.workspace to the location of the fetched files.

Since we know the location of the files - /files in docker image, we’ll simply override deploy:fetch task by creating our own, identically named and setup shipit.workspace appropriately.

ci/shipitfile.js

require('dotenv').config()
const createDeployTasks = require('shipit-deploy')

const PM2_APP_NAME = 'DemoApp'

module.exports = shipit => {
  createDeployTasks(shipit)

  shipit.initConfig({
    default: {
      workspace: 'files',
      deployTo: '/home/deploy/app',
      repositoryUrl: '',
      ignores: ['.git', 'node_modules', 'ci'],
      keepReleases: 3,
      shallowClone: false
    },
    vps: {
      servers: `deploy@${process.env.SERVER_IP}`,
      branch: 'master'
    },
  })

  // Override 'fetch' step of shipit-deploy
  shipit.blTask('deploy:fetch', async () => {
    // Normally this step would've fetch project files from git
    // In this case, codeship provides those files to location defined in
    shipit.workspace = shipit.config.workspace

    shipit.log('Established path to project files.')
  })
}

Now we can create tasks related to installing dependencies and stopping/starting the app. shipit-deploy emits appropriate events after each subtask, so by listening to proper events and starting blocking tasks (blTask), we can inject our own operations in the middle of shipit-deploy workflow.

ci/shipitfile.js

  (...)

  // before publishing, execute following tasks
  shipit.on('updated', () => {
    shipit.start([
      'installDependencies',
      'stopApp',
    ])
  })

  shipit.blTask('installDependencies', async () => {
    await shipit.remote(`cd ${shipit.releasePath} && npm install --production`)

    shipit.log('Installed npm dependecies')
  })

  shipit.blTask('stopApp', async () => {
    try {
      await shipit.remote(`pm2 stop ${PM2_APP_NAME} && pm2 delete ${PM2_APP_NAME}`)
      shipit.log('Stopped app process')
    } catch (error) {
      shipit.log('No previous process to restart. Continuing.')
    }
  })

  // When symlink changes, restart the app
  shipit.on('published', () => {
    shipit.start('startApp')
  })

  shipit.blTask('startApp', async () => {
    await shipit.remote(
      `cd ${shipit.currentPath} && ` +
      ` NODE_ENV=production pm2 start index.js --name "${PM2_APP_NAME}"`
    )

    shipit.log('Started ap process')
  })
}

Push your changes to the repository and monitor your codeship dashboard - your app should deploy successfully.

Full final source code

More possibilities

Configuration described in this article is quite simple, but should get you up and running in the initial state of app development. These are things you might want to do next:

  • install and use nginx as reverse proxy for the app
  • create multiple environments (dev, staging, master)
  • use shipit-shared to share some folder between releases (e.g. uploads)
  • further expand deployment workflow - database migrations, backups etc