Prismatic Digital Blog

A production ready CI CD pipeline for $1 a month with Docker, AWS and Gitlab.

A production ready CI CD pipeline for $1 a month with Docker, AWS and Gitlab.

This is a blogpost of a presentation done for the Auckland continuous delivery meetup, slides can be found at this url: https://docs.google.com/presentation/d/1wQcGelbBA67FYxLw0EwjGz75OutFkp0jr08APK35g84/

What problem to solve ?

With my personal pipeline (This is the one used for this very website) I am trying to:

  • Reduce the time to production
  • Automate all test and all gatekeeping
  • Have a High Availability website
  • Release small, release often
  • Easy way to rollback

With my Solution Architect experience I can say that many many companies are spending to much time testing and releasing instead of developing.

In my opinion with a good pipeline and a good automated testing strategy releasing multiple times per day is not that hard and should be one of your pipeline goal.

Why Docker in my pipeline ?

A long time ago developers were sending emails with individual files to the release manager that was building and releasing the version.

Then version control appeared and git became the de-facto standard for source working and releasing a project. In the meantime a lot of environnement as code started to emerge with Chef, Puppet or Ansible

Docker allows you to version control your environement with the code and ship one artefact containing everything including the Operating System. But because Docker filesystem is smart you do not need to copy a 2 gig artefact over the network every time. Only the last steps are pushed.

Here is the Dockerfile example of this website.

FROM bchabord/django-nginx
COPY ./requirements.txt /files/requirements.txt
WORKDIR /files/
RUN pip install -r requirements.txt && pip install awscli && rm -rf /root/.cache/
COPY conf/nginx.conf /etc/nginx/nginx.conf
COPY conf/nginx_prismatic.conf /etc/nginx/conf.d/prismatic.conf
RUN nginx -t
COPY ./ /files/
EXPOSE 9808
ENTRYPOINT ["./entrypoint.sh"]

  • This Dockerfile depends on another dockerfile (You can find it there: docker-django-nginx) 
  • Then copy the requirements.txt those are python dependencies of the project, change the working dir and run pip install to install those dependencies
  • Copy the 2 nginx configurations and run the nginx config checker
  • Copy all the code in the current repo
  • Expose port 9808 (Nginx is binding to this port)
  • Set the entrypoint to be entrypoint.sh

It is not good practice to run 2 processes in a container, good practice is to run one blocking process per container. In this case I am runing Nginx as a deamon and python as a blocking process in the entrypoint. This is OK for me because if nginx dies then the health-check will fail as well.Here is the entrypoint of my container

#!/usr/bin/env bash
echo "Launching Entrypoint"
export DJANGO_SETTINGS_MODULE="prismatic_website.settings.production"
function inject_env {
${CONFIG_URL?"Need to set CONFIG_URL env variable"}
${CONFIG_URL:?"Need to set CONFIG_URL env variable not empty"}
aws s3 cp $CONFIG_URL /tmp/env_config
export $(cat /tmp/env_config | grep -v ^# | grep '=' | xargs)
rm /tmp/env_config
}
if [[ $# -eq 0 ]]; then
inject_env
echo "webserver started ..."
service nginx start
python manage.py collectstatic --noinput --clear
python runserver.py
else
echo "${@:1}"
if [[ $1 == "test" ]]; then
export DJANGO_SETTINGS_MODULE="prismatic_website.settings.testing"
service nginx start
python manage.py migrate
python manage.py collectstatic --noinput --clear
python manage.py test
else
inject_env
python manage.py "${@:1}"
fi
fi
#export DJANGO_SETTINGS_MODULE="prismatic_website.settings.production" && python manage.py migrate