I'm not sure it's possible to express how much I love Docker. My first Linux experience, with Redhat Manhattan, was set square in the middle of dependency hell. Dependency management has come a long way since then. However, this intensifies the effect of a dependency conflict when one does arise. Docker solves these problems and brings along both service and firewall management to boot. Because of this, I tend to develop and deploy with Docker.
For a web app to be useful, it necessarily needs to be accessible from the internet. Binding each app to the host negates much of the encapsulation Docker provides. So, we'll start by setting up a reverse-proxy into our Docker environment. In the end, our environment should resemble the following diagram.
This setup will require docker, docker-compose, and cron on the host. Let's start by defining our Nginx container with docker-compose.
As you can see, we've defined volumes for Nginx config and TLS certificates, a single network to attach to, and we're binding to ports 80 and 443 on the host. We've also mounted /tmp and /run in a memdisk. Pretty standard stuff.
And we're done, right? docker-compose up And, let's call it a day. 🙌
Not quite. This does run Nginx, but the out-of-the-box config isn't very useful. To make changes to the config we have to move files in and out of our container by hand which is error prone. Let's build some tooling around this so we don't have to remember all the fine details of this deployment every time we add or remove a virtual server, register a new TLS certificate, or whatever else we may need.
We'll start by creating a few directories and then define a few environment variables so we stay organized.
Then let's start by initializing our docker volumes with the latest Nginx config. Then, mount all of our volumes in a temp container for initialization.
Our init script installs git in the temp container so we can pull an Nginx config repo (Maybe, I'll dissect my Nginx config in another post) and moves that config into place. Then, moves our certbot credentials into place and generates unique Diffie-hellman parameters for this deployment.
Now we've got a base Nginx config, but how do we add a server block or TLS certificate to our reverse-proxy? Let's write some scripts to handle these and other common tasks.
To add a site to our reverse-proxy, we take the file path to an Nginx server block as a parameter. Then we launch a temp container mounting both the config fragment and our sites volume, copying the fragment into place. We then reload the Nginx config in the container with docker-compose. Pretty easy stuff, but stuff with fine details that could lead to errors if not automated.
To remove a site, we take the name of a server block as a parameter. We then launch a temp container with the sites volume mounted, remove the config fragment, and reload the Nginx config as before. Again easy, but the fine details could lead to problems if not automated.
We should also provide tools to see what sites are installed and their configuration.
I think this looks pretty good. We have a repeatable setup and we can add sites to and remove sites from Nginx without having to remember all the fiddly details of our deployment config. Our app directory is starting to fill-out nicely.
Now we'll configure EFF's certbot to generate TLS certificates with Let's Encrypt. I love Let's Encrypt. I used StartSSL for years and when they came to a fiery end, Let's Encrypt seemed to magically swoop in to save the day. 🤯
Certbot has many options for automating certificate generation. We'll use the Digital Ocean DNS plugin to verify our identity. We'll place our API key in etc/digitalocean.ini. Scripts will need to be modified if another DNS provider is used.
To generate a certificate, we take a certificate name, list of domains, and a contact email address as parameters. The certificate name is arbitrary, a label used by certbot to reference the certificate. The list of domains is a comma separated list of domains you wish to register. The first domain given becomes the certificate's Common Name, subsequent domains are listed in the Subject Alternative Name. Finally, the contact email address is used for expiry notification and account recovery. Certificates are placed on the certs volume, which is mounted to /opt/certs in the Nginx container.
To delete a certificate, we take the certificate name as a parameter. Certbot removes all the files related to the certificate. Caution: If any Nginx server blocks reference the certificate, Nginx will fail when the config is reloaded.
To renew all eligible certificates, simply run bin/cert.renew. Certbot will renew any installed certificates that expire in less than 30 days.
To force the renewal of a certificate, run bin/cert.renew.one passing the certificate name as a parameter. This will renew the specified certificate regardless of it's expiation date.
To list installed certificates, run bin/cert.list. This returns a list of certificate names.
To view detailed information about a certificate, run bin/cert.view passing the certificate name as a parameter. Certbot returns the certificate name, serial, key type, domains, and expiry date. The certificate and key path are returned as viewed from the Certbot container; swap /etc/letsencrypt/ with /opt/certs/ for use in an Nginx server block.
I think this is shaping up rather nicely. In addition to our repeatable config and site scripts, we can now generate and administer TLS certificates with Let's Encrypt.
Let's automate our Certbot renewal script. To start, we'll strip all the sudo and echo bits from bin/cert.renew; these bits are for interactive use and not necessary when running from cron. Then we'll swap references to environment variables with tags we can replace with sed; we don't want our elevated script to run potentially unsecured code.
Now, we'll add a block to the end of bin/init replacing our tags with the values set in etc/env during the install. Then we set permissions on the script and move it to /etc/cron.daily/. Now our renew script will check and renew certificates once a day.
Startup & Shutdown
There is an issue with Docker Compose that is nearly unforgivable. Docker Compose insists on starting in attached mode with no clean way to detach. To work around this, we'll script our start command and avoid using compose directly. We'll also script our stop command for the sake of consistency.
This is the most basic of things, it's literally just calling docker-compose up -d. But considering the dire consequences if a mistake is made, it is best to modify our process and avoid docker-compose up completely.
And there we have it, a dockerized reverse-proxy server that we can hide our future projects and deployments behind.
We've scripted the most common actions required to make our proxy useful. This reduces the likelihood of mistakes and ensures consistency with our configuration over time.
If this project seems useful, check it out over on github. 😎