November 6, 2018

Clever-clever automated certificate download

Introduction #

A client of mine needs some proper automation on his server. It’s a modest little container-based thing that runs a LAMP stack which powers a few tens of websites. I faced the prospect, for each website that was added to the server, of making the necessary changes to the vhosts configuration, adding sites to the SAN certificate they have, and so on. The manual way of doing this, would be something like:

Download the cert + bundle *.zip to my laptop
Unzip the file
Rename the certs if necessary
Upload the certs to the server
Concatenate the certs if necessary
Blah blah blah
…
Profit!

This is quite a bit of stuff that needs to be remembered every time, so it’s a perfect candidate for automation.

It turns out that this client’s certificate provider, GoDaddy, exposes a well-documented REST API to the rest of the internet. One of the services provided by this API is the ability to interact with certificates

Accessing the API #

To access the API, you need access keys. GoDaddy has a page for that (login required), Click on “Create New API Key” to…create a new API key. Choose “Production” for the environment and add a name that makes sense. The next step shows the API key and secret. I dealt with these as follows.

Add the API key as an environment variable #

In ~/.bashrc, ~/.bash_profile, or wherever you store such things, add a line like this:

GODADDY_API_KEY=922495jklhh_notarealkey;lkfjskdl

Store the API secret securely #

It’s possible to also place the secret in an unencrypted file in the same way, but probably not recommended. I chose to store it using pass. I won’t cover that now, because my friend Mike has already done that quite capably
here.

Something like this should work:

echo "09ujnotarealsecretno9vsd | pass insert godaddy/secret --echo

To make this accessible from the environment, add a line like this in your ~/.bashrc or similar.

GODADDY_SECRET=$(pass show godaddy/secret)

Don’t forget to source ~/.bashrc to read in these environment variables!

Actually accessing the API :) #

The GoDaddy site gives a sample command on its getting started page. It uses curl to send and receive data from the API. To modify it to fit the example above and to access the certificates endpoint, we need:

curl -X GET -H"Authorization: sso-key $GODADDY_API_KEY:$GODADDY_SECRET" https://api.godaddy.com/v1/certificates/

This will give a load of JSON output, which won’t make a lot of sense to begin with. I recommend jq. It’s great for parsing json, but I often find myself resorting to the documentation and a few excellent articles to remember how to use it. To get a list of (godaddy’s internal designation) certificate IDs, the correct incantation is:

curl -X GET -H"Authorization: sso-key $GODADDY_API_KEY:$GODADDY_SECRET" https://api.godaddy.com/v1/certificates/ | jq '.[].certificateId'

The output of that will be something like this:

"rcvqyobnotarealcertidbohikjhkoxx"
"clyszfnnotarealcertidmuqrxacfyym"
"bqolmhdnotarealcertidhu0uxgwixxc"
"uhovvs58notarealcertidnz9n5okxhu"
"tjrltnotarealcertidoyht8vblxmauz"
"jga0o5g5notarealcertidtdwmusq0zp"
"pjghdcv1tpe0h67notarealcertidoxz"
"onotarealcertid6qcnlpd2iplv9shwi"

In this case, I determined that the certificate ID was the first in that list (FIXME: that can probably be automated). Now, we can use a slightly
different request to dig into the info available from the API about that particular cert.

There are also actions that can be called for each certificate ID, of particular interest for this is download/. This produces output like this:

{
  "serialNumber" : "100c0000000000000000",
  "certificateThumbprint" : "45xxx1",
  "pems" : {
    "certificate" : "-----BEGIN CERTIFICATE-----\....xxxx....\n-----END CERTIFICATE-----\n",
    "intermediate" : "-----BEGIN CERTIFICATE-----\n....xxxx....\n-----END CERTIFICATE-----",
    "root" : "-----BEGIN CERTIFICATE-----\n....xxxx....\n-----END CERTIFICATE-----",
    "cross" : null
  }
}

Obviously, the TLS data has been obfuscated above. But the point is….there’s TLS data available.

Enter Ansible #

My automation tool of choice at the moment is Ansible. So, that’s what I went for. Initially, I was going to run some ansible code locally to i) download the certs ii) unzip them iii) go and make a cup of tea iv) etc. etc. But as we now seem to have access to the raw data, why not do that operation on the server itself? We can use ansible to make the client’s webserver contact the godaddy API and use the data it receives to write the certificate files. No messy copying around of files, sounds like a win-win situation.

Ansible has a uri module, which can be used in much the same way we used curl earlier

We can use it to access the API as described earlier, then use the data it receives about the cert as a variable.

The actual bit of code is surprisingly compact:

- name: Grab the certs from the api 
  uri:
    url: "{{ godaddy_api_url_root }}/{{ cert_id }}/download" 
    headers: 
      Authorization: sso-key {{ godaddy_api_key }}:{{ godaddy_secret }}
  register: godaddy_cert_download_response

The {{ godaddy_api_key }} and {{ godaddy_secret }} Ansible variables are passed from the shell to Ansible like this:

godaddy_api_key: "{{ lookup('env', 'GODADDY_API_KEY') }}"
godaddy_secret: "{{ lookup('pipe', 'pass show tokens/godaddy') }}"

See here and here for info about those to lookup plugins…

So now, we have an Ansible variable called {{ godaddy_cert_download_response }} containing the JSON response from the API. We can use normal variable notation to get to this data, as Ansible can use it directly. So, .e.g to get to the certificate data, we would need {{ godaddy_cert_download_response.json.pems.certificate }}

Of course, lookup('env') 'GODADDY_SECRET would also work, but I wanted to showcase the lookup('pipe' pass ...) method, as Mike’s blog post mentioned…

The final piece of the puzzle is to use the template module to place the data in the files that the webserver expects:

- name: Template the bundle file 
  template: 
    src: apache-bundle.j2 
    dest: "{{ godaddy_cert_root_location }}/san-cert-bundle.crt" 
    owner: root 
    group: root 
    mode: 0644 
    backup: yes 
  notify: restart apache2 

- name: Template the certificate file 
  template: 
    src: apache-cert.j2 
    dest: "{{ godaddy_cert_root_location }}/san-cert.crt" 
    owner: root 
    group: root 
    mode: 0644 
    backup: yes 
  notify: restart apache2

apache-bundle.j2:

{{ godaddy_cert_download_response.json.pems.root }}
{{ godaddy_cert_download_response.json.pems.intermediate }}

apache-cert.j2:

 {{ godaddy_cert_download_response.json.pems.root }}

This worked a treat. The files were written correctly. The server contacted the API on its own, using credentials safely stored and securely transmitted from the Ansible control machine. Yay!

FIXME: There is obviously more that can be done here. For instance, we could generate a CSR on the server, and use that to request a cert update automatically, then place that new cert on the server all in one go, etc. WRiting a test for the role wouldn’t hurt either, of course..

Another blog post on that later, if you’re lucky :)

Ansible code here, feel free to submit PRs, etc.

Kudos