Let's run our own CA

Having our own certificate authority would let us generate certificates for our applications and, upon importing them in our OS or browser, provide secure communications. It would also reduce our dependence on third parties as well as the overall technical complexity of everything, which is especially useful for private homelab setup and self-hosted applications that aren't meant for a wider audience.

Before we begin, a bit of background

Nowadays we often count on Let's Encrypt to provide SSL/TLS certificates for our public sites, so that we can use HTTPS, to make communication between our user's browsers and the web server secure. This eliminates a whole group of man-in-the-middle attacks, for example, someone changing certain page contents (such as executable files that you may download), as well as prevents your ISP from injecting tracking scripts or advertisements into the pages.

And yet, there are cases when you might want SSL/TLS certificates for sites that aren't publicly available and thus the popular HTTP-01 challenge type won't work - you won't be able to get certificates from Let's Encrypt. Now, you might be able to use the DNS-01 challenge instead, but it's generally a bit harder to set up, because any automation that you might want to utilize will need to be able to setup the necessary DNS TXT record with your domain registrar.

For example, Apache recently added mod_md functionality for getting certificates without the need for certbot, which feels like a nice simplification as you no longer need two separate pieces of software for this, but even they choose not to get too involved with DNS-01 challenges at the time, instead allowing you to bring your own implementation for this:

Define a program to be called when the dns-01 challenge needs to be setup/torn down. The program is given the argument setup or teardown followed by the domain name. For setup the challenge content is additionally given.

You do not need to specify this, as long as a 'http:' or 'https:' challenge method is possible. However, Let's Encrypt makes 'dns-01' the only challenge available for wildcard certificates. If you require one of those, you need to configure this.

Which in practice might mean whipping out Go or Python or whatever and doing a bit of integrating with whatever API your DNS registrar provides, for example, using the Namecheap API, which might actually turn out to be a bit error prone and risky to do.

There's got to be a better way! And, for our case of mostly private sites, there is - having our own certificate authority and signing our own certificates. Now, the approach that I'll describe here won't have the benefit of automatically rotating certificates every 90 days, but in my case generating a certificate for a year (or 10) will be good enough.

So, how do we make the CA?

For the most part, a CA is essentially just a certificate that we can use for signing other certificates, as well as import its public part into our OS or browser, to trust it. Normally your operating system, browser and even runtimes like JDK provide their own root certificates, whereas in our case we're just going to add another one to the list. For example, here's how the default certificates look in Firefox:

example of root certificates

Now, normally working with certificates would mean using a bunch of OpenSSL commands, to the tune of this:

openssl genrsa -aes256 -out ca.key 4096
openssl req -x509 -new -nodes -key ca.key -sha256 -days 3652 -out ca.crt

Which would be enough for us to get our own key and CA certificate, like the following (after entering a bit of additional data in interactive prompts, because the syntax for that is even worse):

example of generated certificate

The problem here is that to me it doesn't look like the best developer experience, when you want to have simple and visual feedback about what you're doing. This is especially relevant, given that we might have a lot of sites and eventually CSRs to sign, doing which through the command line wouldn't exactly inspire happiness. This will be the case in the near future as well, at least while the CLI experience of OpenSSL will lag behind that of something like the Docker CLI, where every command has nice and easy to use examples and documentation, versus something like tar's user experience.

Oh, and the certificate that we generated isn't really a CA certificate, because there should be a few more identifiers added to it, to indicate its use case, doing which would necessitate a bit more configuration and longer commands.

So how do we make the CA, the lazy way?

Thankfully, there's a piece of software out there that's really easy to use and does most of what we need, thus lessening the risks of making mistakes in the CLI commands and instantly giving us visual feedback about our certificates. This piece of software is called Keystore Explorer and it is available for free. So how hard is generating a CA with it? I'm glad that you asked.

First, we create a keystore, which is where we'll store the certificates that we work with, including the CA one, as well as protect everything with a password. If needed, you can find out more about the keystore types here, but the default JCEKS should be enough for us:

keystore type

We can also immediately save the keystore, even though it's empty for now, and also give it a password of our choosing, ideally something secure and randomly generated:

save empty keystore

After that, we can generate the key pair, for which we have an easy option:

generate key pair

Here, we can also enter the information about our certificate:

entering data for our certificate

(the values here aren't super important, but might help identify the root certificate among others)

In addition, here we can also set those identifiers that we forgot or didn't know how to do previously, which will indicate how the certificate is to be used:

key usage

basic usage

After that, we enter whatever alias we want, as well as a password to protect our CA with and we are done. Then, we can save they keystore again with or CA in it, as well as export the certificate for other software to use, if needed:

export certificate

We'll need to do this, to import the certificate in our browser or OS, but for the most part, we'll be able to do most things inside of Keystore Explorer.

How do we import this CA certificate into the OS?

Suppose we want to trust this certificate for securing sites, with other certificates that it will sign. To do this, we must add it either to our OS, or our browser. One would let us trust it for any application that checks what the OS trusts, the other would be used just for websites.

First, let's finish that export process, in a format that our computer will understand:

export certificate chain

Then, in our OS we can just double click on the file and we'll get certificate details and will be able to import it, if needed:

install root certificate

On Windows we might also need to choose to install it as a trusted root certificate:

trusted root certificate

Which will then show something like this when our attempt succeeds:

now trusted

How do we import this CA certificate into the browser?

Suppose I'd want to import the certificate in the browser instead. To do that, I'd open my browser of choice, which in this case is Firefox, then go to Tools > Settings > Privacy & Security and choose the certificate option:

browser certificates

In the aforementioned window, we can choose the import option, which will then let us pick the file:

import

There, we can customize the import process, what we want to allow the certificate to be used for (or preview the certificate details):

import dialog

And finally, we get it in the list of the trusted certificates:

imported certificate

How do we generate and sign a certificate for a website with the CA?

So, what can we do with the CA and the fact that we trust it with our device? Well, the obvious thing is to generate a certificate for some application, which might run in the browser. Thankfully, Keystore Explorer also makes that easy for us, because honestly we can just use the same keystore that we already have, which would let us list all of our private certificates and see when any of those expire.

Of course, if we were to allow someone else to use our root certificate, the CA private key would stay with us, we'd give them the public certificate, whereas they'd have a private key of their own and they'd give us a certificate sign request for us to sign with our CA, resulting in a trusted certificate that we could return to them. Let's see how that might look! First, we'll generate another keypair for the site that we want to protect:

generating another keypair

We set its common name (CN) to the site domain that we'll use it for, but otherwise just give it an alias in the keystore and a password to protect it with. However, using just the CN for the site domain alone isn't actually supported as of a number of years ago, so if this is all that you'd do, you'd most likely get an error like SSL_ERROR_BAD_CERT_DOMAIN in your browser. Instead, we need to use the Subject Alternative Name (SAN) extension:

subject alternative name

Which, when added, should look a bit like this:

added subject alternative name

Then, we can generate a CSR for it, to sign with our CA:

generate CSR

Here we have a few options, but mostly we can leave these untouched, except maybe decide on a "Challenge" value, which will largely act as a shared nonce, which both parties could have if verification would ever be needed. Then, we get a .csr file which we can then sign with our root certificate:

sign CSR

Once we choose to sign the CSR, we'll be shown a number of options and will be able to decide on the validity period of our signature, which is basically just another certificate, which will form a certification chain:

CSR confirm

After this we get a file, which we can then import back as a CA response, to get our site certificate to have the full and trusted chain:

import CA reply

We can actually check this chain right there, to see whether everything is okay:

certificate chain details

Actually, if we export the certificate (not the private key), we can even check whether it's trusted by our computer. If the root CA is imported correctly and the site certificate is signed by the root CA according to the CSR, then we should get a valid chain of trust:

certificate chain shows up as valid

From there, we can use it with our web server, for securing the site that it's meant for.

How to actually use it?

First, we'll export the certificate chain to use with the web server of our choice:

export certificate chain

We'll also need that particular certificate's private key, in a format our web server can read:

export private key

In our case, I think that the OpenSSL format should be good enough for our needs:

private key export details

Of course, we indicate a password to protect the private key with, though support for that might depend on the web server in question.

Now we have the two files that we need: backups.kronis.eu.cer and backups.kronis.eu.key, which we're now going to copy over to the web server. Once that's done, we can configure the server to use our key and certificate for our site, which will vary based on what kind of a server you're using.

Personally, some of my homelab servers still use the old version of Caddy, which you probably shouldn't use, but the configuration format is delightfully simple, so that's what I'll use as an example. The configuration for it would look a bit like:

https://backups.kronis.eu {
  gzip
  proxy / some_proxied_path {
    transparent
  }
  tls /path/to/file.cer /path/to/file.key
}

And upon web server restart, the certificate should now be used!

In my case, I host it locally but if you could open it whilst having my root CA certificate, you'd see something like this:

certificate is trusted

In addition to that, it's also possible to look at the certificate details in full and see the certification chain:

certificate details

It wasn't all that hard and the software in question even largely guided us through everything, giving us a good idea of what options are actually available for any object in question!

Summary

Now, was this approach perfect? Not quite, however for when you want to manually have a few valid and trusted certificates, it's a pretty nice and relaxed way to go about it. If you needed to issue a hundred or more certificates per day, you'd probably look into a more automatable workflow and would need to work with the OpenSSL CLI. Even without that, it can also be useful to know it for the cases when something like Keystore Explorer won't be available or allowed. But for most other cases, it's a nice piece of software.

As for running our own CA, I'd say that it's a pretty good idea, because:

  • it allows not worrying about exposing our sites publicly or thinking about how to process DNS-01 challenges for Let's Encrypt
  • it simplifies certificate management because anything that needs to use our own certificates only needs to import the CA and the rest will "just work" (as long as you include the full certification chain)
  • Caddy v1 was a really simple web server for something like this and although I will migrate to Apache in the near future, having your own CA works equally well with Nginx (although I've seen it get finicky about the order in which certificate chains are concatenated inside of the file)
  • we can also relatively easily go a step further - for example, I have a basicauth setup for some of the more private URLs across my applications, but I might as well generate a client certificate or two with Keystore Explorer and make the server validate the client browser or device as well

Lastly, I'd like to suggest that having something like GUI software for learning can actually be pretty good - as long as you don't stop at just using it (when there is something as ubiquitous as OpenSSL out there), it can be an easier way to grasp some of the concepts and get immediate visual feedback of what you've achieved! Also, even if I don't know everything there is to know about certificates, something like Keystore Explorer allows me to get plenty of stuff done in a short amount of time. I'd argue that using the GUI is actually far quicker than messing about with the CLI, unless I have scripts already pre-written for it (which many sadly don't) or I've done it a million times and have memorized how this should work.

Update: client certificate example

Actually, I decided to set up a few client certificates for the sake of completeness as well. It turns out that actually it's not too hard and provides a nice layer of additional security (now I have both the client certificates, as well as basicauth for a few private sites).

First, we generate another key pair, this time also choosing to indicate the necessary values under Extended Key Usage:

extended key usage

After that, we go through the regular CSR/sign process, eventually exporting the whole key pair as a PKCS12 format store, because otherwise Firefox might get confused:

export key pair

Finally, we import the client certificate into the browser and enter our password once more:

import client certificate

You can even use the existing site certificates for signing your client certificates, without necessarily getting the root CA involved, which means that each site may request its own set of client certificates, in which case the import will look a bit like this:

client certificates for each site

Edit: of course, if your own certificate isn't meant for signing others, this might only work in some non-standard/lax software. For example, this works in Caddy but not in Apache, which I tested after migrating everything over to Apache shortly after finishing this post.

As for the server side, in the case of Caddy, we simply needed to add our CA (the public certificate) to the server and indicate that we want to require client certificates:

https://backups.kronis.eu {
  gzip
  proxy / some_proxied_path {
    transparent
  }
  tls /path/to/file.cer /path/to/file.key {
    clients require /path/to/ca.cer
  }
}

Provided that I haven't leaked too much information in this post already, the whole setup should be reasonably secure! Of course, one might choose to have intermediate certificates as well, instead of using the CA directly, but for simpler setups the need to do that isn't necessarily there.