Moving from GitLab CI to Drone CI

Recently i posted about how GitLab updates are broken, at least when attempting to self host it and occasionally update it on my VPSes, which necessitated me moving to something better suited to my particular set of circumstances.

Now, at the time of writing this, i've already finished the migration which replaces 90% of what GitLab does for me (though certainly not all of its functionality that others might use) in a more lightweight and more easily manageable deployment format, that's hopefully also slightly less resource hungry.

Thus, i figured that i might just document the process and the outcomes of it all, over a few posts, each of which will focus on particular aspects:

• interacting with the source code
• doing automated CI/CD builds of apps (this article)
• persisting dependencies and container images

Without further ado, let's get going!

Why Drone CI

Previously i used GitLab CI and actually really liked it - in my eyes, it's actually more comfortable and productive to use than many of the alternatives, like GitHub Actions. However, today i've picked Drone CI as my self-hosted replacement for GitLab's offering:

Actually my choice for it was pretty simple: it's largely container native, works off of build step descriptions in files that are much like shell scripts and is wonderfully boring and lightweight. For a moment, i considered the likes of Jenkins, but personally my experience with it has been almost wholly negative - there are far too many plugins and moving parts that tend to break in complex setups, as well as if you're ever using the file system for workspace files or system packages for builds (like JDK or Node), you risk on really screwing yourself over.

Thus, any and all packages that use short lived containers for building apps are one of the better approaches, containers being an almost ideal use case for this in my eyes. Also, Drone CI in particular is fairly feature complete, having almost everything that i could care about and integrates nicely with Gitea and any supported container registry, Sonatype Nexus included.

So, let's get going!

Getting Drone CI up and running

It is actually pretty interesting, in that it uses your source management platform for authentication, so the first step was to actually set up an OAuth2 application inside of Gitea:

After that, their documentation gives you examples of how to launch Drone with Docker:

In my case after a few tweaks i settled on the following Docker Compose stack which i launch with Docker Swarm on my own container cluster with a simple bind mount for the data:

version: '3.3'
services:
  drone:
    image: drone/drone:2.9.1
    environment:
      - DRONE_GITEA_SERVER=https://git.kronis.dev
      - DRONE_GITEA_CLIENT_ID=SOME_VALUE
      - DRONE_GITEA_CLIENT_SECRET=SOME_VALUE
      - DRONE_RPC_SECRET=SOME_VALUE
      - DRONE_SERVER_HOST=ci.kronis.dev
      - DRONE_SERVER_PROTO=https
      - DRONE_USER_CREATE=username:KronisLV,admin:true
    networks:
      - ingress_network
    volumes:
      - ..../drone/data:/data
    deploy:
      placement:
        constraints:
          - node.hostname == SOME_VALUE.servers.kronis.eu
      resources:
        limits:
          memory: 512M
          cpus: '0.75'
networks:
  ingress_network:
    driver: overlay
    attachable: true
    external: true

As you can see, there is actually a part there that makes me the admin of my instance, which i'll explain in more detail later:

DRONE_USER_CREATE=username:KronisLV,admin:true

But once again, because the software is so simple, it launched shortly after figuring out the YAML that Omnissiah demands:

Similarly, my configuration for Drone Runner, the actual component that's needed to run the tasks (the previous one only allowing you to use the UI to change configuration and view the results) was also similarly simple and was also sufficiently described in their documentation:

One Compose file later, i had a Runner up and running, as one would expect given its name:

version: '3.3'
services:
  drone_runner:
    image: drone/drone-runner-docker:1.8.0
    environment:
      - DRONE_RPC_HOST=ci.kronis.dev
      - DRONE_RPC_PROTO=https
      - DRONE_RPC_SECRET=SOME_VALUE
      - DRONE_RUNNER_CAPACITY=2
      - DRONE_RUNNER_NAME=SOME_VALUE
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.hostname == SOME_VALUE.servers.kronis.eu
      resources:
        limits:
          memory: 4096M
          cpus: '1.85'
networks:
  ingress_network:
    driver: overlay
    attachable: true
    external: true

And voila, that was pretty much all i needed.

Figuring out how to make pipelines work

Of course, the question of how to actually do anything remains, so i figured that i might as well test it all out and give you a bit of a walkthrough. I'll actually offer the complete pipeline definition at the bottom of this section, but i'd like to walk you through the full process of getting it up and running.

For that exact purpose, i created a new Git repository in Gitea:

Luckily, it seems like everything's fine there and Git clone still works (after revoking my old password):

They also have some documentation on how the actual pipelines should be structured, which should be simple enough to figure out, so that was definitely helpful. In it, you'll find descriptions for most of the things you might want to do, from simple environment variable and secret management, to shared storge between build steps, parallel execution, DAG and so on:

But let's not get ahead of ourselves, first we just need to enable a repository from the list:

Then, you get some useful configuration that's actually more useful than GitLab's, at least in my opinion, due to how clear and well structured everything is (honestly, Drone is not just really boring and stable, but also pretty to look at AND with good UX):

The configuration covers most of the stuff that you might need, which in my case necessitated adding login credentials for my registry:

Also, they let you do something similar to GitLab CI's schedules, though i'm yet to figure out how to run things at a particular time (they don't seem to give you a regular cron expression input option, maybe to stagger the tasks):

And, after running our first task... we get an error:

But hey, at least we know that the badges work:

This was where i discovered that if you want to use volumes with host directory for any purpose (e.g. time zone, or Docker socket in case you have a really good reason to do this), you need additional configuration:

Of course, my user in Gitea didn't have the necessary permissions to choose that option, which is why i needed the aforementioned DRONE_USER_CREATE option, as their docs once again explain:

After that, it was as easy as enabling the trusted configuration for the project:

Then, you can finally successfully build your apps:

(see below for the task definition)

Another thing that i did set up was a secret for authenticating with all of my Docker Registries inside of Nexus, so pulling any images that i'd need would become far easier:

Because of that, suddenly i could create the base image that i'd need in the later steps, push it in the first and have it be available when necessary later, without persistent local storage, which is ideal if i want to create reusable and sometimes commonly shared images (e.g. one for Docker, one for Maven, one for OpenJDK etc., everything from build tools to images to run apps with):

But it doesn't just end there! Drone also adopts the DAG approach that you might be familiar with from GitLab CI, which allows you to define what depends on what and let the software dynamically figure out the build order:

Maybe it's my familiarity with GitLab CI or the fact that i've actually used Drone CI in the past, but overall the experience was a pretty positive one - i had it up and running, with tasks that could be successfully executed and integrated with my Nexus in less than an hour:

Of course, it doesn't quite end there.

Summary

Not only was Drone easy to get up and running and was surprisingly sane as far as both its UI and UX go, low on resource usage and pretty snappy, but it also even reports build statuses back to Gitea:

While Nexus is maybe a bit of a problematic solution (as described in the previous tutorial, mostly due to its memory usage), overall Gitea and Drone CI play together wonderfully nice and are really well suited for my needs, moreso than GitLab was!