{
    "version": "https://jsonfeed.org/version/1",
    "title": "blog.kronis.dev",
    "home_page_url": "https://blog.kronis.dev/",
    "feed_url": "https://blog.kronis.dev/blog.json",
    "description": "My blog, where I attempt to collect my thoughts and share the occasional interesting topic with others",
    "author": {
        "name": "Kristiāns Kronis"
    },
    "items": [
        {
            "title": "I rebuilt my blog (and then broke it)",
            "date_published": "2026-06-13",
            "id": "https://blog.kronis.dev/blog/i-rebuilt-my-blog-and-then-broke-it",
            "url": "https://blog.kronis.dev/blog/i-rebuilt-my-blog-and-then-broke-it",
            "content_html": "\u003cp\u003eYou might notice that my blog now looks a bit different from how it used to! Let me tell you a bit about the old setup, why and how I got to a new one, and also offer up some basic benchmarks to compare the two with one another. Long story short, I wanted to eliminate as many dependencies as possible, so I'd have to update the thing less often - which led to me making my own static site generator.\u003c/p\u003e\n\u003cp\u003eHere's a quick comparison with the previous look:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-old-blog-with-darkreader.jpg\" alt=\"01-old-blog-with-darkreader\" title=\"01-old-blog-with-darkreader\"\u003e\u003c/p\u003e\n\u003cp\u003eOr, at least, that's what it was like with the \u003ca href=\"https://links.kronis.dev/TfwsWJdICH\"\u003eDark Reader extension\u003c/a\u003e that I usually have enabled on my browser and have gotten used to, to the point where I don't even notice it. In contrast, here's what most of the other users would have seen by default. To avoid a flashbang inline, you can view the file here: \u003ca href=\"02-old-blog-regular-colors.jpg\"\u003e02-old-blog-regular-colors.jpg\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eIt used to run on \u003ca href=\"https://links.kronis.dev/rp653O2RoZ\"\u003eGrav\u003c/a\u003e, a flat file CMS that I still think is quite lovely! One of the reasons for me to initially choose it, was that I could write posts in Markdown (with some custom extensions, e.g. image resizing), and that I wouldn't need to host a full sized database - since WordPress setups, in comparison, are known to be a bit harder to get running well. I didn't care that much about having a huge ecosystem, nor having that many plugins.\u003c/p\u003e\n\u003cp\u003eQuite on the contrary, as are the reasons for me moving away from Grav - I want less, not more. With an ever increasing number of packages getting pwned, I realize one thing: when I wrote that slightly absurdist article called \u003ca href=\"https://links.kronis.dev/razox\"\u003e\u0026quot;Never update anything\u0026quot;\u003c/a\u003e, I might have been a few years early to the party. My Grav setup \u003ca href=\"https://links.kronis.dev/zmv8cIA4jM\"\u003egot vandalized once\u003c/a\u003e and while it wasn't a very big deal, I have reason to think that the vulnerabilities out there will get more exploited as time goes on, not less. The good news is that my blog getting compromized wouldn't cause any serious monetary losses or whatever, the bad news is that as time goes on I have less and less time to actually keep software up to date. It stands to reason that software that has fewer dependencies and features will also have fewer vulnerabilities due to a smaller attack surface - that's also the reason why I'm considering moving away from Nextcloud for something simpler as well.\u003c/p\u003e\n\u003cp\u003eWhen it came to the blog itself, my choice was pretty clear - I'd just write my own static site generator (SSG) so I don't need a large PHP project, so that the attack surface would be smaller, and also because honestly I didn't need either the admin plugin or the analytics, or backups built into Grav, ever since I moved to a fully Git driven approach to publishing stuff: \u003ca href=\"https://links.kronis.dev/Y6W53boNVM\"\u003eMy blog doesn't need quality, it needs to look like it's from the 90s\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eCome to think of it maybe \u0026quot;quality\u0026quot; wasn't the best word choice, something like \u0026quot;modern look\u0026quot; could have been better there, but you get the idea - I went from using the admin UI, to pushing from Git and letting the CI do the builds, to now more or less knowing that I'm fine with a fully static site generator that also has a fast local development loop, and any other features that I might want (like having a back end component for search).\u003c/p\u003e\n\u003cp\u003eThere are quite a few good SSGs out there, but since I want to decrease my dependency on external projects, I just slopped something together over the last week in \u003ca href=\"https://links.kronis.dev/2NkFQhjOkw\"\u003eGo\u003c/a\u003e, here's the new look with a native dark mode:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03-new-blog-dark-mode.jpg\" alt=\"03-new-blog-dark-mode\" title=\"03-new-blog-dark-mode\"\u003e\u003c/p\u003e\n\u003cp\u003eThe default is actually whatever the user selects, so if you prefer a light mode look (e.g. if you spend sufficient time outside in sunlight that's probably better for your eyes), this is what you get instead, same as above, the bright version may be viewed separately: \u003ca href=\"04-new-blog-light-mode.jpg\"\u003e04-new-blog-light-mode.jpg\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eStill the same font stack, slightly different typography, still made to mostly work okay without JS enabled, in addition to supporting mobile decently:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"05-new-blog-mobile-look.jpg\" alt=\"05-new-blog-mobile-look\" title=\"05-new-blog-mobile-look\"\u003e\u003c/p\u003e\n\u003cp\u003eWhile I did iterate a bunch and tweaked quite a few things along the way, I don't exactly imagine that heavily using AI to actually get something done while also being overworked in my day job will result in the perfect project, so I picked an apt name for this new project, Gravis (\u0026quot;Grāvis\u0026quot; being the fully Latvian version, which translates to \u0026quot;ditch\u0026quot;):\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"06-my-own-static-site-generator.jpg\" alt=\"06-my-own-static-site-generator\" title=\"06-my-own-static-site-generator\"\u003e\u003c/p\u003e\n\u003cp\u003eSo, what is it? It's a Go app that uses \u003ccode\u003enet/http\u003c/code\u003e and as much stuff as I could get working out of the box, along with a few good dependencies like \u003ca href=\"https://links.kronis.dev/jdCjvIiy7j\"\u003egoldmark\u003c/a\u003e to produce and serve HTML files, with some bundled CSS and fonts. As much as possible is pushed to the build time and the live version lives in a container and pretty much has a read only workload.\u003c/p\u003e\n\u003cp\u003eFor search, I do use a pre-populated SQLite database file, with their \u003ca href=\"https://links.kronis.dev/5YQupPPIid\"\u003eFTS5 extension\u003c/a\u003e for full text searching. It supports a dev mode where it watches the directory for changes and rebuilds files whenever they change, and in general is a bit kinder to the hardware resources than the PHP app (or at least how I had it set up) was.\u003c/p\u003e\n\u003cp\u003eThere are also things that it doesn't do - since I use \u003ca href=\"https://links.kronis.dev/XlsNfipwqX\"\u003eApache2\u003c/a\u003e as my reverse proxy (personal choice, it's good enough and their docs are great), the web server can handle ACME/Let's Encrypt stuff with \u003ca href=\"https://links.kronis.dev/3b2pTW6xS0\"\u003emod_md\u003c/a\u003e, similarly, Apache2 can also handle setting headers for me, in addition to compression and whatever else makes more sense to do there instead of in Go code.\u003c/p\u003e\n\u003ch2\u003eLoad testing the new blog\u003c/h2\u003e\n\u003cp\u003eI also decided that I probably wanted to load test how well (or badly) each of the instances perform various tasks - opening the homepage, opening the blog posts, using the page listings and also the search.\u003c/p\u003e\n\u003cp\u003eWhile Grav isn't horribly slow itself, I did set it up with \u003ca href=\"https://links.kronis.dev/gowMFPmfrG\"\u003emod_php as one of the simpler ways to get PHP working\u003c/a\u003e, which does get critiqued quite a bit in comparison to \u003ca href=\"https://links.kronis.dev/VXgotrwdZR\"\u003ePHP-FPM\u003c/a\u003e, which is faster and which I also used to use, but decided against in the most recent iteration of rebuilding my own container base images (since there are so many: Node, Java, .NET, Go, PHP, Ruby, ... and I wanted things to be simple and low effort). The \u003ccode\u003emod_php\u003c/code\u003e setup did work, just oddly enough seemed to be worse locally in particular, while trying to edit and preview new pages.\u003c/p\u003e\n\u003cp\u003eIn regard to the actual testing, I made a few concessions: I'd need to test both with load testing tools \u003ca href=\"https://links.kronis.dev/xqHxKVVWus\"\u003esuch as K6\u003c/a\u003e, but also get at least basic metrics from real browser loads of the pages, for which I chose \u003ca href=\"https://links.kronis.dev/hGGaIy0Gxb\"\u003ePlaywright\u003c/a\u003e, which is great and which I can totally recommend you take a look at as well, especially if you ever need to do browser based testing (sure beats Selenium, in my experience, nice API). My own experiences with K6 are a bit more mixed - it's pretty nice for the things regarding HTTP, however the last time when I tried to integrate it with a WebSocket based auction application, it was just endless headaches and I couldn't get it fully working, though maybe things are better in the recent versions.\u003c/p\u003e\n\u003cp\u003eThe goal was to be able to both launch everything locally in Docker containers, as well as test against the remote instances, and get a good look at how well things perform:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"07-load-testing-it.jpg\" alt=\"07-load-testing-it\" title=\"07-load-testing-it\"\u003e\u003c/p\u003e\n\u003cp\u003eA somewhat important note, however, is that this isn't what most people would call a great or very scientific approach to testing - I'm not here to make any strong generalized claims about the overall performance of either solution, especially given my setup details. Plus, I'm not even trying to eliminate all external factors, just look at how I'm re-using the very same CPU for both running the software and also the benchmarks (though at least when I talked to the remote instances having a 1 Gbps Internet connection was helpful to at least eliminate that kind of a bottleneck). Heresy, I tell you:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"08-local-resource-usage.jpg\" alt=\"08-local-resource-usage\" title=\"08-local-resource-usage\"\u003e\u003c/p\u003e\n\u003cp\u003eAt the same time, the tests below are enough to give us a pretty good idea about how they perform in reality (in my specific circumstances). In a word, they let me show you why my quality of life improved a bunch after this migration, when it comes to blogging.\u003c/p\u003e\n\u003cp\u003eOh, and if I'm talking about the methodology, the K6 part is pretty straightforward - I just create a bunch of virtual users (VUs) thanks to K6 that use whatever functionality I need them to, be it the search or browsing or anything else, the data about which is then aggregated and exported, which later on let me create a few graphs that you'll find below soon. Furthermore, the browser based tests load the full pages, with the content that gets served statically. While Apache2 is the common ingress that sits in front of either of the apps on the server, locally I talk to them directly (Grav also uses Apache2, its own instance with PHP, whereas Go serves the static assets itself), and it's nice to see how the overall metrics of either differ.\u003c/p\u003e\n\u003cp\u003eI didn't try to do load tests with the browser instances, because they're far slower and heavier than K6 workers, and because computationally serving those static assets isn't super intensive and is more like pushing a bunch of data through a tube. Consequently, that's also why going from the \u003ca href=\"https://links.kronis.dev/aq9sc\"\u003eTwig templates\u003c/a\u003e on PHP to pre-rendered PHP pages (even if they have their own cache as well) to Go does make at least serving blog posts way simpler, even though the search is still dynamic.\u003c/p\u003e\n\u003cp\u003eEither way, I could at least compare all of the instances as I wanted, though the real blog had to be re-run before the migration when it was still on PHP and after the migration to test the Go implementation:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"09-also-using-real-browser-tests.jpg\" alt=\"09-also-using-real-browser-tests\" title=\"09-also-using-real-browser-tests\"\u003e\u003c/p\u003e\n\u003cp\u003eHere's also a quick example of the browser based test that went through all of the posts, one by one:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"10-going-through-all-the-pages.jpg\" alt=\"10-going-through-all-the-pages\" title=\"10-going-through-all-the-pages\"\u003e\u003c/p\u003e\n\u003cp\u003eAnd here's an excerpt from the logs, showing that those particular tests also fetched all of the static assets (I could automate that with K6 as well, but it'd be a bit more annoying to extract all of the static image links and such, and to estimate how a real browser fetches those):\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"11-more-realistic-tests.jpg\" alt=\"11-more-realistic-tests\" title=\"11-more-realistic-tests\"\u003e\u003c/p\u003e\n\u003ch2\u003eSo, what are the numbers?\u003c/h2\u003e\n\u003cp\u003eSo, let's get into it. I simulated up to 256 virtual users, which was enough for me to get a pretty accurate look into how the instances perform.\u003c/p\u003e\n\u003cp\u003eFirst up, here are the containers at idle (ignore the ones that aren't relevant here), as you can see, the Grav (\u003ccode\u003egrav2\u003c/code\u003e) based setup and Apache2 (\u003ccode\u003eapache2_ingress\u003c/code\u003e) don't consume that many resources at idle and everything seems pretty calm:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"12-grav-containers-idle.jpg\" alt=\"12-grav-containers-idle\" title=\"12-grav-containers-idle\"\u003e\u003c/p\u003e\n\u003cp\u003eHowever, once I applied enough load, it became apparent that Grav was the bottleneck here (thanks to \u003ccode\u003emod_php\u003c/code\u003e):\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"13-grav-containers-under-load.jpg\" alt=\"13-grav-containers-under-load\" title=\"13-grav-containers-under-load\"\u003e\u003c/p\u003e\n\u003cp\u003ePeople used to say that you typically shouldn't worry that much about slight differences in the performance of your web server, however in this case I proved them both wrong and right - wrong, because it's not the reverse proxy (that handles TLS, compression and routes traffic between all of the containers I want) but the application server that's slow here, which ALSO happens to be the same web server, just configured with a slow runtime.\u003c/p\u003e\n\u003cp\u003eHtop also demonstrates this in a visually pleasant way:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"14-grav-htop-idle.jpg\" alt=\"14-grav-htop-idle\" title=\"14-grav-htop-idle\"\u003e\u003c/p\u003e\n\u003cp\u003eYou will notice that the usage doesn't go much past 50%, which is because I place limits on the containers themselves:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"15-grav-htop-under-load.jpg\" alt=\"15-grav-htop-under-load\" title=\"15-grav-htop-under-load\"\u003e\u003c/p\u003e\n\u003cp\u003ePersonally, I'd much rather have containers buckling under load (there are automated restarts if they break entirely in place), rather than the whole server becoming unresponsive, especially since it also runs a bunch of other stuff and containers. I guess my current issue is that I don't have an easy way of telling Docker \u0026quot;Hey, never use more than 95% of the server's CPU\u0026quot; or maybe reserving some resources for the SSH server.\u003c/p\u003e\n\u003cp\u003eWhile I could probably (hopefully?) figure out how to setup that with cgroups, \u003ccode\u003ecpus: '2.0'\u003c/code\u003e is delightfully simple (and generally good enough, though I'd like better support for bursty workloads when the other containers aren't doing anything). Either way, none of this is as bad as \u003ccode\u003ekswapd\u003c/code\u003e deciding to eat 100% of my CPU instead of just doing OOM kills of containers when there's memory over-commit and the swap gets exhausted on Oracle Linux that I couldn't really find a way around, when I had to use that.\u003c/p\u003e\n\u003cp\u003eWhatever the case, Gravis immediately performed better and used less CPU. We more or less saw the opposite compared to the previous situation here: it could process more traffic than Apache2. At lower loads (anything realistic) this wasn't an issue because both sat under the total limits, here's an example of 64 VUs:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"16-gravis-containers-testing-64vu.jpg\" alt=\"16-gravis-containers-testing-64vu\" title=\"16-gravis-containers-testing-64vu\"\u003e\u003c/p\u003e\n\u003cp\u003eHere's an example of 128VUs:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"17-gravis-containers-testing-128vu.jpg\" alt=\"17-gravis-containers-testing-128vu\" title=\"17-gravis-containers-testing-128vu\"\u003e\u003c/p\u003e\n\u003cp\u003eAnd here's an example of 256VUs:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"18-gravis-containers-testing-256vu.jpg\" alt=\"18-gravis-containers-testing-256vu\" title=\"18-gravis-containers-testing-256vu\"\u003e\u003c/p\u003e\n\u003cp\u003eThankfully, this blog doesn't have so many readers to make me deal with that many requests hammering the server all the time, but it's interesting to see that with the static site generation suddenly we see the Go implementation be roughly twice as capable as the Apache2 instance sitting in front of it. I suspect that difference would narrow if Go had to also do compression and TLS termination, while Apache2 would only get marginally slower if it was serving the files itself (which in my setup it cannot do, because it only routes traffic, as ingress should). No matter how you look at it, it's nice to see that neither of the solutions is an order of magnitude slower than the other, meaning that even without a bunch of tuning, the overall setup is decent.\u003c/p\u003e\n\u003cp\u003eHere's some examples of how the server handled the increasing load with Gravis, 64 VUs first:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"19-gravis-htop-testing-64vu.jpg\" alt=\"19-gravis-htop-testing-64vu\" title=\"19-gravis-htop-testing-64vu\"\u003e\u003c/p\u003e\n\u003cp\u003eThen 128 VUs:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"20-gravis-htop-testing-128vu.jpg\" alt=\"20-gravis-htop-testing-128vu\" title=\"20-gravis-htop-testing-128vu\"\u003e\u003c/p\u003e\n\u003cp\u003eAnd finally, 256 VUs:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"21-gravis-htop-testing-256vu.jpg\" alt=\"21-gravis-htop-testing-256vu\" title=\"21-gravis-htop-testing-256vu\"\u003e\u003c/p\u003e\n\u003cp\u003eThe local testing was quite similar, except instead of hitting the live docker containers running on an Ubuntu server with Docker, I was instead talking to local Docker containers running on Windows (through their WSL based engine). Since there's already a few images of logs in this here post, let's jump onwards to some graphs I made.\u003c/p\u003e\n\u003ch2\u003eHow about some pretty graphs?\u003c/h2\u003e\n\u003cp\u003eAll of the testing above also aggregated the data locally in JSON files for me to then process. I figured that with what I had I could provide two groups of results (the load testing and then the browser testing), as well as a few metrics in each. Let's proceed!\u003c/p\u003e\n\u003ch3\u003eK6 testing - Throughput\u003c/h3\u003e\n\u003cp\u003eHere's the most basic of metrics we might care for, essentially, how many requests we can successfully process per second. Here, I found some odd things and had to retest a few times, just to confirm my findings, which seemed to hold.\u003c/p\u003e\n\u003cp\u003eIt was immediately obvious that the Go implementation was way faster (as you'd expect not only from Go in comparison to PHP through \u003ccode\u003emod_php\u003c/code\u003e, but also from the static vs dynamic rendering), however, there was a point at which the throughput on the server instance seemed to collapse:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"22-k6-homepage-throughput.jpg\" alt=\"22-k6-homepage-throughput\" title=\"22-k6-homepage-throughput\"\u003e\u003c/p\u003e\n\u003cp\u003eDon't get me wrong, 1.2k requests is still a lot, and certainly more than the 69 requests/second that the Grav version used to get me, but that seems to suggest that something along the way was starting to go wrong. Apache2 might have been unable to keep up with the load due to its limited parallelism configuration, because it was still sitting in front of the Go application, but it being able to process 3.5k requests/second with 128 VUs and then almost 3 times less at 256 VUs was puzzling.\u003c/p\u003e\n\u003cp\u003eI might have to dig into the logs to understand this better at some point, but thankfully the load isn't very realistic for now, since while hitting the HN front page did make Grav struggle a lot, it's not like the whole site would go down immediately because of it. Oh, and you can also see that the local performance of Grav was quite trash, which held consistent across all the benchmarks - seems like Docker containers on Windows are a really problematic environment for Apache2 running PHP for whatever reason.\u003c/p\u003e\n\u003cp\u003eThe blog posts seemed to scale a bit more linearly and closer to how you'd expect. At lower loads there's not a world of difference here, but at 256 VUs, Gravis absolutely crushes Grav with 1.6k requests/second vs just 70 requests/second:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"23-k6-blog-posts-throughput.jpg\" alt=\"23-k6-blog-posts-throughput\" title=\"23-k6-blog-posts-throughput\"\u003e\u003c/p\u003e\n\u003cp\u003eMore or less the same could be observed with listing the blog pages, where Grav was even slower, the local Gravis instance had a similar collapse of performance, which suggests that it's not Apache2 but something else, because locally there was no Apache2 in front of it, only on the server, whereas the Gravis docker containers on the server had no issues to speak of:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"24-k6-listing-pages-throughput.jpg\" alt=\"24-k6-listing-pages-throughput\" title=\"24-k6-listing-pages-throughput\"\u003e\u003c/p\u003e\n\u003cp\u003eSearch throughput was where it arguably got way worse, though Gravis was still plenty fast in comparison to my Grav setup, so that's still a win:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"25-k6-search-throughput.jpg\" alt=\"25-k6-search-throughput\" title=\"25-k6-search-throughput\"\u003e\u003c/p\u003e\n\u003cp\u003eIt did make me wonder about what might cause the performance degradation, though. On the server, I have HDDs running with a RAID array, whereas locally I have a SATA SSD (NVMe for the OS, but the projects are on a SATA SSD, though sadly it's one of those QLC ones, so the write performance sucks for other workloads), which makes me think that I might be I/O bound in a way that only seems to surface under a greater load.\u003c/p\u003e\n\u003cp\u003eThis wouldn't be impossibly hard to test, I'd just need to load all of the data in memory upon startup - it's a blog so even with all of the images and whatnot it's less than 1 GB and the pages themselves probably fit on a few floppies, so it'd be perfect for being tested that way. On the other hand, the SQLite database isn't really written to and since it's just a regular file, surely it'd end up on RAM as a part of disk caching anyways, right?\u003c/p\u003e\n\u003cp\u003eNot world's biggest mystery to solve, and I probably don't have the free time for it right now (am writing this at around 10-11 PM), but would be nice to look into at some point! It's either that, or some weirdness about either OS sockets or maybe how Go is serving stuff, or the reverse proxy in the case of the server. There, it seems to cap out and work at around 115 requests/second, whereas locally we still see that collapse.\u003c/p\u003e\n\u003cp\u003eI probably should have pulled in I/O stats from the environments themselves, in addition to performance metrics on the app side. It might also be a case of different bottlenecks across the environments and configurations.\u003c/p\u003e\n\u003ch3\u003eK6 testing - p95 response time\u003c/h3\u003e\n\u003cp\u003eIn regards to the p95 response time, we see something else that's interesting - it's not horrible on Grav when it's running on the server, but it absolutely is locally, which just further illustrates how writing blog posts was just ever so slightly annoying locally:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"26-k6-homepage-p95.jpg\" alt=\"26-k6-homepage-p95\" title=\"26-k6-homepage-p95\"\u003e\u003c/p\u003e\n\u003cp\u003eThe blog posts p95 still shows that Grav is struggling, whereas Gravis locally also seems to similarly collapse, but works just fine on the server:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"27-k6-blog-posts-p95.jpg\" alt=\"27-k6-blog-posts-p95\" title=\"27-k6-blog-posts-p95\"\u003e\u003c/p\u003e\n\u003cp\u003eListing pages seem to have no issues with Gravis:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"28-k6-listing-pages-p95.jpg\" alt=\"28-k6-listing-pages-p95\" title=\"28-k6-listing-pages-p95\"\u003e\u003c/p\u003e\n\u003cp\u003eAnd the search also does pretty well:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"29-k6-search-p95.jpg\" alt=\"29-k6-search-p95\" title=\"29-k6-search-p95\"\u003e\u003c/p\u003e\n\u003ch3\u003eK6 testing - error rate\u003c/h3\u003e\n\u003cp\u003eNext up, the error rates in percent of all requests. I might have needed to instead show the failed request counts, but luckily the percentage values are largely good enough, because the overall picture is very clear here. We had almost no errors at lower concurrency, and they really only showed up at 256 VUs:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"30-k6-homepage-err.jpg\" alt=\"30-k6-homepage-err\" title=\"30-k6-homepage-err\"\u003e\u003c/p\u003e\n\u003cp\u003eAs it was in the homepage test, it's also more or less the same in the blog posts:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"31-k6-blog-posts-err.jpg\" alt=\"31-k6-blog-posts-err\" title=\"31-k6-blog-posts-err\"\u003e\u003c/p\u003e\n\u003cp\u003eAnd also in the page listings:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"32-k6-listing-pages-err.jpg\" alt=\"32-k6-listing-pages-err\" title=\"32-k6-listing-pages-err\"\u003e\u003c/p\u003e\n\u003cp\u003eAnd finally also in the search:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"33-k6-search-err.jpg\" alt=\"33-k6-search-err\" title=\"33-k6-search-err\"\u003e\u003c/p\u003e\n\u003cp\u003eOn the server, things were mostly fine with both Grav and Gravis just slowing down, whereas locally with enough parallelism the requests would start failing. Thankfully, even if that were to be the same failure mode on the server, 256 VUs slamming the server corresponds to a much, much higher real user count - because typically a user would just load a page and read it, a few more images would lazily load in as they scroll down, meaning that this can easily support anywhere between 2k-10k real users looking at it before things get bad, traffic that this blog usually doesn't even generate.\u003c/p\u003e\n\u003cp\u003eAs I said, the testing isn't very scientific and while there's not enough data for me to draw exact conclusions from, at least it gives me an overall idea - the new setup won't be worse than Grav, with the added benefit of me needing to worry a bit less about the dependencies and updates. I can also only write and include the features that I need, as well as re-test things later and optimize the codebase more, if I figure out exactly what the current obstacles are.\u003c/p\u003e\n\u003cp\u003eSimultaneously, if it's proven at any point that the bottleneck now is truly just Apache2, at that point I can evaluate whether I even need to push further and switch to a different web server like Caddy, or whether at that point it'd just be optimization for the sake of optimization, instead of for real loads that my blog is likely to see.\u003c/p\u003e\n\u003ch2\u003eBrowser testing - other metrics\u003c/h2\u003e\n\u003cp\u003eIn addition to the above, I also did the aforementioned browser based testing, where I checked some additional metrics, to see how the apps perform with real browsers. First up, there was the page weight - since I was looking at the same pages, with the same images and same fonts, you can see that the averages are also very similar:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"34-page-weight-kb.jpg\" alt=\"34-page-weight-kb\" title=\"34-page-weight-kb\"\u003e\u003c/p\u003e\n\u003cp\u003eThe differences are mostly due to me having a bit less CSS and JS in Gravis, whereas the local and remote pages differ ever so slightly due to some templating changes in the configuration. In general, however, you can see that a full page load is around 250 KB (in no small part thanks to my choice to self-host the fonts as well), whereas the blog posts add space on top of that with images:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"35-time-to-first-byte.jpg\" alt=\"35-time-to-first-byte\" title=\"35-time-to-first-byte\"\u003e\u003c/p\u003e\n\u003cp\u003eThe first contentful paint in a real browser demonstrates some of the more annoying aspects that I had to deal with in regards to testing the Grav setup locally, where for whatever reason the render blocking resources take a while to load, at least compared to all the other setups:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"36-first-contentful-paint.jpg\" alt=\"36-first-contentful-paint\" title=\"36-first-contentful-paint\"\u003e\u003c/p\u003e\n\u003cp\u003eAnd finally, we see something similar in regards to the full page load, working locally just wasn't pleasant:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"37-full-load-time.jpg\" alt=\"37-full-load-time\" title=\"37-full-load-time\"\u003e\u003c/p\u003e\n\u003cp\u003eDespite the throughput being better, the Gravis setup still takes more overall time to load on the actual server, than in the PHP based Grav setup, though it wasn't a big difference.\u003c/p\u003e\n\u003ch2\u003eSummary\u003c/h2\u003e\n\u003cp\u003eIn summary, I am both pleased that I managed to get this done, as well as a bit puzzled by the initial results.\u003c/p\u003e\n\u003cp\u003eGrav is pretty nice, but writing my own SSG was a pretty nice choice - now there are fewer dependencies for me to manage, the overall setup is simpler, it does everything I need it to (and nothing I don't), it fits well within my stack and local development is quite the joy. For example, when I add a new sentence or paragraph to this file in Zed and hit save, I see this and the new page is available for preview before I can manage to refresh it in the browser:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003eUpdating...\n  I rebuilt my blog (and then broke it)\n  Updating listings, feeds, search, sitemap, robots.txt...\nDone.\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eI could maybe even explore a setup similar to what I have for a presentation tool I wrote, where there's a script within the page that talks to the dev server and automatically refreshes the page, so I don't need to.\u003c/p\u003e\n\u003cp\u003eAt the same time, the testing showed that under realistic loads Gravis will be a pretty good replacement for Grav and won't have issues serving similar loads, but that under heavier loads the performance and throughput can collapse. It's generally still better than the old baseline, but the failure modes seem to be a bit different across the different environment. What's worse, it seems to happen even when the task is to just serve up index.html to a lot of concurrent users - a pre-rendered file that should fit within any sort of a file cache that the OS has.\u003c/p\u003e\n\u003cp\u003eWhile I no longer have the old Grav setup to compare against since I've replaced it with Gravis now, I'll probably need to do some additional testing another day when I have the time and maybe trace down exactly why the results look a bit funky. Until then, a lot of the above is guesswork, on one hand pulling in more metrics from the actual environments would have helped a bunch - but on the other, that'd extend this well into the Sunday, not just Saturday, I still have plenty of stuff to do tomorrow, unrelated to this.\u003c/p\u003e\n\u003cp\u003eThe ideal circumstance would be finding out that it was just due to me running MOST of my tests on my main PC being the culprit, for better testing I'll need to reinstall my homelab Ubuntu servers to distribute the load more (OR temporarily just get some beefy VPSes because I'll only need them for an hour).\u003c/p\u003e\n\u003cp\u003eI also fed in the raw data into Claude and there are some promising findings there already, it might indeed be Apache2 (I've done similar configuration changes for work, it can be tuned quite a bit even if the defaults are conservative), alongside the local setup being a bit borked cause of how Docker works on Windows too:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"38-what-the-raw-data-shows.jpg\" alt=\"38-what-the-raw-data-shows\" title=\"38-what-the-raw-data-shows\"\u003e\u003c/p\u003e\n\u003cp\u003eA more rigid test harness is probably the right way to go about it and would be fun to do sometime, though I'm not reconfiguring Apache2 to test that just now. The current setup should be good enough, especially for my next post where I'll probably complain about the way how LLMs do writing.\u003c/p\u003e\n"
        },
        {
            "title": "Apple is increasing my cortisol levels",
            "date_published": "2026-05-09",
            "id": "https://blog.kronis.dev/blog/apple-is-increasing-my-cortisol-levels",
            "url": "https://blog.kronis.dev/blog/apple-is-increasing-my-cortisol-levels",
            "content_html": "\u003cp\u003eI'm creating a simple developer utility to make managing Claude Code profiles (e.g. running it with DeepSeek, or some OpenRouter models) a little bit easier.\u003c/p\u003e\n\u003cp\u003eEdit: I just did the first release, which you can check out on \u003ca href=\"https://links.kronis.dev/wXc1SBTXMo\"\u003eccode.kronis.dev\u003c/a\u003e, or go directly to the \u003ca href=\"https://links.kronis.dev/NAnEME3Kqt\"\u003eItch.io page\u003c/a\u003e to either download or buy the pre-built binaries or look at the source code. It's a simple utility and it's early on (consider getting it for free first and only paying later, if it feels useful), but currently the code is not signed.\u003c/p\u003e\n\u003cp\u003eThe utility is written in the \u003ca href=\"https://links.kronis.dev/2NkFQhjOkw\"\u003eGo language\u003c/a\u003e, and the tooling there makes it really easy to compile for various platforms - I get a static executable that I can put anywhere I want. Even before the release, I wanted to see how easy it would be to ship it.\u003c/p\u003e\n\u003cp\u003eIt works just fine for distributing Linux software (same deal, after \u003ccode\u003echmod +x\u003c/code\u003e).\u003c/p\u003e\n\u003cp\u003eIt works sort of fine for distributing Windows software (I get an .exe, SmartScreen might have a word or two, though you can click through it in the same pop-up).\u003c/p\u003e\n\u003ch3\u003eDistributing Mac software\u003c/h3\u003e\n\u003cp\u003eIt does not just work for macOS and my MacBook instead shows me this:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-quarantine.jpg\" alt=\"01-quarantine\" title=\"01-quarantine\"\u003e\u003c/p\u003e\n\u003cp\u003eWhat you see is their \u003ca href=\"https://links.kronis.dev/SLnzZORtzR\"\u003equarantine kicking in\u003c/a\u003e for downloaded software, even if I share it with myself over Nextcloud.\u003c/p\u003e\n\u003cp\u003eTechnically, you can ask your users to override it manually, in the terminal:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02-manual-override.jpg\" alt=\"02-manual-override\" title=\"02-manual-override\"\u003e\u003c/p\u003e\n\u003cp\u003eMost developers might be willing to do that. It is not, however, good user experience and might raise some eyebrows.\u003c/p\u003e\n\u003cp\u003eDoesn't seem like such a big deal, right? I'll just enroll in their \u003ca href=\"https://links.kronis.dev/Cum6GUANtA\"\u003eApple Developer Program\u003c/a\u003e, sign the executable and be on my way, right?\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03-enrollment-requirements.jpg\" alt=\"03-enrollment-requirements\" title=\"03-enrollment-requirements\"\u003e\u003c/p\u003e\n\u003ch3\u003eGiving Apple money, and failing\u003c/h3\u003e\n\u003cp\u003eWait, they want how much money for the account?\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04-the-pricing.jpg\" alt=\"04-the-pricing\" title=\"04-the-pricing\"\u003e\u003c/p\u003e\n\u003cp\u003eAnd it's a yearly subscription? My brother in Christ, I intend to release a utility maybe a dozen or two dozen people are going to download, tops, for like 7 USD on \u003ca href=\"https://links.kronis.dev/R9lv8e8ELW\"\u003eItch.io\u003c/a\u003e with a \u003ca href=\"https://links.kronis.dev/OYAnXlHoxc\"\u003epay-what-you-want model\u003c/a\u003e, meaning that most of those people will probably choose the price of 0 USD instead (since I don't intend to be like Apple, people have various circumstances).\u003c/p\u003e\n\u003cp\u003eThat means that even if it works out that much, there's going to be VAT and Itch.io will also take a cut so out of those maybe 50 USD I'll get about 25 USD, which funds me about 3 months of that Apple Developer Program price. I guess the reason for it being priced like that lies somewhere between greed and wanting to gatekeep hobbyists out and only support Serious Users™, but it seems a bit stupid. Oh well, I already had to get the overpriced MacBook for another freelance thing, because they also won't let me compile macOS/iOS apps on Windows or Linux, so I guess this is just them spitting on me after slapping me in the face.\u003c/p\u003e\n\u003cp\u003eWhat I get from that is that articles like \u003ca href=\"https://links.kronis.dev/Fz5THjhuO8\"\u003eAn app can be a home-cooked meal\u003c/a\u003e are cool but don't take the economics of wanting to release something publicly into account - unless you're developing something that you'll add a bunch of monetization to, you'll be losing money. For desktop software there is \u003ca href=\"https://links.kronis.dev/qD951nec51\"\u003eHomebrew\u003c/a\u003e but that also means that you couldn't charge a few bucks for it even if you wanted to (or that you'd need to add \u003ccode\u003emac-homebrew-install-instructions.txt\u003c/code\u003e to the Itch.io downloads page when doing the pay-what-you-want approach, which would feel awkward).\u003c/p\u003e\n\u003cp\u003eI don't like that the economics are pushing software and app development in a direction where releasing a package (that might be non-open-source or just source-available, but you want to release binaries) costs money, though I also acknowledge that there would be other issues, like insane amounts of spam, with not doing that.\u003c/p\u003e\n\u003cp\u003eThen, we get to the actual verification process - it's understandable that they'd want to verify my ID. The problem is that on the MacBook they also expect me to use its webcam to take a picture. I will admit that my M1 MacBook Air is getting dated at this point, but regardless of what lighting I tried, I could just not get a good picture of the document. It's not like they were like \u0026quot;Oh hey, we've detected that your own iPhone is connected to the same local network as this MacBook, would you like to use it as a camera?\u0026quot;, so for about 10 attempts, this is what I saw:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04-upload-error.jpg\" alt=\"04-upload-error\" title=\"04-upload-error\"\u003e\u003c/p\u003e\n\u003cp\u003eEventually, I moved over to trying to use my main webcam for that, since their built in one just doesn't work:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"05-custom-camera.jpg\" alt=\"05-custom-camera\" title=\"05-custom-camera\"\u003e\u003c/p\u003e\n\u003cp\u003eWhy they can't just let me upload a scan of the document eludes me. I mean, I guess I can imagine a few reasons why, but it'd probably be easier to forge my own ID so it's not as glossy rather than having to turn my small kitchen table into this. Pictured for maximum frustration, a dongle that I needed:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"06-jank-setup.jpg\" alt=\"06-jank-setup\" title=\"06-jank-setup\"\u003e\u003c/p\u003e\n\u003cp\u003eEven that wasn't good enough, because understandably it doesn't have autofocus for something that you hold close. Not only that, but every 2nd failure seemed to just give me a generic error and I'd have to start the whole enrollment process from the beginning again:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"07-whole-process-fails.jpg\" alt=\"07-whole-process-fails\" title=\"07-whole-process-fails\"\u003e\u003c/p\u003e\n\u003cp\u003eLuckily I realized that I can install the app on my iPhone directly. There, it worked on the first try. I guess it must really suck if you don't have an iPhone or a fancy webcam, better spend some more money so you can give them money! The payment went through okay, soon after I had an activated developer account.\u003c/p\u003e\n\u003cp\u003eExcept of course I didn't, look, the app tells me to await an e-mail (which I seemingly already received?):\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"08-failed-status-update.jpg\" alt=\"08-failed-status-update\" title=\"08-failed-status-update\"\u003e\u003c/p\u003e\n\u003cp\u003eAnd the desktop app doesn't care at all either, it doesn't even know that I've tried the enrollment, and offers me to start the whole thing over again, despite me being signed into the exact same account:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"09-desktop-app-doesnt-care.jpg\" alt=\"09-desktop-app-doesnt-care\" title=\"09-desktop-app-doesnt-care\"\u003e\u003c/p\u003e\n\u003cp\u003eIt's probably a case of eventual consistency and some background processes or whatever, but it's also quite frustrating and, in a word, stupid.\u003c/p\u003e\n\u003ch3\u003eApple is kind of frustrating\u003c/h3\u003e\n\u003cp\u003eApple, I think you make hardware with pretty good build quality and the M-series chips made for pretty much the perfect notebook for me - and I'm sure they're great main dev machines for those that can afford the higher spec versions.\u003c/p\u003e\n\u003cp\u003eI think that's nice and I genuinely enjoy having the iPhone SE 2022, at least before learning that you \u003ca href=\"https://links.kronis.dev/jzDLXZwUj3\"\u003ekilled off the budget series altogether\u003c/a\u003e (your new e-series are more expensive) and \u003ca href=\"https://links.kronis.dev/ZYxKZEBAJL\"\u003eremoved the nice silent mode toggle on the side\u003c/a\u003e and \u003ca href=\"https://links.kronis.dev/MeRO2FfJjE\"\u003eremoved TouchID\u003c/a\u003e. That's before we even start talking about the 3.5mm jack and frankly all of that makes me question whether my next phone shouldn't just be an Android again.\u003c/p\u003e\n\u003cp\u003eI can deal with needing software like \u003ca href=\"https://links.kronis.dev/uQ44WVmbdy\"\u003eAutoRaise\u003c/a\u003e and \u003ca href=\"https://links.kronis.dev/3SEIO7e8zz\"\u003eRectangle\u003c/a\u003e and \u003ca href=\"https://links.kronis.dev/jLFv5BrBG1\"\u003eDiscreteScroll\u003c/a\u003e alongside others to customize your OS to my liking because you won't let me do that myself like most Linux distros do. I can even deal with your window focus needing an extra click across multiple monitors and AutoRaise being nice but perhaps too aggressive, since the developers are at least trying to make the experience nicer!\u003c/p\u003e\n\u003cp\u003eI can deal with your keyboard shortcuts being odd and not even having a \u0026quot;Cut\u0026quot; option in your Finder program.\u003c/p\u003e\n\u003cp\u003eI can deal with your weird Control/Command button setup which even breaks remote desktop software.\u003c/p\u003e\n\u003cp\u003eI can deal with your weird \u0026quot;programs you close aren't actually closed\u0026quot; approach even though you sold me a MacBook with 8 GB of RAM just so I could develop software in your walled garden ecosystem.\u003c/p\u003e\n\u003cp\u003eBut to first vendor lock me to your ecosystem for developing apps, then demanding a whole bunch of money so I can sign my software and it not get quarantined all while I'm not too well off financially, then refuse to let me submit my documents to you because your hardware produces pictures that are not good enough and make me have to install the app on a phone that's also expensive and that not even everyone has, then to still make me wait and have your apps not even show that I've submitted my application?\u003c/p\u003e\n\u003cp\u003eYou know what? Apple, fuck you and your forsaken ecosystem. This sucks.\u003c/p\u003e\n\u003ch3\u003eA more sane world\u003c/h3\u003e\n\u003cp\u003eI can use \u003ca href=\"https://links.kronis.dev/OXnmFq4Odg\"\u003eSmartID\u003c/a\u003e to verify my ID (and age) in about 20 seconds when buying an energy drink at the local grocery store.\u003c/p\u003e\n\u003cp\u003eI can use \u003ca href=\"https://links.kronis.dev/AxxPbX6kQV\"\u003eeParaksts\u003c/a\u003e to digitally sign documents in about a minute, from either my PC with a card reader (using my government issued ID card), or my phone with their app, ending up with a proper cryptographic signature either attached to the \u003ca href=\"https://links.kronis.dev/LYTBIKYURY\"\u003eEDOC container (ASIC-E)\u003c/a\u003e or a PDF file directly.\u003c/p\u003e\n\u003cp\u003eI'm sure that other countries also have plenty of similar services for ID and age verification, signing documents, and other digital services. I acknowledge that that's not all of them, and that things are all over the place in this regard (alongside the credit card mafia holding a lot of the world's payment infrastructure hostage), but come on, surely it's possible to create something that works better than my experience did.\u003c/p\u003e\n\u003cp\u003eHaving a bunch of scrappy Baltic software packages working better than those by a multi-billion dollar company feels silly.\u003c/p\u003e\n\u003ch3\u003eUpdate\u003c/h3\u003e\n\u003cp\u003eYou know what, Apple isn't the only company where things are somewhat messed up.\u003c/p\u003e\n\u003cp\u003eIf you want to sign some code for Windows, you can find \u003ca href=\"https://links.kronis.dev/SnspFF71Di\"\u003eCertum offering code signing\u003c/a\u003e, but that also costs around 209 EUR per year, all while they're supposed to be one of the affordable options out there! I'm not singling them out as much as acknowledging that they're one of the cheaper options and that many others out there are worse. What the fuck?\u003c/p\u003e\n\u003cp\u003eAnd then you look at \u003ca href=\"https://links.kronis.dev/E9qW0evYDn\"\u003eAzure Artifact Signing\u003c/a\u003e noticing that their \u003ca href=\"https://links.kronis.dev/GEIhyFcxGT\"\u003ebasic tier only costs 8.54 EUR per month\u003c/a\u003e and for a bit feel happy that someone has at least tried to disrupt the extortionate prices - until you set up your Azure account and notice that you cannot sign certificates as an \u003cem\u003eindividual\u003c/em\u003e if you're outside of US \u0026amp; Canada, in the EU only organizations can sign code through them.\u003c/p\u003e\n\u003cp\u003eThis feels about as bad as getting TLS certs was before \u003ca href=\"https://links.kronis.dev/Hr2ldIHvA9\"\u003eLet's Encrypt\u003c/a\u003e displaced most of that rent seeking behavior - the problem being that there aren't many alternatives or competitors to them, which on one hand means that we're moving towards a massive point of failure, and on the other hand means that if they ever decide to demand money, then a large part of the Internet will be straight up fucked.\u003c/p\u003e\n\u003cp\u003eI thought that maybe I'm overreacting a bit - since my previous issues were mostly about the Apple signup process being annoying and the way they've treated their users being worse than it could be, but no, I should be more angry - the whole code signing space is stupidly expensive for what it is. You have arguments in the opposite direction? Just know that they said the exact same kind of stuff about TLS before Let's Encrypt and how you have to pay 100 EUR a year for a cert that's not even a wildcard because of reasons™.\u003c/p\u003e\n\u003cp\u003eJust let me sign my code with my governmental ID card and be done with it, jeez.\u003c/p\u003e\n"
        },
        {
            "title": "Setting up a Git Bash alias in Windows",
            "date_published": "2026-05-07",
            "id": "https://blog.kronis.dev/blog/setting-up-a-git-bash-alias-in-windows",
            "url": "https://blog.kronis.dev/blog/setting-up-a-git-bash-alias-in-windows",
            "content_html": "\u003cp\u003eHere's a quick tutorial on how to work around an annoying issue on Windows.\u003c/p\u003e\n\u003cp\u003eSuppose that I have \u003ca href=\"https://links.kronis.dev/18tcd\"\u003eGit Bash\u003c/a\u003e installed on Windows that I got with \u003ca href=\"https://links.kronis.dev/AragQWRw3m\"\u003emy install of Git\u003c/a\u003e, and that I like to use to run various shell scripts. The problem there, is that if I also have \u003ca href=\"https://links.kronis.dev/9JAHXRfx2B\"\u003einstalled WSL\u003c/a\u003e, then I have approximately 0 idea how to run Git Bash from a PowerShell terminal session, for example, if a development tool has a terminal tab and opens that directly, or if I'm in a regular terminal session and just want to run a Bash script, BUT maintain access to the tooling I have on the system directly, NOT inside of WSL.\u003c/p\u003e\n\u003cp\u003eIn this case, opening \u003ccode\u003ebash\u003c/code\u003e opens WSL, not Git Bash:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-bash-opens-wsl.jpg\" alt=\"01-bash-opens-wsl\" title=\"01-bash-opens-wsl\"\u003e\u003c/p\u003e\n\u003cp\u003eAt the same time, maybe I don't want to mess around and overwrite the \u003ccode\u003ebash\u003c/code\u003e behavior, so I need a new alias. Here's how we can do that.\u003c/p\u003e\n\u003cp\u003eFirst up, we make sure that there is a PowerShell profile set up (but don't overwrite it if it already exists) and open it for editing:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003eNew-Item -ItemType File -Path $PROFILE\nnotepad $PROFILE\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eThere, we add our new alias:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003efunction gitbash { \u0026amp; 'C:\\Program Files\\Git\\bin\\bash.exe' @args }\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eThen we save the file, and reload the profile (needs to be done once, no restart of the whole PC necessary):\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003e. $PROFILE\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eOnce done, now you can open Git Bash with the aliased command:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003egitbash\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eHere's what it looks like when you run it:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02-what-it-looks-like.jpg\" alt=\"02-what-it-looks-like\" title=\"02-what-it-looks-like\"\u003e\u003c/p\u003e\n\u003cp\u003eThis will also work in any terminal sessions inside tools, for example Zed, \u003ca href=\"https://links.kronis.dev/es7Q8rJitg\"\u003ewhich I also wrote about today\u003c/a\u003e, but which doesn't seem to support easy terminal switching on a per-tab basis:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03-works-inside-tools.jpg\" alt=\"03-works-inside-tools\" title=\"03-works-inside-tools\"\u003e\u003c/p\u003e\n\u003cp\u003eNot the longest blog post, but I'm writing this down here, because I can guarantee that I'll forget what the exact commands and syntax was for all this. Either way, at least here's something annoying about my setup out of the way!\u003c/p\u003e\n"
        },
        {
            "title": "Zed is pretty nice",
            "date_published": "2026-05-07",
            "id": "https://blog.kronis.dev/blog/zed-is-pretty-nice",
            "url": "https://blog.kronis.dev/blog/zed-is-pretty-nice",
            "content_html": "\u003cp\u003eRecently, the \u003ca href=\"https://links.kronis.dev/g8903rmBTK\"\u003eZed\u003c/a\u003e editor had its 1.0 release. While I can't say what it's going to be like in a few years, even now it has largely replaced \u003ca href=\"https://links.kronis.dev/geejq\"\u003eVisual Studio Code\u003c/a\u003e as my editor for non-IDE tasks (though I still keep \u003ca href=\"https://links.kronis.dev/jwth4\"\u003eNotepad++\u003c/a\u003e around for some more persistent tabs, like notes).\u003c/p\u003e\n\u003cp\u003eThis won't be a super formal or structured review, but just my first impressions, setup and some thoughts so far. I will probably test drive it for a few months and see how it goes! It might also be a really nice replacement for Visual Studio Code on my M1 MacBook Air, because it doesn't have that much memory and Zed even on Windows is using about 159 MB of RAM in total right now.\u003c/p\u003e\n\u003cp\u003eIf anyone is curious, I set it up in a pretty comfy way and here's what it looks like:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-zed-is-pretty-nice.jpg\" alt=\"01-zed-is-pretty-nice\" title=\"01-zed-is-pretty-nice\"\u003e\u003c/p\u003e\n\u003cp\u003eSo, what's the selling point?\u003c/p\u003e\n\u003cp\u003eEssentially, a lot of why I was looking in the direction of \u003ca href=\"https://links.kronis.dev/34juz\"\u003eCudaText\u003c/a\u003e, a self-contained editor that doesn't need a frickload of plugins to be useful, while at the same time having most of the features you'd want out of the box, alongside great performance. CudaText had a few oddities about it and while I liked that project, it felt more like a replacement for Notepad++, while Zed feels like a replacement for Visual Studio Code - most of the stuff you're used to there, is also available here. In addition, it is also really fast (personally I already found Visual Studio Code to be nicely optimized, but this is one step further).\u003c/p\u003e\n\u003ch3\u003eSome AI features\u003c/h3\u003e\n\u003cp\u003eThey even support a bunch of AI stuff out of the box, if you're into that kind of a thing - rather than just running TUI/GUI based agents separately, to have something right in your editor. It's nice to have an editor where you don't need to switch between a bunch of awkward plugins (also \u003ca href=\"https://links.kronis.dev/PaNRkaheEt\"\u003eRIP RooCode\u003c/a\u003e), I really liked that one to be fair, felt like a step up above \u003ca href=\"https://links.kronis.dev/K5oB0AMWv5\"\u003eCline\u003c/a\u003e, but not as cluttered as \u003ca href=\"https://links.kronis.dev/Q3doIZbDgK\"\u003eKiloCode\u003c/a\u003e:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02-integrated-ai-chat.jpg\" alt=\"02-integrated-ai-chat\" title=\"02-integrated-ai-chat\"\u003e\u003c/p\u003e\n\u003cp\u003eOf course, it isn't exactly perfect in all the ways. For example, when using their integrated Zed agent, it seems to have trouble doing file edits with some models, like DeepSeek in particular kept corrupting fairly simple HTML templates more than once (while \u0026quot;LSP Edit\u0026quot; would show up as tabs):\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03-zed-isnt-perfect.jpg\" alt=\"03-zed-isnt-perfect\" title=\"03-zed-isnt-perfect\"\u003e\u003c/p\u003e\n\u003cp\u003eIt's odd, because the \u003ca href=\"https://links.kronis.dev/VBp4dtfKdx\"\u003eDeepSeek V4 Pro\u003c/a\u003e model itself is quite good - to the point, where I might replace my current Anthropic 100 USD subscription with it (or probably downgrade to the 20 USD tier and use it more sparsely), given that \u003ca href=\"https://links.kronis.dev/sCEosR1tEL\"\u003etheir API prices\u003c/a\u003e are also really, really great. After some digging around, it looks like the issues are all on the default Zed agent, because the code edits work just fine in OpenCode, it can even fix the previous corruption:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04-deepseek-works-fine-though.jpg\" alt=\"04-deepseek-works-fine-though\" title=\"04-deepseek-works-fine-though\"\u003e\u003c/p\u003e\n\u003cp\u003eI do have to note that running \u003ca href=\"https://links.kronis.dev/1IsO9tgOD4\"\u003eOpenCode inside of WSL\u003c/a\u003e is an exercise in frustration in and of itself if you have to use Windows, though, so if you're curious about OpenCode, probably just install it directly with npm instead of following their instructions. Otherwise you'll need to replicate your entire dev environment in WSL and also many other integrations with OpenCode (that don't know about WSL) won't work locally, this will become relevant just in a second:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"05-opencode-in-wsl.jpg\" alt=\"05-opencode-in-wsl\" title=\"05-opencode-in-wsl\"\u003e\u003c/p\u003e\n\u003cp\u003eWhy do I mention that? Because I found that having OpenCode installed on your PC directly and having Zed use that (yes, it can do that), is a way better setup for AI. They support \u003ca href=\"https://links.kronis.dev/5JieWY8S6V\"\u003eAgent Client Protocol (ACP)\u003c/a\u003e, which essentially means that an IDE can talk to an AI agent directly. It actually works pretty nicely in practice and is easy to setup, you just add the agent from their registry:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"06-use-opencode-from-zed.jpg\" alt=\"06-use-opencode-from-zed\" title=\"06-use-opencode-from-zed\"\u003e\u003c/p\u003e\n\u003cp\u003eThey support a whole bunch, by the way, you can probably look up the tools that you're already using:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"07-lots-of-agents-supported.jpg\" alt=\"07-lots-of-agents-supported\" title=\"07-lots-of-agents-supported\"\u003e\u003c/p\u003e\n\u003ch3\u003eThe more basic features\u003c/h3\u003e\n\u003cp\u003eThe AI doesn't really mean anything, if the other editor features would suck, though! So, do they?\u003c/p\u003e\n\u003cp\u003eI'd say that no, albeit it does take a little bit of getting used to. For example, by default things seem to open in a single instance/window. You do get a nice switcher at the top and your active terminal and chat sessions seem to all be preserved when doing these switches, but I'm kind of used to opening multiple separate instances personally:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"08-workspace-switcher.jpg\" alt=\"08-workspace-switcher\" title=\"08-workspace-switcher\"\u003e\u003c/p\u003e\n\u003cp\u003eIt's a bit odd, in the sense that they don't seem to have something like the \u003ca href=\"https://links.kronis.dev/CdQztGhPyO\"\u003eVisual Studio Code workspace\u003c/a\u003e files, where I can just save such a file in a repository and then double click it and have a new window with the workspace open. Zed does have a nice context menu option, but it also seems to just add the folder to the open workspaces in the main window:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"08-context-menu.jpg\" alt=\"08-context-menu\" title=\"08-context-menu\"\u003e\u003c/p\u003e\n\u003cp\u003ePersonally, I think it would be quite nice to have TWO options added to the context menu:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eOpen in Zed\u003c/li\u003e\n\u003cli\u003eOpen in Zed (New Window)\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThe good news is, that I dug around the configuration and found this:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"08-cli-default-open.jpg\" alt=\"08-cli-default-open\" title=\"08-cli-default-open\"\u003e\u003c/p\u003e\n\u003cp\u003eI can confirm, that changing that default setting does actually resolve the issue, something I found out about while writing this and having the thought: \u0026quot;Hey, I wonder if they support changing that?\u0026quot; Now, one could feasibly argue that the context menu and CLI might need to work differently in some cases, but honestly it's not a big deal, and I'm happy that I can get a really close approximation of what I want!\u003c/p\u003e\n\u003cp\u003eThis more or less matches my experience with Zed so far - it feels thoughtful and like it's actually made to be used daily, not as a loose collection of various plugins that don't fit together well. It might not have every feature under the sun, but I don't think it needs that, just the features people will use.\u003c/p\u003e\n\u003cp\u003eSpeaking of which, it does support double tapping the SHIFT key (I'm using the JetBrains keymap, by the way; it also supports Visual Studio Code, Sublime and some others, alongside custom configuration) to bring up a command menu where I can easily search for the commands I have available to me right now. I'm pretty sure that almost any GUI software package would benefit from something like this - not even just ones like Blender, that aren't really that approachable without (due to having a lot of features in there), but this feels immediately better than having to rely on only navigating bunches of nested menus:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"09-new-window.jpg\" alt=\"09-new-window\" title=\"09-new-window\"\u003e\u003c/p\u003e\n\u003cp\u003eSimilarly, there's a pretty nice integrated file search/switch functionality, where I can just press CTRL+SHIFT+N and get a popup where I can look for files - both by the filename, as well as the directory where they are:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"10-file-search.jpg\" alt=\"10-file-search\" title=\"10-file-search\"\u003e\u003c/p\u003e\n\u003cp\u003eThere's also a regular code search (CTRL+SHIFT+F), which optionally supports regex and various case options if you're into that, but otherwise is just a new tab (that you can come back to later, your search results are saved there) that will show which files have the results that you seek, in some ways this is way better than what JetBrains IDEs do:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"11-code-search.jpg\" alt=\"11-code-search\" title=\"11-code-search\"\u003e\u003c/p\u003e\n\u003cp\u003eThe one thing I didn't really like that much, was the way they do shell integration. In Visual Studio Code, for example, it's easy to switch between the various different shells, you can even have something like one PowerShell tab, and three Git Bash tabs, and then another CMD tab. Over here, you just have a default one, which more or less pushed me in the direction of having to setup an alias for Git Bash in Windows (since typing in \u003ccode\u003ebash\u003c/code\u003e just opens WSL, which again doesn't have all the tools available through Git Bash directly on the machine):\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"12-default-shell.jpg\" alt=\"12-default-shell\" title=\"12-default-shell\"\u003e\u003c/p\u003e\n\u003cp\u003eWhat's more, they don't really seem to have presets for the most common shells, they just have an option to invoke a program with some arguments if you'd like. It's nice that you can configure it, but some more convenience would go a long way! Luckily, it's not a dealbreaker and I've also included the PowerShell instructions in \u003ca href=\"https://links.kronis.dev/oo9N3qS0rX\"\u003eanother short blog post\u003c/a\u003e, if you'd like.\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"13-settings-page.jpg\" alt=\"13-settings-page\" title=\"13-settings-page\"\u003e\u003c/p\u003e\n\u003cp\u003eSpeaking of the settings, they're pretty good! You can search for what you want and can change most of the things you'd like in an editor - I also enjoyed that they have presets for line heights, so instead of having to figure out exactly what 10 or 1.0 means in how they render fonts, I could just pick \u0026quot;Standard\u0026quot; line height instead of \u0026quot;Comfortable\u0026quot; and get a nice amount of lines per a screen of text. That's a good example of the convenience and common sense I talked about! Secondly, you can change both the code and also the UI fonts, which is a nice bit of customization, alongside the sizes and most of the other things you'd expect:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"14-settings-panel.jpg\" alt=\"14-settings-panel\" title=\"14-settings-panel\"\u003e\u003c/p\u003e\n\u003cp\u003eIn addition, they also have some autocomplete if you come from Cursor or are looking for something new \u003ca href=\"https://links.kronis.dev/zDYsJfzQH4\"\u003eafter GitHub Copilot decided to change their terms\u003c/a\u003e and moved to usage based billing:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"15-inline-autocomplete.jpg\" alt=\"15-inline-autocomplete\" title=\"15-inline-autocomplete\"\u003e\u003c/p\u003e\n\u003cp\u003eZed still expects you to \u003ca href=\"https://links.kronis.dev/jaFqKFc219\"\u003eget a subscription from them\u003c/a\u003e for unlimited edit predictions, but that makes a decent amount of sense to me, since the editor itself is free and they don't seem to be gating the actual editor features behind paywalls. So as a business, I don't have a problem with them so far.\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"16-zed-subscription.jpg\" alt=\"16-zed-subscription\" title=\"16-zed-subscription\"\u003e\u003c/p\u003e\n\u003cp\u003eCome to think of it, I had a closer look at the terms and they say that you get 2'000 \u0026quot;accepted\u0026quot; edit predictions for free, which makes me think that ones that pop up and you don't need/accept don't count towards the limit and therefore don't even waste what you get for free - I'm not sure how much sense it makes for them to do that, given that those predictions still need to be paid for by them in the form of compute, but on the bright side, most of these autocomplete models are quite small, so it's not like I'm raising the global temperature by 0.001C every time it pops up. In other words, if you do most of your development with agentic tools and only occasionally need to do manual edits, and from those only occasionally accept the suggestions - it will essentially be free for you forever (or at least until they alter the limits or terms themselves).\u003c/p\u003e\n\u003ch3\u003eSummary\u003c/h3\u003e\n\u003cp\u003eThese are my first impressions with Zed - it takes a bit of getting used to, but this is the first time in a bit, where I've felt a sense of joy and pleasant surprise, looking at a new editor. If anything, they're proving that you can build new code editors in 2026 even when ones like Visual Studio Code are so well established, and still succeed in making something good!\u003c/p\u003e\n\u003cp\u003eI just hope that they also succeed in the financial sense, and can support this project for the years to come - JetBrains also tried doing something similar with their \u003ca href=\"https://links.kronis.dev/aulwf\"\u003eFleet\u003c/a\u003e editor, but sadly that \u003ca href=\"https://links.kronis.dev/CM8dmwyAQV\"\u003edidn't really work out\u003c/a\u003e. Their blog post titled \u0026quot;The Future of Fleet\u0026quot; then proceeds to explain how Fleet does not have a future, a bit sad and ironic, given that I also wanted Fleet to succeed.\u003c/p\u003e\n\u003cp\u003eI guess the difference here is that Fleet felt like a worse Visual Studio Code - it was kind of slow and sometimes awkward, even if it had a lot of potential to be a good competing project/product. I could have imagined that with a few years of polish it could have indeed gotten up to speed and I could easily imagine myself daily driving it, though maybe there were technical underlying issues for why it'd never achieve the same level of performance and comfort as VSC, I'm not sure.\u003c/p\u003e\n\u003cp\u003eAt the same time, Zed has already replaced Visual Studio Code for this trial period - I've straight up uninstalled it, because Zed already feels good enough for daily driving. I'm not sure how two different groups of people can develop similar types of software and have such wildly different outcomes. Either the Zed people are those mythical 10x developers you hear about, or they picked a really good set of technologies for building their editor, or they had a slightly better architectural idea and vision of what they want to build and how, or maybe it's just a mix of all of those.\u003c/p\u003e\n\u003cp\u003eEither way, it's pretty cool and you should maybe check it out for yourself!\u003c/p\u003e\n"
        },
        {
            "title": "[LV] LATA 2026 Konference",
            "date_published": "2026-04-21",
            "id": "https://blog.kronis.dev/blog/lv-lata-2026-konference",
            "url": "https://blog.kronis.dev/blog/lv-lata-2026-konference",
            "content_html": "\u003cblockquote\u003e\n\u003cp\u003eThis post is in Latvian, because the conference and the accompanying slides were also in Latvian.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eNesen bija kārtējā LATA konference, nu jau 2026. gada versija: \u003ca href=\"https://links.kronis.dev/8UgVcQvKsd\"\u003eDigitālā burbuļošana. Procesi, datu centri, mākslīgais intelekts.\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eSlaidus viņi vēl nenopublicēja, bet šo to norises laikā piefiksēju. Iedomājos, ka uztaisīšu sava veida kopsavilkumu par to, kas šķita vissaistošākais. Nebūs te tik daudz info par konkrētajiem runātājiem vai LATA gada balvu, vairāk par pašu saturu.\u003c/p\u003e\n\u003cp\u003eSavukārt, ja interesē pilnais video ieraksts, to var apskatīties šeit: \u003ca href=\"https://links.kronis.dev/wUnmiDPZIG\"\u003eYouTube: \u0026quot;Digitālā burbuļošana. Procesi. Datu centri. Mākslīgais intelekts.\u0026quot;\n\u003c/a\u003e\u003c/p\u003e\n\u003ch3\u003eJēgpilna IKT pārvaldība: no birokrātiskas kontroles līdz attīstības ceļakartēm\u003c/h3\u003e\n\u003cp\u003e\u003cem\u003ePrezentēja: Gatis Ozols, VARAM, Valsts sekretāra vietnieks digitālās transformācijas jautājumos\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eNo sākuma bija prezentācija par IKT pārvaldību valstī. Vārdu sakot, VARAM (Viedās Administrācijas un Reģionālās Attīstības Ministrija) mēģina izstrādāt plānu, kā valstī pārvaldīt sistēmu izveidi un uzturēšanu, viņi to nosauca par \u0026quot;IKT būvvaldi\u0026quot;, ar domu to pārvaldi arī taisīt vairākos slāņos:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01_01-ikt-buvvalde.jpg\" alt=\"01_01-ikt-buvvalde\" title=\"01_01-ikt-buvvalde\"\u003e\u003c/p\u003e\n\u003cp\u003eJa interesē, tad atradu prezentāciju kur par to var vairāk paskatīties: \u003ca href=\"https://links.kronis.dev/cKraMJM7NN\"\u003eIKT būvvalde un digitālās pārvaldes arhitektūra\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003ePapildus tam, arī pašā mājaslapā ir vairāk info: \u003ca href=\"https://links.kronis.dev/BZwOO2TJWk\"\u003eDigitālās pārvaldes arhitektūra\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eProtams, ikdienā gan jau tā ir birokrātija, bet censties sakārtot to kā valstī sistēmas tiek izstrādātas, vismaz ideja ir atzīstama!\u003c/p\u003e\n\u003cp\u003ePaši arī par to varēja palielīties:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01_02-ikt-buvvalde.jpg\" alt=\"01_02-ikt-buvvalde\" title=\"01_02-ikt-buvvalde\"\u003e\u003c/p\u003e\n\u003cp\u003ePapildus tam, viss izskatās ka ir ne tikai \u0026quot;top-down\u0026quot; pieejā, bet arī cenšas drusku dabūt info no nozares:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01_03-ikt-buvvalde.jpg\" alt=\"01_03-ikt-buvvalde\" title=\"01_03-ikt-buvvalde\"\u003e\u003c/p\u003e\n\u003cp\u003eNo vienas puses, lietas centralizēt ir riskanti, jo var rasties situācijas kur domēnam piedāvā/rekomendē/uzspiež nepiemērotus risinājumus kas no tehniskās puses vienkārši nedarbotos labi (OS, DB, API). Bet no otras puses, reizēm lietas izdodas tīri smuki. Var paskatīties uz to, kā Lielbritānijā izveidoja priekš valdības saitiem konsistentu dizaina sistēmu, lai nebūtu problēmas ar accessibility un lietotājiem būtu vieglāk strādāt (UI elementi, izskats un darbība laika gaitā pazīstami): \u003ca href=\"https://links.kronis.dev/MSFXZI6wmD\"\u003eDesign your service using GOV.UK styles, components and patterns\u003c/a\u003e\u003c/p\u003e\n\u003ch3\u003eGet ahead. Stay ahead.\u003c/h3\u003e\n\u003cp\u003e\u003cem\u003ePrezentēja: Aigars Mačiņš, Emergn, Risinājumu arhitektūras prakses vadītājs\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003ePēc tam bija viena praktiskāka prezentācija, kas runāja par to, cik daudz ir mainījies tas kā uzņēmumi strādā, populārākiem kļūstot AI rīkiem. Detaļas bija visai daudz, bet lielās līnijās viss vērsts uz to, ka uzņēmumi un vispār izstrādātāji kas izmanto AI, var lietas apgūt un iterēt daudz ātrāk:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02_01-ai-impact.jpg\" alt=\"02_01-ai-impact\" title=\"02_01-ai-impact\"\u003e\u003c/p\u003e\n\u003cp\u003eLaika gaitā tas visticamāk novedīs pie tā, ka tie uzņēmumi kam nav Claude/Codex/Gemini subscription vienkārši atpaliks, nespēs taisīt prototipus, nespēs ātri taisīt jaunus projektus (īpaši startup). Projektu pārvaldes un ieguldījumu sakarā arī uzsvars uz to, ka vairāk jāeksperimentē un jādarbojas iteratīvi, nevis jāizplāno kaut kādi ļoti lielie projekti daudziem gadiem uz priekšu:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02_02-portfolio-management.jpg\" alt=\"02_02-portfolio-management\" title=\"02_02-portfolio-management\"\u003e\u003c/p\u003e\n\u003cp\u003eNevajag arī pārsarežģīt, jāskatās kāds risinājums ir spējīgs ģenerēt vērtību, būtībā \u003ca href=\"https://links.kronis.dev/zV5uR3nOnD\"\u003eMinimum Viable Product\u003c/a\u003e pieeja. Varbūt pat ne tikai tas, bet arī lielām sistēmām jāskatās kura funkcionalitāte reāli ir vajadzīga, kuru izmanto, cik lielus uzturēšanas riskus nevajadzīgā rada. Būtībā no tehniskās puses to pašu gribētos teikt par arhitektūru un DevOps lietām reizēm:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02_04-overall-view.jpg\" alt=\"02_04-overall-view\" title=\"02_04-overall-view\"\u003e\u003c/p\u003e\n\u003cp\u003eVispār interesanti, viņi uztaisīja (būtībā vibe coded) produktu projektu pārvaldībai, palaida tirgū un jau par to pelna naudu: \u003ca href=\"https://links.kronis.dev/bP5IFVNFy1\"\u003ePraxis by Emergn\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02_05-what-others-are-doing.jpg\" alt=\"02_05-what-others-are-doing\" title=\"02_05-what-others-are-doing\"\u003e\u003c/p\u003e\n\u003cp\u003ePar koda un kopējā risinājuma kvalitāti un to biznesa ideju neteikšu pats neko ne labu, ne sliktu, bet pajautājiet sev:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eCik bieži \u003cem\u003emēs\u003c/em\u003e uzņēmumā palaižam paši savus SaaS projektus publiskai lietošanai?\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eTas pats \u003ca href=\"https://links.kronis.dev/vTEBE33J9o\"\u003eEmergn\u003c/a\u003e arī ir uzņēmums kurš izskatās ka nodarbojas ar konsultēšanu. Ja viņi var, kas to liedz darīt jums? Ja kādreiz ir kāda laba ideja, kāpēc kavēties? Var kaut ko uzsist gaisā un ļaut tirgum pateikt, labs produkts (un visiem $$$) vai nē (un tad meklēt citus). Tāpēc arī runā par ātriem izstrādes cikliem, nevis liela mēroga ilgajiem projektiem.\u003c/p\u003e\n\u003cp\u003eMazāk kaut ko, kas praksē izskatās pēc \u0026quot;waterfall\u0026quot; un vairāk \u0026quot;īsto agile\u0026quot;, tostarp arī \u003ca href=\"https://links.kronis.dev/fNWin4N6Se\"\u003eproduktu domāšanu\u003c/a\u003e vajag:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02_06-how-things-should-look-like.jpg\" alt=\"02_06-how-things-should-look-like\" title=\"02_06-how-things-should-look-like\"\u003e\u003c/p\u003e\n\u003cp\u003eTur var teikt ka AI tehnoloģijas tam visam nebūt nav centrālas, bet vienkārši rīks kas ar to var palīdzēt veiksmīgāk strādāt. Personīgā pieredze arī rāda, ka ar Claude Code varu strādāt pie vairākiem projektiem produktīvi, vairāk gan kā pats, gan kā citi kolēģi. Ja iegulda laiku (un naudu) lai to visu labi atstrādātu, tad lietas lido uz priekšu.\u003c/p\u003e\n\u003cp\u003eProblēma, par ko tur tik daudz nepastāstīja: nevar būvēt sistēmas slikti un gaidīt ka AI ļaus 10x pavairot produktivitāti.\u003c/p\u003e\n\u003cp\u003eSistēmām ir jābūt testējamām. Ja tur ir pliki biznesa loģikas testi kam tomēr vajag celt gaisā veselas lietotnes kontekstu (piem. Spring Boot) jo biznesa loģikas kods ir cieši sasaistīts ar tehnisko risinājumu VAI arī ja ir jātaisa mocki pus lietotnei un nav iespējams testā uzrakstīt dažus objektus ko iebarot savam kodam un testēt to, tad bottleneck ir pats projekts. Vēl sliktāk, ja testi nav vispār un projekts tikai \u0026quot;izskatās\u0026quot; ka strādā un cilvēkiem ir bailes kaut ko mainīt, ja nu salūzt.\u003c/p\u003e\n\u003cp\u003eIzstrādātājiem ir jāspēj iterēt brīvi, salauzt lokālās izstrādes vides (tādas vajag, ne tikai kaut kādu dalīto DB), eksperimentēt un kustēties uz priekšu. Ja projekts neļauj smuki palaist paralēli 3-10 aģentus dažādiem uzdevumiem, tad bottleneck ir reāli pats projekts. Līdzīgi, ja tur DB shēma ir kosmoss un nav nekādas izstrādātāju dokumentācijas, coding conventions un stila lietas utt. ir viss \u0026quot;jāpatur galvā\u0026quot; un par to tikai kāds kolēģis ar garu baltu bārdu ieminēsies code review laikā, tad bottleneck ir viss izstrādes process.\u003c/p\u003e\n\u003cp\u003eVajag dokumentāciju turēt tuvāk kodam, jā, arī visu dīvaino/īpatnējo/sarežģīto/svarīgo kodā dokumentēt ar komentāriem (100% self explanatory code ir būtībā mīts), ieguldīt laiku rīkos - gan publiski pieejamos, visādos formatteros, linteros, code style check rīkos, koda statiskās analīzes rīkos, bet arī automatizēt lietas pašiem ar saviem rīkiem, visu kas ir specifisks projektam. Codegen kur var, savukārt kur nevar tad vismaz kādu harness kas 80% no \u0026quot;Vai šis kods ir ok?\u0026quot; varēs pilnīgi automatizēti pateikt, kas nozīmē ka visi tie paralēlie aģenti uzreiz arī dabūs pa saviem virtuālajiem pirkstiem un būs spiesti kodu rakstīt \u0026quot;labāk\u0026quot;, tāpat kā kolēģi.\u003c/p\u003e\n\u003cp\u003eTie, kas šo sapratīs un izmantos, lidos. Tie kuriem vides ir kaut kādi manuāli pārvaldīti serveris un Oracle DB ar loģiku ko pat Oracle XE nevar pacelt lokāli, nekādiem README.md vai citiem setup skriptiem un nekādu automatizāciju, tāpat lēnām muļļāsies uz priekšu. Būtībā nekā jauna - slikts kods iešauj kājā kā pašiem, tā AI.\u003c/p\u003e\n\u003ch3\u003eAtvērtība un brīva konkurence valsts pārvaldē. Vai atvērtajai VPS sekos pārējie?\u003c/h3\u003e\n\u003cp\u003e\u003cem\u003ePrezentēja: Edžus Žeiris, Dativa, direktors\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eTālāk bija prezentācija par open source, vārdu sakot, uzmanība jāpievērš MK 367: \u003ca href=\"https://links.kronis.dev/5GMtFE39DL\"\u003eInformācijas sistēmu vispārējās tehniskās prasības\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03_01-code-and-supply-chain.jpg\" alt=\"03_01-code-and-supply-chain\" title=\"03_01-code-and-supply-chain\"\u003e\u003c/p\u003e\n\u003cp\u003eTur interesantās daļas no likuma ir:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e2.2.1. informācijas sistēmas prasības un projektējumu attiecībā uz datiem, par kuru pārvaldību informācijas sistēmas pārzinis ir atbildīgs un kas ir saistoši un noderīgi sabiedrībai un ir klasificēti kā vispārpieejama informācija, veido atbilstoši principam \u0026quot;atvērts pēc noklusējuma\u0026quot;, paredzot, ka datus atvērto datu formātā kopā ar metadatiem vai tikai datu kopu metadatus publicē Latvijas atvērto datu portālā \u003ca href=\"https://links.kronis.dev/SLW0yTDY9X\"\u003ehttps://data.gov.lv\u003c/a\u003e (turpmāk – atvērto datu portāls) vai valsts vienotajā ģeotelpiskās informācijas portālā \u003ca href=\"https://links.kronis.dev/8S3IG0TJVe\"\u003ehttps://geolatvija.lv\u003c/a\u003e;\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eun\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e2.4.5. izstrādājot jaunas informācijas sistēmas, izmanto atvērtā koda platformas un risinājumus, kuri atbilst mūsdienīgas – modulāras, sadarbspējīgas un informācijas un komunikācijas tehnoloģiju infrastruktūru efektīvi izmantojošas – informācijas un komunikācijas tehnoloģiju arhitektūras prasībām saskaņā ar Vides aizsardzības un reģionālās attīstības ministrijas tīmekļvietnē publicētajām specializētās lietojumprogrammatūras tehnoloģiskās arhitektūras vadlīnijām un kuriem ir demonstrētu pielietojumu vēsture un pastāvīga attīstības kopiena:\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eProtams, tur vēlāk ir ierastās atrunas kas nozīmē ka šiem punktiem nesekos tik daudz kā vajadzētu, bet saprātīgā pasaulē tas rezultētu:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eatvērto datu portālā būtu arvien vairāk informācija par Latviju pieejama laika gaitā, datu zinātniekiem būtu ko darīt un pētīt\u003c/li\u003e\n\u003cli\u003eja taisām klienta-servera risinājumu, tad pēc default gan jau tas būtu FreeBSD, NetBSD, OpenBSD vai kāda no Linux distribūcijām: nekāds Windows Server ar dārgajām licencēm, būtībā pat kāpēc lai mēs par RHEL maksātu (par nodokļu maksātāju naudu); pat ne tāpēc ka viņi slikti būtu, vienkārši mums nav tik liels mērogs lai to \u003cem\u003evajadzētu\u003c/em\u003e\u003c/li\u003e\n\u003cli\u003eja taisām sistēmu kurā vajag DB, tad pēc default gan jau tā būtu PostgreSQL vai MariaDB (vai nosacīti MySQL) vai SQLite, vai jebkas cits ko var BRĪVI uzcelt kur vajag, cik instancēs, testēšanai, izstrādei un prod, ne dārgās Oracle, vai SQL Server, vai IBM DB2 vai citu risinājumu licences (arī par nodokļu maksātāju naudu)\u003c/li\u003e\n\u003cli\u003etāpat ar teju jebkuru specializēto programmatūru, piemēram Valkey priekš key-value store, RabbitMQ vai NATS priekš ziņojumu rindām, Garage vai SeaweedFS priekš kā S3 saderīga, Apache2 vai Nginx vai Caddy kad vajag tīkla serveri utt.\u003c/li\u003e\n\u003cli\u003evārdu sakot, tiešām brīvība izstrādē, nevis sasietas rokas un lielā naudas tērēšana; tikpat labi par \u0026quot;support\u0026quot; var maksāt pašiem, lai nauda ir LV\u003c/li\u003e\n\u003cli\u003eatvērtā pirmkoda programmatūra arī ir ļoti daudz laba, vajag tik viņu spēt izmantot UN dot savu darbu atpakaļ visa projekta uzlabošanai\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eTātad, minēja vienoto pašvaldību sistēmu, redz kur vairāk info (ZZ Dats gan pārsauca par Dativa tagad): \u003ca href=\"https://links.kronis.dev/0KXj5zgY3B\"\u003e“ZZ Dats” atver vienoto pašvaldību informācijas sistēmu – turpmāk pieejama zem EUPL atvērtā koda licences\n\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03_02-pasvaldibas.jpg\" alt=\"03_02-pasvaldibas\" title=\"03_02-pasvaldibas\"\u003e\u003c/p\u003e\n\u003cp\u003eLielās līnijās domāšana pareizā virzienā, patiesībā lielu daļu valsts un privātā sektora projektu vajadzētu pie reizes padomāt arī izplatīt tālāk, vai ko nevar uztaisīt tā, ka var pārdot, piemēram tiem pašiem Igauņiem vai Lietuviešiem, vai citām EU valstīm:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03_03-open-source.jpg\" alt=\"03_03-open-source\" title=\"03_03-open-source\"\u003e\u003c/p\u003e\n\u003cp\u003eLīdzīgi arī ar izstrādes valodām, runtime, satvariem un bibliotēkām, būtībā visu; reāli neredzu iemeslu kāpēc nevarētu aiziet soli tālāk un pateikt ka visi tagad kodu liks uz kāda git.gov.lv (uzcelt lielu GitLab vai Gitea vai Forgejo instanci gaisā) savām sistēmām un katrs Latvijas iedzīvotājs kas par to sistēmu ar saviem nodokļiem maksā varēs pieslēgties un izstrādātājiem submitot pull request, kad atkal sistēmās ir bugi un lietas nestrādā. Līdzīgi ar drošības caurumiem, labāk tos pamanīt drošības speciālistiem un taisīt responsible disclosure, nevis Krievijas hakeriem.\u003c/p\u003e\n\u003cp\u003ePrezentāciju klausoties, gan palika nedaudz dīvaini:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03_04-open-source-but-not-really.jpg\" alt=\"03_04-open-source-but-not-really\" title=\"03_04-open-source-but-not-really\"\u003e\u003c/p\u003e\n\u003cp\u003eIevērojāt problēmu? Preses relīze ir, stāsta ka open source... a kur ir kods? Kur ir jūsu Git repo, vai GitHub links? Te laikam konkrētā iniciatīva sabrūk jo meklēju GitHub un atradu tikai šos:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"https://links.kronis.dev/P4kPHAi2xL\"\u003eGitHub - zzdats\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://links.kronis.dev/c3l6rO6Bjq\"\u003eGitHub - dativa-lv\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eTajā rakstā augstāk komentāros cilvēki arī zobojas:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003ePubliskā licence nenozīmē, ka viss ir izlikts publiski. Publiski ZZdats ir izlikts tikai LX – Vue.js freimworks. To gan jebkurš var gan lietot, gan piedāvāt savus papildinājumus. Es tās sistēmas esmu redzējis – tas, ka no ārpuses tas viss saucas Vienotā pašvaldības sistēma, nenozīmē, ka tur apakšā ir viena megasistēma. Vienots ir tikai logins, kamēr apakšā ir ašpadsmit dažādas sistēmas no 30 gadus vecām EXĒm līdz jaunām un svaigām web un mobilajām aplikācijām.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eIzklausās pēc atrunām.\u003c/p\u003e\n\u003cp\u003eFSF \u003ca href=\"https://links.kronis.dev/Ln0SNmMxHp\"\u003epasaka skaidri\u003c/a\u003e:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eA program is free software if the program's users have the four essential freedoms:\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThe freedom to run the program as you wish, for any purpose (freedom 0).\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThe freedom to study how the program works, and change it so it does your computing as you wish (freedom 1). Access to the source code is a precondition for this.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThe freedom to redistribute copies so you can help others (freedom 2).\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThe freedom to distribute copies of your modified versions to others (freedom 3). By doing this you can give the whole community a chance to benefit from your changes. Access to the source code is a precondition for this.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eTātad, tu nevari teikt \u0026quot;atvērtais pirmkods\u0026quot;, ja tas kods tiem kas sistēmu izmanto vai saņem tās pakalpojumus nav pieejams.\u003c/p\u003e\n\u003cp\u003ePat viņu izvēlētā \u003ca href=\"https://links.kronis.dev/br8DopJAU8\"\u003eEUPL licence\u003c/a\u003e to pasaka skaidri:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThe Licensor may provide the Work either in its Source Code form, or as Executable Code. If the Work is provided as Executable Code, the Licensor provides in addition a machine-readable copy of the Source Code of the Work along with each copy of the Work that the Licensor distributes or indicates, in a notice following the copyright notice attached to the Work, a repository where the Source Code is easily and freely accessible for as long as the Licensor continues to distribute or communicate the Work.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eun tālāk:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eProvision of Source Code: When distributing or communicating copies of the Work, the Licensee will provide a machine-readable copy of the Source Code or indicate a repository where this Source will be easily and freely available for as long as the Licensee continues to distribute or communicate the Work.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eKo tur nozīmē tā saziņa? Vienkārši, līdzīgi AGPL:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e— ‘Distribution’ or ‘Communication’: any act of selling, giving, lending, renting, distributing, communicating, transmitting, or otherwise making available, online or offline, copies of the Work or providing access to its essential functionalities at the disposal of any other natural or legal person.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eTātad, visi kurus skar tā pašvaldību sistēma ir pelnījuši piekļuvi pirmkodam. Man šķiet tie puiši paziņoja ka tagad ir atvērtais pirmkods, bet ja nav piekļuve tam kodam, tad ir liela iespēja ka šobrīd paši pārkāpj licences noteikumus un tiem neseko, vienalga kā to centīsies pasniegt. Nevar paņemt un tā pārdefinēt labi zināmus terminus, lai tie nozīmētu kaut ko pavisam citu, tikai tāpēc, ka pašiem tā ir ērtāk.\u003c/p\u003e\n\u003ch3\u003eDatu centri: Kritiskā infrastruktūra, kuras nozīmi un ietekmi uz nākotni mēs nenovērtējam\u003c/h3\u003e\n\u003cp\u003e\u003cem\u003ePrezentēja: Pēteris Elksnis, Lailio, izpilddirektors\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eBija arī prezentācija par datu centriem, vārdu sakot, tur ir prātam neaptverami ieguldījumi:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04_01-data-centre-investments.jpg\" alt=\"04_01-data-centre-investments\" title=\"04_01-data-centre-investments\"\u003e\u003c/p\u003e\n\u003cp\u003ePašsaprotami ka tekošajā AI ērā ir daudz enerģijas patēriņš tieši GPU, kā arī dzesēšanai:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04_02-data-centres.jpg\" alt=\"04_02-data-centres\" title=\"04_02-data-centres\"\u003e\u003c/p\u003e\n\u003cp\u003eIzskatās, ka tas tikai aug plašumā. Izskatās ka ja tas \u0026quot;burbulis\u0026quot; kādreiz plīsīs, tad varētu globālo ekonomiku paraut līdzi aptuveni kā 2008. gadā:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04_03-ai-infrastructure.jpg\" alt=\"04_03-ai-infrastructure\" title=\"04_03-ai-infrastructure\"\u003e\u003c/p\u003e\n\u003cp\u003eTam vajag ļoti daudz elektrības, savukārt vairumā valstu tam vienkārši nav pietiekami liels tīkls, lai to visu paceltu:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04_04-electricity-use.jpg\" alt=\"04_04-electricity-use\" title=\"04_04-electricity-use\"\u003e\u003c/p\u003e\n\u003cp\u003eBija ļoti interesants salīdzinājums ar naftu, šobrīd datu centri (drīzāk gan GPU compute, nekā dati) ir ar lielākiem ieguldījumiem:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04_05-oil-business-comparison.jpg\" alt=\"04_05-oil-business-comparison\" title=\"04_05-oil-business-comparison\"\u003e\u003c/p\u003e\n\u003cp\u003eLatvijā, savukārt, ir mazāka mēroga projekti, dēļ ierobežojumiem:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04_06-latvia-situation.jpg\" alt=\"04_06-latvia-situation\" title=\"04_06-latvia-situation\"\u003e\u003c/p\u003e\n\u003cp\u003eTajā pat laikā minēja arī riskus saistībā ar centralizāciju:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04_07-latvia-scale.jpg\" alt=\"04_07-latvia-scale\" title=\"04_07-latvia-scale\"\u003e\u003c/p\u003e\n\u003cp\u003eViens no komentētājiem teica ka izklausās \u0026quot;šizofrēniski\u0026quot; jo iepriekš runāja par vienotu pieeju izstrādei un pārizmantošanu bet šajā prezentācijā atkal par to ka vajag vairākus piegādātājus utt. - moderators to jautājumu gan nepacēla, jo visticamāk nebija īsti pieklājīgi uzrakstīts.\u003c/p\u003e\n\u003cp\u003eEs gan pacelšu - ja ir SSO tikai ar vienu piekļuvi (piem. Latvija.lv) tad tas ir masīvs single point of failure. Ja tur piekabina klāt eParakstu, Smart-ID, internetbankas un varbūt vēl kaut ko, tad uzreiz ir labāk. Tā arī ir galvenā doma tam, ka vairāki provideri ir labāk. Papildus tam, pat ja ir viena platforma, viņu var taisīt ar \u003ca href=\"https://links.kronis.dev/vvoTCSS6es\"\u003eHigh Availability pieeju\u003c/a\u003e (HA), lai pat ja nobrūk vesels datu centrs, lai risinājums vismaz ar ierobežotu kapacitāti turpinātu strādāt.\u003c/p\u003e\n\u003cp\u003eKāpēc Latvijā sistēmas joprojām brūk? Lielās līnijās:\u003c/p\u003e\n\u003cp\u003eA) nemāk uztaisīt HA sistēmas, nav kompetences, savukārt tie kam ir nestrādā valsts sektorā, bet pie tiem kas daudz maksā\u003c/p\u003e\n\u003cp\u003eB) nav budžets ko ieguldīt, lai to uztaisītu kārtīgi, vai arī ir izdomāti fake deadline un nav laiks\u003c/p\u003e\n\u003cp\u003eC) šķērdē skaitļošanas resursus ar sliktām arhitektūrām un sliktām realizācijām, vecām pieejām utt.\u003c/p\u003e\n\u003cp\u003eEsmu pats redzējis sistēmas kas nobrūk pie 400 RPS, lai gan paskatoties ko \u003ca href=\"https://links.kronis.dev/T6EIPmLbBz\"\u003eciti spēj dabūt gatavu\u003c/a\u003e, tas nav labākais ko moderni dzelži spēj.\u003c/p\u003e\n\u003cp\u003eTas ir tas pats, kāpēc daudzi raksta kodu ar N+1 problēmām (vieglāk acīmredzot, vai arī nav rīki kas to noķer), kāpēc nemāk izmantot vai vienkārši netaisa slodzes testus ar tādiem rīkiem kā K6 (vai neuzraksta savējos, jo jāatzīst ka K6 support WebSocket ir slikts), kā arī sataisa sistēmu arhitektūras kas vienkārši nav mērogojamas. Līdzīgi, industrijā rakstām nevis normālu desktop programmatūru arī, bet shippojam Electron lietotnes kas ir vesels pārlūks lai atrādītu UI, un pat OS šķērdē resursus pa labi un pa kreisi.\u003c/p\u003e\n\u003cp\u003eNo vienas puses, jāatceras: \u003ca href=\"https://links.kronis.dev/niqkn\"\u003eWirth's law\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eNo otras puses, ok, nesaku ka katra sistēma ir jāraksta Rust vai baigi jātestē visas no viņām. Reizēm pietiek kaut ko sasviest kopā Ruby vai Python, vai pat neuzrakstīt optimālu kodu Spring Boot vai ASP.NET lietotnē un galvenais lai strādā - bet ir atšķirība, vai sistēmai būs varbūt 100 lietotāji, vai arī 1'000'000. Tam ievērojami jāietekmē gan arhitektūru, gan arī izstrādes un testēšanas procesu. Vienkāršām sistēmām, savukārt domāt par tādu scaling reāli ir laika tērēšana. Tik uzmanāmies, ka sistēma no vienas kategorijas nepārvēršas par otru, tad ir auzas.\u003c/p\u003e\n\u003ch3\u003eMI kā automatizācijas instruments kiberdrošības nodrošināšanā\u003c/h3\u003e\n\u003cp\u003e\u003cem\u003ePrezentēja: Juris Pūce, PEAKDEFENCE, partneris\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eBija arī prezentācija par pašu LLM pielietojumu, par modeļiem, kas lielās līnijās atbilst paša pieredzei:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"05_01-translation-issues.jpg\" alt=\"05_01-translation-issues\" title=\"05_01-translation-issues\"\u003e\u003c/p\u003e\n\u003cp\u003eApskatīja arī pieeju par kuru domāju pats, piemēram, kāds lokālais Qwen modelis apstrādā uzdevumu, bet ar EuroLLM pārtulkot Latviski pareizi (te pat JSON nevajadzētu, būtībā var no viena modeļa uz otru datus stumdīt, tikai harness vajadzētu):\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"05_02-workaround.jpg\" alt=\"05_02-workaround\" title=\"05_02-workaround\"\u003e\u003c/p\u003e\n\u003cp\u003eUn protams, arī runāts par datu kvalitāti un formātu, lai nebūtu variants \u0026quot;garbage in, garbage out\u0026quot;:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"05_03-need-data.jpg\" alt=\"05_03-need-data\" title=\"05_03-need-data\"\u003e\u003c/p\u003e\n\u003cp\u003eKo tas nozīmē katram kurš grib lokāli laist LLM un kāda ir mana pieredze?\u003c/p\u003e\n\u003cp\u003eSakarīgākais jautājums:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eVai Tev ir aptuveni 100'000 EUR ko ieguldīt dzelžos?\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eJa atbilde ir jā, tad varēsi pacelt vidēja lieluma modeļus (\u0026gt;200B) ar labu precizitāti un kontekstu, un gan jau tikt sveikā cauri dažādiem uzdevumiem ar normālu ātrumu. Ja atbilde ir nē, tad nāksies izvēlēties arvien vairāk kompromisu: mazākus modeļus (kas ir derīgi tikai specializētiem uzdevumiem, un ne pārāk sarežģītiem), kvantizētās versijas, mazāki konteksta izmēri, sliktāka veiktspēja. Tas nenozīmē ka ar tiem nevarēs izdarīt neko, pat ar Qwen klases 35B MoE modeļiem uz parastas aparatūras var dabūt aptuveni 40-60 tokenus/sekundē un veikt plānveidīgus uzdevumus... bet tie nekad nebūs tik labi kā mākoņmodeļi.\u003c/p\u003e\n\u003cp\u003eVar skatīties uz visādām kastēm ar \u0026quot;unified memory\u0026quot;, bet viņas būs lēnas. Var skatīties uz mazākas jaudas GPU, piemēram Nvidia L4 lai nebankrotētu, kamēr MoE apmierina. Var arī skatīties uz augšu un sapņot par investīcijām lai dabūtu vēl vairāk naudu un tādus dzelžus ar kuriem varētu palaist GLM 5.1 modeli: \u003ca href=\"https://links.kronis.dev/1H0tRvDqCO\"\u003eunsloth/GLM-5.1-GGUF\u003c/a\u003e kam pat 4 bitu kvantizētajā versijā vajadzētu aptuveni 465 GB atmiņas + vēl KV cache pa virsu + vēl compute un bandwidth saziņai (tostarp NVLink vai tml., ar PCIe būtu bottleneck).\u003c/p\u003e\n\u003cp\u003eUn tad uzreiz ir otrs jautājums:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eBet vai vajag?\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eMan šķiet daudziem uzņēmumiem pieeja AI, un pieeja ar ko ir grūti strīdēties, būs:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u0026quot;Our AI solution is just a cloud version of GPT-5.X in a trench coat.\u0026quot;\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eUn ziniet kur ir patiesība? Tiem, kas izvēlēsies to pieeju būs labāki iznākumi, jo lokālie modeļi bez ievērojamām investīcijām nekad nekonkurēs ar tiem mākoņmodeļiem.\u003c/p\u003e\n\u003cp\u003eTie kas izmanto Codex, vai Claude Code, vai Gemini CLI / Antigravity vai jebko citu iegūs daudz vairāk spējas, nekā tie kuri izmantos sliktus modeļus sarežģītiem uzdevumiem. Tie kas kontrolē tos datu centrus un platformas, lielā mērā kontrolē spējas tiem, kas AI arī izmanto - un agrāk vai vēlāk, tas gan nozīmēs to, ka viņi uzskrūvēs cenas un samazinās usage limits dažādiem subscription... un tas joprojām būs lētāk kā censties visu laist lokāli.\u003c/p\u003e\n\u003cp\u003ePēkšņi tā naftas analoģija nav nevietā, īpaši ja apdomājam ģeopolitiku: mums vispār no AI lielajiem uzņēmumiem Eiropā ir kas... Mistral? Labi ka viņi ir, bet kaut kā vajadzētu vairāk, īpaši pēdējā laikā apzinoties, ka ASV nebūt vienmēr nav draugi, kas gribētu mums visu to labāko.\u003c/p\u003e\n\u003ch3\u003eCould public cloud be considered as AI accelerator?\u003c/h3\u003e\n\u003cp\u003e\u003cem\u003ePrezentēja: Bogdans Jarbuss, Accenture, prakses vadītājs\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eTajā pat laikā, AI pielietojums EU pieaug:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"06_01-ai-usage.jpg\" alt=\"06_01-ai-usage\" title=\"06_01-ai-usage\"\u003e\u003c/p\u003e\n\u003cp\u003eUn ierobežojumi ir tie ko minēju, papildus kam klāt nāk visādi līgumi un to ierobežojumi, neesošas kompetences, slikta datu pārvaldība un \u0026quot;legacy\u0026quot; sistēmas, kā arī nav iniciatīvas vai līdzekļi eksperimentācijai:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"06_02-limits-to-ai.jpg\" alt=\"06_02-limits-to-ai\" title=\"06_02-limits-to-ai\"\u003e\u003c/p\u003e\n\u003cp\u003eTādā ziņā Latvija tomēr atpaliek no citām EU valstīm, bet par laimi tomēr ir uzņēmumi kuriem ir iniciatīvas, kas cenšas saprast kā un kur tehnoloģijas var pielietot, un kādas:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"06_03-latvia-limits.jpg\" alt=\"06_03-latvia-limits\" title=\"06_03-latvia-limits\"\u003e\u003c/p\u003e\n\u003ch3\u003ePaneļdiskusijas\u003c/h3\u003e\n\u003cp\u003eTam visam sekoja arī paneļdiskusijas, šeit viņas apkopošu vienuviet, un padalīšos ar savām domām.\u003c/p\u003e\n\u003cp\u003ePar to, kādā līmenī valstī ir dzelži, dzirdēju aptuveni:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u0026quot;LVRTC ir 10 statnes ar Nvidia kartēm, līdz gada beigām būs 20.\u0026quot;\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eUn dažus teikumus vēlāk:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u0026quot;Darba vidē to bieži vien vispār nenodrošina, pat ne lokālos rīkus.\u0026quot;\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eVārdu sakot, LVRTC to GLM 5.1 gan jau dabūtu gaisā, bet tāds prieks ir reti kuram uzņēmumam IKT sfērā, kā arī daudzi vispār nenodrošina pat minimumu - kas noved pie tā, ka kopē privātos DeepSeek un līdzīgu AI kontos. Liek datus ko nevajadzētu iekšā mākoņrīkos, ko nevar izkontrolēt (AI platformas arī ir daudz) un ko visticamāk nevarēs izkontrolēt.\u003c/p\u003e\n\u003cp\u003eLokālos modeļus mēģina laist un lietot daudzi, bet pat lieliem uzņēmumiem neiet pārāk labi - visiem vairāk patīk mākoņmodeļi, pat ar to pašu programmatūru. Uzņēmumiem ar to ciešas integrācijas, reizēm būvē risinājumus kur runā ar publiskajiem modeļiem bet laiž pieprasījumus caur savu infrastruktūru no sākuma, lai varētu vismaz noauditēt. Sava veida hibrīdais modelis.\u003c/p\u003e\n\u003cp\u003eProtams, piemēram, maksāt par Anthropic modeļiem per-token (AWS Bedrock, vai viņu API pa taisno, vai OpenRouter utt.) vienmēr būs dārgāk kā paņemt viņu subscription: praksē atšķirība varētu būt 10-20x. Ja tu vari par 100 USD paņemt uz mēnesi subscription, tad tos limitus sasniedzot ar pieeju kur maksā par tokeniem, tie drīzāk būtu 1000 EUR, jo uzņēmumi subscription servisus paši subsidizē.\u003c/p\u003e\n\u003cp\u003eTajā pat laikā, laba ideja bija Pēterim Jurčenko (Piekļūstamības eksperts, Balsu talkas aktīvists): ieteikums katram paņemt un pamēģināt Claude Code (vai Cowork), Codex, vai līdzīgos risinājumus.\u003c/p\u003e\n\u003cp\u003eViņiem uzņēmumā bija pieeja, kur reizi nedēļā pārrunāja, kādas lietas ar AI var novienkāršot un uzlabot, kādas problēmas atrisināt. Kad to sāka darīt, tad tam aizgāja varbūt 7 minūtes, bet kā stāstīja, tagad jau tās ir 40 minūtes - jo ar modernajiem rīkiem un modeļiem, tiem ir ievērojamas spējas.\u003c/p\u003e\n\u003cp\u003eĻoti kodolīgi (aptuveni ko teica):\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u0026quot;Nav tā ka uzņēmumi un iestādes ar lieliem projektiem nevarētu atļauties Claude Code subscription.\u0026quot;\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eTur arī uzradās skeptiķis, Egils Rupenheits (ESET kiberdrošības speciālists):\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u0026quot;Nav tā ka visu tie rīki izdarīs, viņi nav bez robežām.\u0026quot;\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eTe gan gribētu pielikt savas domas: nevienam no tiem rīkiem nevajag kosmosu. Vajag lai tie rīki var spēt realizēt uzdevumus:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003epietiekami labi\u003c/li\u003e\n\u003cli\u003epietiekami ātri\u003c/li\u003e\n\u003cli\u003epietiekami lēti\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eMan šķiet ka nevaram ignorēt to ka tuvākajos 10 gados daudzi zaudēs savu darbu, jo kapitālistiskā sistēmā daudzi uzņēmumi izvēlēsies viņus aizstāt ar AI, pat ja kvalitāte nebūs ideāla, tad šiem uzņēmumiem tā būs pietiekami laba. Citi atkal nevarēs konkurēt ar tīro darba apjomu, kuru var paveikt ar AI, ja paši to neizmantos - no vienas puses tas palielinās konkurētspēju un produktivitāti visiem, kam tā tehnoloģija ir pieejama, no otras puses, iespējams devaluēs darbu. Īpaši saistoši cilvēkiem kas raksta saturu, tulko, transkribē video/audio, kā arī dažāda veida māksliniekiem un jā - arī programmētājiem.\u003c/p\u003e\n\u003cp\u003eTas ir aptuveni tāpat, kā kāpēc maksāt 150'000 EUR gadā par ASV izstrādātāju, ja var par tādu naudu paņemt veselu offshore veselu komandu. Tur vēl ir kultūras un komunikācijas jautājumi - bet AI gadījumā, modeļi kļūst labāki, viņš nekad neguļ, viņš nenogurst, viņš labi seko instrukcijām. Programmētājiem tas var būt sava veida spēju vairotājs, bet tajā pat laikā no izglītības iestādēm nāks ārā cilvēki kuri pa īstam to saturu nav iemācījušies, bet arvien vairāk ir prasījuši AI. Tāpat, jāskatās uzmanīgi kas būs ar jaunajiem izstrādātājiem - no vienas puses, kompānijas varbūt mazāk tādiem pievērsīs uzmanību jo produktivitāte visiem pārējiem pieaugs, bet no otras puses arī jaunie izstrādātāji varētu \u0026quot;sist uz augšu\u0026quot; vienu soli, jo AI var palīdzēt nepieļaut dažādas kļūdas (kamēr modelis ir pietiekami labs un nesaka visu laiku \u0026quot;Yes, you are absolutely right!\u0026quot; par muļķībām) un darīt vairāk, kā viņu pašu prasmes ļautu.\u003c/p\u003e\n\u003cp\u003eVārdu sakot, ir risks drusku nosliekties distopijas virzienā, bet runājot par pašu produktivitāti, kad rīku un modeļu kombinācija mums iedod \u0026quot;good enough\u0026quot; kvalitāti, tad skats uz konkrēto industriju mainās. Man šķiet programmētājiem tieši tādas izmaiņas bija 2025. gada beigas un kad iznāca Opus 4.5 modelis, ko kombinācijā ar atbilstošu rīku nodrošinājumu un kontroles metodēm var pataisīt par efektīvu kolēģi - kamēr es rakstu šo te postu, man viņš fonā jau 3 projektos uztaisīja refaktoringu, pēc mana plāna, kā arī palaida visus sakonfigurētos koda kontroles rīkus, salaboja kļūdas, palaida 3 paralēlas review sub-aģentu iterācijas, cikliski salaboja visas atrastās problēmas, līdz viss ir pieņemamā stāvoklī.\u003c/p\u003e\n\u003cp\u003ePar pašu paneļdiskusiju runājot, atkal labi ka drusku izteica abu veidu argumentus, gan par, gan pret - jo lai gan konfliktējoši viedokļi, nevajag no tā baidīties, un taisnība bija abiem. Arī tas pats Egils Rupenheits atgādināja, ka lokālo modeļu kvalitāte varētu arī noteiktiem uzdevumiem nebūt \u0026quot;good enough\u0026quot;. Tātad, pie tā vēl jāstrādā, jāgaida kamēr modeļi kļūs labāki (piem. var salīdzināt Qwen 2.5 un Qwen 3.5 un vispār cik liels progress notiek gadu mijā), kā arī jāstrādā pie inženierijas - kā jau teicu, pat ja ir spējīgi modeļi, lai nebūtu vibe coded sviests, tāpat vajag testus un veidu kā kodam piespiest izskatīties un strādāt tā, kā mēs to gribam.\u003c/p\u003e\n\u003ch3\u003eKā pašiem izmēģināt AI tehnoloģijas\u003c/h3\u003e\n\u003cp\u003ePapildus tam, ja nu kāds grib palaist kādu modeli lokāli arī, te būs īsās instrukcijas no manas puses. Tas gan nebūs production use case, bet vairāk pašu interesei.\u003c/p\u003e\n\u003cp\u003eNo sākuma vajadzēs programmatūru lai modeļus laistu, piemēram \u003ca href=\"https://links.kronis.dev/99b8RkGKKN\"\u003eOllama\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003eKad tas ir uzinstalēts, tad var paskatīties kādi modeļi interesē: \u003ca href=\"https://links.kronis.dev/9mixa\"\u003eModels\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eLokālām ierīcēm tie varētu būt MoE (mixture of experts, aktivizē tikai daļu neironu) modeļi un pēc izmēra nelieli: piemēram Qwen 3.6, Gemma 4, Qwen 3.5 (ir mazākas versijas kā 3.6), GLM 4.7 Flash, Devstral Small 2.\u003c/p\u003e\n\u003cp\u003eKad modelis ir izvēlēts, tad atliek sekot \u003ca href=\"https://links.kronis.dev/9KXa2oZc67\"\u003eQuickstart instrukcijām\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003eAr mazajiem modeļiem caur API gan vairums programmatūras izstrādes rīku strādās visai slikti, man labākā pieredze bija tieši ar \u003ca href=\"https://links.kronis.dev/K5oB0AMWv5\"\u003eCline\u003c/a\u003e, \u003ca href=\"https://links.kronis.dev/w6hrt\"\u003eRooCode\u003c/a\u003e un \u003ca href=\"https://links.kronis.dev/Q3doIZbDgK\"\u003eKiloCode\u003c/a\u003e (būtībā vairāki fork sākotnējam projektam ar savām fīčām), praksē labāk nekā OpenCode, jo vairāk pabiksta modeli pareizajā virzienā. Iesaku paņemt un uz Visual Studio Code viņus uzstādīt, paskatīties vai caur Ollama API strādā ok.\u003c/p\u003e\n\u003cp\u003ePapildus tam, ja nu gribas ko automatizēt, var paņemt arī \u003ca href=\"https://links.kronis.dev/2n801d4SDq\"\u003eLiteLLM bibliotēku priekš Python\u003c/a\u003e, strādās labi.\u003c/p\u003e\n\u003cp\u003eGalvenais ko saprast: svarīgi ir nevis paši modeļi, bet gan tas ka OpenAI saderīgo API (kad pārejat no Ollama API uz to, agrāk vai vēlāk; pats Ollama atbalsta abus) varēs saslēgt arī mākoņmodeļiem pēc vajadzības. Vārdu sakot, modelis ir kaut kas ko pēc vajadzības var mainīt, piemeklēt kas ir konkrētam uzdevumam piemērots un pašiem pa makam.\u003c/p\u003e\n\u003cp\u003eJa nu gribas laist ko glaunāku par Ollama, te būs daži atslēgas vārdi: LM Studio, llama.cpp, vLLM.\u003c/p\u003e\n\u003cp\u003eUn ja vajag vairāk GGUF formāta vai jebkāda cita formāta modeļus, redz kur saits: \u003ca href=\"https://links.kronis.dev/nGkfCddqXG\"\u003eHuggingFace - Models compatible with the GGUF library\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eKonkrētāk, ļoti labas kvantizētās versijas ir \u003ca href=\"https://links.kronis.dev/ymDC2Ywdn9\"\u003eUnsloth\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003eSavukārt, ja gribat vienkārši paskatīties kā strādā State of the Art (SOTA) modeļi, tad redz kur, var paņemt ~20 USD (ar šo gan varētu būt par īsu) vai 100 USD subscription: \u003ca href=\"https://links.kronis.dev/AVm9QbpD1U\"\u003eClaude Pricing\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eIr arī lētākas opcijas, piemēram GLM 5.1 varat sameklēt, bet arī daļa no šī pēc gada vai diviem varētu būt out of date.\u003c/p\u003e\n\u003ch3\u003eKopsavilkums\u003c/h3\u003e\n\u003cp\u003eKopumā konferenci labi paklausīties, redzēt ka Latvijā arī kaut kas notiek, vienīgais ka tā nākotne tāda neskaidra paliek.\u003c/p\u003e\n\u003cp\u003eVisi ignorēja grūtos jautājumus, tādi optimistiski paši, par to problēmu ka AI tehnoloģijas mēs paši bieži nekontrolējam (un kad kontrolējam, nekonkurējam) neviens īsti nerunāja.\u003c/p\u003e\n\u003cp\u003eBet no otras puses, tas pasākums arī ir sava veida marketing padarīšana, nevar gaidīt ka tur būtu īstā vieta vai laiks izcilāt sarežģītos jautājumus.\u003c/p\u003e\n"
        },
        {
            "title": "On Anthropic",
            "date_published": "2026-02-28",
            "id": "https://blog.kronis.dev/blog/on-anthropic",
            "url": "https://blog.kronis.dev/blog/on-anthropic",
            "content_html": "\u003cp\u003eIt's been a while since I last looked over the pond, at what the Americans are doing. The problem is that they just keep doing more stuff, to the point where if I were to write about them, it would be a long post and it becomes less and less possible to hide behind sarcasm and not condemn that whole administration.\u003c/p\u003e\n\u003cp\u003eAt the same time, in absolute terms I am nobody so that gives me some safety, but as a European, it's also hard to hide my disgust at blatant disregard for human rights, human life and our European values - especially when there's foreign interference that aims to meddle with the EU, NATO and also support far right, and borderline fascist, parties like AfD.\u003c/p\u003e\n\u003ch3\u003eA measured take on AI and safety\u003c/h3\u003e\n\u003cp\u003eToday, however, I'm not doing a long political post, rather I'd like to just reference this one post that Anthropic recently made: \u003ca href=\"https://links.kronis.dev/KAYKp29Vup\"\u003eStatement from Dario Amodei on our discussions with the Department of War\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eIn the post, they do a little patriotic blurb about wanting to help Americans and keep them safe, which I don't take an issue with. Even though I'd say that the administration is trampling over the legacy of what was once a democratic ally, I doubt anyone would take an issue with wanting to keep your people safe - I don't have a problem with Americans in general, the same way I have some friends in Israel while holding similar disdain for the actions of that state.\u003c/p\u003e\n\u003cp\u003eAnthropic then proceeds to reiterate that they don't have \u0026quot;ad hoc\u0026quot; limitations on the use of their models, but that they take an issue with two particular use cases, where it is impossible to use the technology responsibly:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-anthropic-limitations.jpg\" alt=\"01-anthropic-limitations\" title=\"01-anthropic-limitations\"\u003e\u003c/p\u003e\n\u003cp\u003eThey do share a considerable amount of detail, but I'd say that their assessment is correct. With the current state of AI, using it for either mass surveillance or fully autonomous weapons systems would be deeply problematic and in quite a few cases, also severely illegal - though it's not that it has stopped the current administration before.\u003c/p\u003e\n\u003cp\u003eNo matter how you look at it, their response is fairly measured and in any other governmental system, that should be met with nothing other than agreement from the powers that be, while a lot of other use cases could be pursued, money exchanged, and Americans kept safe. However, that was not the response that they got. Almost immediately, there was backlash from the government, which is ridiculous - since the only things they could have opposition on is that they do, as a matter of fact, actually want to use AI for those very unsafe and illegal use cases.\u003c/p\u003e\n\u003cp\u003eLuckily, a lot of people from both OpenAI and Google got together and signed an open letter in protest of that: \u003ca href=\"https://links.kronis.dev/ZLyuanaaSN\"\u003eWe Will Not Be Divided\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eHere's a snapshot:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02-we-will-not-be-divided.jpg\" alt=\"02-we-will-not-be-divided\" title=\"02-we-will-not-be-divided\"\u003e\u003c/p\u003e\n\u003cp\u003eAgain, the position of not letting technologies with no fundamental possibility to take responsibility over its actions kill people feels like the bare minimum. How did that \u003ca href=\"https://links.kronis.dev/Xn8PTM58mj\"\u003eold warning from IBM in the late 70s\u003c/a\u003e go?\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eA computer can never be held accountable, therefore a computer must never make a management decision.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eObviously, that was talking about management - things like possibly wrongfully terminating someone from their work position, we're talking about something way more severe than that. So surely the people in power understand something that was known decades ago, right?\u003c/p\u003e\n\u003ch3\u003eAn unhinged take on AI and safety\u003c/h3\u003e\n\u003cp\u003eRight?\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"05-president-response.jpg\" alt=\"05-president-response\" title=\"05-president-response\"\u003e\u003c/p\u003e\n\u003cp\u003eWhat a bunch of drivel.\u003c/p\u003e\n\u003cp\u003eIt's as disgusting, as it is embarrassing:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eclaiming that anything you don't like is \u0026quot;radical left\u0026quot; or \u0026quot;woke\u0026quot;, exaggerating for the sake of having an easy out to hate it\u003c/li\u003e\n\u003cli\u003eclaiming that taking an issue with illegal mass surveillance and autonomously killing people is somehow \u0026quot;strong arming\u0026quot;\u003c/li\u003e\n\u003cli\u003eclaiming that those are somehow codified in the US constitution\u003c/li\u003e\n\u003cli\u003eclaiming that not wanting to do those things is somehow selfish or endangers your people\u003c/li\u003e\n\u003cli\u003eopenly saying that you will use and abuse whatever power you have to strong arm them instead, into allowing both illegal and ethically bankrupt actions\u003c/li\u003e\n\u003cli\u003eopenly threatening to prosecute them for having a shred of human decency\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eI'm happy I cut the few Trump supporters out of my life, since you have to be either delusional or plain evil to support someone like that.\u003c/p\u003e\n\u003cp\u003eI think it's useless to point out individual lies when everything that is said is lies. The new status quo seems to be a \u003ca href=\"https://links.kronis.dev/NH23GuNDNx\"\u003egish gallop\u003c/a\u003e of bullshit and attempts to shift the \u003ca href=\"https://links.kronis.dev/d6hf4RGHKO\"\u003eoverton window\u003c/a\u003e so far right that explaining why mass surveillance, letting drones autonomously kill people or, quite possibly, eventually repressing anyone who wants to stand up against all that somehow puts the burden of proof on you - as if all of those are perfectly normal and patriotic.\u003c/p\u003e\n\u003cp\u003eThat's exactly the kind of thing that people who want to see the US turn into Russia 2.0 would say and strive for. The lack of humanity and basic decency on display is apalling.\u003c/p\u003e\n\u003ch3\u003eCorporate profit seeking and capitalism\u003c/h3\u003e\n\u003cp\u003eUnfortunately, because of the clown world that we live in, not long after, OpenAI filled in that spot, in search of a nice big fat chunk of money to let them sleep at night no matter what their systems will do. Read it from the source: \u003ca href=\"https://links.kronis.dev/HCPrN6J3t3\"\u003eSam Altman on Twitter\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03-openai-agreement.jpg\" alt=\"03-openai-agreement\" title=\"03-openai-agreement\"\u003e\u003c/p\u003e\n\u003cp\u003eMaybe that's why a page where you can delete your OpenAI account is on the top of HN right now (and they're predictably faking deletion errors to prevent at least some people from successfuly leaving them):\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04-hacker-news-front-page.jpg\" alt=\"04-hacker-news-front-page\" title=\"04-hacker-news-front-page\"\u003e\u003c/p\u003e\n\u003cp\u003eAt the same time, you have people who are in the comments explaining how they're employees on HN and don't see this as a reason to leave their employment there. It will probably be increasingly saddening to see how the voices of the employees mean nothing, but also how some of those will have been just performative - because I don't expect everyone on that open letter signatories list to have resigned by the end of the week and having a cushy position at Anthropic or Google.\u003c/p\u003e\n\u003cp\u003eThe thing is, that I can't even easily blame the employees at those companies. When push comes to shove, they are as dependent on money to make a living as anyone else - and uprooting your own life over principle, especially when many others won't do the same, is hard to ask of anyone. For example, if I was also making a 6 or even 7 figure income, I'd also find it similarly hard to take action like that if I didn't know what the future would hold, even if other large orgs would be more than happy to throw me a whole bunch of cash and comfort if I helped them instead of the previous org. It's just unfortunate to see that, as it is others applauding such actions.\u003c/p\u003e\n\u003ch3\u003eJust go with Anthropic\u003c/h3\u003e\n\u003cp\u003ePersonally, I voted with my wallet and now have an \u003ca href=\"https://links.kronis.dev/PDHIljAFkK\"\u003eAnthropic Max subscription\u003c/a\u003e - with it, in the past few weekends I've done more than some other people do during an entire week or more of work. I can work on 4 projects in parallel and, when coupled with proper safeguards such as prebuild scripts to enforce the architecture and conventions that I want, linters and code tests, it's a real force multiplier in both software development and elsewhere. It's actually so good and they give me so much usage, that I can just throw Opus 4.6 at all problems and having a powerful model always available has eliminated me needing to swap between various different ones.\u003c/p\u003e\n\u003cp\u003eI also benefit greatly from \u003ca href=\"https://links.kronis.dev/OZRJD1CUw3\"\u003eClaude Code\u003c/a\u003e - since it seems to stick to the working directory a lot better than even the same models running on AWS Bedrock in the EU when coupled with OpenCode, in addition to having pretty great support for sub-agents and parallel tool calling. They also have a desktop version and while I don't enjoy it being written in Electron and sometimes being sluggish, it's great from an observability point of view - I can view a list of all the parallel projects being worked on, tool calls and permissions, and manage it all way better than I could with just a parallel terminal windows.\u003c/p\u003e\n\u003cp\u003eEven their \u003ca href=\"https://links.kronis.dev/1IDNokJy8n\"\u003eClaude Cowork\u003c/a\u003e seems promising and I quite like the idea of having a nice UI for agentic work like that (even though Claude Code achieves a lot of the same), albeit the limitation of it only being able to work on my OS drive and home folder feels a bit arbitrary. I am not actually quite sure whether direct computer use will ever quite pan out due to how much we rely on GUI software for all sorts of things, which makes me think that the world should embrace CLI tools that GUI tool just sit in the front of for convenience a bit more - structuring our apps more like the MVC pattern, with the GUI solutions being just the view/controller component in some regards, but I digress.\u003c/p\u003e\n\u003cp\u003eI will say that the end goal for me there would be to have something similar to Jenkins (just an example, not actual Jenkins) or Woodpecker CI, where I could install an orchestrator of some sort on a server and let it take care of directing my homelab servers and personal PC to work on tasks, and occasionally check in on that with my phone or laptop when I'm on the move. That would make it as close to perfect as possible, alongside good permissions settings and sandboxing (e.g. commands that are verboten and running the agents in Docker containers, similar to Woodpecker CI runs). Either way, people already seem to be working on \u003ca href=\"https://links.kronis.dev/dhUlvcj6Sx\"\u003eConductor\u003c/a\u003e and numerous similar projects, as well as \u003ca href=\"https://links.kronis.dev/cPNzOsgQVI\"\u003eOpenHands\u003c/a\u003e, so it's probably just a matter of time until I get what I want.\u003c/p\u003e\n\u003cp\u003eI will admit that Anthropic's subscription replaced pretty much all others for me, since I really don't need to spend around 200 EUR per month on these technologies, after finding one that is good enough. Perhaps that's also largely why they're trying so hard to be loss leaders in the industry and secure as big of a userbase as possible - because now Cerebras Code doesn't get any of my money, even if I otherwise found their products and performance to be great, because now I have one that, even if slower, covers all of my use cases well enough.\u003c/p\u003e\n\u003ch3\u003eSummary\u003c/h3\u003e\n\u003cp\u003eIn an ideal world, we'd see \u0026quot;Anthropic EU\u0026quot; emerge, with strategic funding from the greater EU and partnerships with Mistral, to push EU towards AI innovation and give the company a safe area to operate in, which better aligns with its values and the greater European culture, a safe haven during this tumultuous time in the US. That could provide a similar push towards innovation and improvements, as the first emergence of DeepSeek from the East did towards the then stagnating Western models and technologies.\u003c/p\u003e\n\u003cp\u003eSadly, there are complexities there that I believe wouldn't be easy enough to navigate and make the whole idea unfeasible - even if I like to daydream about a world where the EU leadership is competent, quickly sets aside 50-100B EUR (I would gladly have my tax money going towards this) and gets an offshoot off of the main company going here with expedited visas and residency permits and housing whoever needs them, with the EU being the owner of this newfound company, alongside pushing for partnerships and merging with Mistral, making EU be a considerable force when it comes to this technological innovation and implementation, but one that cares about human rights and safety.\u003c/p\u003e\n\u003cp\u003eIt would take the equivalent of one year's GDP of the Baltic countries to dominate AI in the world. That's a no brainer. Like, I'm going to be somewhat poor all my life anyway, we might as well take the output of my work and use it for something meaningful.\u003c/p\u003e\n\u003cp\u003eIf there's one thing we should take from the Americans, then it's their entrepreneurial spirit and the risk taking behavior - building our own sovereign tech instead of just giving our hard earned money away to a bunch of corpos and consultants and contractors with no long term ownership over everything. It's the same how a lot of the digital infrastructure lives under the shoe of Microsoft and AWS, instead of embracing Linux (and contributing back to the community) and supporting regional businesses such as Hetzner. It's like they think that reducing the history of the European nations down to a slow decline in the name of safety and predictability is somehow better than risking it all to prosper and not taking a shot at greatness, sovereignty and success.\u003c/p\u003e\n\u003cp\u003eWhy can't we live in a world like that? Then again, in such a world, the help towards Ukraine would also look way different, as would the stances on numerous other issues - and that might take tightening our belts (not that RAM prices and rampant capitalism don't do that already, we really should produce our own tech), alongside the populace being both principled, ethical and educated. I want to live in that sort of a world.\u003c/p\u003e\n\u003cp\u003eIt's not even that elements of nationalism are inherently bad, but rather that those other people are blatantly corrupt and more often than not plain evil. I hope Americans wake up and vote them out, or put the ones who have committed crimes in prison. Sadly, I won't be holding my breath on that, it seems like the rule of law and increasingly, common sense, both seem to be dead or dying. What a world to live in.\u003c/p\u003e\n"
        },
        {
            "title": "Buying some drives from Datablocks",
            "date_published": "2026-02-12",
            "id": "https://blog.kronis.dev/blog/buying-some-drives-from-datablocks",
            "url": "https://blog.kronis.dev/blog/buying-some-drives-from-datablocks",
            "content_html": "\u003cp\u003eA while back, I learnt of the practice of buying white label and recertified hard drives: the idea being that large batches of good drives sometimes get returned to OEMs by hyperscalers. They can then be retested and sold either with the branding, or with the branding removed (although can sometimes have small scratches and such).\u003c/p\u003e\n\u003cp\u003eEssentially, if I decide to get a white label drive, then the original manufacturer is off the hook in regards to what happens with it, however, it can still very much be a good drive with a low RMA rate (less than a percent), so I can actually get a really good deal.\u003c/p\u003e\n\u003cp\u003eOne such vendor that is available in Europe is \u003ca href=\"https://links.kronis.dev/a8qW8x2EOK\"\u003eDatablocks\u003c/a\u003e, I think I heard about them on HackerNews and saved the link for later. Eventually, I decided to get a few 1 TB HDDs from them because I found out that the regular Seagate 1 TB drives I usually got from a local e-commerce store went up in price from like 40-50 EUR all the way to 110 EUR, which I think is pretty insane. Not sponsored by them in any way, just decided to share my experience.\u003c/p\u003e\n\u003ch3\u003eThe good\u003c/h3\u003e\n\u003cp\u003eI don't really use high capacity drives, because I get them in multiples for backup reasons and having spares, and since I compress my data and don't really have that much of it, 1 TB drives still make sense (unless there's a good deal on 2 TB drives, which there wasn't).\u003c/p\u003e\n\u003cp\u003eThere were still some in stock, so I went for the desktop drives:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"00-datablocks-drives.jpg\" alt=\"00-datablocks-drives\" title=\"00-datablocks-drives\"\u003e\u003c/p\u003e\n\u003cp\u003eLook at that: that's 35 EUR for a 1 TB drive, even cheaper than Seagate drives used to be back when the prices were actually decent. There was shipping I had to pay to get them delivered to Latvia, which was short of 30 EUR total - that wasn't too pleasant, but since I got two drives for about 70 EUR, the total came out to around 100 EUR, which is still somehow cheaper than a single drive from a local store.\u003c/p\u003e\n\u003cp\u003eShipping was done through DPD and the package arrived within a week, safely:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-the-box.jpg\" alt=\"01-the-box\" title=\"01-the-box\"\u003e\u003c/p\u003e\n\u003cp\u003eThere was some of that packing paper inside (maybe the box itself is a bit too big for the contents), and the drives themselves were wrapped in bubble wrap nicely:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02-the-packaging.jpg\" alt=\"02-the-packaging\" title=\"02-the-packaging\"\u003e\u003c/p\u003e\n\u003cp\u003eHere are the drives, nothing special about them, they came in sealed packages, just like new ones do:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03-the-drives.jpg\" alt=\"03-the-drives\" title=\"03-the-drives\"\u003e\u003c/p\u003e\n\u003cp\u003eFor comparison's sake, here's one of the Seagate drives (actually the last new one I had), as you can see, they look pretty similar and there's also no obvious scratches or defects that I can notice:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04-drive-comparison.jpg\" alt=\"04-drive-comparison\" title=\"04-drive-comparison\"\u003e\u003c/p\u003e\n\u003cp\u003eSame on the back, visually, everything looks okay:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"05-drive-comparison-back.jpg\" alt=\"05-drive-comparison-back\" title=\"05-drive-comparison-back\"\u003e\u003c/p\u003e\n\u003cp\u003eNow, at work I've run into issues with using HDDs, mostly due to random reads and writes being bad due to their nature: in particular, when trying to host an instance of S3 compatible software, such as \u003ca href=\"https://links.kronis.dev/np6IrwR22V\"\u003eGarage\u003c/a\u003e. There, the read and write performance plummets when the files are chunked, or alternatively, you have to give up storage efficiency by increasing the chunk sizes.\u003c/p\u003e\n\u003cp\u003eRegardless, for my personal use, HDDs are more than enough, given that they have decent capacity and are still cheaper than equivalent SSDs most of the time - as of the time of writing this, 1 TB SSDs would cost about 120 EUR each, which is way outside of my budget. For my homelab servers, I usually get a small SATA SSD for the boot drive and everything else, including databases, lives on the HDDs, because thankfully I don't really run anything public or at least in high demand on those boxes.\u003c/p\u003e\n\u003cp\u003eFor my main PC, I have a 1 TB SATA SSD for projects and games, as well as another 1 TB NVMe SSD for the boot drive, but my hand was more or less forced there, because I ran out of SATA ports - since I have both a backup drive locally for each HDD (nothing fancy like RAID, just synchronizing them with \u003ca href=\"https://links.kronis.dev/5lefa\"\u003eFreeFileSync\u003c/a\u003e), and then further replicate that storage remotely.\u003c/p\u003e\n\u003cp\u003eEither way, this is more or less what a full on homelab server setup would look like for me, and I'm happy to be able to procure it on a budget:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"06-next-to-boot-drive.jpg\" alt=\"06-next-to-boot-drive\" title=\"06-next-to-boot-drive\"\u003e\u003c/p\u003e\n\u003cp\u003eEven though those drives will sit in a drawer for a while until I need them, I decided to at least boot them up and see if they work - whether any of them are dead on arrival. For that, since all of the SATA ports on my main PC are taken up, I just pulled out my USB 3.0 HDD enclosure (which comes with its own power plug, neat) and put the drives there. You can see it here with one of the Seagate drives (I had one that I previously used with \u003ca href=\"https://links.kronis.dev/nwvq2\"\u003eClonezilla\u003c/a\u003e in addition to the new one), which I also spun up just to compare the performance:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"07-drive-enclosure.jpg\" alt=\"07-drive-enclosure\" title=\"07-drive-enclosure\"\u003e\u003c/p\u003e\n\u003cp\u003eLet's look at some numbers, shall we?\u003c/p\u003e\n\u003ch3\u003eSome data and numbers\u003c/h3\u003e\n\u003cp\u003eThe Seagate drive actually showed that this is its first time being powered on, as you'd expect with a completely new drive. Other than that, no errors or anything to report, since obviously it hasn't had any time to accumulate data, even if there was anything wrong with the drive, shown in \u003ca href=\"https://links.kronis.dev/fah21J4tKY\"\u003eCrystalDiskInfo\u003c/a\u003e:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"08-seagate-crystal-disk-info.jpg\" alt=\"08-seagate-crystal-disk-info\" title=\"08-seagate-crystal-disk-info\"\u003e\u003c/p\u003e\n\u003cp\u003eFor a quick comparison, here's both of the drives I got from Datablocks, which interestingly enough showed that this is their second time being powered on, but otherwise also didn't really have much in regards to accumulated data. Oh, and also, turns out that they're 5400 RPM drives, compared to my Seagate 7200 RPM drive, here's drive #1:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"09-datablocks-crystal-disk-info-1.jpg\" alt=\"09-datablocks-crystal-disk-info-1\" title=\"09-datablocks-crystal-disk-info-1\"\u003e\u003c/p\u003e\n\u003cp\u003eAnd here's drive #2:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"10-datablocks-crystal-disk-info-2.jpg\" alt=\"10-datablocks-crystal-disk-info-2\" title=\"10-datablocks-crystal-disk-info-2\"\u003e\u003c/p\u003e\n\u003cp\u003eI also decided to run some tests with \u003ca href=\"https://links.kronis.dev/PNLZJx4QAj\"\u003eCrystalDiskMark\u003c/a\u003e. Normally you'd want to run it on a large amount of data, but frankly I didn't have that much time, so here's a quick 2 GB read and write test showing us what we already know, that the drives are good for sequential reads and writes, but crumble under random workloads.\u003c/p\u003e\n\u003cp\u003eHere's the Seagate drive:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"11-seagate-crystal-disk-mark.jpg\" alt=\"11-seagate-crystal-disk-mark\" title=\"11-seagate-crystal-disk-mark\"\u003e\u003c/p\u003e\n\u003cp\u003eHere's the Datablocks drive #1:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"12-datablocks-crystal-disk-mark-1.jpg\" alt=\"12-datablocks-crystal-disk-mark-1\" title=\"12-datablocks-crystal-disk-mark-1\"\u003e\u003c/p\u003e\n\u003cp\u003eHere's the Datablocks drive #2:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"13-datablocks-crystal-disk-mark-2.jpg\" alt=\"13-datablocks-crystal-disk-mark-2\" title=\"13-datablocks-crystal-disk-mark-2\"\u003e\u003c/p\u003e\n\u003cp\u003eIt actually surprised me that the 5400 RPM drives did more or less the same as the 7200 RPM drive in a way where it doesn't seem like a particularly good or bad run on either of their parts. Maybe it's the cache that they have, but even so, unless you are storing a bunch of movies in high quality, I don't imagine you usually move much more data than a few dozen GB at a time. To be honest, even for prolonged tasks like syncing my whole local drive with a homelab drive, I don't think it matters that much even if the new drives were to be a bit slower, given that this isn't something I'd feel every day.\u003c/p\u003e\n\u003cp\u003eOn the other hand, if I ever tried using one of these drives to play a game like Arma Reforger, I'd very quickly find out why you really want an SSD for that kind of a workload - because model LODs and textures would just fail to load in properly in time, like actually happened to me before I moved to having my games be on an SSD, though sadly at the time I didn't really take any pictures.\u003c/p\u003e\n\u003cp\u003eEither way, I got the drives, I'd probably need to use them for a while to see if they're actually good or whether I got unlucky, other than that - I'm a pretty happy customer! The ordering process wasn't too difficult, the prices were good, the shipping was fast, it went well.\u003c/p\u003e\n\u003cp\u003eSo, was there anything that was lacking?\u003c/p\u003e\n\u003ch3\u003eThe bad\u003c/h3\u003e\n\u003cp\u003eFirst up, I think there is an AI slop background image on the homepage. While the company and the drives are real, seeing low quality crap like that makes me doubt their trustworthiness, if I'd randomly stumble upon the site, I'd be even more inclined to just click away.\u003c/p\u003e\n\u003cp\u003eSecondly, the stock kind of sucks for lower capacity drives. At the time of writing, there were only 1 TB ones available and not slightly bigger ones like 2 TB, 4 TB and so on, the next actually available group of drives were 8 TB which are totally outside of my budget. The specs page for the drive I picked also didn't mention RPM anywhere, so I was a bit surprised upon seeing it after receiving it.\u003c/p\u003e\n\u003cp\u003eA customer might be reasonably upset at that:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"14-low-stock.jpg\" alt=\"14-low-stock\" title=\"14-low-stock\"\u003e\u003c/p\u003e\n\u003cp\u003eThey have a page with more specs, but trying to find the details of that particular drive yields nothing of use, instead I see that it might match a 4 TB drive, which is plain wrong:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"15-no-specs.jpg\" alt=\"15-no-specs\" title=\"15-no-specs\"\u003e\u003c/p\u003e\n\u003cp\u003eThat more or less concludes my complaints, nothing I couldn't handle when they effectively allow me to save 2x on drive costs, or at least have some backup drives in this time of uncertainty.\u003c/p\u003e\n\u003ch3\u003eSummary\u003c/h3\u003e\n\u003cp\u003eOverall, I'd say that it's cool that companies like this exist, since it probably also reduces e-waste somewhere along the way. Plus, if you actually do need larger capacity drives, they have plenty of those available as well:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"16-high-stock.jpg\" alt=\"16-high-stock\" title=\"16-high-stock\"\u003e\u003c/p\u003e\n\u003cp\u003eI did look at their blog and it seems like their last restock was around the end of 2024 which does raise some questions, but go figure - it's not like drives go bad quickly and since I doubt they have a super big budget for marketing, maybe the stocks don't dry up that fast anyways.\u003c/p\u003e\n\u003cp\u003eI'm still undecided where they land between \u0026quot;a few people making a buck and having a few boxes of drives in storage somewhere\u0026quot; and a bigger company, but that hardly matters, since it worked out pretty well for me. Who knows, maybe some day I'll go for the bigger drives, or perhaps they'll even get SSDs in stock.\u003c/p\u003e\n\u003ch3\u003eUpdate\u003c/h3\u003e\n\u003cp\u003eA person actually reached out and linked what I think might have been the original \u003ca href=\"https://links.kronis.dev/aEmoS5rjM5\"\u003eHN submission\u003c/a\u003e that I learnt of Datablocks from! They had a blog post that goes into more detail and has an update 6 months later; also, they tried out the 18 TB drives! I kinda made my own post mostly for visibility, and because I think it's cool that businesses like that exist in the first place.\u003c/p\u003e\n"
        },
        {
            "title": "Why Europe needs open source",
            "date_published": "2026-02-08",
            "id": "https://blog.kronis.dev/blog/why-europe-needs-open-source",
            "url": "https://blog.kronis.dev/blog/why-europe-needs-open-source",
            "content_html": "\u003cp\u003eOkay, so I'm a little bit late with this one.\u003c/p\u003e\n\u003cp\u003eA while back, there was \u003ca href=\"https://links.kronis.dev/ECfsr6dVKL\"\u003ea post on LWN\u003c/a\u003e about the European Commission calling for evidence on open source. If you want, you can have a look at the \u003ca href=\"https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=intcom%3AAres%282026%2969111\"\u003eEUR-Lex\u003c/a\u003e post for yourself:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-eur-lex.jpg\" alt=\"01-eur-lex\" title=\"01-eur-lex\"\u003e\u003c/p\u003e\n\u003cp\u003eThe gist of it is, that Europe has become quite dependent on foreign tech, which means risks both in regards to the supply chain itself, as well as the overall governance - and given the state of the world, isn't a really good situation to be in:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThe EU faces a significant problem of dependence on non-EU countries in the digital sphere. This reduces users' choice, hampers EU companies' competitiveness and can raise supply chain security issues as it makes it difficult to control our digital infrastructure (both physical and software components), potentially creating vulnerabilities including in critical sectors.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eIn the last few years, it has been widely acknowledged that open source – which is a public good to be freely used, modified, and redistributed – has the strong potential to underpin a diverse portfolio of high-quality and secure digital solutions that are valid alternatives to proprietary ones. By doing so, it increases user agency, helps regain control and boost the resilience of our digital infrastructure.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eFor a long time, it has been happening silently in the background, various systems being built, integrated and operated, or sometimes changing ownership to that of presumed allied countries, with nobody paying it much attention - such as the \u003ca href=\"https://links.kronis.dev/gC1EFVJzeg\"\u003eNetherlands DigiD identity system\u003c/a\u003e almost getting sold to the US.\u003c/p\u003e\n\u003cp\u003eIt's not even the case of various large systems that might get treated like public utilities, but rather even the foundational building blocks, from OSes like Windows Server and RHEL (still controlled by a foreign company, despite being more open than Windows), to databases like DB2, SQL Server, Oracle and others, alongside a huge amount of proprietary solutions, frameworks and even libraries.\u003c/p\u003e\n\u003cp\u003eThe thing is, that the risks of vendor-lock have been known for a long time, though sadly we have to contend with adages such as:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eNobody ever got fired for choosing IBM.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eReplace IBM in that sentence with any mainstream large tech company, be it Microsoft, Google, or maybe even entire platforms like AWS. People keep waving their hands around and saying that proprietary technology is good, actually, since it often comes with support (not that you can't be the support for an open-source solution, or even pay someone to support you, but apparently that eludes most people; it might just be about covering your ass in case something goes wrong, but I'll explain why it doesn't actually work). Couple that with the sales departments of those companies having a lot of resources at their disposal and large govt. contracts being right up their alley, and the friction for using that tech as opposed to FOSS or even source-available software will often be lower.\u003c/p\u003e\n\u003cp\u003eNow, four weeks have passed since the call was open, so it's not like I would want to submit anything formal, but at the same time I at least wanted to share my thoughts on open source and why it matters to Europe. As an opinion piece, this won't really be that facts heavy, since I've had quite the busy weekend as well. My opinions are my own, my experiences are what I've lived through.\u003c/p\u003e\n\u003ch3\u003eWhy we need Open Source\u003c/h3\u003e\n\u003cp\u003eFirst up, I don't believe that any of those companies are evil. They are companies - they are there to make money. It's just that that very assumption is likely to hurt you. They are not your friends, they are only interested in ensuring your success insofar as they'd be able to do further business with you. This implies vendor lock. They hold the cards in regards to control over the technology, licensing it, and might alter the deal whenever they please. That's just the reality that you have to accept when dealing with them, before sovereignty of any sort or security risks even enter the conversation.\u003c/p\u003e\n\u003cp\u003eSecondly, a lot of that \u0026quot;choose X to not get fired\u0026quot; is both a lack of imagination and spine - for technically inclined developers, not having access to source will almost always be more difficult to work with than being able to peel back the abstractions and see what's inside the solution, and make any alterations as necessary. Or, in lieu of having those skills themselves, being able to pull up a GitHub discussion and see what other brilliant people have done to find a workaround for a particular issue. Anyone who has been on StackOverflow, or even used AI to help with issues that have been publicly acknowledged, will know that.\u003c/p\u003e\n\u003cp\u003eThe whole bit about covering your ass in case something goes wrong, it doesn't actually help you as much as you think. If you need support, there are plenty of companies that would be more than happy to support your open source installs of whatever software you can imagine. Often times, it can be even simpler than going for the proprietary software. You can get \u003ca href=\"https://links.kronis.dev/6x9xh\"\u003eMattermost\u003c/a\u003e installs that will even be run for you, same for \u003ca href=\"https://links.kronis.dev/agAovUSkkE\"\u003eZulip\u003c/a\u003e if you need some sort of a communication tool. You can use \u003ca href=\"https://links.kronis.dev/u6nbEksuRa\"\u003eJitsi Meet\u003c/a\u003e for free until you have a need to self host, unlike Zoom. You can get managed installs of \u003ca href=\"https://links.kronis.dev/bDO6nwhn51\"\u003eNextcloud\u003c/a\u003e if you want, you can get \u003ca href=\"https://links.kronis.dev/XJE5yy2Ux2\"\u003eCollabora Office\u003c/a\u003e in the cloud and use \u003ca href=\"https://links.kronis.dev/e8gwc\"\u003eLibreOffice\u003c/a\u003e locally. You can self host all of the above when you need to, when you want to - the ability to do so is there.\u003c/p\u003e\n\u003cp\u003eThere's also a lot of software out there, that you can get and run completely for free, that will be similar enough to the upstream offerings not to matter much - the likes of \u003ca href=\"https://links.kronis.dev/ZkJMOcEC4V\"\u003eRocky Linux\u003c/a\u003e if you need something RHEL-compatible, or maybe you can just choose to use Ubuntu LTS or Debian, which have never had that much of a commercial component to them, outside of additional management tools or getting longer EOL support, which you can do if you need to! Most of the above can also be easily set up and run as a Docker container, and the same applies to most types of software out there - from source code management platforms like \u003ca href=\"https://links.kronis.dev/gjvgz\"\u003eGitea\u003c/a\u003e or \u003ca href=\"https://links.kronis.dev/KDRqRNtsLd\"\u003eGitLab\u003c/a\u003e, CI/CD solutions like \u003ca href=\"https://links.kronis.dev/0jpg2\"\u003eWoodpecker CI\u003c/a\u003e, the excellent \u003ca href=\"https://links.kronis.dev/0wRyhDchTW\"\u003eGitLab CI\u003c/a\u003e, or heck, even \u003ca href=\"https://links.kronis.dev/iefvy\"\u003eJenkins\u003c/a\u003e if you want to go in that direction, to issue tracking software like \u003ca href=\"https://links.kronis.dev/p6r2z\"\u003eOpenProject\u003c/a\u003e or \u003ca href=\"https://links.kronis.dev/pb8c7\"\u003eKanboard\u003c/a\u003e, or APM tools like \u003ca href=\"https://links.kronis.dev/PCuBxjnaMK\"\u003eApache Skywalking\u003c/a\u003e, or analytics solutions like \u003ca href=\"https://links.kronis.dev/vQEEO2j8UW\"\u003eMatomo Analytics\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003eThe same also extends to those same building blocks: you can get good relational databases, like \u003ca href=\"https://links.kronis.dev/9AX1bqJQd9\"\u003ePostgreSQL\u003c/a\u003e and \u003ca href=\"https://links.kronis.dev/nqu3w\"\u003eMariaDB\u003c/a\u003e, use \u003ca href=\"https://links.kronis.dev/Yh3p6YBZTP\"\u003eSQLite\u003c/a\u003e for more local or simple use cases, can get S3 storage in the form of \u003ca href=\"https://links.kronis.dev/np6IrwR22V\"\u003eGarage\u003c/a\u003e and key-value store in the form of \u003ca href=\"https://links.kronis.dev/4lj8i\"\u003eValkey\u003c/a\u003e, message queues in the form of \u003ca href=\"https://links.kronis.dev/hu2st\"\u003eRabbitMQ\u003c/a\u003e or \u003ca href=\"https://links.kronis.dev/mXTQLLZb1D\"\u003eNATS\u003c/a\u003e and there's a slew of open source solutions for both full-stack work, front end and back end: you could build a \u003ca href=\"https://links.kronis.dev/8jp2e\"\u003eDjango\u003c/a\u003e app that has \u003ca href=\"https://links.kronis.dev/ThJQzfNWcL\"\u003eVue\u003c/a\u003e on the front end with \u003ca href=\"https://links.kronis.dev/3a9o5\"\u003ePrimeVue\u003c/a\u003e today, if you wanted to, and get it running in the cloud behind \u003ca href=\"https://links.kronis.dev/uGZeGTKaQg\"\u003eApache2\u003c/a\u003e with \u003ca href=\"https://links.kronis.dev/Hr2ldIHvA9\"\u003eLet's Encrypt\u003c/a\u003e certs (\u003ca href=\"https://links.kronis.dev/73rNpALZwb\"\u003emod_md\u003c/a\u003e; some might prefer Caddy) without spending money on any of the tech. The only thing keeping you from doing that is making the choice not to, and let me tell you - often the engineers are not the ones making the choice to go proprietary.\u003c/p\u003e\n\u003cp\u003e(Note: okay, Let's Encrypt being under ISRG is a bit of a risk, EU probably should provide a similar service, though if the global CA trust falls apart, then I guess the Internet would also be severely fragmented and hopefully nobody wants that, so maybe that's a bit paranoid to say; either way, I moved my domains at least from \u003ca href=\"https://links.kronis.dev/FUbPmnrsP1\"\u003eNameCheap\u003c/a\u003e to \u003ca href=\"https://links.kronis.dev/4viRrBLtsS\"\u003eINWX\u003c/a\u003e to support more local business)\u003c/p\u003e\n\u003cp\u003eI know so, because I have done so in the past - and when issues arose, I just resolved them, because I'm an engineer and that's what I do. I don't claim to be some genius or an expert in any one particular area, so that if I can do that, then hopefully the many people working in the public sector can, as well. And in regards to the software, it doesn't even matter that much, who originally developed it - because if they try to rugpull you at any point in time, you can just say \u0026quot;No, thank you, I will make a fork of your project and set off in my own direction.\u0026quot; That happened with Redis and Valkey, as it did with many other solutions - and standards like S3 only help this, because when MinIO starts going in a more commercial direction, which might not be a problem for a government project but is for me, I can just thank them for their effort and go with Garage instead.\u003c/p\u003e\n\u003cp\u003eMy argument isn't even that you can't buy software, or use foreign tech as utilities when it makes sense (e.g. US based AI inference when you really need it, while not forgetting that \u003ca href=\"https://links.kronis.dev/E2LfN25Hwe\"\u003eMistral\u003c/a\u003e exists and is actually pretty dang good) - I've gotten \u003ca href=\"https://links.kronis.dev/xoe6Wedn6G\"\u003eiframe-resizer\u003c/a\u003e licenses for a project at work once, when it made sense and we needed to save time, it worked okay. My argument is that I should never be (figuratively) held at a gunpoint and made to fork over cash for \u003ca href=\"https://links.kronis.dev/L7RKMMyzg6\"\u003eKendo UI\u003c/a\u003e when my project doesn't really call for it.\u003c/p\u003e\n\u003cp\u003eHiding behind support and the status quo is nothing other than cowardice. Saying that you can't use something open source because your procurement department can't \u0026quot;buy\u0026quot; it is plain stupid. There, I said it. I'm not saying that about the people themselves, but rather their choices, mind you.\u003c/p\u003e\n\u003cp\u003eFor most of the apps there, if you can get Oracle working as your DB, it will most likely work fine on MariaDB or PostgreSQL - with the added benefit that you can do whatever you want, with no restrictions. Need to set up a new development environment? Run a copy of the whole database locally? Build and run some containers across a wide variety of hardware configurations? If your system runs on the aforementioned open source DBs, licensing doesn't even enter the equation, whereas going with any of the proprietary solutions in some cases will tie your hands.\u003c/p\u003e\n\u003cp\u003eI can already hear people saying:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eBut open source solution X doesn't have feature Y.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eI hate to be the obnoxious person telling you this, because it's unhelpful at a glance - but you don't need it and you can work around it. There is no single feature that's so foundational that your entire success depends on it and it alone, at least in a way where it doesn't exist in another software package in some shape or form. If it truly doesn't, then shout about it from the rooftops, or better yet, write it yourself.\u003c/p\u003e\n\u003cp\u003eYes, switching would be a pain in the ass. Yes, most people might find the UI of LibreOffice a bit dated (honestly their customization is lovely and I like that look over the MS Ribbon, very, very much), and that the experience around pivot tables and some of the formulas will be a bit awkward. Heck, if you switched over to a Linux distro like \u003ca href=\"https://links.kronis.dev/5CctaA1rVg\"\u003eLinux Mint\u003c/a\u003e, the overall user experience would be pretty solid, even if there'd be absolutely nothing you could do to avoid some level of switching pains, for the people that are used to Windows. They'd still get over it.\u003c/p\u003e\n\u003cp\u003eBecause otherwise, you are hostages. If you have no alternatives, Microsoft can charge you whatever they want. If they're a business, treat them as such - have enough leverage to negotiate better prices for their software, if you ever decide that you truly need it, \u003ca href=\"https://links.kronis.dev/y7pM92eYOq\"\u003elike the Germans did\u003c/a\u003e. You need to explore the other options, lest you end up with none.\u003c/p\u003e\n\u003cp\u003eIt's not even a matter of complacency, or realpolitik, but also of cost.\u003c/p\u003e\n\u003ch3\u003eWhy B-tier Europe can't afford anything else\u003c/h3\u003e\n\u003cp\u003eI will be the first to admit that Latvia and many other EU countries like it are pretty poor. The tone in places above is a bit inflammatory, because going for expensive proprietary solutions is like burning money that we don't really have. Countries like Germany and France, or smaller ones that are still wealthy like Denmark and Netherlands can afford to do that, but we are just not like them. If they're A-tier Europe, then we are B-tier Europe: I'm still thankful that we're a part of EU, but our day to day reality is quite different when it comes to economic matters.\u003c/p\u003e\n\u003cp\u003eIt's a bit like comparing how much a software dev might earn in the US vs EU. If you are curious, a developer over here in Latvia makes on average around 23 EUR an hour, or about 3700 EUR per month, before taxes, \u003ca href=\"https://links.kronis.dev/J0LorF8w6j\"\u003eaccording to government data\u003c/a\u003e:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02-software-developer-salaries.jpg\" alt=\"02-software-developer-salaries\" title=\"02-software-developer-salaries\"\u003e\u003c/p\u003e\n\u003cp\u003eThe data is also corroborated by some other sources as well, like the site \u003ca href=\"https://links.kronis.dev/hMCfOGidab\"\u003ealgas.lv\u003c/a\u003e where people report their own salaries:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03-software-developer-salaries-algas.jpg\" alt=\"03-software-developer-salaries-algas\" title=\"03-software-developer-salaries-algas\"\u003e\u003c/p\u003e\n\u003cp\u003eThat site is actually more telling and suggests that what most of the people make, the values closer to the median rather than the skewed averages, fall in line somewhere between ~1300 and ~3700 EUR, after taxes. This actually gives us a more realistic look into how much people make here. Per year, that is between 15k and 44k EUR for the majority of people here.\u003c/p\u003e\n\u003cp\u003eWhy focus on that? Because it's a pretty comparable example for the people that might read this. At the same time, our public sector is underfunded - teachers don't receive nearly enough money, for how important their work is. Medical nurses don't either, nor do policemen or firefighters. It is tough for everyone out there. Pensioners struggle to afford their necessities and medicine.\u003c/p\u003e\n\u003cp\u003eThen why do we act as if we can waste millions on consulting companies and expensive software and platforms that we don't have the money for? Yes, there are a lot of consulting companies here that pay the local developers and I'm glad for that - it's bringing at least some money into the economy where we otherwise don't have a strong tech sector. It's at least something and lets our developers work on a wide variety of projects - but it's something that the \u0026quot;rich\u0026quot; part of Europe should purchase.\u003c/p\u003e\n\u003cp\u003eWe shouldn't act like we can afford Microsoft licenses for everything. We shouldn't act like we can afford to run things on AWS or pay for Cloudflare. We shouldn't act like we can afford Oracle or DB2 licenses and treat software development projects like a black hole that we just throw money into and hope for the best.\u003c/p\u003e\n\u003cp\u003eWe should invest as much as we can into our people and developers and only reach for that software when the engineers in charge decide that it's what we truly need. If a project crashes and burns, then so be it - don't try to cover your ass by shifting the blame, because at the end of the day it doesn't matter whether you have support by a vendor or don't, the solution either works or doesn't and that is that.\u003c/p\u003e\n\u003cp\u003eStop building your platforms in others' walled gardens and start using open source properly. And when possible, contribute back to it. The same goes for the rich part of Europe - because currently, if you pay a vendor a bunch of money for some proprietary software, they're gonna take that money, spend it on whatever they do and make their closed software better. That's it, nobody else other than them will benefit from it. And yet, if you sponsored the development of PostgreSQL and improved the query performance by 1%, you are improving that for everyone using PostgreSQL, all across the world, in perpetuity.\u003c/p\u003e\n\u003cp\u003eMost of the software out there is developed \u003ca href=\"https://staltz.com/software-below-the-poverty-line.html\"\u003ebelow the poverty line\u003c/a\u003e:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04-software-below-poverty-line.jpg\" alt=\"04-software-below-poverty-line\" title=\"04-software-below-poverty-line\"\u003e\u003c/p\u003e\n\u003cp\u003e(image sourced from the blog post, there's a higher quality version there)\u003c/p\u003e\n\u003cp\u003eThe person maintaining \u003ccode\u003elog4j\u003c/code\u003e had \u003ca href=\"https://links.kronis.dev/f8FQKx7NTq\"\u003ethree GitHub sponsors\u003c/a\u003e:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"05-log4j-github-sponsors.jpg\" alt=\"05-log4j-github-sponsors\" title=\"05-log4j-github-sponsors\"\u003e\u003c/p\u003e\n\u003cp\u003eI don't know about you, but I don't think Michael, Glenn and Matt should support the development of a large part of the logging infrastructure for Java by themselves, as nice as it was from them.\u003c/p\u003e\n\u003cp\u003eThe person maintaining \u003ccode\u003esudo\u003c/code\u003e \u003ca href=\"https://links.kronis.dev/DjQ6vd7EUu\"\u003erecently said they could really use some money\u003c/a\u003e:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"06-sudo-maintainer-needs-some-money.jpg\" alt=\"06-sudo-maintainer-needs-some-money\" title=\"06-sudo-maintainer-needs-some-money\"\u003e\u003c/p\u003e\n\u003cp\u003eAgain, a utility that's used in almost every server running some variety of GNU/Linux out there, and the person is struggling financially and the project needs support. Remind me, just how much money do the corporations that provide Linux cloud services earn?\u003c/p\u003e\n\u003cp\u003eThe longer I live, the more it feels like \u003ca href=\"https://links.kronis.dev/5rrcy\"\u003ethis XKCD\u003c/a\u003e is completely correct and representative of our reality:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"07-state-of-open-source-xkcd.jpg\" alt=\"07-state-of-open-source-xkcd\" title=\"07-state-of-open-source-xkcd\"\u003e\u003c/p\u003e\n\u003cp\u003eWhile even a single person that's tirelessly working on the software you all depend on is struggling financially, I cannot ever say that you should reach for paying a bunch of corpos that will just take the money. These people are working on the backbone of the technology that runs a good part of the world and... they still have to worry about making ends meet? What the fuck is wrong with the world?\u003c/p\u003e\n\u003cp\u003eMost of the commits to the Linux kernel are corporate-backed, that much is true. So live in that reality: when considering buying software from one of those large orgs, ask them if they're going to make the source available to you together with the purchase, for you to use in whatever capacity you desire. If they say no, you say no.\u003c/p\u003e\n\u003cp\u003eThere's no reason why you shouldn't use something like Visual Studio Code (maybe ideological reasons, but I'm more concerned about realpolitik here), but there is relatively little friction in regards to switching to something else, when compared with a foundational relational DB that is at the core of your system and has 5-10 years of work running to it right now. The dependencies that cause risks include the entirety of your tech stack, all the way to the OSes and hardware you use.\u003c/p\u003e\n\u003cp\u003eYou don't have to go all-in on the Free Software movement, just please take the first steps towards a saner world:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eInvest in talent when you can, instead of trying to replace it with a pre-packaged software solution.\u003c/li\u003e\n\u003cli\u003eDon't give away control over what you'll build just because of rosy promises.\u003c/li\u003e\n\u003cli\u003eUnless you have a set of very specific requirements, go for open source solutions first.\u003c/li\u003e\n\u003cli\u003eTake ownership over the implementation and contribute back when you can.\u003c/li\u003e\n\u003cli\u003eSeriously, stop burning my tax money, please. I'd much rather it go to devs than corpos.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThere are second order effects to this, as well. Right now, there may or may not be a project somewhere that is about migrating from one proprietary form solution to another and it's behind, largely because they just didn't go with an open source solution and didn't build some of the primitives themselves. They gave up control in exchange for promises and the promises won't get them far. They instead have an over-complicated mess of XML and two packages that are hard to work with and no source for either. It will probably be late.\u003c/p\u003e\n\u003cp\u003eIt's probably easy for me to speak, because when your country has no more than 2 million people in it total, you could even get away without having huge DB clusters in many cases, especially for the systems that aren't used by too many of those and are primarily there for the purposes of basic digitization - in which case, you should prefer monoliths to complex microservices and it can probably run on a single server on-prem (hopefully with regular and tested off-site backups).\u003c/p\u003e\n\u003cp\u003eWhatever the case, I hope Europe wakes up. We need a cultural shift, to have proper digital sovereignty and create jobs to build our digital future. This doesn't mean turning away from the rest of the world in the slightest, but to not turn away from projects that are developed in the open and benefit everyone across the world.\u003c/p\u003e\n"
        },
        {
            "title": "My dad passed away",
            "date_published": "2026-02-01",
            "id": "https://blog.kronis.dev/blog/my-dad-passed-away",
            "url": "https://blog.kronis.dev/blog/my-dad-passed-away",
            "content_html": "\u003cp\u003eNear the end of the last year, my father, Pēteris Kronis, passed away. It has been months since and I was thinking about whether I should write about it or not, but in the end decided to put this out there anyway. It was a sad, humbling and human experience. I wanted to write something, to both share in that experience and to also remember him, in writing.\u003c/p\u003e\n\u003ch3\u003eWhat happened\u003c/h3\u003e\n\u003cp\u003eI live in his old city apartment and look after it, while I work in the city as a software developer, whereas he and mom lived together in the countryside home. It just so happened that this was one of the few times when mom had been out of the house in recent memory - visiting another lady, a family friend, in Italy for a week or two, to see the sights and nature. I had joked that this was a sort of \u0026quot;vacation\u0026quot; for her, because otherwise a lot of her time was spent looking after the house, keeping it more clean than both me and dad would otherwise, cooking and looking after the dogs. Some time after her leaving, dad sounded more sickly on the phone.\u003c/p\u003e\n\u003cp\u003eI was planning to visit him at the end of the work week, for the weekend, as I often did with my parents, being on the phone with both him and mom in the days leading up to it - how he was feeling, what medicine to better bring to him and so on. Me and mom had both told him that if it gets bad, then he should drive to the nearby city (a 15 minute drive, approximately) and see the doctor there, yet he had pretty consistently responded with: \u0026quot;I'll wait and see.\u0026quot; It wasn't completely out of the ordinary, because we all had previously had things like a common cold, that did not seem to actually require much medical attention, and he wouldn't really say that it was much different this time.\u003c/p\u003e\n\u003cp\u003eI had been on the phone with him just that Friday, but by Saturday, the day when I was actually getting into the bus and going to visit him, he would no longer pick up his phone. A part of me felt like this shouldn't be that big of a deal, because mom would sometimes also not pick up when busy with something or outside, yet a part of me couldn't shake the feeling that it was a sort of bad omen. By the time I arrived, he had already passed. I found him in his bed.\u003c/p\u003e\n\u003cp\u003eI called mom about what to do. I called the emergency number after, they sent an ambulance, they confirmed that he had passed away, I guess \u0026quot;dead with no signs of violence\u0026quot; is the term that gets put on the document in such cases. One of them told me that his chest was blue in color, and that it was most likely something to do with his lungs - where a person tries to take a breath but doesn't get enough oxygen, and that even if he would have been rushed to hospital while still alive (or gone himself), then it would have been hard on him. The people from the crematorium were also nice and took away his remains - after he and mom had previously discussed that he doesn't want a regular burial.\u003c/p\u003e\n\u003cp\u003eIn the time leading up to that, I let the dogs into his bedroom, so they'd also understand what had happened. The person from the ambulance told me that I probably shouldn't stay alone in the house and should visit some friends or relatives for a day or two. After his remains were taken away, I brought in some firewood and got the stove going. I cleaned up the place a bit, so that by the time mom would get home it would be a bit more presentable - her flight back was scheduled in just a few days, and dad had not washed his dishes. So I did all of those. I changed the water for the dogs and put out enough food so that they'd be okay until mom would get back, since I'd be heading back to the city the next day.\u003c/p\u003e\n\u003cp\u003eIt felt quite a lot like going on autopilot. I did sit down on the floor in my old room, next to the wood stove and together with dogs, and spent some time crying, just letting it all out. I felt what I had to feel in that moment, no reason to keep up appearances. It was odd - my dad was over 70 years old, I knew that something like that was bound to happen and logically I knew that it was a part of life. There wasn't much of a \u0026quot;bargaining\u0026quot; or \u0026quot;denial\u0026quot; stage of grief to be had. That didn't make any of it hurt less. It wasn't horrible in the traditional sense, there was no sense of doom or horror, but a profound emptiness and loss, a sorrow.\u003c/p\u003e\n\u003cp\u003eFor much of the weeks following, that sense of being on autopilot persisted - figuring out all of the documents, taking over paying all of the bills, like electricity and water, gas in the city apartment, phone and TV bills for mom, moving providers to make the money go a bit further in the time of uncertainty. Notifying the relatives, eventually also starting proceedings in regards to the will - even though the people who provided electricity and other utilities were kind enough to let me take over paying them without technically having the ownership rights over anything yet. Everyone was nice along the way. I was reminded of the fact that the society that we live in has the ability to lend you support when you need it - systems in place that nobody would normally think much about.\u003c/p\u003e\n\u003cp\u003eThen, came the relatives - his brothers, and then also my sister and brother from his previous marriage. We got in touch, we talked a bit, as you do in cases like this. They offered to help, but thankfully the immediate concerns were already addressed. In a sense, I felt bad about never really getting to know them that well, though they are all grown people with their lives all around the country and in some cases, other countries. The family friend even offered to come live with mom a bit, at least until she'd have the fridge stocked and also the house cleaned out of various things dad had accumulated, not really collected, over the years. Old clothes to give to charity, bunches of old chargers, a silly amount of dead batteries, a bunch of documents and whatnot, that kind of stuff.\u003c/p\u003e\n\u003ch3\u003eMy dad and our life\u003c/h3\u003e\n\u003cp\u003eIt has now been months since and I feel like I've come more to terms with what happened, but it still hurts when I remember it. The other day, I saw something on the news and thought that I should tell dad about this, yet when my hand reached for the phone, I remembered. It will probably hurt for a long time, and that's okay.\u003c/p\u003e\n\u003cp\u003eI am thankful that I don't have many regrets. Sometimes when a person passes away, those around them say they wished they had been on better terms. Me and dad loved and respected each other, that much I'm happy about. Sure, over the years we had some generational disagreements - he said I should socialize more, and look for a partner, while I was just buried in work to do and things to learn. That same family friend also shared the sentiment, saying that if she were my age, she'd probably be backpacking through Europe or something, not working as much as I do. He also talked a lot about having to do more in the way of sports and exercise. It's not even that I think any of them are wrong, just that my circumstances are a bit different and hopefully I'll have time for everything later.\u003c/p\u003e\n\u003cp\u003eThough, on some level, I am sad that I didn't know more about my dad. We did talk plenty on the phone when I was living in the city, I visited around every other weekend, but a lot of it was talking about surface level topics like world politics and economy, whereas the visits would often involve a lot of various chores and fixing broken stuff - as life in the countryside demands.\u003c/p\u003e\n\u003cp\u003eObviously, he had told me about his youth, some of the places he'd visited and things he'd done, admittedly he had lived a life way more interesting than mine has been up to this point, as many of the people of his generation did - he had even participated in the barricades of 1991 shortly before which our country became independent from Soviet rule, yet worried about OMON and Soviets trying to take it back. Back in the day, he even had his company and had travelled both through the Soviet Union, as well as abroad. He once told me that he just narrowly decided on coming back to live in Latvia, instead of moving to Australia permanently. While he had seen his fair share of times being tough in his life, he was a patriot regardless. In this country, he met my mom and built his house. He lived a long and good life, and passed away in the house that he had built. Yet it's not like I have a year by year retelling of his life, since he kept a lot of it to himself. When the relatives were recalling their own past and interactions with him, I couldn't help but to feel that his own perspective was missing, almost like if it's wrong to talk about a person's life without their own words.\u003c/p\u003e\n\u003cp\u003eOn some level, I do wish I had talked more to him in general, but these past years had been kind of rough. I'm pretty sure that he might have gone through COVID or something similar - though after the crisis situation in the country due to it was resolved, the variants of the illness that came afterwards were less deadly and he just spent a week in the countryside, so I'm not sure if he did a formal test. His energy levels seemed lower, he didn't have the initiative that he once did. Maybe he was just getting old.\u003c/p\u003e\n\u003cp\u003eWhen I was younger, I remember him telling me that you have to do something every single day. He pretty much lived by that mantra, often times (figuratively) dragging me outside to work on something around the house, help him with the car, or the tractor, or put up a diving board for the pond, or fix water pumps, mow grass, chop firewood or any number of other things. I might not have learnt as much as he knew, but looking back at it, I'm thankful I had those experiences - in a way, it was bonding.\u003c/p\u003e\n\u003cp\u003eEven collecting these plants that grow on the sides of forest roads (chamaenerion angustifolium) to dry out and make tea out of, something that my mom rolled her eyes at because you can buy that sort of tea in the store, I'm happy that I went with him. The Latvian practice of parenie and making those brooms out of branches of Birch trees, tapping the trees for \u0026quot;Birch juice\u0026quot;, some of it felt a bit silly, but I'm glad I got to participate in that part of his view on what a good life should be.\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-chamaenerion-angustifolium.jpg\" alt=\"01-chamaenerion-angustifolium\" title=\"01-chamaenerion-angustifolium\"\u003e\u003c/p\u003e\n\u003cp\u003e(not my own picture, but these are what you make the tea out of; though oftentimes in Latvia the sides of the roads are way more overgrown)\u003c/p\u003e\n\u003cp\u003eAnd yet, as I said, the past few years weren't as kind to him. I talked to his family doctor a bit more while handling the documents in the weeks following, and she told me that he had been experiencing some respiratory issues previously, apparently something that he might have had medicine for, but didn't really tell the details of to me or mom much. More recently, he had gotten a flu vaccine which can be somewhat straining on the immune system and it's possible that during the recovery from that he caught something else that was too much to deal with - yet, at his age catching flu might very well have had a similar outcome. I think it hardly matters anymore.\u003c/p\u003e\n\u003cp\u003eI think of my own health and how I probably should see a doctor for a thing or two, and just how easy it is to put off until later and be busy with the everyday passage of time - and I can see too much of him in me. I think I had pleurisy for a bit a few years ago and also didn't get an X-Ray for it in the end, because on the day when I went the machine was broken and after waiting for a few hours, I just went with the parents back to the countryside, since they had been shopping for groceries and were waiting for me in the car. It's easy to brush it aside and say how I'm \u0026quot;fine\u0026quot; now myself, yet what if there'd be something that might actually need attention? Far too many people, for some reason men, especially, don't seek out enough medical help when they really should. One might get away with it when they are younger, yet we shouldn't bet on that.\u003c/p\u003e\n\u003cp\u003eEither way, in the last few years, he no longer was waking me up, telling me that we have to go outside to work on things. I could tell that he was struggling and doing things in the countryside was becoming harder. So I'd come visit and would be mowing the grass or chopping some of the firewood (though we'd also have hired help, thankfully), and he'd be on the couch, scrolling TikTok on his phone, or watching stuff on the TV indoors.\u003c/p\u003e\n\u003cp\u003eSometimes he'd talk about things he wants to do, additional structures to build, like we had built an elevated water container next to the greenhouse, that'd heat up in the sun and would give you warm water to water the plants with, but it was clear that it most likely wouldn't happen - I'm not sure he also understood that, or just preferred not to think about it. Now that I think back on it, I wish we could have talked more, but it also hurts, knowing that he wasn't the person that he once was and that the final years were unkind to him.\u003c/p\u003e\n\u003cp\u003eEither way, he lived a good life and I'm thankful for the years we had together, how he raised me - he did his best and was a good father. I don't think that he was too religious, but believed in some sort of a higher power, be it the Christian god or something like it. If there is a heaven, I'm sure that he is there. If not and that all that expects us after passing is nothingness, at least he's getting some well deserved rest now. He did leave me and mom with a bit of a mess afterwards, not just the dishes, but the house to clean and documents to sort out, yet we don't blame him one bit. If anything, that's a bit like him, well intentioned but a bit messy - you'll have to forgive me for the comedic relief, because I am once again tearing up.\u003c/p\u003e\n\u003ch3\u003eLife goes onwards\u003c/h3\u003e\n\u003cp\u003eIt still hurts. It's not that I can't accept what's happened, or don't know how life will go onwards now. It just will, one day at a time. I caught laryngitis a while later and for a bit my colleagues at work had to deal with me sounding like a mobster out of an 80s movie, and being quite terse with them - not because I was upset with them or anything, but rather because speaking hurt. I'm a bit better now, life goes on, there are projects to work with, more documents to sort out. I have plans of replacing the gas water heater in the city apartment with a more modern one, and rewiring the old aluminium wires with more safe copper ones for my PC, as well as the kitchen oven. Painting the apartment's windows from the outside to prevent water damage, since the kitchen still has wooden windows. Visiting mom when I can and helping around with everything, bringing more fresh groceries and the magazines you can't subscribe for and receive in the mailbox, and so on. Keeping in touch and making good choices for the future, one day at a time.\u003c/p\u003e\n\u003cp\u003eIt's the normalcy of it all that's almost painful. That something like that can happen, a person's life just end. Not even with an accident or something else that would commonly be viewed as a \u0026quot;tragedy\u0026quot;, but peacefully. And yet, I will never talk to my dad again. I will never be able to ask him for advice or support, or vice versa. He is just gone, and will be gone forever. And most people out there will have similar experiences sooner or later, and will have to accept it, and pick things back up and keep going through life. On some level, it feels like I had to \u0026quot;grow up\u0026quot; very fast, despite having my own job, finished education and all that, this feels different, like a different chapter in my life.\u003c/p\u003e\n\u003cp\u003eI am thankful that up until now I've led a somewhat Spartan lifestyle - not splurging on things I don't need and having savings for a rainy day, if something were to happen. So at least I can and have taken over all of the bills and will be able to help mom as much as necessary, to take care of her, so that whatever life may look like from here on out, it's going to be okay. Dad and mom each loved and respected each other for decades - both of them good people, that I care for a lot. She has been taking this as well as anyone could, given the circumstance, even if she'll similarly need some time to process everything, as we've talked about, both in person and on the phone since. We will get through this and I will take care of her, that's what dad would have wanted.\u003c/p\u003e\n\u003cp\u003eBut when all is said and done, I will miss you, dad. Rest well.\u003c/p\u003e\n"
        },
        {
            "title": "Sometimes Dropbox is just FTP: building a link shortener",
            "date_published": "2026-01-21",
            "id": "https://blog.kronis.dev/blog/sometimes-dropbox-is-just-ftp-building-a-link-shortener",
            "url": "https://blog.kronis.dev/blog/sometimes-dropbox-is-just-ftp-building-a-link-shortener",
            "content_html": "\u003cp\u003eThere is this \u003ca href=\"https://links.kronis.dev/JOx5cgqmsa\"\u003eone old HackerNews post\u003c/a\u003e that sometimes people reference, when talking about how engineers view software:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-dropbox-is-just-ftp.jpg\" alt=\"01-dropbox-is-just-ftp.jpg\" title=\"01-dropbox-is-just-ftp\"\u003e\u003c/p\u003e\n\u003cp\u003eIt's an observation of how the trend goes:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003ea new product or service comes out\u003c/li\u003e\n\u003cli\u003ean engineer looks at it and says \u0026quot;Anyone could build this for themselves\u0026quot;\u003c/li\u003e\n\u003cli\u003eit proceeds to do really well and becomes popular regardless\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThere's even \u003ca href=\"https://links.kronis.dev/OIvXqZe2gZ\"\u003ean XKCD about this\u003c/a\u003e and you can \u003ca href=\"https://links.kronis.dev/onYsklvfEB\"\u003elook at the Dropbox revenue\u003c/a\u003e for yourselves:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02-dropbox-revenue.jpg\" alt=\"02-dropbox-revenue.jpg\" title=\"02-dropbox-revenue.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eSometimes these products or services more or less outlive the relevance of the software that already existed and could be used to build something like that at the time. Why? Well, in part, I don't doubt that engineers underestimate the staying power of something that is pleasant to use and also solves a problem that the user has, even if it's nothing groundbreaking on a technical level.\u003c/p\u003e\n\u003cp\u003eSometimes it's not even completely new ideas or software, either. For example, look at \u003ca href=\"https://links.kronis.dev/UN6cBUPLlj\"\u003eLinear\u003c/a\u003e, it is just a slightly better \u003ca href=\"https://links.kronis.dev/AXOW19lB80\"\u003eJira\u003c/a\u003e that already existed before, which in turn is maybe a bit better than \u003ca href=\"https://links.kronis.dev/nXIs5AONF4\"\u003eRedmine\u003c/a\u003e. Similarly, \u003ca href=\"https://links.kronis.dev/yOKZJ9gOkN\"\u003eObsidian\u003c/a\u003e is just Notepad++ with Dropbox, which could just be Notepad++ with SFTP.\u003c/p\u003e\n\u003cp\u003eAnd yet, all of those are useful and are doing pretty well! Except in some cases, the engineer is right and Dropbox is, indeed, just SFTP.\u003c/p\u003e\n\u003ch3\u003eMy previous setup\u003c/h3\u003e\n\u003cp\u003eLet's talk about link shorteners. I use them on this very blog, as you can see above - in part due to the CMS that's running this blog sometimes having weird behavior around complex URLs, other times because a URL gets super long and I want to include something nice instead, e.g. a URL to a map to tell a delivery driver \u003cem\u003eexactly\u003c/em\u003e how to get to my place when delivering something, or maybe some instructions for an event, or how to access a particular document.\u003c/p\u003e\n\u003cp\u003eThere are some out there that you can just use, however they might not be around forever, might inject ads and other stuff before the redirect (they have to earn money somehow as well), in addition to there always being the option of self-hosting something yourself. For a while, that's exactly what I did. There is a pretty cool open source project out there, that is called \u003ca href=\"https://links.kronis.dev/a8e8h\"\u003eYOURLS\u003c/a\u003e. It has served me well for a while and was pretty much perfect for not making me rely on external services.\u003c/p\u003e\n\u003cp\u003eHere's what it looks like, in case you're curious:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03-yourls-main-ui.jpg\" alt=\"03-yourls-main-ui.jpg\" title=\"03-yourls-main-ui\"\u003e\u003c/p\u003e\n\u003cp\u003eIt even supports a pretty nice statistics view, in case you're curious about where your traffic is coming from and all sorts of other stuff:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04-yourls-statistics.jpg\" alt=\"04-yourls-statistics.jpg\" title=\"04-yourls-statistics\"\u003e\u003c/p\u003e\n\u003cp\u003eAt the same time, as an engineer, I view it as a liability. I trust that the developers are doing their best, but what if there's a vulnerability, maybe not even in their code, but in one of the dependencies? What about keeping the database that it needs up to date? What if one day it decides to break, like I've had happen plenty of times with PostgreSQL WAL getting corrupted and needing manual intervention? Even if nothing breaks there, sooner or later the updates will need newer PHP versions, speaking of which, what if there's a PHP or configuration related issue?\u003c/p\u003e\n\u003cp\u003eObviously, I don't actually store anything important in this instance, but anything that might help me focus less on running and administering software (which seems to be an ever increasing amount), leaves me free to spend more of my time and attention on more important things.\u003c/p\u003e\n\u003cp\u003eSo, let's build our own link shortener.\u003c/p\u003e\n\u003ch3\u003eBuilding just what I need\u003c/h3\u003e\n\u003cp\u003eAt its core, a link shortener just receives a request and if it matches something that should be redirected, does so with the \u003ca href=\"https://links.kronis.dev/3Vc6cRVl4B\"\u003eLocation header\u003c/a\u003e and either a \u003ccode\u003e301\u003c/code\u003e (permanent redirect) or \u003ccode\u003e302\u003c/code\u003e (temporary redirect) status code, depending on whether the link is meant to be unchanging, which it will be most of the time, or something that could change often:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"05-how-a-link-shortener-works.jpg\" alt=\"05-how-a-link-shortener-works.jpg\" title=\"05-how-a-link-shortener-works\"\u003e\u003c/p\u003e\n\u003cp\u003eI quite quickly realized that I don't actually care much about the analytics or statistics, similarly to how I mostly blog to just put my voice out there, but don't care much about back-and-forth or the sometimes inflammatory comments I might get with a public comments section (it's kind of unfortunate, but anyone who's seen YouTube comments will get it, maybe outside of more moderated communities like HackerNews, those have different kinds of biases despite generally being more pleasant).\u003c/p\u003e\n\u003cp\u003eSometimes people e-mail me with kind messages and I appreciate those, funnily enough I've even gotten a few job offers thanks to my public presence, but in general I'm saying that I don't really care that much about analytics and number crunching (aside from maybe setting up analytics to have a proof of concept to apply at work later).\u003c/p\u003e\n\u003cp\u003eEither way, it became quite obvious that, much like how Dropbox is just SFTP, a link shortener could just be a web server and some shell scripts for generating the links, thanks to \u003ccode\u003e/dev/urandom\u003c/code\u003e.\u003c/p\u003e\n\u003cp\u003eTo prove that, here's my link shortener:\u003c/p\u003e\n\u003cpre\u003e\u003ccode class=\"language-bash\"\u003e#!/bin/bash\nCONFIG_FILE=\u0026quot;apache/etc/apache2/sites-enabled/000-default.conf\u0026quot;\nCODE_LENGTH=10\ngenerate_shortcode() {\n    local chars='a-zA-Z0-9'\n    cat /dev/urandom | tr -dc \u0026quot;$chars\u0026quot; | head -c \u0026quot;$CODE_LENGTH\u0026quot;\n}\nis_shortcode_in_use() {\n    local shortcode=\u0026quot;$1\u0026quot;\n    grep -q \u0026quot;\\\u0026quot;/${shortcode}\\\u0026quot;\u0026quot; \u0026quot;$CONFIG_FILE\u0026quot; 2\u0026gt;/dev/null\n    return $?\n}\nshortcode=\u0026quot;\u0026quot;\nwhile [ -z \u0026quot;$shortcode\u0026quot; ]; do\n    candidate=$(generate_shortcode)\n    if ! is_shortcode_in_use \u0026quot;$candidate\u0026quot;; then\n        shortcode=\u0026quot;$candidate\u0026quot;\n    fi\ndone\necho \u0026quot;$shortcode\u0026quot;\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eI will take \u003ca href=\"https://links.kronis.dev/tKjF8CPhU5\"\u003e100 million dollars per year\u003c/a\u003e, please.\u003c/p\u003e\n\u003cp\u003eObviously, I say that in jest, but the above, with a few additional scripts and Apache2 running, alongside some CI/CD so it'd be redeployed once I push some new changes to the Git repo, covers most of my needs.\u003c/p\u003e\n\u003cp\u003eI actually got around to doing this because I reinstalled \u003ca href=\"https://links.kronis.dev/hqsqrBxgj2\"\u003eOpenCode\u003c/a\u003e and could bootstrap the whole project based on my pre-existing configuration examples and templates really quickly, making that friction of starting something new near-0:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"06-opencode-bootstrap.jpg\" alt=\"06-opencode-bootstrap.jpg\" title=\"06-opencode-bootstrap\"\u003e\u003c/p\u003e\n\u003cp\u003eIn the future, that will probably let me migrate from Drone CI over to Woodpecker CI, which I've actually been meaning to do for a while now, but for now it helps me create any number of scripts and test them out, with the previously mentioned Cerebras code subscription (or Codex/Claude Code/Gemini or whatever, should the other models fuck up):\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"07-generation-script.jpg\" alt=\"07-generation-script.jpg\" title=\"07-generation-script\"\u003e\u003c/p\u003e\n\u003cp\u003eI did tweak a bunch of things along the way, but also ran into a problem, which was that I still had the old database and it's not like I can just discard 900 links and break most of my blog:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"08-the-old-database.jpg\" alt=\"08-the-old-database.jpg\" title=\"08-the-old-database\"\u003e\u003c/p\u003e\n\u003cp\u003eI needed to carry all of those over to my Apache2 container image configuration file, which thankfully isn't too hard, because pretty much all of my software runs in containers and the data of which can easily be exported, downloaded and ran locally:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"09-launching-db-locally.jpg\" alt=\"09-launching-db-locally.jpg\" title=\"09-launching-db-locally\"\u003e\u003c/p\u003e\n\u003cp\u003eWith a local instance and \u003ca href=\"https://links.kronis.dev/ee5fymPWVz\"\u003eDataGrip\u003c/a\u003e (or MySQL Workbench or honestly any other software out there), I could then look at the local data pretty easily:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"10-viewing-db-data.jpg\" alt=\"10-viewing-db-data.jpg\" title=\"10-viewing-db-data\"\u003e\u003c/p\u003e\n\u003cp\u003eNow, thankfully, YOURLS had a pretty simple DB structure with no surprises along the way, but that still got me thinking about whether all of the links would follow a format with no trailing slashes and a simple \u003ca href=\"https://links.kronis.dev/uXaXIr8lhE\"\u003eRedirect directive\u003c/a\u003e would work in Apache2, or whether I'd need the more complex \u003ccode\u003eRewriteEngine\u003c/code\u003e and \u003ccode\u003eRewriteRule\u003c/code\u003e combination, which I'd rather avoid.\u003c/p\u003e\n\u003cp\u003eThankfully, over here the AI once again came in helpful, executing a bunch of \u003ccode\u003egrep\u003c/code\u003e commands faster than I could come up with them (especially given the patterns):\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"11-reviewing-my-actual-blog.jpg\" alt=\"11-reviewing-my-actual-blog.jpg\" title=\"11-reviewing-my-actual-blog\"\u003e\u003c/p\u003e\n\u003cp\u003eI did later identify some other issues that had to be handled separately, but this was a pretty promising start!\u003c/p\u003e\n\u003cp\u003eIt's been said before, that you should \u003ca href=\"https://links.kronis.dev/65QuqLtVbI\"\u003ealways bet on text\u003c/a\u003e, which in practice means that if I can export my data from inside of MariaDB or PostgreSQL or wherever into a file I can browse on the file system in a simple manner, then suddenly operating on it becomes trivial:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"12-always-bet-on-text-data.jpg\" alt=\"12-always-bet-on-text-data.jpg\" title=\"12-always-bet-on-text-data\"\u003e\u003c/p\u003e\n\u003cp\u003eSomewhere along the way I realized that we're all pretty cooked, once I realized that even the somewhat limited near-SOTA \u003ca href=\"https://links.kronis.dev/XQ1Sea1pNV\"\u003eGLM-4.7 model\u003c/a\u003e can:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003enot only use tools to navigate my file system (even if I give it my entire Projects directory, with the caveat that it might go insane and sooner or later delete everything, making me pull it from my server Git repos anew)\u003c/li\u003e\n\u003cli\u003enot only write code with a plethora of available tools, like inside of Cline/RooCode/KiloCode or OpenCode\u003c/li\u003e\n\u003cli\u003ebut also that the tech works closer to being autonomous at this point, integrating with various linting and LSP tools:\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"13-we-are-so-cooked.jpg\" alt=\"13-we-are-so-cooked.jpg\" title=\"13-we-are-so-cooked\"\u003e\u003c/p\u003e\n\u003cp\u003eAs I'm sure you're aware, I acknowledge the issues with AI and ethics, but also don't have the burning passion that some of the people I know have towards AI. I view it as a tool and at this point can (perhaps unfortunately) say that it produces code faster AND better than many of the people I've worked with would, when properly guided and validated.\u003c/p\u003e\n\u003cp\u003eThere are fuckups that humans wouldn't make, sure, but it also is capable of catching some edge cases in code that humans wouldn't think of - I still use it as a tool, but the people slowly raising alarms over the fact that a lot of junior developers will get replaced by greedy orgs that don't want to train them seems more and more likely. In my case, I could achieve something in 1-2 hours that would take me 3-4 hours normally, precisely because of this level of automation (and the ability to validate any one bit of output as correct/incorrect quite quickly):\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"14-validation-scripts.jpg\" alt=\"14-validation-scripts.jpg\" title=\"14-validation-scripts\"\u003e\u003c/p\u003e\n\u003cp\u003eThat said, it's not like we can or should discard formal methods of software engineering, for example, nor the Silica Animus, nor did the scripts it wrote catch the fact that one of the links stored in the YOURLS database was NOT a link at all (which also escaped my own view when I put it there through their web UI), but Apache2 refused to start because of it, helping me catch it:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"15-identifying-errors.jpg\" alt=\"15-identifying-errors.jpg\" title=\"15-identifying-errors\"\u003e\u003c/p\u003e\n\u003cp\u003eVery much seems to me like everyone benefits from some guardrails, tests and other stuff, human and machine alike:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"16-i-messed-up-previously.jpg\" alt=\"16-i-messed-up-previously.jpg\" title=\"16-i-messed-up-previously\"\u003e\u003c/p\u003e\n\u003cp\u003eNow, obviously the file can get quite long, and under normal circumstances I will not want to edit it manually. Not like it's hard to do, but convenience scripts are so much nicer:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"17-adding-new-urls.jpg\" alt=\"17-adding-new-urls.jpg\" title=\"17-adding-new-urls\"\u003e\u003c/p\u003e\n\u003cp\u003e(I still played around with the link lengths a bit, I think 10 symbols is more than enough, especially given that I can easily check for duplicates already in the file)\u003c/p\u003e\n\u003cp\u003eNot only did I migrate all of the links over, but I could even get a Python script running, which would check every single line in the CSV file and check whether \u003ccode\u003ecurl\u003c/code\u003e can get the \u003ccode\u003eLocation\u003c/code\u003e header back:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"18-validating-if-everything-works.jpg\" alt=\"18-validating-if-everything-works.jpg\" title=\"18-validating-if-everything-works\"\u003e\u003c/p\u003e\n\u003cp\u003eBut wait, there's more! I had a bit of technical debt laying around! You see, before YOURLS, I had a bunch of regular, direct links. Although those didn't tend to break since I had only ran into simpler ones back then, I decided that migrating them as well might not be too hard to do, so with a bunch of parsing, preparation and updates, I could make every single URL in my blog (unless I messed up here and there) use this new link shortener:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"19-migrating-old-links.jpg\" alt=\"19-migrating-old-links.jpg\" title=\"19-migrating-old-links\"\u003e\u003c/p\u003e\n\u003cp\u003eIt's actually immensely useful to have two repos locally and to be able to have scripts from one repo call scripts in the other repo - to iterate through my blog posts and automatically add what needs to be added to the link shortener configuration.\u003c/p\u003e\n\u003ch3\u003eWhat's next?\u003c/h3\u003e\n\u003cp\u003eSo, the link shortener is up and works pretty well! It did take me a while, but that's because I ran into a slight, yet manageable, amount of scope creep, as tends to sometimes happen. The good news is that this setup is good enough to hopefully last me for another decade without too many issues along the way, probably longer than that, given the manageable amount of links my blogging (and other stuff) creates.\u003c/p\u003e\n\u003cp\u003eOh, also, the container image builds and deploys in about 5-10 seconds. It's kind of the sweet spot of KISS and YAGNI.\u003c/p\u003e\n\u003cp\u003eLooking forwards to the future, I think that I might also consider the possibility of moving away from Grav altogether, if it ever becomes a problem - the current setup is good enough and I get the benefits from both a CMS and being able to treat it as something headless normally: it's built and deployed from my Git repo, I'm writing this post in Visual Studio Code, yet it's hard to beat the security and simplicity of a bunch of HTML files.\u003c/p\u003e\n\u003cp\u003eFor some documentation at work we already moved to \u003ca href=\"https://links.kronis.dev/1mP9aaON8o\"\u003emdBook\u003c/a\u003e and in practice it works very well! Maybe it's better suited for Wiki-style sites, but a part of me also daydreams of getting rid of PHP and a bunch of dynamic logic currently needed by Grav, until some day the whole site is built on my server and sent away for deployment.\u003c/p\u003e\n\u003cp\u003eOr I could go down the rabbit hole of writing my own CMS from scratch, because even if I'd make something perhaps more horrible than Grav, it would still be a cool experience. Either way, even if I wouldn't hold my head high and confidently tell people that \u0026quot;Dropbox is just FTP\u0026quot;, sometimes you can indeed build something that's good enough for you with relatively few issues along the way.\u003c/p\u003e\n"
        },
        {
            "title": "I blew through 24 million tokens in a day",
            "date_published": "2025-10-20",
            "id": "https://blog.kronis.dev/blog/i-blew-through-24-million-tokens-in-a-day",
            "url": "https://blog.kronis.dev/blog/i-blew-through-24-million-tokens-in-a-day",
            "content_html": "\u003cp\u003eSuppose I have a legacy project, or even just an older project that I worked on previously and now want to carry over the mechanisms that made it work, to a new one.\u003c/p\u003e\n\u003cp\u003eAll sorts of custom functionality, wrappers around the underlying library components with custom functionality, like being able to tell when any input field or input element has been touched and a form should be considered dirty (prompt before navigating away), as well as links and navigation logic that integrates with this, custom utilities for i18n built on top of pre-existing solutions, validators, date and number formatting utilities (maybe with \u003ca href=\"https://links.kronis.dev/30jlb\"\u003emoment.js\u003c/a\u003e and \u003ca href=\"https://links.kronis.dev/krpxj\"\u003ecurrency.js\u003c/a\u003e integrated) and so much more.\u003c/p\u003e\n\u003cp\u003eThere might be dozens, or sometimes hundreds of files in a project that otherwise has over a thousand source files, obviously not coupled as loosely as it might be (e.g. a collection of separate packages), because clearly nobody has the time for setting that up and that's never how these things evolve over projects that span 5 or more years.\u003c/p\u003e\n\u003cp\u003eYet, I don't want the business functionality. I don't want the constants from that project. I don't want the router routes, but I want the logic for having nested routes, I want the logic for highlighting the route group in the navbar/sidebar when a route below that is active, I want at least some of the permission checks and the more generic bits of redirects to the login page, error handlers and error boundaries and so much more.\u003c/p\u003e\n\u003cp\u003eYou'll notice that this example is front end centric, but it might as well be back end centric as well. In either case, what I have a lot of are requirements, since otherwise I'd be writing everything from scratch and that never works out well, especially when there are perfectly serviceable implementations somewhere for me to reference, instead of rediscovering the edge cases anew. But maybe I want to migrate it all to \u003ca href=\"https://links.kronis.dev/lic07\"\u003eTypeScript\u003c/a\u003e, maybe I want to move from \u003ca href=\"https://links.kronis.dev/avt02\"\u003eVuetify\u003c/a\u003e to \u003ca href=\"https://links.kronis.dev/d6zla\"\u003eQuasar\u003c/a\u003e, perhaps I even want to explore implementing the same functionality with \u003ca href=\"https://links.kronis.dev/uyxto\"\u003ePinia\u003c/a\u003e, or get rid of it.\u003c/p\u003e\n\u003cp\u003eThe one thing I don't have, however, is time. Nor do I think I necessarily have enough motivation or working memory to go through 100 components and update all of them in a specific way. Everyone has deadlines and even if it's a personal project, the evenings are only so long. So, it's the perfect use case for generative AI, right?\u003c/p\u003e\n\u003ch3\u003eGenerative AI and its main problem\u003c/h3\u003e\n\u003cp\u003eWell, sort of. Having generative AI at your fingertips does a few good things:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eit follows boilerplate adjacent tasks pretty well\u003c/li\u003e\n\u003cli\u003eeven more so if you can give examples of what to do and not to do\u003c/li\u003e\n\u003cli\u003efurthermore, whatever it spits out isn't too hard to verify as correct/incorrect\u003c/li\u003e\n\u003cli\u003eit doesn't get tired and demotivated, you can also steer it in a particular direction easily\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eIn other words, it's like having a very motivated junior developer that sometimes does completely erroneous things, but at the same time has like no ego and will carry out your bidding, which you will never get with real human beings (nor should you expect that). In other words, it's a tool.\u003c/p\u003e\n\u003cp\u003eUnfortunately, hundreds of files is still a lot, even for most of the AI solutions out there. Not if you're swimming in money, like 1000 EUR a month to spend on it and above, \u003ca href=\"https://links.kronis.dev/pzzwj\"\u003eas these people did\u003c/a\u003e a while back:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eFrustrated [with the LLM-isms, like changing failing tests into skipped tests, overcomplicating changes, unnecessary dependencies], I tried switching from Claude Sonnet to the new o3 thinking model. I knew o3 was painfully slow, so I took the time to write out exactly what I knew, and what I wanted the solution to look like, and gave it some time to work. To my surprise, the response was… great?\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThe more I tried it, the more I found o3’s improved ability to use tools, assess progress, and self-correct led to results that were actually worth the wait. I found myself expanding what terminal commands I allowed the agent to run, helping it get further than ever before. When I completed a hard “o3-grade” task and moved on to something simpler, I was increasingly tempted to leave it on o3 instead of switching back. Sonnet was faster in theory. But o3 was faster in practice.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThe only problem was, it was costing a fortune.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eDepending on the task, my o3 conversations were averaging roughly $5 of Cursor requests each, or about $50 a day. That… is a lot of money.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eStill, we were in a hurry. And what is a startup if not a series of experiments? So I turned to my co-founder.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eA: Jenn, I have a proposal. You’re going to hate it. ... So you know I’ve been getting really good results from o3. I propose we try just defaulting to o3 for the next 3 weeks until our demo, and increase our Cursor spending cap to $1000/mo.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eB: That’s a lot of money.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eA: I know. It’s ridiculous. This is ridiculous. But also, when we hire a Founding Engineer, they will cost a lot more than that. Like, 10x more.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eB: …Okay. Let’s try it. If it’s not worth the cost, we’ll go back.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eA: So we tried it. And, to both of our horror, it was worth the cost.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eThey go into a bit more detail, if you're interested in their experience, go read that post: but I'm linking it because it's an interesting look at things. \u0026quot;This tool that still needs a human to manage it and is sometimes flawed still has value in it, and even if the sticker price might give some people shock, it's still far cheaper than a full time employee.\u0026quot;\u003c/p\u003e\n\u003cp\u003eUnderstandably, that kind of thing might make sense if you're a business and if you want to support your developers. I would honestly expect most businesses out there to make the best tools available to them, as per \u003ca href=\"https://links.kronis.dev/v8i6q\"\u003eThe Joel Test\u003c/a\u003e, although don't overlook the part where I say \u0026quot;available\u0026quot; instead of \u0026quot;forced to use\u0026quot;, which I'd apply to very few tools - other than maybe automatic code formatting and linters with fixes applied whenever possible, for the sake of consistency and getting out of your way otherwise.\u003c/p\u003e\n\u003cp\u003eI, however, am not a company with 1000 EUR to burn per month, even spending 100 EUR on Google Gemini API tokens hurt. Yet, at the same time, as the task above might let you know, I want to work with larger amounts of data than most users of generative AI will, to the point where most models I could run on-prem just wouldn't have enough memory for the context, even with 4bit KV caches and 4bit quantized models, aside from the models themselves not being very good.\u003c/p\u003e\n\u003cp\u003eInevitably, I have to look at cloud options and my main problem with generative AI isn't that it's unethical or that it's stealing, or that it might replace my job and those of countless people and lead to an economic crash that could turn out way worse than that of 2008, all while killing human creativity and possibly cognitive abilities in the process - it's that there just isn't enough AI at my disposal. Yes, I sometimes write in an absurdist style.\u003c/p\u003e\n\u003cp\u003eSo, I looked for what are my options, realistically.\u003c/p\u003e\n\u003ch3\u003eShilling for Cerebras Code\u003c/h3\u003e\n\u003cp\u003eMost of the services out there go out the window, Claude Code and Gemini and ChatGPT subscriptions are all too restrictive in regards to the rate limits and how many tokens you can shuffle around - with them, my only option would be paying per token and while I can do that for a smaller amount of queries, it just wouldn't do for my scale.\u003c/p\u003e\n\u003cp\u003eEventually I found a subscription by a company that's making their own chips that are quite different from Nvidia's GPUs - \u003ca href=\"https://links.kronis.dev/s65xn\"\u003eCerebras\u003c/a\u003e:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"00-cerebras.jpg\" alt=\"00-cerebras\" title=\"00-cerebras\"\u003e\u003c/p\u003e\n\u003cp\u003eI won't try to get into the technical aspects of it all, nor try to figure out why they aren't 10x bigger than they currently are, but the bottom line is that they can run LLMs at speeds that sometimes are 10-20x faster than other providers. Their materials say up to 30x, but that's probably some theoretical figure or varies on a case by case basis, but the bottom line here is: they're fast.\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"00-cerebras-speed.jpg\" alt=\"00-cerebras-speed\" title=\"00-cerebras-speed\"\u003e\u003c/p\u003e\n\u003cp\u003eSimilarly, see the Qwen3 Coder model they mention? It's the 480B variety, that scores pretty favourably in comparison with most of the other models you can find out there and holds its own against Google's offering, both in my personal experience, as well as \u003ca href=\"https://links.kronis.dev/ywjvc\"\u003esome of the benchmarks on SWE-bench\u003c/a\u003e:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"00-swebench.jpg\" alt=\"00-swebench\" title=\"00-swebench\"\u003e\u003c/p\u003e\n\u003cp\u003eI am slightly frontloading this to explain that this setup isn't the cutting edge when it comes to the actual model quality, but it's more than competent and at that speed needing 2 or 3 attempts to get things right is a matter of a few seconds.\u003c/p\u003e\n\u003cp\u003eEssentially, that resolves my issue of now having access to enough AI, because they have a plan that's 50 USD a month:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04-subscriptions.jpg\" alt=\"04-subscriptions\" title=\"04-subscriptions.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eIt also includes some pretty good limits in regards to what you can do (more than any other provider), just look at that amount of tokens you can use:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"05-limits.jpg\" alt=\"05-limits\" title=\"05-limits.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003e131k of context isn't great, but is decent for most low-medium complexity programming tasks, even when you need to reference related functionality and so on, even if sometimes you'll need to make the model write its progress into a text file and reference that, especially after context compression.\u003c/p\u003e\n\u003cp\u003eAt the same time, they now let you use 24'000'000 tokens per day. 24 million. That is seriously a lot. For comparison, if I needed that amount and I would be paying for some decent models, here's \u003ca href=\"https://links.kronis.dev/l7ygc\"\u003ehow much that would cost per input/output tokens\u003c/a\u003e for me:\u003c/p\u003e\n\u003ctable\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth\u003eModel\u003c/th\u003e\n\u003cth\u003eCost per 24M Tokens\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n\u003ctr\u003e\n\u003ctd\u003eSonnet 4.5 (input)\u003c/td\u003e\n\u003ctd\u003e72 USD\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eGemini 2.5 Pro (input)\u003c/td\u003e\n\u003ctd\u003e30 USD\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eGPT-5 (input)\u003c/td\u003e\n\u003ctd\u003e30 USD\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eSonnet 4.5 (output)\u003c/td\u003e\n\u003ctd\u003e360 USD\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eGemini 2.5 Pro (output)\u003c/td\u003e\n\u003ctd\u003e240 USD\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eGPT-5 (output)\u003c/td\u003e\n\u003ctd\u003e240 USD\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eEven assuming an 80% input and 20% output split, it'd be:\u003c/p\u003e\n\u003ctable\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth\u003eModel\u003c/th\u003e\n\u003cth\u003eCost per 24M Tokens\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n\u003ctr\u003e\n\u003ctd\u003eSonnet 4.5\u003c/td\u003e\n\u003ctd\u003e130 USD\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eGemini 2.5 Pro\u003c/td\u003e\n\u003ctd\u003e72 USD\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eGPT-5\u003c/td\u003e\n\u003ctd\u003e72 USD\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eAnd that's per day. If we extrapolate per month (assuming hitting those limits every day, unlikely but for the sake of comparison):\u003c/p\u003e\n\u003ctable\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth\u003eModel\u003c/th\u003e\n\u003cth\u003eMonthly Cost\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n\u003ctr\u003e\n\u003ctd\u003eSonnet 4.5\u003c/td\u003e\n\u003ctd\u003e3880 USD\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eGemini 2.5 Pro\u003c/td\u003e\n\u003ctd\u003e2160 USD\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eGPT-5\u003c/td\u003e\n\u003ctd\u003e2160 USD\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eCerebras\u003c/td\u003e\n\u003ctd\u003e50 USD\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eI rounded some of the results, but the difference here is clear. Plus, with Cerebras letting you do 50 requests per second, you can just let your IDE or whatever solution you use for agentic type of work fly and not worry too much about artificial rate limits. Maybe 1 second delay between requests if you want to be super sure that you won't see rate limit requests.\u003c/p\u003e\n\u003cp\u003eI cannot overstate how cool having a subscription like this is and I earnestly hope that they don't discontinue it. It seems to me like a great way to attract more attention towards Cerebras, even though I can't begin to make any guesses about the financial feasibility of it on their part, given how much a single of their chips costs to begin with.\u003c/p\u003e\n\u003cp\u003eAs an individual user, I'd say give them a shot! Just don't look at how expensive the other enterprise-oriented plans are:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"00-enterprise-plans.jpg\" alt=\"00-enterprise-plans\" title=\"00-enterprise-plans\"\u003e\u003c/p\u003e\n\u003cp\u003eSo, with that in mind, I have a new daily driver, a subscription that doesn't make me feel like I'm being ripped off, all while also leaving the door open for me to pay for other APIs when I need to (like getting feedback on a problem from multiple different LLMs, given that training data differs).\u003c/p\u003e\n\u003cp\u003eSurely 24 million tokens is such an outrageously large amount that I could never run into rate limits, right?\u003c/p\u003e\n\u003ch3\u003eLet's burn down some forests\u003c/h3\u003e\n\u003cp\u003eWrong. It took me just a few hours:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-rate-limit.jpg\" alt=\"01-rate-limit\" title=\"01-rate-limit.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eThe error you see here is from \u003ca href=\"https://links.kronis.dev/w6hrt\"\u003eRooCode\u003c/a\u003e, one of my presently favorite solutions for agentic development, that has both chat and code modes, alongside others for planning and a bunch of other useful functionality (multi-file reads, multi-file edits, allows reading large files bit by bit as needed etc.).\u003c/p\u003e\n\u003cp\u003eI was still surprised to see it. I thought that it was the request count related warning, that the IDE had decided to be a bit too speedy for what's allowed, but no, even after a minute, the error was still there and the Cerebras dashboard confirmed that my daily allowance was up:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02-cerebras.jpg\" alt=\"02-cerebras\" title=\"02-cerebras.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eIt's helpful to remember that it didn't really cost me much to do that, essentially those 24 million tokens were a bit less than 2 USD given the subscription that I'm on (a bit more in practice because my actual utilization is usually below those rate limits). Even with the good price for me, it might seem like kind of an insane amount.\u003c/p\u003e\n\u003cp\u003ePerhaps, but I did make a whole bunch of progress, got about 90 components carried over and about 40-60 utilities and project files, alongside a few dependencies (that I had to confirm manually, because I don't allow terminal interaction without my confirmation), a few passes of changes in those files as needed, some restructuring and so on. Even with 150 files that still comes out to around 160'000 tokens per file, which might seem a lot until you consider that it needed to read an instructions file and update a few progress files that persist after the context being condensed, re-read the instructions after the context is partially wiped, as well as reference library type information and other old code.\u003c/p\u003e\n\u003cp\u003eI was surprised at how quickly I burned through those tokens, but I'm afraid it's already over for me - the very next day, the very same thing happened when I decided to refactor everything and also ensure that all of the TypeScript type checks work and some other linter issues are resolved. On another note, fuck the Vue SFC Compiler and its \u003ca href=\"https://links.kronis.dev/xldy9\"\u003einability to handle complex TypeScript types\u003c/a\u003e, conditionals etc.\u003c/p\u003e\n\u003cp\u003eI'm like those people who drink multiple cups of coffee (or energy drinks) a day and then wonder why they're always feeling a bit anxious and why their sleep quality is not great. Out of curiosity, \u003ca href=\"https://links.kronis.dev/sf2rs\"\u003eto run that model locally\u003c/a\u003e, I'd need around 10 Nvidia H100 cards even at 8bit quantization:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"00-nvidia-h100.jpg\" alt=\"00-nvidia-h100\" title=\"00-nvidia-h100\"\u003e\u003c/p\u003e\n\u003cp\u003eLet me remind you, that one of those cards costs around 25-30k USD, so I'd be looking at a total of around 250'000 to 300'000 USD to buy those cards, and then a total power draw of around 3500W. Even with all that, I'm not sure if I'd reach Cerebras speeds, but I'd certainly do better than \u003ca href=\"https://links.kronis.dev/bkbtn\"\u003emost other providers\u003c/a\u003e since I'd have it all for myself, but even so, the time to generate 24 million tokens would be a lot:\u003c/p\u003e\n\u003ctable\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth\u003eTokens per second\u003c/th\u003e\n\u003cth\u003eHours\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n\u003ctr\u003e\n\u003ctd\u003e100\u003c/td\u003e\n\u003ctd\u003e66\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e200\u003c/td\u003e\n\u003ctd\u003e33\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e300\u003c/td\u003e\n\u003ctd\u003e22\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e400\u003c/td\u003e\n\u003ctd\u003e16\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e1000\u003c/td\u003e\n\u003ctd\u003e6\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e2000\u003c/td\u003e\n\u003ctd\u003e3\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eIf that means the cards running at full power for the duration, that comes out to around 10.5 kWh and 231 kWh, or enough energy to boil somewhere between 112 to 2480 liters of water. Every single day (except in the case where it'd take more than 24 hours, obviously). That's like having a bonfire burning whenever all of those cards are going full send.\u003c/p\u003e\n\u003cp\u003eIt's some back of the napkin maths that's probably not entirely accurate, but with less efficient cards, one can definitely imagine a bit more power usage than one would expect and figures somewhere inside of that super wide range, so in my case I'm both thankful that Cerebras exists but also a bit concerned about the future of our environment.\u003c/p\u003e\n\u003ch3\u003eThe line must go up\u003c/h3\u003e\n\u003cp\u003eYet, as I said, I crave more.\u003c/p\u003e\n\u003cp\u003eParallel agents, each spending tens of millions of tokens a day. Agents controlling other agents, doing quality control on every change. Other agents calling tools like linters and type checks and builds to see whether the output is good. Agents taking my text and turning it into fully fledged spec documents, others reporting progress on the tasks and delegating down. Just endless iteration to check and cross check and verify, improve and fix and enforce. Loop after loop after loop until it looks like something out of Warhammer 40k. A gargantuan system burning our forests and boiling our oceans, I crave to look upon the gray and brown skies and know that I will never see the sun again, for its light is swallowed in all of the smog, only its rays heating the greenhouse gases with each passing day.\u003c/p\u003e\n\u003cp\u003eAll of that, to make working on shitty CRUD apps easier while I sip whatever drink I'm set on that day.\u003c/p\u003e\n\u003cp\u003eThis should read as a critique of AI, of all of the consequences and empty promises made by people trying to sell it, all of the slop that's created, all of the jobs of translators and people who transcribe audio ruined, all of the graphical design that can no longer be someone's passion because their bosses are pushing for more AI slop and genuinely couldn't care less for the craft, artists having their entire styles copied and stolen, same with books, same with code... at the end of the day, however, I like the power.\u003c/p\u003e\n\u003cp\u003eThe problem is that what we want to do might need an order of magnitude, or maybe two orders of magnitude more resources, time, training, research and all that - not even to get close to AGI, but just make most use out of the technology. I genuinely see how I could make programming be more about planning and less about resolving stupid compilation errors, even if I'd still have to look through all of the files and my job would become very code review centric. Even if the tools now aren't excellent, even if people aren't familiar with the problems that LLMs are good and bad for, even if it all burns down at some point and takes our economy with...\u003c/p\u003e\n\u003cp\u003eCerebras gave me a glimpse into how good things could be. A bit like running a Spring Boot app and it compiles and runs in about 10 seconds instead of 2 minutes. One of those leaves you pleasantly surprised, the other makes you want to jump off of a bridge. Presently, most other LLM providers are somewhere in the middle of that spectrum.\u003c/p\u003e\n\u003ch3\u003eSome genuinely cool stuff\u003c/h3\u003e\n\u003cp\u003eOkay, I'm done being dramatic, time to shake things up!\u003c/p\u003e\n\u003cp\u003eThere's also cool stuff that you can do, for example, did you know that \u003ca href=\"https://links.kronis.dev/qaie7\"\u003eSentinel-2\u003c/a\u003e data is available to everyone and is updated around every 5-10 days:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"07-ndvi.jpg\" alt=\"07-ndvi\" title=\"07-ndvi.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eIt's not just regular pictures, but multi-spectral data, meaning that you can calculate all sorts of data, like \u003ca href=\"https://links.kronis.dev/qgjo1\"\u003eNDVI (Normalized Difference Vegetation Index)\u003c/a\u003e, which can tell you what is the state of the vegetation on the ground in any given area!\u003c/p\u003e\n\u003cp\u003eNot only that, but if you have historical data available (you do), you can even figure out \u003ca href=\"https://links.kronis.dev/99x3f\"\u003ewhat types of crops are growing\u003c/a\u003e in a given area and combine that with the other figures and see the overall health of every single field in your country, if you're so inclined.\u003c/p\u003e\n\u003cp\u003eWhy AI? Not just because of the methods used in the classification system (less generative AI, more formal methods), but rather that I could whip up a quick prototype to generate simple NDVI visualizations based on the real data within less than an hour:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"08-prototype.jpg\" alt=\"08-prototype\" title=\"08-prototype.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eNotice how the latitude and longitude figures are all screwed up? The data itself is okay, but I'd definitely have to fix that - here's the important part though, at least I'd have a clear starting point to iterate upon! Oh and you'll see that the colors are different, but that's mostly due to the scales not being the same.\u003c/p\u003e\n\u003cp\u003eAnyone who is saying that the technology isn't useful clearly hasn't seen how it makes the friction of starting new projects or doing those initial iterations, or handling menial repetitive tasks dissipate in front of your eyes. It's amazing.\u003c/p\u003e\n\u003ch3\u003eSummary\u003c/h3\u003e\n\u003cp\u003eYou still need to babysit AI. It is flawed and it has very real limitations. However, it can also be immensely useful and trying out Cerebras was an eye opening experience - it's like it's on a completely different level of its own.\u003c/p\u003e\n\u003cp\u003eAt the same time, it's almost inevitable that corpos will eventually try to squeeze as much money out of us as possible, because currently the AI companies are simply competing to be loss leaders and are nowhere near to turning a profit (albeit that's because they are in a race to make better models, all while inference by itself would be profitable).\u003c/p\u003e\n\u003cp\u003eYet, I'm convinced that the technology underneath it all is solid and while execs all over the place have the solution for a problem they haven't yet found (yet are trying to sell regardless), I don't care and I benefit already. This is in contrast to the former hype of blockchain, which still remains niche (with one or two use cases here and there), as well as cryptocurrency, which often ends up being a massive rugpull, even if in theory can simplify money transfers a whole bunch (even if transfers inside of the EU are thankfully already fast and easy).\u003c/p\u003e\n\u003cp\u003eAt the same time, it would be cool if we didn't actually need to ruin our ecosystems, I guess I'll have to satisfy that craving of mine by playing \u003ca href=\"https://links.kronis.dev/85qv9\"\u003eFactorio\u003c/a\u003e instead:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"09-factorio.jpg\" alt=\"09-factorio\" title=\"09-factorio.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003e(I think there's a certain kind of charm that in the game ruining the local ecosystem with the pollution from your factory makes alien bugs come after you, gives you an actual reason to build more base defenses. Bit of a tangent, but hey, it's my blog.)\u003c/p\u003e\n"
        },
        {
            "title": "AIOs are superb, thermal pastes are the same?",
            "date_published": "2025-10-13",
            "id": "https://blog.kronis.dev/blog/aios-are-superb-thermal-pastes-are-the-same",
            "url": "https://blog.kronis.dev/blog/aios-are-superb-thermal-pastes-are-the-same",
            "content_html": "\u003cp\u003eHere's a short post: I think AIOs or All-In-One coolers are the superior cooling solution, both to air coolers and custom liquid cooling loops for the average user!\u003c/p\u003e\n\u003cp\u003eRecently, I got some new thermal paste, because my CPU temps were still a bit higher than I'd like under full load. While gaming they'd hover around 70 C and under some benchmarks they'd go up to 80 C whereas with Prime95 they'd rise so far that the CPU would thermal throttle to prevent damage. This is on a Ryzen 7 5800X, a series which is known to run a bit hot, but at the same time I wondered whether I could improve things a bit.\u003c/p\u003e\n\u003cp\u003eI realized that I more or less dreaded the idea of having to take off the CPU cooler and to repaste it and assemble everything again, mostly due to the mounting mechanisms that I've seen a lot of AM4 coolers use - those annoying pressure latches that are hard to secure properly, as well as the fact that good air coolers can be quite bulky and I sometimes just get cut or scraped by the fins on them.\u003c/p\u003e\n\u003ch3\u003eAIOs are very nice, you should get one\u003c/h3\u003e\n\u003cp\u003eBut then I remembered that I moved over to having an AIO which leaves it as a really simple errand and the case doesn't even feel all that crowded:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-aio.jpg\" alt=\"01-aio\" title=\"01-aio.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eNow, the fact that everything is a bit dusty and that I use tape for managing some cables (don't worry, nothing has melted yet, it might look like a mess but it works) aside, you can see for yourself - there are some screws that I can easily unscrew or rescrew with my hands without getting a screwdriver out. It's about as easy as installing an AIO can get.\u003c/p\u003e\n\u003cp\u003eI got the Aigo ACSE 240 on AliExpress a while back for about 54 EUR and I have to say that it was money well spent, they also have or at least had some of the more affordable case fans as well! This setup pretty much replaced me needing like 5 case fans and the temps are pretty good, all while there isn't too much noise.\u003c/p\u003e\n\u003cp\u003eI will admit that I wouldn't recommend them over other brands at the time because it seems like most prices on AliExpress went up a good 40% and I've literally 0 idea why - for that price you can probably find something on local e-commerce stores for cheaper, even from more reputable or at least mainstream brands:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-aio-store.jpg\" alt=\"01-aio-store\" title=\"01-aio-store.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eEither way, the argument remains that as long as you go for literally any AIO with good reviews that fits in your case and your budget, you'll probably have a pretty decent time. And yes, although they might need a refill in like 5 years or might need a replacement if and when the pump dies, they are still affordable enough for this not to be a super big problem.\u003c/p\u003e\n\u003cp\u003eSo what's this whole thing about the thermal paste? Well, the temps were still a bit high for my liking, so I figured that I'd get rid of the Gembird paste I had on the cooler up until now. You might gasp in horror at seeing how there's a bit too much paste on the cold plate, but honestly that's mostly a combination of pretty high mounting pressure and it being better to have a bit too much instead of too little, especially given how the cheaper pastes don't really spread out that well:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02-paste-aio.jpg\" alt=\"02-paste-aio\" title=\"02-paste-aio.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eOkay, I admit it - I made a bit of a mess and used too much. But it was easy to clean off, because the radiator and the fans didn't even have to be removed from the case, I just unscrewed the cooler head from the CPU mount and pulled it aside and could wipe it down as much as I needed. That's really, really convenient.\u003c/p\u003e\n\u003cp\u003ePaste aside, notice how those standoffs are still in place, meaning that I don't have to worry about the backplate falling off or anything, it's not like a big deal or anything, but nice to see common sense engineering:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03-paste-motherboard.jpg\" alt=\"03-paste-motherboard\" title=\"03-paste-motherboard.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eOf course, I'm slightly exaggerating here, since even if you do use a bit too much thermal paste and it gets squeezed out from the sides, it's just a matter of a few minutes of wiping and it's all clean. Plus, since the thermal compounds we use are typically not conductive, there isn't really that much to worry about. On some level, I could even say that the surface of the IHS being covered just means that the maximum surface area is available for heat transfer, but realistically people only care about the hotspot and the surrounding area, so that'd be a silly thing to say.\u003c/p\u003e\n\u003cp\u003eTo heck with it, I paid for the whole tube, I'm going to use the whole tube. /jokes\u003c/p\u003e\n\u003cp\u003eI will say that I did just use some TP for wiping it down, which you technically shouldn't do because of lint, but I didn't feel like making a mess of a microfiber cloth (I need those for my glasses), alongside not really having any alcohol around the place, so I did need a bit more work and patience. But look, the end result was still pretty decent, even if not perfect and it took a few minutes to get it cleaned up:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04-aio-clean.jpg\" alt=\"04-aio-clean\" title=\"04-aio-clean.jpg\"\u003e\u003c/p\u003e\n\u003ch3\u003eThe new thermal paste\u003c/h3\u003e\n\u003cp\u003eNow, what's the old thermal paste I wanted to replace and what with?\u003c/p\u003e\n\u003cp\u003ePreviously, I didn't pay it much mind, thinking that most of them should perform in the same ballpark, so \u003ca href=\"https://links.kronis.dev/7pmi3\"\u003ethe old one was from Gembird\u003c/a\u003e - it was dirt cheap and actually had worked pretty nicely on 65 W CPUs previously, but seemingly wasn't quite up to the job when paired with a 105 W CPU, alongside being advertised for heatsinks, which is kind of the right thing, but perhaps not the least confusing wording to put on the package:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"05-arctic-mx-4.jpg\" alt=\"05-arctic-mx-4\" title=\"05-arctic-mx-4.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eWhile the nondescript name might raise a few eyebrows, I assure you, it worked mostly fine.\u003c/p\u003e\n\u003cp\u003eAnd the replacement, as you can see is \u003ca href=\"https://links.kronis.dev/d4cee\"\u003ethe good old Arctic MX-4\u003c/a\u003e, something that's been around for ages and is pretty well regarded by pretty much everyone: still an option that won't break the bank and is generally proven to work pretty well. I also opted for the 8 gram package, because if you seal it up properly, it's not like they have a short shelf life or anything.\u003c/p\u003e\n\u003cp\u003eLiterally my only critique is that they didn't give me that little applicator spatula thing to make it easier to spread it on the CPU, which I stole from the Gembird kit instead:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"06-spatula.jpg\" alt=\"06-spatula\" title=\"06-spatula.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eI did use a bit less this time around, you're just going to have to trust me on this one, because I was so focused on getting equal mounting pressure when putting the cooler back on, I totally forgot that I should have taken a picture beforehand.\u003c/p\u003e\n\u003cp\u003eAs an aside, personally I think Intel had a really good mounting mechanism on their stock coolers, those four latches that rotate, but I guess screws give you the ability to apply a bit more mounting pressure when you really need it.\u003c/p\u003e\n\u003cp\u003eThey did make me a little bit squeamish, because I have accidentally destroyed motherboards with my bare hands before - after a really stubborn 24-pin ATX connector latch made me bend the motherboard while just trying to disconnect it. That's honestly why I sometimes cut those latches off with pliers, because normally there is enough friction between all of the pins to lead to a tight fit anyways (of course, it depends on the individual PSU and motherboard and the manufacturing tolerances).\u003c/p\u003e\n\u003cp\u003eAt the same time, I don't mount and unmount the AIO that often, so I did put a little bit of strength into screwing the AIO down, but not to the point where it'd get risky or anything. All of it, including cleaning the old thermal paste off, which was the most annoying part, took me maybe 15 minutes. Plus, not a single scrape on my fingers, no annoyances about mounting mechanisms.\u003c/p\u003e\n\u003cp\u003eIf you don't have a custom water cooling loop already or an Intel stock cooler or something similarly easy to install and just unobstructive, you probably should get an AIO. It can also replace a few case fans, which I think is just lovely.\u003c/p\u003e\n\u003cp\u003eNow, what about the thermal paste, did it matter in the grand scheme of things?\u003c/p\u003e\n\u003ch3\u003eHow much does thermal paste matter?\u003c/h3\u003e\n\u003cp\u003eSurprisingly, it doesn't seem like the Arctic MX-4 is that much better than the cheap Gembird thermal paste. There is a bit of a burn in period of sorts until it cures fully, but those differences usually are around a few degrees, so seeing \u003ca href=\"https://links.kronis.dev/79z7d\"\u003eCPU-Z\u003c/a\u003e stress tests settle on around the same 80ish degrees as before was a bit surprising:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"07-cpuz.jpg\" alt=\"07-cpuz\" title=\"07-cpuz.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eThere might be a difference between the thermal pastes, but if it's there, it will probably be below 3-5 degrees C in most cases. I won't say that this is super disappointing or anything, actually a bit of the opposite - it's kind of cool that the more generic brand is good enough and can put up a fight even against more established brands.\u003c/p\u003e\n\u003cp\u003eIt does make me want to experiment with a thermal pad instead of paste between the CPU and the cold plate, but I've been advised that it'd generally be not a very good idea, because pads generally struggle a bit more with heat transfer, which would be perfectly fine for a laptop or my 35 W homelab server CPUs (that are also AM4 ones, but passively cooled with a radiator and no fan, which is nice in its own way, because it's suddenly less noise coming from those two boxes), but might not be something I want to try on my main PC in the near future.\u003c/p\u003e\n\u003cp\u003eDoubly so, because \u003ca href=\"https://links.kronis.dev/3np4z\"\u003ePrime95\u003c/a\u003e still very much brings the system to its knees and it starts to thermal throttle:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"08-prime95.jpg\" alt=\"08-prime95\" title=\"08-prime95.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"09-prime95-peak.jpg\" alt=\"09-prime95-peak\" title=\"09-prime95-peak.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eYeah, it hits 90 C and has to throttle back to prevent damage. Not great to see, but also way better than the chip just frying itself or something, so I'll take that. The good part here is the fact that if everything works correctly, you won't really have any shutdowns or weird instability - just the CPU acting like a fully loaded piece of hardware, which it is at that point.\u003c/p\u003e\n\u003cp\u003eNow, you might be thinking about why the system is running so hot. Did I apply the paste wrong, or maybe the AIO isn't too good? The answer there is that all of the parts involved are generally pretty decent, it's just that I'm pushing the system a bit harder in an effort to make it last until 2030 (unless I fry it). I want to settle on getting pretty close to the maximum amount of juice that I can get for the squeeze and leave it like that. Pushed harder than stock, yes, but only to such a degree where it's stable.\u003c/p\u003e\n\u003cp\u003eInstead of 4.7 GHz the CPU boosts to around 4.9 GHz, I even tried letting it do 5 GHz for a bit and while generally it was fine, I did run into some stability issues in specific games instead of stress tests, oddly enough. It might have been related to the RAM timings which weren't exactly like the manufacturer would like before I got the 3600 MHz sticks and set it up properly, but in general 4.9 GHz is fine and those frequencies should be maintainable under workloads like games that only need some cores to do things quickly, especially because at this point I'm doing negative per-core curve optimizer (CO) offsets:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"10-cpu-core-frequencies.jpg\" alt=\"10-cpu-core-frequencies\" title=\"10-cpu-core-frequencies.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eAs you can see, PBO2 changes the clock rates as necessary and the cores that need the power get priority and can maintain pretty high speeds, which is pretty much what you want for games, because they'll rarely load all of the cores, but executing code on a few cores as quickly as possible is highly beneficial.\u003c/p\u003e\n\u003cp\u003eThe good news here is that I haven't really found a real workload that stresses the system as much as the Prime95 did, either - because both gaming, video encoding and most other stuff I do on the system don't really make it go past 85 C even under full load. In general, higher temps might lead to some CPU degradation over years of constant usage, but it's nothing on the level of the voltage issue that was killing the 13th and 14th gen Intel CPUs. Rather, a few years down the line, maybe it won't be able to hold the higher clock speeds super well, but that's not a super big deal.\u003c/p\u003e\n\u003ch3\u003eSummary\u003c/h3\u003e\n\u003cp\u003eLooking at the results above, sadly I don't think I'll ever do 4.9 GHz sustained, but oh well, you win some, you lose some. I actually used \u003ca href=\"https://links.kronis.dev/1e0zh\"\u003esome of the advice in this guide\u003c/a\u003e for my overclock, though it was a pretty aggressive setup on their end. Even my setup is probably going well into the diminishing returns and risking turning the CPU into a little space heater.\u003c/p\u003e\n\u003cp\u003eAfter a bit of trial and error, I'm overall pretty satisfied with the current setup. But also, AIOs are great, you should at least consider getting one, especially now that the prices are pretty reasonable (at least for the models coming out now, not sure what's going on with AliExpress).\u003c/p\u003e\n\u003cp\u003eIt was a bit surprising how the thermal pastes performed pretty close to one another, but this also does match what I've seen in some videos online, like this one by JayzTwoCents: \u003ca href=\"https://links.kronis.dev/rqg9z\"\u003eDoes what Thermal Paste you use, REALLY matter?\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eRealistically, I'm probably holding on to that unopened package of the Gembird thermal grease as well, there's nothing wrong with it. As long as the mounting pressure is okay and there isn't too little on (or too much, if you don't like cleaning it up later), it will be fine.\u003c/p\u003e\n\u003cp\u003eAt the same time, major props to Arctic for \u003ca href=\"https://links.kronis.dev/lzh29\"\u003ebeing pretty upfront in their marketing\u003c/a\u003e about their products. That sort of integrity is nice to see, instead of them just slapping some high imaginary number on the box.\u003c/p\u003e\n"
        },
        {
            "title": "I'm an e-waste consumer",
            "date_published": "2025-10-11",
            "id": "https://blog.kronis.dev/blog/im-an-e-waste-consumer",
            "url": "https://blog.kronis.dev/blog/im-an-e-waste-consumer",
            "content_html": "\u003cp\u003eIn my previous article about my investments on Revolut in 2025 (plot twist: AMD's value just spiked and fell a bit afterwards, that's how those things go), I mentioned that I live a fairly Spartan lifestyle and get the things I need, rather than the ones I might want.\u003c/p\u003e\n\u003cp\u003eA part of this is finding the pieces of hardware that fit my needs without being exorbitantly expensive. For example, my main PC currently has these parts in it:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eASRock B450 Pro4 R2.0 motherboard (AM4)\u003c/li\u003e\n\u003cli\u003ea Ryzen 7 5800X 8c/16t CPU\u003c/li\u003e\n\u003cli\u003e2x16 GB of G.Skill 3600 MHz DDR4 RAM\u003c/li\u003e\n\u003cli\u003ean Intel Arc B580 with 12 GB of VRAM\u003c/li\u003e\n\u003cli\u003ean assortment of various drives, HDDs for data due to historical discounts, nowadays more SATA SSDs and also an NVMe boot drive after the SATA boot SSD melted (blog post about that later)\u003c/li\u003e\n\u003cli\u003ean Aigo ACSE 240 AIO cooler that I got off of AliExpress\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eI've settled on the setup with incremental updates and replacing failing parts over the years, without spending bank on any single part. My previous CPUs (and the ones still running in my homelab servers, a pair of low TDP 200GE Athlons that draw up to just 35W) sometimes came off of AliExpress, whereas sometimes there was new old stock in some e-commerce stores in my country for good prices as well.\u003c/p\u003e\n\u003cp\u003eIn a word, I try to get parts with good value and then to squeeze as much out of them as possible - tuning the CPU OC so it can punch a little bit above its weight class, picking up the new faster RAM sticks when the DDR4 prices are finally nice (in opposition to DDR5 prices right now), as well as experimenting with a dual GPU setup, which sadly didn't work out well, but let me settle on a setup that's either way hopefully good enough to last me until around 2030.\u003c/p\u003e\n\u003cp\u003eHowever, I've got an admission to make.\u003c/p\u003e\n\u003ch3\u003eI am responsible for some e-waste\u003c/h3\u003e\n\u003cp\u003eThings are way more shaky when it comes to peripherals. While getting a mainstream CPU or HDD/SSD will generally be a fairly consistent experience, when it comes to buying keyboards, mice, headsets and microphones, as well as webcams, it's like the wild west out there. There are decent quality products that you can get for cheap out there: and I don't mean\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u0026quot;Oh hey, spend 100 EUR on this Logitech keyboard.\u0026quot;\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003echeap, instead I mean\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u0026quot;Spend \u0026lt;60 EUR for a keyboard that will last you for years\u0026quot;\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003echeap. A good tradeoff between the value the product brings and its actual cost, which is harder to do than you might imagine. Some of the things I've found out were good purchases in that regard include:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eFifine microphones (currently rocking a K669)\u003c/li\u003e\n\u003cli\u003eSavio webcams (the CAK-01, at that price point everything else looks like trash)\u003c/li\u003e\n\u003cli\u003epreviously the wired Genesis Argon 100/120 and 200 headphones (good sound, dirt cheap)\u003c/li\u003e\n\u003cli\u003ebut now the wired White Shark Ocelot GH-2042 headphones (mic is not good on any of them, but the sound quality is great for the price, again better than most others around 30 USD)\u003c/li\u003e\n\u003cli\u003evarious wired mechanical keyboards with Kailh and Outemu keys (I prefer red/brown) that give you good lifespan and typing feel, without the price of Cherry MX or other niche ones; like the Ozone Strike X30 from back in the day or Spc Gear GK650K which looks a lot like Endorfy Omnis, and maybe soon a Preyon keyboard\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003e(note: there are no links to specific products over here, because I'm not really doing the whole affiliate thing, nor do I know how long specific listings would stay up for; in addition to almost nobody reviewing these products, at least not in English)\u003c/p\u003e\n\u003cp\u003eAt the same time, there is one thing that I have never been able to nail down: computer mice.\u003c/p\u003e\n\u003cp\u003eThe ones currently in my apartment look like this:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-mice.jpg\" alt=\"01-mice\" title=\"01-mice.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eApologies about everything looking kinda dusty: I pulled some of those out from a drawer that's meant for parts to connect to a homelab server when I need to access it directly (if SSH is down), and one of those is going into the trash afterwards due to having recently died, which is why I swapped my main PC over to the green one. Oh and the plastic is the variety that shows fingerprints, but oh well.\u003c/p\u003e\n\u003cp\u003eBut you'll notice a few things: all of them look very similar, despite some being from different manufacturers. In addition to that, I also have some backups of that exact same kind of mouse, as if they're not going to last that long. The truth is - they never do, that particular variety of mouse. I don't keep super exact track of these things, but I'd reckon that they don't usually last a whole year and I have to swap them out.\u003c/p\u003e\n\u003cp\u003eAt the same time, they're pretty much dirt cheap, when compared to most other computer mice you can get:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02-esperanza.jpg\" alt=\"02-esperanza\" title=\"02-esperanza.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eI will say that the prices from the different sellers vary, despite it being pretty close to being the same product (with some slight visual variation), so once I run low on them (yes, they're basically an expendable resource), I just have to order a few from whichever is the cheapest seller at the time:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03-gembird.jpg\" alt=\"03-gembird\" title=\"03-gembird.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eWhy on Earth would I do this, instead of just buying a reputable Logitech mouse and using that for years? The answer there is pretty simple - because in a way, that'd be overpaying. I've experimented with getting gradually more expensive mice and the quality isn't really there. Sometimes the weight is customizable and there's stuff like rapid fire buttons which I don't really ever use, and while I did enjoy the Canyon Hazard mouse I got for like 4x the cost of the cheapest of the mice varieties above, it still died within like a year.\u003c/p\u003e\n\u003cp\u003eIt's not like I treat the mice super roughly or that they suddenly fail, but there definitely are breakdowns - in the case of the Canyon mouse the wheel would start misbehaving and I'd have to deal with the super annoying issue of websites scrolling up when I want to go down, whereas with the mice above the left mouse button which sees the most use eventually becomes inconsistent, sometimes not registering clicks, sometimes holding it down resulting in a random release and second click.\u003c/p\u003e\n\u003cp\u003eAt the same time, when I buy one of those, I can be pretty sure that it's going to be a good experience while it lasts:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003ethey are pretty lightweight, which I personally find to be okay\u003c/li\u003e\n\u003cli\u003ethe tracking quality is good on the desk mats or mouse pads that I use\u003c/li\u003e\n\u003cli\u003ethe buttons and mouse wheel all feel good if just a little bit loud\u003c/li\u003e\n\u003cli\u003eat this point I've gotten used to the exact DPI settings, the 2nd one from the lowest is perfect\u003c/li\u003e\n\u003cli\u003eadmittedly, I did have to experiment a little bit with getting a mouse pad that minimizes the friction for aiming in games due to the bottom of the mouse being just a few smooth plastic shapes, but it also turned out that a 2 EUR Gembird mousepad was pretty much perfect, so it all worked out\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eSo now I just use these mice and once they start going bad, I throw them into the trash and grab a new one from the pile. And once the pile gets small enough, I order some more from whatever manufacturer is shipping the same whitelabel stuff from China. The current one is from Poland I think, in the future that will probably change, but I've been using more or less the same mouse design for about 6-8 years at this point and it hasn't really failed or annoyed me too much.\u003c/p\u003e\n\u003ch3\u003eWhy not just go to the source?\u003c/h3\u003e\n\u003cp\u003eNow, is it stupid to do so? Honestly, I couldn't tell you. I'm sure that if I pay a bit more money to some middleman company in the EU and get the product from a local e-commerce store that has pretty fast delivery, there is some semblance of supporting the local economy while getting that bit of additional convenience instead of just waiting on AliExpress orders to ship and make their way into my country.\u003c/p\u003e\n\u003cp\u003eThat said, if I don't need the mice on super short notice, I might as well do some sleuthing and find the mice on AliExpress as well, since that exact pattern seems to be available there as well, sometimes for a bit cheaper:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04-aliexpress-1.jpg\" alt=\"04-aliexpress-1\" title=\"04-aliexpress-1.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eObviously it's the internals of the mouse that I enjoy that just loosely seem to correspond to that one particular body type, but so far that claw look has been a good enough indicator that I'm looking at the real deal, like so:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"05-aliexpress-2.jpg\" alt=\"05-aliexpress-2\" title=\"05-aliexpress-2.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eHowever, the savings there wouldn't be that great compared to just grabbing more of the exact same ones that I already have tested in the past, so maybe I don't need to look for ways to save a few more Euros after all. I'm kind of curious about why I can't find the more \u0026quot;plain\u0026quot; variants on the Chinese sites though, maybe I'd need to look in the direction of Alibaba or something.\u003c/p\u003e\n\u003ch3\u003eSummary\u003c/h3\u003e\n\u003cp\u003eTo be honest with you, this feels like being stuck in some local maximum of \u0026quot;good enough\u0026quot;. Who knows, maybe if a friend was to let me use their super snazzy Logitech mouse I'd also splurge a bunch of money on it... and then if it fails or just gets worn out due to natural use, still feel super bummed out because at that point it's a bigger investment.\u003c/p\u003e\n\u003cp\u003eSo on some level, it happens to be the case that I treat my computer mice as one would servers in the age of cloud native apps - if one doesn't work, get rid of it and replace it with a new one, vs the older approach of nurturing a single one and hoping that it's going to be okay. I know these mice will wear out and fail, but at that point I'll be ready for it.\u003c/p\u003e\n\u003cp\u003eAnd in the day to day, they're perfectly fine for my needs.\u003c/p\u003e\n\u003ch3\u003eUpdate\u003c/h3\u003e\n\u003cp\u003eWell, technically a fake update, because I decided to look around Alibaba before posting this, but I did found a listing that's pretty much the exact kind of mouse that I want. Obviously I don't care about customization for it and while this type of mouse isn't the most popular kind on the site, this isn't the only listing:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"06-alibaba-listing.jpg\" alt=\"06-alibaba-listing.jpg\" title=\"06-alibaba-listing.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eExcuse me while I go and make an Alibaba account, this might actually cover my needs for the next decade without breaking bank.\u003c/p\u003e\n"
        },
        {
            "title": "My investments in 2025 so far",
            "date_published": "2025-10-09",
            "id": "https://blog.kronis.dev/blog/my-investments-in-2025-so-far",
            "url": "https://blog.kronis.dev/blog/my-investments-in-2025-so-far",
            "content_html": "\u003cp\u003eI thought I'd make a casual blog post and talk a little bit about the investments I have done in 2025 so far and how they've turned out. I guess sometimes talking about our finances is a bit of a faux pas, but I don't really care about it that much and I think it's an interesting topic. Secondly, I am obviously not a financial advisor and none of what I say is actual financial advice, just my personal look at things.\u003c/p\u003e\n\u003cp\u003eIn general, I live a fairly Spartan lifestyle - most of my money goes into either my savings or investments and I generally only buy the things I really need. For example, even though I use a computer daily, it's not some top of the line rig, but rather a setup with a Ryzen 7 5800X (OCed a bit and with a budget AIO, but still) and an Intel Arc B580, by all accounts pretty mid or entry level hardware. I did end up upgrading to RAM that's a bit faster at 3600 MHz and also had to move over to an NVMe boot drive, but that's because the old SSD I was running on melted. There will be a blog post about this later, but my expectations are that this exact setup will last me until 2030. Similarly, I don't really wear designer clothes or even have a car and most of the games and other entertainment I enjoy is also somewhat budget oriented (like buying games on a Steam sale a few years after release).\u003c/p\u003e\n\u003cp\u003eWhy live frugally? Because my expectations are that the economy won't always be in a very good state and I generally value the ability to not stress over what I will do financially next month or even year more than I do about having a lavish lifestyle. Secondly: I live in Latvia and as a consequence most people here, myself included, aren't doing amazingly financially. You can compare the average developer salaries in Latvia with either the rest of the EU or even US and weep for me. There's of course the ability to start your own business, but my risk tolerance is a bit too low for that, so I generally look for ways to invest money without doing too high risk investments, nor let inflation eat it all up.\u003c/p\u003e\n\u003cp\u003eAnd it doesn't end with just me. If my parents or even friends need that sort of a help, I'm happy to be able to help them - for example, one of my friends ended up with a cancer diagnosis and while she's undergoing chemo, she can't really work and in her part of the world the treatment itself is covered by insurance, but it's not like the world around her just stands still and she couldn't really cover everything without a bit of help. Where governments and the systems around them fail, people have to just pick up the slack.\u003c/p\u003e\n\u003cp\u003eBut back to the investments, in the January of this year, I put around 14'000 EUR into a bunch of different investments through \u003ca href=\"https://links.kronis.dev/h5dyw\"\u003eRevolut\u003c/a\u003e. I'm not really advertising them, but it was one of the easier ways for me to put my money into individual companies. Meanwhile, most of my other investments are handled by my bank, \u003ca href=\"https://links.kronis.dev/Xoz1WuIo5j\"\u003eSEB\u003c/a\u003e, with various sector funds - less volatility, but also less in the way of profits, or at least the potential for profits. The idea here was to also be able to compare how things have performed in the last 10 months at the end. With that out of the way, let's see what groups of companies and companies (as well as crypto and commodities) I put my money in, and how it turned out!\u003c/p\u003e\n\u003ch3\u003eTech companies, OSes\u003c/h3\u003e\n\u003cp\u003eI went for a lot of tech and tech adjacent investments. The first group that I thought of were companies that have their own OSes (and sometimes hardware). Why? Because they don't have a crazy amount of competition, getting into that space is pretty difficult and yet everyone has a computing device of some sort.\u003c/p\u003e\n\u003ch4\u003eApple\u003c/h4\u003e\n\u003cp\u003eA long while back, I ended up getting a MacBook Air and an iPhone SE for a freelance project and I have to admit that both the software and hardware are quite good! With macOS also having way more of a market share than Linux, it felt like a pretty good investment:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03-apple.jpg\" alt=\"03-apple.jpg\" title=\"03-apple.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eSadly, I appear to have been wrong about this, because their stock price has been more or less stagnating for most of the year. While their chips and the build quality of their devices are still pretty much amazing, it seems like the industry does not share my positive outlook, at least in a way that'd matter. I guess the launch of their \u003ca href=\"https://links.kronis.dev/f6cyb\"\u003eLiquid Glass\u003c/a\u003e design also wasn't anything that'd wow people, but at this point I'm not sure why most companies do some pointless redesign every few years, when things already work okay.\u003c/p\u003e\n\u003ch4\u003eMicrosoft\u003c/h4\u003e\n\u003cp\u003eMicrosoft is also very much guilty of the same kind of faffing about and recently announced that \u003ca href=\"https://links.kronis.dev/e0ppt\"\u003ethey are making having offline accounts harder\u003c/a\u003e, which I think is fairly stupid, user hostile and generally a bad move, alongside them consciously sometimes ruining what could be a good OS, alongside underwhelming services like Teams, yet it seems that they did better somehow:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04-microsoft.jpg\" alt=\"04-microsoft.jpg\" title=\"04-microsoft.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eI couldn't tell you why they did that much better, seems like things went upwards after they posted their earnings, but at the same time they had \u003ca href=\"https://links.kronis.dev/lmhos\"\u003ea lot of layoffs\u003c/a\u003e, shifted focus to AI and are still pushing a lot of Office 365 services. When you think about it, it's kind of upsetting that taking away the livelihoods of thousands of people is described as \u0026quot;savings\u0026quot;.\u003c/p\u003e\n\u003ch3\u003eTech companies, CPUs \u0026amp; GPUs\u003c/h3\u003e\n\u003cp\u003eI also decided to invest into companies that make their own hardware: CPUs and GPUs in particular. Again, it's a case of those largely just being too big to fail despite any sorts of bad product launches or trends of stagnation. You can be pretty much sure that there will be \u003ca href=\"https://links.kronis.dev/mb5c0\"\u003ebailouts\u003c/a\u003e if it's within the strategic interests of some of the largest governments in the world.\u003c/p\u003e\n\u003ch4\u003eNVIDIA\u003c/h4\u003e\n\u003cp\u003eThe first idea was to go for Nvidia. I don't really like them as a company much, because their drivers on Linux aren't known to be great (yet remain the best option for hassle free enjoyment of games on Windows), though are more or less printing money due to the AI boom that we're in the middle of:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"05-nvidia.jpg\" alt=\"05-nvidia.jpg\" title=\"05-nvidia.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eThe overall trend is positive and they did make YTD +40% but the issue is that I'm a bit late to the party. The bubble hasn't quite popped yet and we're not in a financial crash, but even if I can make a profit off of the state of things, it won't be as much as those that bought in early:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"05-nvidia_2.jpg\" alt=\"05-nvidia_2.jpg\" title=\"02-nvidia_2.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eAs usual, luck isn't fully on my side and the right time to invest in Nvidia might have been somewhere in 2023, before they got really big. Imagine if instead of 40% profits I had instead made over 1000%. But hey, I don't think anyone could have predicted just how big AI has gotten (sometimes undeservedly so).\u003c/p\u003e\n\u003ch4\u003eAMD\u003c/h4\u003e\n\u003cp\u003eNow, AMD is a bit more of a darling company. Despite their pricing strategy for their consumer products sometimes being \u0026quot;Intel - 50 USD\u0026quot;, or them never missing a chance to miss a chance (just look at how their ROCm struggles to catch up to CUDA), at least they're trying to provide viable alternatives to the overpriced Nvidia and often power hungry Intel. Their Ryzen chips are still some of the best you can get.\u003c/p\u003e\n\u003cp\u003eUnfortunately, their stock prices were stagnating for most of the year, however very recently saw a significant jump:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"06-amd.jpg\" alt=\"06-amd.jpg\" title=\"06-amd.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eAll of that actually came from a recent \u003ca href=\"https://links.kronis.dev/p5ktk\"\u003epartnership with OpenAI\u003c/a\u003e, where AMD will ship them a lot of GPUs. It increasingly feels like for all of these companies, the consumer segment is close to a rounding error, but if it gives them more money to work with, I guess I should cheer them on. Maybe in a few years ROCm will also be good and honestly if the RX 9060 XT 16 GB GPU supported 4 monitors instead of just 3, that'd probably be my next upgrade from my Intel Arc B580 (or whatever would be the next series that's out by 2030).\u003c/p\u003e\n\u003ch4\u003eIntel\u003c/h4\u003e\n\u003cp\u003eNow Intel's a bit of an odd one. I largely decided to invest in them both because they're too large to fail, even after \u003ca href=\"https://links.kronis.dev/19wlp\"\u003ethrowing Pat Gelsinger out\u003c/a\u003e and their new CEO largely saying that they \u003ca href=\"https://links.kronis.dev/b3855\"\u003ewon't even try to be competitive\u003c/a\u003e, and the blunders of the 13th and 14th gen, as well as the \u003ca href=\"https://links.kronis.dev/wbp7w\"\u003esomewhat disappointing Core Ultra\u003c/a\u003e launch, as well as layoffs... actually, there is so much wrong with Intel, yet somehow they're doing BETTER than Nvidia from the lows:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"07-intel.jpg\" alt=\"07-intel.jpg\" title=\"07-intel.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eIn this case, it definitely was a higher risk investment that paid off, even if you zoom out and look around, it doesn't reflect all that well on the long term performance of the company at all:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"07-intel_2.jpg\" alt=\"07-intel_2.jpg\" title=\"07-intel_2.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eA part of my decision to invest in them might have been the fact that their Intel Arc GPUs provide some much needed competition to the other companies. A part of it is the understanding that if they go under and the only serious manufacturer of x86 chips becomes AMD, then they'll stagnate similarly if not worse than Intel did, because there will be very little preventing them from cranking up the prices.\u003c/p\u003e\n\u003cp\u003eBut a part of it is the understanding that maybe they could be a good company if they stopped shooting themselves in the foot at every opportunity, worse than AMD does. For example, their Core Ultra 285K chip came out at a price of 589 USD. By all accounts, despite the new architecture and lower power consumption, it was more or less a sidegrade - sometimes being outpaced both by their prior 14th gen chips (that had really high power consumption and ran hot), as well by the current AMD chips, despite otherwise scoring pretty well in various productivity benchmarks.\u003c/p\u003e\n\u003cp\u003eYou know when most consumers wouldn't have been angry at them? If they competed with AMD on price: their 9800X cost 499 USD on release and their 9700X cost 359 USD. What if Intel stopped being delusionally greedy and released their new CPU at a price of 480 USD? Ask yourself, what is better: to make less profit, or to sell no chips at all? Suddenly, there'd be a new platform that's really good at productivity, okay at gaming, has good power draw and thermals... and they just didn't do that.\u003c/p\u003e\n\u003ch3\u003eServices companies\u003c/h3\u003e\n\u003cp\u003eThe next group of companies, and perhaps the largest, that I invested in were various services based companies. Sometimes they offer web related services, other times it's about e-commerce, game engines, or even consulting. Even if I can't predict the future, I can at least diversify enough to make most individual failures (short of an economy collapse) tolerable.\u003c/p\u003e\n\u003ch4\u003eShopify\u003c/h4\u003e\n\u003cp\u003eFirst up, I wanted something kinda \u0026quot;legacy\u0026quot; - an established company that's been around for a while, that I've heard of even if I don't use their services directly a lot. Shopify fits the bill and in general did pretty well:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"08-shopify.jpg\" alt=\"08-shopify.jpg\" title=\"08-shopify.jpg\"\u003e\u003c/p\u003e\n\u003ch4\u003eGoDaddy\u003c/h4\u003e\n\u003cp\u003eAnother such company was GoDaddy, they do a bunch of web hosting, sell domains and so on. There's lots of other companies like that out there, I've personally moved from NameCheap and Porkbun over to \u003ca href=\"https://links.kronis.dev/oqokw\"\u003eINWX\u003c/a\u003e because I wanted something EU based instead. But as far as publicly traded companies go, they seemed like a safe bet. Surprisingly, I was wrong:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"09-godaddy.jpg\" alt=\"09-godaddy.jpg\" title=\"09-godaddy.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eWho knows, maybe I should have left the investments alone for a few more years, but it seems that their value actually peaked around the start of 2025 and if I sold now I'd basically be doing a \u0026quot;buy high, sell low\u0026quot; strategy, which is not really what you want to do:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"10-godaddy-5y.jpg\" alt=\"10-godaddy-5y.jpg\" title=\"10-godaddy-5y.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eIn other words, it might actually be a good idea to keep the investment there and see whether they can recoup the value... or maybe they'll just stagnate and never recover, who knows. I can't really think of that many news with them recently either.\u003c/p\u003e\n\u003ch4\u003eCloudflare\u003c/h4\u003e\n\u003cp\u003eOn the complete opposite end of the spectrum, Cloudflare seems to have done amazingly this year so far, they actually had the most increase in stock value out of all my investments! My rationale for picking them is that they're also largely too big to fail and are too entrenched in the structure of the Internet at some point, providing a bunch of services for an ever increasingly large part of it, to the point where they've been critiqued as holding it hostage in a way.\u003c/p\u003e\n\u003cp\u003eRegardless, their figures are great:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"11-cloudflare.jpg\" alt=\"11-cloudflare.jpg\" title=\"11-cloudflare.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eI'm now pondering whether I should just sell and cash in, or wait for a bit longer and see what happens.\u003c/p\u003e\n\u003ch4\u003eAccenture\u003c/h4\u003e\n\u003cp\u003eI also wanted a company that does software development for others, more or less an organization that does consulting. Accenture fit the bill and I thought that they have a lot of government contracts and such despite their occasional bit of controversy, but I'm not really here to talk about their ethics, or even the idea of consulting in general, vs just building the damn software in-house.\u003c/p\u003e\n\u003cp\u003eA part of picking them was that they also have a large presence in my country, over here in Latvia we don't really have a strong software development scene with investors and such, so what ends up happening is that a lot of the work you do get is basically being a body shop of software devs that write code for large multinational orgs and make pennies on the dollar. Not a dignified existence, but hey, it pays the bills.\u003c/p\u003e\n\u003cp\u003eI've actually personally seen some investors look at a Latvian startup that wants to compete with Bolt, decide that they want a higher likelihood of success and the whole initiative dying in the water then and there, while it could still be developed for peanuts, compared to what would be needed with your typical unicorn in SV.\u003c/p\u003e\n\u003cp\u003eIf we can still build software and can do so way cheaper than in the US, then why do we get relegated to doing mercenary work instead of attracting actual investments, having a good tech scene and propping up the economy in ways that would sure as hell be more useful in the long term than some AI bubble!?\u003c/p\u003e\n\u003cp\u003eAnyways, I digress, here's Accenture:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"12-accenture.jpg\" alt=\"12-accenture.jpg\" title=\"12-accenture.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eThey kinda bombed. That said, their stock prices have been volatile for years:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"13-accenture-5y.jpg\" alt=\"13-accenture-5y.jpg\" title=\"13-accenture-5y.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eMore so than with the other companies, actually. I guess if I keep the investments in them for a few years they might as well bounce back, since I feel like they still very much fit the bill of being too big to fail.\u003c/p\u003e\n\u003ch4\u003eUnity\u003c/h4\u003e\n\u003cp\u003eUnity's an interesting one!\u003c/p\u003e\n\u003cp\u003eI hate Unreal Engine with a passion because it's ruining games with their horrible unoptimized technology that makes most hardware out there struggle a lot and at this point both upscaling and framegen are necessary to run the slop games that studios put out, while \u003ca href=\"https://links.kronis.dev/ypm6a\"\u003etrying to gaslight gamers\u003c/a\u003e about their bad games somehow actually supposedly being good and it being people's eyes that are wrong.\u003c/p\u003e\n\u003cp\u003eFuck that engine, fuck that trend, fuck the developers saying that crap, get out of here. Instead of people online ranting incorrectly about how inclusion or whatever is ruining games, the real stuff ruining them is right under your noses - studios prioritizing being able to ship faster instead of optimizing games properly or picking reasonable tech, like an extreme case of \u003ca href=\"https://links.kronis.dev/niqkn\"\u003eWirth's law\u003c/a\u003e, except mandated by a bunch of managers.\u003c/p\u003e\n\u003cp\u003eSo, Unity is not Epic and that's reason enough to like them at least a bit. The underlying tech has historically been pretty good, they support a lot of devices, from mobile phones to consoles and desktop platforms (and even web), the engine is nice to use and uses a sane language like C#, has a frickload of assets available in a store and historically has resulted in both a lot of bad games (because of how easy it is to make one), as well as pretty notable hits, which shows in their stock price:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"14-unity.jpg\" alt=\"14-unity.jpg\" title=\"14-unity.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eExcept no, I'm kind of lying here, look at the last 5 years:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"14-unity_2.jpg\" alt=\"14-unity_2.jpg\" title=\"14-unity_2.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eWhile the tech itself has been pretty okay, the org behind it has been plagued by all sorts of issues. From \u003ca href=\"https://links.kronis.dev/4j48w\"\u003eGigaya being cancelled\u003c/a\u003e, which was supposed to be a full game that shows how to make good stuff with the engine - imagine what sort of a message that sends \u0026quot;Oh yeah, literally impossible, so we cancelled it.\u0026quot;, to technological things like \u003ca href=\"https://links.kronis.dev/i8kuj\"\u003eDOTS\u003c/a\u003e being half baked for the longest time, and them stumbling about with their \u003ca href=\"https://links.kronis.dev/xgjxw\"\u003erender pipelines\u003c/a\u003e and largely the feeling being that they don't know what they want the engine to be, leading to a lot of half baked functionality that almost sort of works, but not really.\u003c/p\u003e\n\u003cp\u003eAdd to that their \u003ca href=\"https://links.kronis.dev/0439x\"\u003emerger with IronSource\u003c/a\u003e which drew a lot of negative press, alongside their \u003ca href=\"https://links.kronis.dev/x4pAMfKeqI\"\u003eUnity Runtime Fee\u003c/a\u003e which is now cancelled after literally everyone dragged them through the mud, all the way to \u003ca href=\"https://links.kronis.dev/BUqtnW9941\"\u003etheir CEO being fired\u003c/a\u003e because of it.\u003c/p\u003e\n\u003cp\u003eBut the thing is, much like with Intel, everyone is wrong. Why? Because even \u003ca href=\"https://links.kronis.dev/vZA6vmHCdJ\"\u003eback when the Runtime Fee happened, I wrote plainly that it was just a badly concealed sales funnel\u003c/a\u003e - all of those extreme numbers that people had outrage about would have gone away if you'd just upgrade to their higher subscription plan, even aside from how stupid the idea of the runtime fee was and how bad their messaging around that was. What ruined them wasn't just being greedy, it was the community having the worst possible interpretation, content creators chasing drama and blowing it out of proportion and the company might never recover from that.\u003c/p\u003e\n\u003cp\u003eInstead, the engine genuinely could be good and more or less be the \u0026quot;default\u0026quot; engine for the next decade, much how Blender is more or less the default 3D modelling program outside of either legacy or niche orgs (most gamedev studios kind of fitting that description, what an odd world). Their 3D support infinitely more mature than the likes of Godot and doesn't run like crap like Unreal does, they have the asset store, the docs, the examples and the overall ecosystem. If they just locked in for 5 years and did good work on the core tech, things would be good.\u003c/p\u003e\n\u003cp\u003eBut of course, in reality, they'll probably be about as greedy as Intel, because we can't have nice things. For all I know, the company might as well be in a slow death spiral and we'll all need 300W GPUs to just run the latest Unreal games with upscaling from 720p and framegen from 30 FPS.\u003c/p\u003e\n\u003ch4\u003eAmazon\u003c/h4\u003e\n\u003cp\u003eThen there's Amazon:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"15-amazon.jpg\" alt=\"15-amazon.jpg\" title=\"15-amazon.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eI don't really have much to say for them, aside from the fact that I thought that they'd do better by kissing up to the current US administration. Guess not quite, go figure.\u003c/p\u003e\n\u003ch4\u003eAirbus\u003c/h4\u003e\n\u003cp\u003eI also picked Airbus and after the US election results maybe should have invested more heavily in EU companies (though oddly enough I couldn't find \u003ccode\u003eAIR1\u003c/code\u003e on Yahoo Finance, though AIR.PA did similarly well), but they had a pretty nice performance overall:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"16-airbus.jpg\" alt=\"16-airbus.jpg\" title=\"16-airbus.jpg\"\u003e\u003c/p\u003e\n\u003ch3\u003eCommodities\u003c/h3\u003e\n\u003cp\u003eI also saw the fact that Revolut allows you to trade in commodities like gold or other rare materials, except there very much was a disclaimer about it being quite unregulated. That said, I decided that since I'm doing 1000 EUR (or in the case of US stocks, 1000 USD) per one investment, I might as well risk some money on those as well.\u003c/p\u003e\n\u003ch4\u003eGold\u003c/h4\u003e\n\u003cp\u003eFor that, I picked gold, which is the only thing that has been more or less consistently going up, maybe aside from the Cloudflare investments:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"17-gold.jpg\" alt=\"17-gold.jpg\" title=\"17-gold.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eI wonder how it will look in a year, I might also keep this investment around, since I'm not sure what should happen in the world to make gold lose its value. Oh yeah, since I'm not doing screenshots of the Revolut app (for whatever reason, they won't run well in a landscape orientation on my iPhone), I looked up the data in another website rather than Yahoo Finance, since they didn't let me find Gold prices. This does seem to more or less match what's in Revolut, though, so good enough.\u003c/p\u003e\n\u003ch3\u003eCrypto\u003c/h3\u003e\n\u003cp\u003eNow, if I'm comparing as many ways of throwing your money into the void and hoping some of it comes back out, I did also decide to do some crypto investments. I guess like with many things, I have a bit of an utilitarian stance when it comes to it. It's an okay way of making payments without involving the banking system too much (or middlemen like Wise when doing transfers across the borders, like why the hell should I give up so much of the money I want to send to a friend to some company for just using their service) and some cryptocurrencies could be a deflationary means of storing assets... but in practice also carries pretty great risks and high volatility.\u003c/p\u003e\n\u003cp\u003eAt the same time, there's a nearly infinite amount of rugpulls, that at this point I'm not even sure whether they're money laundering, a way to take bribes out in the open, or just profit off of stupid people, look at stuff like this:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"19-trump.jpg\" alt=\"19-trump.jpg\" title=\"19-trump.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eIn my case, I went for some regular crypto coins that are more mainstream, I didn't intend to speculate a lot but rather take the \u0026quot;hodl\u0026quot; approach and just put some money in a wallet on chain, \u003ca href=\"https://links.kronis.dev/emo91\"\u003emake backups of said wallet\u003c/a\u003e and see where it all is in a year or ten.\u003c/p\u003e\n\u003ch4\u003eBitcoin (BTC)\u003c/h4\u003e\n\u003cp\u003eMy first choice was Bitcoin. I briefly looked at Bitcoin Cash, which was an older project that also attempted to have low transfer fees, but with the \u003ca href=\"https://links.kronis.dev/sBiXdOk3FR\"\u003eLightning network\u003c/a\u003e mainstream Bitcoin no longer really has that problem and BCH has largely been displaced:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-btc-eur.jpg\" alt=\"01-btc-eur.jpg\" title=\"01-btc-eur.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eOverall, this went both a bit better and a bit worse than expected. Better, since there wasn't a frickload of volatility and I didn't end up with no money whatsoever. Worse, because there also weren't any significant profits, either.\u003c/p\u003e\n\u003cp\u003eActually, by now you've probably noticed how near the April pretty much everything crashed. It was largely due to the tariffs that Trump announced and what the administration did to the economy. Personally, I'm not even sure whether it was an attempt at throwing things off balance so their rich buddies could capture even more of the wealth both in the country and the global economy, an attempt at strongarming other nations, including the EU, to trade on terms that are more favorable to them (no, we don't want their cars), or a perverse mix of both.\u003c/p\u003e\n\u003cp\u003eIn the end, however, what I got was instead of describing a bit of gradual growth with some events that lead to more profits, was a massive dip in pretty much everything in the middle. Imagine opening up the app and one day seeing this:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"17-everything-in-the-red.jpg\" alt=\"17-everything-in-the-red.jpg\" title=\"17-everything-in-the-red.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003ePut plainly, I don't like that administration, I don't like what they're doing with the social fabric of the country, I've heard from my trans friends that it's no longer safe for them there, nor do I think it's safe for anyone who looks remotely Mexican, or even anyone who wants a democratic country instead of a dictatorship. They were supposed to be a partner nation to the rest of the West, instead they do things like withdraw from the Paris Agreement, mock NATO, try to screw over our economy and are basically good for nothing. The post I wrote about what things might look like by 2030 has predictions more or less coming true... except we're still just in 2025.\u003c/p\u003e\n\u003cp\u003eIt was inevitable that this would have some politics in it, but the takeaway here from the image above is that even crypto is not immune to those sorts of shifts - see the dip in price even against Euro, at the time of the tariffs? In other words, if we end up having another crash like in 2008, be it due to AI or something the administration pulls, it's likely that crypto investments won't be some panacea either.\u003c/p\u003e\n\u003ch4\u003eEthereum (ETH)\u003c/h4\u003e\n\u003cp\u003eI also did some Ethereum investments because way back they seemed like a cool project with the potential for all sorts of programmatic use cases:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02-eth-eur.jpg\" alt=\"02-eth-eur.jpg\" title=\"02-eth-eur.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eRight now, however, they didn't perform much better or worse than Bitcoin. As with most crypto (that survives for years and isn't a rugpull), it very much feels like the earlier you get in, the better things will be. I guess the same applies to rugpulls as well (get in and out before the \u0026quot;pull\u0026quot; part), but in general I'd much prefer most of my investments to be in companies that actually make stuff.\u003c/p\u003e\n\u003cp\u003ePersonal preference, though. There is definitely some strength in a diversified portfolio.\u003c/p\u003e\n\u003ch3\u003eSummary\u003c/h3\u003e\n\u003cp\u003eTo sum it all up, I was actually thinking about how best to visualize the data. I recently moved over to using \u003ca href=\"https://links.kronis.dev/gh38m\"\u003eOnlyOffice\u003c/a\u003e instead of LibreOffice, because I found out that the software package is actually made in Latvia, so I was pretty curious to try it out!\u003c/p\u003e\n\u003cp\u003eI will say that it looks nice and is quite pleasant to use, but definitely they're not a dedicated data visualization solution, the charts are easy to create but not super advanced. Either way, without jumping into writing code, I managed to find a type of graph (radar chart) that was pretty useful for what I'd like to show:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"18-summary.jpg\" alt=\"18-summary.jpg\" title=\"18-summary.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eThe red shape is the starting price of each investment in EUR (some were 100 EUR, others were 1000 USD, so a bit less), whereas the green shape is how much money each of the investments made, loosely in the same order as described above.\u003c/p\u003e\n\u003cp\u003eI haven't seen this visualization be used for this type of dataset that often, but I think it's pretty useful - where red is above green, I'm losing money, whereas the polygon reaches out the furthest away from the center, the highest the current value is.\u003c/p\u003e\n\u003cp\u003eHere you can very much see, that the most profitable investments ended up being:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eCloudflare: 1681 EUR\u003c/li\u003e\n\u003cli\u003eAMD: 1633 EUR\u003c/li\u003e\n\u003cli\u003eIntel: 1622 EUR\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThere were also some in the middle that performed pretty decently:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eUnity Software: 1380 EUR\u003c/li\u003e\n\u003cli\u003eShopify: 1336 EUR\u003c/li\u003e\n\u003cli\u003eGold: 1279 EUR\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThere were also some that made a profit, but not that much at all:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eBitcoin: 1160 EUR\u003c/li\u003e\n\u003cli\u003eEthereum: 1160 EUR\u003c/li\u003e\n\u003cli\u003eNvidia: 1146 EUR\u003c/li\u003e\n\u003cli\u003eAirbus: 1105 EUR\u003c/li\u003e\n\u003cli\u003eMicrosoft: 1062 EUR\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eAnd then there's some that just lost money:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eApple: 915 EUR\u003c/li\u003e\n\u003cli\u003eAmazon: 866 EUR\u003c/li\u003e\n\u003cli\u003eAccenture: 597 EUR\u003c/li\u003e\n\u003cli\u003eGoDaddy: 589 EUR\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eYou'll notice that some of those figures are a bit different from the images above, but that's because this data is straight from Revolut and also based on currency conversions as necessary, plus some fees apply. Either way, that should be enough to demonstrate the principle. I might have also sold some of the crypto along the way, but this is how it'd look if I had held onto it.\u003c/p\u003e\n\u003cp\u003eIn general, I think that investing is a good idea, since it can help you prevent your assets by being eaten up by inflation, if you don't intend to spend all of that money. Investing into individual companies carries higher risk than just sector funds, whereas crypto and commodities will probably be similarly hit or miss. I'd say it all depends on what your risk tolerance is like.\u003c/p\u003e\n\u003cp\u003eAs for what to do now, I'll probably keep the money in there and see whether I need to sell the investments at some point. As I said, it's there primarily for a rainy day, though I did splurge on a 35 EUR mechanical keyboard recently (yeah, a surprisingly cheap one and with Outemu Brown switches, I hope it's going to serve me well for a while).\u003c/p\u003e\n\u003cp\u003eAs for Revolut, I had a bit of a mixed experience: the app itself is nice, annoys me sometimes with making me confirm my on-chain transactions when I don't want to let them keep my crypto for me, but instead want to transfer it to my own wallet (\u0026quot;suspicious transaction\u0026quot;, pffft), and the fact that they display the current prices in either EUR or USD but not in the currency that you originally purchased shares in is a bit annoying. Oh well, they're better than nothing. Sure beats keeping all of my money just in SEB.\u003c/p\u003e\n"
        },
        {
            "title": "The great container crashout",
            "date_published": "2025-08-31",
            "id": "https://blog.kronis.dev/blog/the-great-container-crashout",
            "url": "https://blog.kronis.dev/blog/the-great-container-crashout",
            "content_html": "\u003cp\u003eAt the start of this month, I had an outage where more or less everything I host went down. In part, it was due to failing builds, in part due to bad networking, in part due to broken cloud dependencies, alongside software just being plain finicky. The good news is that I resolved most of the issues, everything was back up and running and eventually I even figured out \u003ca href=\"https://links.kronis.dev/g0j7o\"\u003ethat you need to setup static routes if you want servers on Contabo to be able to talk to one another directly\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003eUnfortunately, this weekend, everything broke again. My homepage was down. My blog was down. I couldn't even connect to some of my servers. For a second I thought that maybe it was finally the time for me to be hacked and someone to steal all of my data or something and take over the servers... but nope, it was just more of software being obtuse garbage. Today, I'll tell you a bit more about the perils of self-hosting, though admittedly situations like this make me write increasingly profanity laden posts, which I don't normally do.\u003c/p\u003e\n\u003ch3\u003eTailscale is broken\u003c/h3\u003e\n\u003cp\u003eEither way, allons-y, everything is down:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-no-containers.jpg\" alt=\"01-no-containers\" title=\"01-no-containers.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eNo containers are running, quite possibly because the servers can't reach the leader node for the Docker Swarm setup, whereas Tailscale shows that we're logged out, which explains that. I've also got host records setup so that nodes can access sites hosted on each other directly through the Tailscale IP addresses (if I ever decide to cut off public access to anything I host, so things would keep working), which is a bit of a problem because suddenly they also can't pull new container versions.\u003c/p\u003e\n\u003cp\u003eWhy are we logged out? The most I've done is update the Tailscale version a few times, but it's kind of horrible when such a base networking component is pulled out from under your feet, not unlike someone doing that to a rug that you're standing on. It seems like there are \u003ca href=\"https://links.kronis.dev/b6mou\"\u003ea few people experiencing similar issues\u003c/a\u003e with the latest version, but I honestly couldn't tell you the cause at the moment.\u003c/p\u003e\n\u003cp\u003eEither way, we log in:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02-tailscale-login.jpg\" alt=\"02-tailscale-login\" title=\"02-tailscale-login.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eGreat! Except it would be nice if a service that's supposed to be fairly automated wouldn't randomly need human intervention. Nothing changed about the nodes in the question, yet I was still logged out. I didn't even get an e-mail along the lines of: \u0026quot;Hey, nodes A, B and C have been disconnected from the tailnet due to reason X, please log back in if necessary.\u0026quot; Nor could I even connect to some of the nodes because me primarily using Tailscale to access them in the first place.\u003c/p\u003e\n\u003cp\u003eI did work around all that, but it's pretty clear that Tailscale can't be the backbone of all my networking.\u003c/p\u003e\n\u003ch3\u003eDocker is broken\u003c/h3\u003e\n\u003cp\u003eExcept even after a server restart (just in case), the Docker service shows \u003ccode\u003einactive (dead)\u003c/code\u003e under its status. It's not even that the containers won't run, but rather the solution to run the containers is dead. Another server restart did eventually help in all of those cases across multiple servers, though I've no idea why another one was needed:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03-docker-is-dead.jpg\" alt=\"03-docker-is-dead\" title=\"03-docker-is-dead.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eOnce I got Docker up and running, it was time to run into our beloved old friend, tasks getting rejected on the servers:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04-tasks-rejected.jpg\" alt=\"04-tasks-rejected\" title=\"04-tasks-rejected.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eThe good news is that the nodes can talk to one another and actually try to schedule the tasks, so the Swarm network itself is okay after fixing Tailscale. The bad news? Well, the fact that it's complaining about containers not existing:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"05-the-dreaded-no-such-image.jpg\" alt=\"05-the-dreaded-no-such-image\" title=\"05-the-dreaded-no-such-image.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eThis is an issue that has been pestering me for years at the most inopportune of times. Sometimes an aggressive Docker cleanup removes an image on the system that should be there and as a consequence the container can't be run, other times the registry that you try to pull the image from is down and you can't download it, I've also seen cases where the image is present locally, but running it is still refused because the remote registry is unreachable.\u003c/p\u003e\n\u003cp\u003eActually, it looks very much like this:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"06-the-image-is-there.jpg\" alt=\"06-the-image-is-there\" title=\"06-the-image-is-there.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eWhich is funny, because we also get the other class of issues right alongside that:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"07-but-not-on-some-servers.jpg\" alt=\"07-but-not-on-some-servers\" title=\"07-but-not-on-some-servers.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eIn such circumstances, the reasonable thing is to at least try to pull the latest version from said registry, except that we get a proxy error when we try to do that:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"08-pulling-also-fails.jpg\" alt=\"08-pulling-also-fails\" title=\"08-pulling-also-fails.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eNow, why would there be a proxy error?\u003c/p\u003e\n\u003ch3\u003eIt's always DNS (and web servers)\u003c/h3\u003e\n\u003cp\u003eMy worst fear was that it'd be some sort of a byzantine error, where the ingress image has gotten wiped due to an automated cleanup and can't be pulled from the registry, because such an ingress is needed in front of the registry, meaning that none of it can be brought up as it is. One solution is to use a public Nginx/Apache2/Caddy image in front of the registry, though that's problematic when you want to use the same images in front of 20-30 sites across all your servers and build them yourself.\u003c/p\u003e\n\u003cp\u003eAnother alternative is just to export it as a ZIP that you can re-import into the system manually if ever needed, instead of relying on those automated cleanups not ruining things for you sooner or later (you're kind of forced to run them, because if you build containers often, without them you'll run out of disk space fairly soon, at least on your average VPS). You can even name the image something like \u003ccode\u003e127.0.0.1/my-project/my-image\u003c/code\u003e so that Docker wouldn't try to reach out to your registry at all, in addition to perhaps deploying them through SFTP/SCP as a push, instead of a pull.\u003c/p\u003e\n\u003cp\u003eI'm actually doing something similar at work, which works nicely, but haven't gotten around to fixing my own setup. The good news (somewhat) is that this time it was a plain DNS error inside of the Apache2 container:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"09-proxy-errors-still-happen.jpg\" alt=\"09-proxy-errors-still-happen\" title=\"09-proxy-errors-still-happen.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eThis is also puzzling. I got the registry image working BUT it wasn't being resolved inside of Apache2 despite it existing. My best guess is that either the web server container cached the fact that no such container name could be resolved (internal domain name, such as \u003ccode\u003emy-registry\u003c/code\u003e), or that expecting to be able to \u0026quot;hot plug\u0026quot; new containers doesn't always work that well, in circumstances that are both unclear and inconsistent, because that usually works.\u003c/p\u003e\n\u003cp\u003eRegardless, after restarting the web server, everything suddenly started working:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"10-after-a-few-restarts.jpg\" alt=\"10-after-a-few-restarts\" title=\"10-after-a-few-restarts.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eIt's a bit unfortunate that so often it boils down to:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eHave you tried turning it on and off again?\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003ebecause I'm quite sure that for a long time people were talking both about graceful degradation, as well as being able to bring systems up after failures, so that they'd be more self-healing. This experience right here proves that software is still quite brittle and that that line of reasoning, in practice, sometimes ends up being plain lies.\u003c/p\u003e\n\u003ch3\u003eWhat to do about it\u003c/h3\u003e\n\u003cp\u003eThe first step, is to acknowledge that I can't fully automate everything. Not with the time and resources (or lack thereof) that I'm dealing with at the present moment. The simplest step is to disable the overly aggressive Docker cleanup:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"11-pruning-might-be-an-issue.jpg\" alt=\"11-pruning-might-be-an-issue\" title=\"11-pruning-might-be-an-issue.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eThe \u003ca href=\"https://links.kronis.dev/ciivz\"\u003ecommand above\u003c/a\u003e essentially resolves to:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eremove all unused containers, networks and images that are dangling (untagged and not in use)\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003e-f\u003c/code\u003e skip confirmation prompt, which is what we want when we automate things\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003e-a\u003c/code\u003e remove all unused images as well, basically the ones that do have tags, but aren't in use\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003e--volumes\u003c/code\u003e do the same for any volumes that might be left over after the containers that used them are gone\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThe problem with the above it, is that it's lies. You saw it yourself, the ingress containers were removed even though they were very much scheduled to run on the nodes and just weren't up for whatever reason. The fact that there were no running containers doesn't mean that the Swarm cluster should suddenly forget what should run where or even the fact that when it was LAST CONTACTED something was supposed to be running on the node and therefore no removals should be allowed.\u003c/p\u003e\n\u003cp\u003eBut they were. This has happened multiple times in the past. I don't even care whether it counts as a bug or some badly described behavior - it's stupid and I want it gone. So now, if an image is tagged, I'll only ever remove it manually.\u003c/p\u003e\n\u003cp\u003eActually, I could even disable the untagged cleanups and just clean up manually, when there are too many intermediate build images or what have you. But first, this should be a pretty decent compromise to hopefully break the setup less often.\u003c/p\u003e\n\u003cp\u003eSecondly, I also disabled the automated Nexus cleanup:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"15-nexus-cleanups.jpg\" alt=\"15-nexus-cleanups\" title=\"15-nexus-cleanups.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eThat mechanism never worked well. I've seen Nexus instances gleefully ignore the fact that there are image layers stored in the blob stores and even when the cleanup pattern is \u003ccode\u003e*\u003c/code\u003e, nothing is cleaned up (with all of the \u003ca href=\"https://links.kronis.dev/iv2ln\"\u003ecorrect tasks for cleanup\u003c/a\u003e run in order, multiple times).\u003c/p\u003e\n\u003cp\u003eOn the other hand, if there's a cleanup policy to delete images older than a month and you try to run a build just slightly ahead of that cleanup schedule, you're not exactly leaving yourself a lot of leeway if the builds decide to fail due to Tailscale rugpulling you:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"14-build-failures.jpg\" alt=\"14-build-failures\" title=\"14-build-failures.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eThe same rugpull that also prevents e-mails about those failures going out, as well as those by the uptime monitoring system. I expected my sites or the web server to fail, not the stupid Docker setup. It seems like my choices are either to fork over money for an external monitoring solution, or to just accept the fact that my monitoring setup will never be perfect, or just run it against an e-mail address that belongs to Google or some other 3rd party.\u003c/p\u003e\n\u003cp\u003eBack to my previous point, Nexus is also garbage, don't use it. Except you don't get the choice because there aren't that many alternatives out there. You either \u003ca href=\"https://links.kronis.dev/33krs\"\u003epay Docker Hub\u003c/a\u003e to host your private repos, or setup something a bit more complex like \u003ca href=\"https://links.kronis.dev/3vw1s\"\u003eHarbor\u003c/a\u003e or just struggle. We need a version of the \u003ca href=\"https://links.kronis.dev/07azw\"\u003eregular Registry image\u003c/a\u003e, but with a decent auth and management solution around it. I'll just keep dreaming.\u003c/p\u003e\n\u003ch3\u003eThe timeline\u003c/h3\u003e\n\u003cp\u003eThis time around, the issues weren't present for all that long, but once more I feel like it was a combination of factors and brittle software that just shouldn't have happened in the first place:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"12-the-timeline.jpg\" alt=\"12-the-timeline\" title=\"12-the-timeline.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eEven the timeline itself is incomplete, because the uptime monitoring software was in the same cluster that went down, hence you see the difference between there being no data about the last day whatsoever and then suddenly errors appearing:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"13-monitoring-output.jpg\" alt=\"13-monitoring-output\" title=\"13-monitoring-output.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eAlas, it cannot ever be truly self-hosted, if I don't have enough nodes for that and want to run everything as a part of the same cluster.\u003c/p\u003e\n\u003ch3\u003eSummary\u003c/h3\u003e\n\u003cp\u003eIn summary, this sucks. It's simple enough for me to understand on a surface level and fix, but I just wanted to write some code over the weekend, not debug stupid things like that all over again. At this rate, I'll probably have gray hair before I hit the age of 30. I think the clearest solution here is to simplify things further, until such issues become more unlikely. What's my biggest step towards that? Well, I recently looked at \u003ca href=\"https://links.kronis.dev/xm36a\"\u003ethe Hetzner server auctions\u003c/a\u003e and noticed that there are some pretty good offerings on dedicated servers:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"16-hetzner-server-auction.jpg\" alt=\"16-hetzner-server-auction\" title=\"16-hetzner-server-auction.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eThe fact of the matter is that I don't want something hyper-scalable for my personal setup. I just want a large box somewhere with a stable OS on it, alongside a pretty good backup setup that pulls data back to my homelab boxes. Something in the cloud that has good networking and can run 24/7 unlike my own servers in the apartment which sleep roughly when I do. Then, inside of that one larger dedicated server, I can run a few VMs that will represent prod and dev \u0026quot;servers\u0026quot;, just so that me messing with the ingress or any software on either won't break everything.\u003c/p\u003e\n\u003cp\u003eIn that regard, having a single dedicated server will actually be better towards making the underlying infrastructure pretty simple, while also giving me a decent amount of bang for the buck, plenty of storage so I can tell those cleanup policies to frick off and do stuff once a month or two manually, alongside still being able to limit the blast range of me messing things up.\u003c/p\u003e\n\u003cp\u003eAlso, since I'm not sponsored by Hetzner or anything, I can tell you that the internal UI for those dedicated servers absolutely sucks, but the prices and the hardware seem okay. In that regard, they're not unlike Contabo, though they do have better reputation, presumably due to their support and track record. Either way, I also look forwards to the day when I get rid of the pull approach for getting Docker images and can maybe even throw Nexus out into the trash and just send images directly to the server over SCP/SFTP, maybe even use something like \u003ca href=\"https://links.kronis.dev/x9caz\"\u003eunregistry\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003eBut until then, I'm crashing the heck out - weekend is kinda ruined and I didn't even learn that much useful, outside of the fact that Tailscale has the odd failure mode of sometimes just disconnecting your servers and asking you to reconnect manually. I love that Tailscale exists, but WTF. Oh and also that I was not at all wrong in calling a lot of the software out there brittle.  Well, okay, maybe the thing that's simplest for me would be to open my wallet and make all of the technical bit someone else's problem and just focus on building software, but that's slightly beyond my means for now, so I learn, I tinker and sometimes I suffer.\u003c/p\u003e\n\u003cp\u003eWho knows, maybe I'm damned to experience the darker side of \u003ca href=\"https://links.kronis.dev/2jdgl\"\u003eMurphy's law\u003c/a\u003e, like how the protagonist in the game \u003ca href=\"https://links.kronis.dev/dg8wy\"\u003eReceiver 2\u003c/a\u003e exists in a world where anything that can go badly, will. I don't really blog about video games, but if I did, I'd definitely do a post about that game - because the writing and worldbuilding in it is pretty interesting. The counterpoint, of course, is that I've self-selected for trying to get my jank setup working with minimal resources.\u003c/p\u003e\n\u003ch3\u003eUpdate\u003c/h3\u003e\n\u003cp\u003eTurns out that Tailscale has \u003ca href=\"https://links.kronis.dev/hnwpl\"\u003ea documentation page\u003c/a\u003e that explains the key expiry, although I'm not sure whether that was the cause or something similar to the issue mentioned above, since there was no messaging about any expiry to me. I get that it's a security feature, but still, some notifications about that would be nice. A bit like how Let's Encrypt used to do certificate expiry reminders over e-mail, but for whatever reason \u003ca href=\"https://links.kronis.dev/vxrf3\"\u003ediscontinued that service\u003c/a\u003e, which I find to be a bummer.\u003c/p\u003e\n\u003cp\u003eOh and fun thing: it seems that not only Portainer decided to upgrade to a new version (since that's one of the few containers where I just went with the LTS tag, expecting updates for something so fundamental to be quite important), but also wiped my defined webhooks, so I had to update the CI configuration for redeploying this very blog as well. Issues are just piling on, it seems.\u003c/p\u003e\n"
        },
        {
            "title": "Building brittle software",
            "date_published": "2025-08-05",
            "id": "https://blog.kronis.dev/blog/building-brittle-software",
            "url": "https://blog.kronis.dev/blog/building-brittle-software",
            "content_html": "\u003cp\u003eSo, for most of the week, my blog, homepage and some other sites have been down, so let's talk about brittle software!\u003c/p\u003e\n\u003cp\u003eIt all started, when I wanted to have a quick look at the font stack on my homepage, since I've forgotten exactly what it was and there's \u003ca href=\"https://links.kronis.dev/xcdnb\"\u003ethis one lovely site\u003c/a\u003e that gave me some inspiration. Unfortunately, when I went to open the site, my browser refused to do that:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-site-down.jpg\" alt=\"01-site-down\" title=\"01-site-down.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003e(you'll notice that a lot of the details in this post are redacted, I got a bit curious about how many environment details I'd still accidentally let slip by if I tried to do that for a post, so let's see)\u003c/p\u003e\n\u003cp\u003eIt wasn't an issue of the site loading slowly, it wasn't a database connection failing, or even Apache2 failing to reverse proxy the requests to the Docker container that is responsible for the exact site. Instead, the whole thing was just down.\u003c/p\u003e\n\u003cp\u003eThat's quite odd, since I do have monitoring set up, \u003ca href=\"https://links.kronis.dev/12m1b\"\u003eUptime Kuma\u003c/a\u003e, which has generally worked pretty well and previously had alerts to Mattermost set up, but since I no longer run my own Mattermost instance (so I'd have less software to keep patching constantly), was hooked up to my mail server.\u003c/p\u003e\n\u003cp\u003eWas the mail server also down, that notifications had failed to appear in my inbox? Aside from the fact that in the future I might also need to hook the monitoring up to some 3rd party mailbox (rate limits and privacy be damned), opening the monitoring site itself on another server showed that everything had been down more or less since the start of the month:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02-down-for-a-while.jpg\" alt=\"02-down-for-a-while\" title=\"02-down-for-a-while.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eNormally, that'd be pretty terrible! It's like most of my online presence had been wiped out for a little bit, but the good news is that it's not particularly important - it's not like people can't receive healthcare or other essential services during this downtime, it mostly just hosts this blog, my homepage, as well as a few sideprojects here and there.\u003c/p\u003e\n\u003cp\u003eIn that sense, not having SLAs is freeing, but on the other hand - also annoying. You see, I purposefully pick \u003ca href=\"https://links.kronis.dev/i7iga\"\u003esoftware and stacks that are quite boring\u003c/a\u003e, with the idea that I'll have fewer surprises along the way.\u003c/p\u003e\n\u003ch3\u003eDocker Swarm issues\u003c/h3\u003e\n\u003cp\u003eUnfortunately, there are still plenty of those, regardless of what I do:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03-no-such-image.jpg\" alt=\"03-no-such-image\" title=\"03-no-such-image.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eSo, it was complaining that the Docker images for my software could not be found neither locally, nor in my custom Docker Registry. That should never happen. Even if the registry is down, there's no reason for the local image to disappear.\u003c/p\u003e\n\u003cp\u003eYet, this had been just one such time. Digging around in the logs, I could also see this for some of the services:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003enetwork sandbox join failed: subnet sandbox join failed for \u0026quot;10.0.2.0/24\u0026quot;: error creating vxlan interface: file exists\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eFor what it's worth, it seems like folks online have run into \u003ca href=\"https://links.kronis.dev/1lnv0\"\u003esimilar issues\u003c/a\u003e, seems like it's just one of those things that you run into if you have a cluster for long enough, not exactly a point in the favor of using Swarm in my eyes, but I've seen similar issues with Kubernetes in the past as well, so I can't exactly complain that much.\u003c/p\u003e\n\u003cp\u003eEither way, I cleaned everything up... and the issue didn't disappear.\u003c/p\u003e\n\u003ch3\u003eBuild issues\u003c/h3\u003e\n\u003cp\u003eIt seems like what had happened, was that since the containers couldn't start, the images were considered not in use. That's when the daily/weekly/whatever container cleanup (\u003ccode\u003edocker system prune\u003c/code\u003e) picked up on it and wiped the images from the local nodes, meaning that they're just not there.\u003c/p\u003e\n\u003cp\u003eWell, why not just pull them from the registry, then? Because they're not in the registry either! You see, since I don't want to manually recreate the blob stores or manually clean them, the registry has a cleanup schedule as well. While it's also setup to automatically rebuild images so the latest versions are always there (but not old builds from months ago), clearly that wasn't working:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04-no-build-prerequisites.jpg\" alt=\"04-no-build-prerequisites\" title=\"04-no-build-prerequisites.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eSince the blog is running on \u003ca href=\"https://links.kronis.dev/gxdhl\"\u003eGrav\u003c/a\u003e, it needs a PHP base image to build it. Since I'm building my own (for layer reuse and consistency's sake), it's attempting to get it from the registry... and failing.\u003c/p\u003e\n\u003cp\u003eSo what's the cause for it not being there? The fact that the task that's building the images is failing due to some dependencies that are missing, as a part of me doing some refactoring:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"05-cant-build-images.jpg\" alt=\"05-cant-build-images\" title=\"05-cant-build-images.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eYou see, I was going to have some MariaDB 11 based images previously, but because \u003ca href=\"https://links.kronis.dev/y2abv\"\u003ethere are some issues\u003c/a\u003e with MySQL drivers against it, I figured that I'd probably have to go back to having a regular MySQL 8 image or whatever is the LTS, because MySQL has more drivers available and even the instructions for some stacks say that you should just use the MySQL driver for MariaDB - which is obviously no longer viable.\u003c/p\u003e\n\u003cp\u003eOkay, fine, so I fixed the issue and everything should now be okay, right? Wrong, now that same task also can't build the OpenJDK image, because the Maven binaries are no longer available \u003ca href=\"https://links.kronis.dev/hly09\"\u003eon the site from which I download them\u003c/a\u003e, leading to more failures:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"06-cant-install-maven.jpg\" alt=\"06-cant-install-maven\" title=\"06-cant-install-maven.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eWhy download them from the Apache site directly, instead of just using the OS package manager? Well, the thing is that I tried doing that initially, but the version included with the OS (at least back then) was too old, meaning that some dependencies would fail to install properly and some plugins and such would fail to work.\u003c/p\u003e\n\u003cp\u003eSo instead of hoping that they packaged the correct version in the latest Ubuntu LTS, I just decided to outright copy the .tar.gz into my Git repo and to use it directly, giving me even more control over what I want to include in my containers:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"07-fixing-maven-install.jpg\" alt=\"07-fixing-maven-install\" title=\"07-fixing-maven-install.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eIt seems like that finally fixed the builds and while I was waiting for them to finish, I noticed some e-mails about failures in my mailbox after all:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"08-actually-had-an-email-about-this.jpg\" alt=\"08-actually-had-an-email-about-this\" title=\"08-actually-had-an-email-about-this.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eThis example mail was from when I removed my own Ansible image as well, because for whatever reason it seems like I needed Docker to be installed inside of the containers alongside it (probably to use the Docker or Swarm tasks), but after a Ubuntu LTS version upgrade it was failing with \u003ccode\u003ePEP 668\u003c/code\u003e related issues and frankly I wasn't using it that much outside of work, so it got cut.\u003c/p\u003e\n\u003cp\u003eThere were also a few more messages closer to the exact issues pictured here, but you get the idea - the cause was forgetting a line or two in CI/CD config, probably in some late evening due to not having a second set of eyes to look over my personal changes, nor bothering to feed it all into AI for code review either, probably being a bit too swamped with work the next morning to notice the e-mail about the failure.\u003c/p\u003e\n\u003cp\u003eAnd since Docker Swarm deploys that fail also don't have any alerting for them, this whole thing just kept getting worse. No idea why my uptime monitoring didn't properly alert me at all, maybe there was something going on with the mail server as well? It's probably a good example of why self-hosting the monitoring for your self-hosted apps isn't a good idea on the same servers, but I'm absolutely in no position to pay for more servers, so it's not like there's a good solution for this.\u003c/p\u003e\n\u003cp\u003eBut at least now everything would be finally solved, right?\u003c/p\u003e\n\u003cp\u003eThe images were building correctly once more, and the Swarm stack was about to be redeployed with those images, once they'd be done downloading from the registry:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"09-eureka.jpg\" alt=\"09-eureka\" title=\"09-eureka.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eEureka! Or at least it seemed so...\u003c/p\u003e\n\u003ch3\u003eMore issues with networking\u003c/h3\u003e\n\u003cp\u003eExcept of course they wouldn't get fully downloaded, because I can't have nice things:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"10-not-really.jpg\" alt=\"10-not-really\" title=\"10-not-really.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eEssentially, it once more complained that the image didn't exist, even though the build logs very much told me that it's uploaded! Not only that, but from my local PC, I could pull the image with no issues whatsoever:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"11-pulling-works-locally.jpg\" alt=\"11-pulling-works-locally\" title=\"11-pulling-works-locally.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eYet, trying to pull it on the server would just return timeout errors:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"12-but-not-on-server.jpg\" alt=\"12-but-not-on-server\" title=\"12-but-not-on-server.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eTrying to increase those values didn't help at all. So, I got debugging and one thing lead to another and eventually I got down to the issue. Admittedly, this looks like something out of declassified NSA documents, but here you go:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"13-cant-reach-server.jpg\" alt=\"13-cant-reach-server\" title=\"13-cant-reach-server.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eWhat's visible here is that I can get the DNS records for the domain after migrating from NameCheap to \u003ca href=\"https://links.kronis.dev/oqokw\"\u003eINWX\u003c/a\u003e, since I also wanted an EU based alternative to US services that I had been using up until now, as much as possible. However, when I try to actually ping the server, or get data from it, nothing happens...\u003c/p\u003e\n\u003cp\u003eIt's not a connection that's refused, it's not a port that's closed, it just... stalls. Now, I did a bunch of exploration and from what I could tell, Contabo VPNs had suddenly decided to NOT be able to talk to one another over their public IPs. For example, if I have server A that reaches out to server B to retrieve some data, that would just no longer work. I could reach the servers from my PC as normal, but they couldn't talk to one another.\u003c/p\u003e\n\u003cp\u003eThis is increasingly odd, especially since I \u003cem\u003eshouldn't\u003c/em\u003e need a private network for this (public IPs should be fine in a zero trust environment), nor could I find any odd iptables rules or regular firewall rules or anything like that. Because I don't have as much free time anymore these days, I just ended up throwing \u003ca href=\"https://links.kronis.dev/spfn3\"\u003eTailscale\u003c/a\u003e at the problem and more or less called it a day.\u003c/p\u003e\n\u003cp\u003eOf course, that doesn't quite solve all of my issues. You see, I need to access my site through \u003ccode\u003ehttps://my-registry.my-domain.com/some/path\u003c/code\u003e and the exact domain matters, because that's what I'm getting a certificate for through \u003ca href=\"https://links.kronis.dev/8d9u3\"\u003eLet's Encrypt\u003c/a\u003e. Arguably, I should probably use a wildcard certificate so there aren't as many attacks thanks to the likes of \u003ca href=\"https://links.kronis.dev/nr86f\"\u003ecertificate transparency logs\u003c/a\u003e, but at the same time Apache2 \u003ca href=\"https://links.kronis.dev/bso36\"\u003edoesn't support wildcards with mod_md\u003c/a\u003e so that will have to wait.\u003c/p\u003e\n\u003cp\u003eEither way, I could specify the host header directly even when talking to a node through its Tailscale IP, a bit like:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003ecurl -H \u0026quot;Host: my-registry.my-domain.com\u0026quot; http://\u0026lt;tailscale_ip_or_name\u0026gt;/some/path\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eBut the problem is that clearly I can't do that for every web request from every piece of software that I use. So, short of running my own DNS server (which also adds a bunch of complexity and misconfiguration risks), I decided that I am going to patch the \u003ccode\u003e/etc/hosts\u003c/code\u003e file as a good enough solution, since I don't have that many nodes.\u003c/p\u003e\n\u003cp\u003eThere's another problem there, though: I use \u003ca href=\"https://links.kronis.dev/0l3qg\"\u003eCNAME records\u003c/a\u003e to have a human friendly name for each of my servers even when it comes to their DNS records. For example, to access the registry example domain, I'd have something like:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003emy-registry.my-domain.com --CNAME--\u0026gt; my-server-5.my-domain.com --A--\u0026gt; 10.110.0.5\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eThis is also sometimes helpful when I want to move software between the servers, so I can just change the CNAME to point to another server and don't have to think about IPs at all.\u003c/p\u003e\n\u003cp\u003eThe problem there is that it doesn't seem to be enough to just change the server A records, I also need to specify the CNAME records, otherwise some software won't correctly retrieve the Tailscale node IP addresses I want to specify. So basically I am replicating my DNS zones in the hosts file, with the added disadvantage that it doesn't support anything like a CNAME, so it all ends up being quite awkward.\u003c/p\u003e\n\u003cp\u003eWhile that did solve my immediate issue of nodes not being able to communicate (since now the traffic would be routed through Tailscale and would work correctly), it doesn't feel like a proper solution and it also feels like I shouldn't have had to mess around with this in the first place, since there's no good reason (other than maybe a lot of attacks originating from their servers) for Contabo to restrict traffic like that. I'm not saying that they're doing that, maybe there's some other weirdness going on, who knows, traceroute just threw me into a void after a few hops.\u003c/p\u003e\n\u003cp\u003eWhat's worse, this doesn't really apply to my Docker containers: they don't inherit the full \u003ccode\u003e/etc/hosts\u003c/code\u003e contents from the host node, meaning that if they ever get those same issues communicating with any of my software through the public IPs (though I generally just use internal networks within the Swarm, after \u003ca href=\"https://links.kronis.dev/7agg9\"\u003epatching the MTU settings and getting everything working with Tailscale\u003c/a\u003e), I'd either need to have a lot of stuff in \u003ccode\u003eextra_hosts\u003c/code\u003e, or look for other options.\u003c/p\u003e\n\u003cp\u003eBy now, I did receive some monitoring e-mails, first about the site going down and later about it coming back up:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"14-monitoring-emails.jpg\" alt=\"14-monitoring-emails\" title=\"14-monitoring-emails.jpg\"\u003e\u003c/p\u003e\n\u003ch3\u003eHow we got here\u003c/h3\u003e\n\u003cp\u003eBy now, the service is largely restored. As for how it got this bad?\u003c/p\u003e\n\u003cp\u003eHard to say, a part of it is probably historically having issues with not enough storage and container builds (Ubuntu LTS based ones, because Alpine had some compatibility issues despite their smaller container sizes) and that leading to too aggressive cleanup policies, alongside the fact that sometimes Sonatype Nexus and similar software just kinda fails to cleanup container images, given that the layer system seemingly isn't the most trivial thing in the world.\u003c/p\u003e\n\u003cp\u003eLuckily, at least with Contabo, it's not as much of an issue, especially since I opted for a few bigger servers instead of multiple smaller ones:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"15-too-scared-of-storage.jpg\" alt=\"15-too-scared-of-storage\" title=\"15-too-scared-of-storage.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eRegardless, that's quite the tally for one day:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003emonitoring and alerting issues\u003c/li\u003e\n\u003cli\u003econtainer cluster issues\u003c/li\u003e\n\u003cli\u003edependency issues\u003c/li\u003e\n\u003cli\u003eautomated process issues\u003c/li\u003e\n\u003cli\u003econtainer build issues\u003c/li\u003e\n\u003cli\u003econtainer deployment issues\u003c/li\u003e\n\u003cli\u003enetworking issues\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThe silver linings are the fact that I had invested enough time to be able to rebuild all of the images that had gotten cleaned up. Within a few hours, everything is back up, at least until the next time something like this happens again. So what's my plan to prevent such situations?\u003c/p\u003e\n\u003cp\u003eLargely nothing.\u003c/p\u003e\n\u003cp\u003eSure, I'll look in the direction of better alerting, maybe something like \u003ca href=\"https://links.kronis.dev/x9caz\"\u003eunregistry\u003c/a\u003e, for being able to push images directly to the servers, maybe even figure out a way to have custom rules for cleaning up containers, maybe even run a bare registry image with some auth in front of it (because Harbor would be overkill and Nexus kind of sucks)... but all of these issues happened in the first place, because my hands are largely tied.\u003c/p\u003e\n\u003cp\u003eUnregistry wouldn't work for when I need to build images that have pre-requisites: unless I want to send all of my OpenJDK images and Python images and everything else to every single CI server every time I rebuild them (though with storage being cheap, maybe that is more viable, especially since I have just a few CI nodes).\u003c/p\u003e\n\u003cp\u003eThe registry choice itself was just based on what I could easily enough setup for free, even the lack of tooling around cleanup policies and such is just a matter of not having enough time and resources to sit down, integrate with Docker or other APIs directly and write code to handle it like any other automation issue. Same for Docker Swarm not doing alerting for when containers fail to startup, because I had to get rid of Mattermost, because that also takes time and effort to manage, so I'd probably be stuck with e-mail alerting, which is easier to overlook for whatever reason (or also might need an external mailbox for this).\u003c/p\u003e\n\u003cp\u003eSame for hosting the monitoring infrastructure on the same servers - it helps in cases where a site or two refuse to work, but not when network weirdness also brings down everything, though it's not like I'm getting another node just for that on another cloud, since literally all servers are kind of expensive out there and I earn nothing from these.\u003c/p\u003e\n\u003cp\u003eIt just feels like a lot of the software out there is going to have edge cases and will end up brittle, no matter what you do.\u003c/p\u003e\n\u003ch3\u003eThe perfect world\u003c/h3\u003e\n\u003cp\u003eI had this old post of mine: \u003ca href=\"https://links.kronis.dev/razox\"\u003eNever update anything\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eIt was written to be a bit absurd and since the post has had a few updates, my blog has had a few updates, pretty much every piece of software or hardware that I use has had a few updates; except for a cheap notebook I have laying around somewhere, that one has the Wi-Fi break with any updates and I'd need to re-compile this random GitHub repo I found that lets it work, although at this point its battery is also kinda dead to the point where it dies in an hour off the plug.\u003c/p\u003e\n\u003cp\u003eBut I digress, I was going to say that everything around me has had updates... and that that's also the problem, very much so. Because with updates, changes, things break. Oftentimes in ways that are hard to anticipate ahead of time and all of which will take time and effort, in fighting against the churn.\u003c/p\u003e\n\u003cp\u003eIn a perfect world, things would go more like this:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eyou get a CD/DVD/HDD/SSD/USB memory stick with installation media\u003c/li\u003e\n\u003cli\u003eyou install the OS, all of the first party packages are also available on the media\u003c/li\u003e\n\u003cli\u003eupdates, if any, are security oriented and tested (as much as SQLite) to not be breaking\u003c/li\u003e\n\u003cli\u003eif anything breaking needs to happen, there are fully automated install scripts that carry things over to a known good sane configuration for moving forwards (possibly with approval if done interactively)\u003c/li\u003e\n\u003cli\u003ethere is a set of packages that are well tested and well documented and proven to work together: a web server, a container runtime, a database solution, something for key-value storage and queues, maybe something for document storage, something for blob storage etc.\u003c/li\u003e\n\u003cli\u003ethere's also a hypervisor and/or a container runtime so everything you run is more or less isolated from the core system and won't mess it up\u003c/li\u003e\n\u003cli\u003ethe experience is somewhere between Docker, AppImage and Flatpak\u003c/li\u003e\n\u003cli\u003eeverything is generally geared towards just working together and keeping working\u003c/li\u003e\n\u003cli\u003eyou setup software in 2025, it runs pretty well until 2050, then you spend a day doing updates and are good until 2075\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eHonestly, 25 years of software running well should be the starting point, not the end goal. But alas, you try running Windows XP in the modern day and age, connect it to a network and see what happens. Even RHEL is supported for about 10 years until EOL makes you upgrade your distro. You just can't do that, unless you like being a part of a botnet.\u003c/p\u003e\n\u003ch3\u003eSummary\u003c/h3\u003e\n\u003cp\u003eIt's not the fact that updates are bad per se, but that dealing with them leads to churn and things breaking, which more or less puts an upper limit on the amount of software that you can run yourself, given a certain amount of free time you have.\u003c/p\u003e\n\u003cp\u003eHonestly, keep a system running for long enough and everything that can go wrong probably will anyways, so for now, the best idea is to just try to isolate components as much as possible, make them easy to carry over to another node (yes, this includes backups) and generally err on the side of boring and simple tech choices.\u003c/p\u003e\n\u003cp\u003eI've linked \u003ca href=\"https://links.kronis.dev/w5ffm\"\u003eService as a Software Substitute\u003c/a\u003e before, but while it still rings somewhat true, the people playing for SaaS or PaaS solutions probably are spending less time on things like these and are out there running awesome software and shipping more often.\u003c/p\u003e\n\u003cp\u003eIf I had to summarize everything with a slightly vulgar meme, I'd go for this one:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"16-tl-dr.jpg\" alt=\"16-tl-dr\" title=\"16-tl-dr.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eOf course, maybe the meme above also doesn't quite work because it seems like most people would indeed just reach towards something like Railway or Render or Fly.io or one of those other platforms, but after doing self-hosting for a while, I indeed have more appreciation for services that exchange some of your freedoms in exchange for ease of use and fewer frustrations (until they raise their prices).\u003c/p\u003e\n\u003cp\u003eYou do learn a bunch self-hosting stuff, but it can be a bit tiring, even in simpler setups without something like Kubernetes, because of how brittle things can be, even trying to follow the happy path as much as possible. I do realize that I'm basically saying that I need something like FreeBSD, but the thing is that the DX of Docker has also kinda spoiled me. Guess there is no silver bullet.\u003c/p\u003e\n\u003cp\u003eOh well.\u003c/p\u003e\n"
        },
        {
            "title": "AI, artisans and brainrot",
            "date_published": "2025-06-27",
            "id": "https://blog.kronis.dev/blog/ai-artisans-and-brainrot",
            "url": "https://blog.kronis.dev/blog/ai-artisans-and-brainrot",
            "content_html": "\u003cp\u003eRecently, I went to a software development event in Germany, where, at the conclusion of one of the days, I was hanging out in a hotel room with a friend of mine. Earlier, she had been exploring ways to get full VMs running on her Linux system easily, for doing some development work and experimentation with Docker on them.\u003c/p\u003e\n\u003cp\u003eAnother friend had suggested \u003ca href=\"https://links.kronis.dev/9w6xu\"\u003eVirtualBox\u003c/a\u003e as a starting point.\u003c/p\u003e\n\u003cp\u003eI looked at what she was trying to do, and suggested that maybe \u003ca href=\"https://links.kronis.dev/01k7z\"\u003eVagrant\u003c/a\u003e might also be a good fit, due to her wanting to make some of the output of her experimentation easily transferable and reproducible elsewhere (the Vagrant files might be a little bit better for that, than just some Bash scripts that talk to VirtualBox), in addition to wanting a bunch of stuff to be executed during startup.\u003c/p\u003e\n\u003cp\u003eObviously, both of us were actually kind of new to Vagrant, due to me having found Docker (and OCI containers in general) more suitable for my needs, also using cloud VPSes for anything longer term and also Ansible for automating configuring my boxes as needed, whereas she hadn't worked that much on the infrastructure side and didn't really feel like paying for cloud resources for something like this. There's nothing wrong with VMs and her choice here was nice, especially because she felt like wanting to learn more about various setups and experiment.\u003c/p\u003e\n\u003cp\u003eThat's where we hit our first roadblock. I also told her about \u003ca href=\"https://links.kronis.dev/pahgc\"\u003eDocker Swarm\u003c/a\u003e as a pretty lightweight and simple to use, setup and operate orchestrator. There is nothing wrong with Swarm itself, but we did hit a snag when trying to get the cluster initiated. What we needed to do was:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eretrieve the correct IP address on the leader node\u003c/li\u003e\n\u003cli\u003euse that IP address in \u003ccode\u003e--advertise-addr\u003c/code\u003e when \u003ca href=\"https://links.kronis.dev/9syr4\"\u003einitializing the cluster\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003eretrieve and save the join token after the cluster is initialized\u003c/li\u003e\n\u003cli\u003eon the other follower nodes, retrieve and use this token to join the cluster\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eShe was going through the docs bit by bit and trying things that didn't really work (yet) and that were frustrating. At one point, she had decided to take a pause for a bit and suggested I give it a shot. Admittedly, I didn't care much about Vagrant: I knew \u003ca href=\"https://links.kronis.dev/g0apk\"\u003ewhat we wanted to do\u003c/a\u003e and the exact details of \u003cem\u003ehow\u003c/em\u003e we'd do that on the line-by-line level felt less important, so I just threw a prompt, a bit like the list above, at one of the AI tools at my disposal.\u003c/p\u003e\n\u003cp\u003eFor this, I needed a few iterations and I got decently far: the AI generated code used the correct IP address within the script with \u003ccode\u003e#{ip}\u003c/code\u003e, initialized the cluster correctly AFTER actually waiting for Docker to startup in the container (another snag which I hit along the way, which needed another iteration) and could seemingly retrieve the join token. Where I had gotten stuck was that I'd still need to make it available to the other nodes and they were sitting in a waiting loop in their own script, waiting to retrieve the keys but never getting them. I think a good approach here would be to use \u003ca href=\"https://links.kronis.dev/6h791\"\u003esome shared storage\u003c/a\u003e and just watch for changes there, or something along the lines of that. With a bit more iteration and familiarity with her computer (e.g. keyboard layout), I could have had something working in about 15-20 minutes.\u003c/p\u003e\n\u003cp\u003eBut that didn't matter. Tragedy had already struck. I had committed a cardinal sin in her eyes, by reaching for the \u003ca href=\"https://links.kronis.dev/q6ifv\"\u003eabominable intelligence\u003c/a\u003e.\u003c/p\u003e\n\u003ch3\u003eAI and learning\u003c/h3\u003e\n\u003cp\u003eYou see, a lot of people out there hate AI. Not dislike, not think that it's a fad, but straight up hate it. Not just because the term AI is more or less misappropriated when describing LLMs, glorified token prediction machines with some emerging agentic capabilities that will probably never directly lead to \u003ca href=\"https://links.kronis.dev/9uw0q\"\u003eAGI\u003c/a\u003e, but because there are fundamental issues with what they represent - the tools in the image generation space often being trained on others' stolen art, whereas LLMs \u003ca href=\"https://links.kronis.dev/mp7hi\"\u003econsuming as much data as possible\u003c/a\u003e, sometimes resorting to piracy, as well as straining websites out there with annoying crawlers.\u003c/p\u003e\n\u003cp\u003eThe arguments against these technologies also go further, with some studies showing that \u003ca href=\"https://links.kronis.dev/03r48\"\u003ebrain activity decreases\u003c/a\u003e when you use tools like ChatGPT for common tasks. The friend herself suggested that she's seen people's development skills deteriorating and that using those tools will eventually lead to developers being worse, yet capable of producing things that seem good enough in the moment, until something does break, at which point they'll be helpless. Her guess was that this would probably be hurting both the career of software development as a whole, as well as taking away any artisanal aspects that might have once been there to the craft.\u003c/p\u003e\n\u003cp\u003ePersonally, it's not like I wholly disagree with her take.\u003c/p\u003e\n\u003cp\u003eBut I don't think it's that simple - the cat is out of the bag in regards to these tools and they'll be adopted more and more, across increasingly wide use cases. I do believe that maybe currently the technologies are near the top of a \u003ca href=\"https://links.kronis.dev/65ai3\"\u003ehype cycle\u003c/a\u003e and that eventually we'll settle on a more reasonable take, where their applications won't exceed their capabilities as much, in part due to more mature technology, as well as due to plenty of experience of what they're good at and what not:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-ai-hype-cycle.jpg\" alt=\"01-ai-hype-cycle\" title=\"01-ai-hype-cycle\"\u003e\u003c/p\u003e\n\u003cp\u003eShe said that it's more of a fundamental change, however. I mentioned how often you don't read all of the underlying code that you're reading with (e.g. the framework you use) but will more often read the documentation, something like StackOverflow, or look at other codebases or examples of how to use it, but her stance is that these AI tools represent a more fundamental shift - that \u003ca href=\"https://links.kronis.dev/fiu4l\"\u003eGitHub Copilot\u003c/a\u003e and similar tools can do the work for you, not making you apply your brain to the problem much at all, in addition to them being crammed in wherever possible. A runaway hype cycle, if you will.\u003c/p\u003e\n\u003cp\u003eMy conclusion from that, however, is that anyone not using these tools will put themselves at a disadvantage - when most developers start using these tools, the effects will be similar to those of someone writing their code in Notepad++ or Vim directly (which they can only do with any degree of success due to learning the language better) vs someone who is similarly smart but is also using a fully featured IDE like some of the JetBrains products, which give you various code inspections and syntax checks and other suggestions live, as well as advanced language aware refactoring capabilities. Suddenly those not using these AI tools will slow themselves down somewhat and be at a disadvantage, especially junior developers, especially with companies that aren't interested in spending a bunch of their money in training them to become better developers eventually.\u003c/p\u003e\n\u003cp\u003eEssentially, the choice becomes one between gradual brainrot vs falling by the wayside - the same way how someone might use social media as one of the few viable ways of keeping in touch with their friends and the social scene in their location nowadays, while also being at the risk of a shortened attention span, comparing themselves to the highlights of the lives of others and the many other risks that social media can present (including, but not limited to, \u003ca href=\"https://links.kronis.dev/acprk\"\u003edisinformation\u003c/a\u003e).\u003c/p\u003e\n\u003cp\u003eThere are people who suggest that you can use the AI tools responsibly and not use them to write all of your code for you, merely help you discover some perspectives or edge cases that you might not have considered yourself, however I remain unconvinced - while I don't deny any utility that these tools have, I've noticed that the more you rely on resources that give you a closer approximation of a solution, the less you need to dig into a given problem to understand what's going on. For example, consider someone giving you instructions on how to import certificates into a Java keystore (like \u003ccode\u003ecacerts\u003c/code\u003e), vs going off on a little research exercise by yourself and learning more about TLS and PKI yourself.\u003c/p\u003e\n\u003cp\u003eI do wonder, what will be faster - the gradual decline of our cognitive abilities due to using the tools, or their integration within software development and other industries to the point where those not using them can't compete. If things align, maybe those avoiding them will keep their wits about them while also remaining more productive on average. The thing, however, is that we don't have limitless time and in the real world not everyone is as curious. Most people just care about solving whatever problem they are paid to solve and not much more beyond that.\u003c/p\u003e\n\u003ch3\u003eThe real world and balls of mud\u003c/h3\u003e\n\u003cp\u003eI wanted to express that nuance to her, the perspective that she perhaps wasn't considering.\u003c/p\u003e\n\u003cp\u003eThat there are plenty of projects out there that these technologies would still be useful for and would ensure better outcomes - if I have a large monolithic codebase that spans a million lines of code or thereabout, there's no way I'll be able to easily review all of it, or even that my searches will reveal everything about it that I might need to know, vs something more advanced than a simple code search could achieve.\u003c/p\u003e\n\u003cp\u003eSuppose that I'm your average developer for a second, and that I'm working on introducing some auditing logic within such a large codebase. I might have done a cursory code search but didn't quite catch that there are 3 other similar bits of functionality under different names, that should either be refactored or that my implementation should just use one of the ones that already exist there. Agentic tools have caught similar things in the past, either when using proper \u003ca href=\"https://links.kronis.dev/g4z6l\"\u003eembeddings\u003c/a\u003e across the whole codebase, or just searching through it with models that allow for hundreds of thousands of tokens in the context. Why? Because these tools can quickly process a bunch of information and have more working memory than the human brain does at any given time. Maybe not the ability to reason properly, but that's not \u003cem\u003ealways\u003c/em\u003e what things hinge on.\u003c/p\u003e\n\u003cp\u003eSimilarly, for various refactoring tasks, I might already be holding 20-30 files in my head and the more I pile on, the more fuzzy everything becomes and unless I make task lists of everything I might want to do, I'll quickly lose track of certain tasks or details that are important, because of which I might not want to do additional refactoring or even risk changes that could be breaking over similar fears. The AI tools I've used don't seem to have such issues and will happily hold upwards of 50 files in their context and be able to reason about them and do refactoring - reviewing the changes in which is easier than having to take the step by step manual approach.\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02-regular-context.jpg\" alt=\"02-regular-context\" title=\"02-regular-context\"\u003e\u003c/p\u003e\n\u003cp\u003eThese tools have also caught edge cases that I wouldn't have thought of (e.g. when doing data validations), are really good for writing boilerplate code (because nobody can be bothered to setup codegen for everything, just ask how many of the people out there can fully generate JPA mappings from your live DB schema and from your versioned migrations), things like utility scripts and one off data transformations, as well as a bunch of stuff.\u003c/p\u003e\n\u003cp\u003eI make no strong claims about any of these, if you're curious you can see for yourself, though of course the more you stray from the popular approaches and technologies out there, the worse your personal experiences will be. My friend, however, she had a different approach to me talking about the other side of the coin. She said that codebases like that should never exist in the first place, that code should be written in such a way where it fits in the working context of the human brain. Or that the developers in question should get better.\u003c/p\u003e\n\u003cp\u003eThe issue is, that we don't live in that world, even though I'd love to have a memory that I'd be able to call \u0026quot;good\u0026quot;. Or not work on complex codebases. Saying that bad or even complex codebases shouldn't exist and therefore that the idea of having to maintain them doesn't need time put into it is like saying that jobs like transcription or translation shouldn't exist because we should be able to automate them (yes, both with formal methods and all sorts of forms of AI) - while the statements themselves aren't wholly wrong, they're also quite ignorant of reality.\u003c/p\u003e\n\u003cp\u003eImagine saying that to someone, which doesn't resolve any of their issues and just puts down tools that might help them in some ways, in addition to proceeding to call either their codebase, their job or both, shit. Then, what do you think are your chances of being able to somehow convince them of the point you want to make without making them feel offended? It doesn't work that way. It's also not like they can go to their place of work the next week and rewrite a codebase that has been around for a decade to suit whatever is the present best understanding of sensible software development.\u003c/p\u003e\n\u003cp\u003eNor is the particular argument something anyone should make from a position of authority - since there are drawbacks to introducing unreliable networks in the middle as well as a bunch of cruft to push data around, as well as some domains are just huge, nor are almost any of the languages that we use well suited for unreliable network calls as first class citizens.\u003c/p\u003e\n\u003cp\u003eBy that reasoning, should planes run on microservices? In some loose sense, maybe - with systems having modularity and redundancy and something like \u003ca href=\"https://links.kronis.dev/g9qkn\"\u003eARINC\u003c/a\u003e being used to push data around, albeit it does not look like the Wild West in web development and might work closer to a microkernel system, except we also all know how \u003ca href=\"https://links.kronis.dev/mst9e\"\u003eGNU Hurd\u003c/a\u003e worked out (or rather, didn't), so at that point it's talking about theory and something that gets increasingly detached from what you'll be doing next Monday.\u003c/p\u003e\n\u003cp\u003eWhile the line about the large codebase was said to me verbatim with the profanity, the thing is, that I don't even fully disagree - there's lots of arguably bad codebases, architectures, entire teams and projects out there, none of which should have existed or at least ended up in the form that they did. I've seen plenty. However, you shouldn't dismiss the constraints or the processes that lead to those circumstances just because they shouldn't exist.\u003c/p\u003e\n\u003cp\u003eWhen someone is stuck in a hole, you don't grandstand about how they should have never fallen in there, or how there should actually be an escalator out of the hole - you give them a fucking shovel.\u003c/p\u003e\n\u003cp\u003eI'm not actually upset or mad at her, more like the entire industry that brought us to this point, that these AI tools are even necessary, that more formal methods never materialized that much outside of academia, otherwise we'd all design our systems with ample help of \u003ca href=\"https://links.kronis.dev/bwx74\"\u003eUML\u003c/a\u003e and \u003ca href=\"https://links.kronis.dev/bz6cu\"\u003eC4 models\u003c/a\u003e, estimate what needs to be done with \u003ca href=\"https://links.kronis.dev/ytp9k\"\u003eCOCOMO 2\u003c/a\u003e and even treat our infrastructure as a bunch of boxes to be dragged around, putting as many low code or no code tools as possible in there.\u003c/p\u003e\n\u003cp\u003eCome to think of it, it doesn't sound right either. I think my problem is that there are so many tools that try to get rid of writing code, instead of making writing code easier and more bulletproof. Writing bad code should be hard in the first place, it's not that you shouldn't write any at all. So maybe more along the lines of various code inspections, linter rules, automatic code formatting and codegen, instead of additional abstractions.\u003c/p\u003e\n\u003ch3\u003eArtisans and engineering\u003c/h3\u003e\n\u003cp\u003eThat is why I kind of took issue with something else that she said: the claim that software development is a craft of artisans.\u003c/p\u003e\n\u003cp\u003eFrom where I stand, it seems like every next week, there is some new approach to writing front end components, some new DB technology, some new back end framework, only a few of which will survive for a decade and many of which will leave people in deep mud once the support runs out. Even technologies like \u003ca href=\"https://links.kronis.dev/ctzu7\"\u003eAngularJS\u003c/a\u003e that were once backed by a big company like Google lost support and caused lots of headaches when people needed to migrate to something else. The same happened with React when the paradigm shifted from class based components to hooks, albeit one could understand some of the advantages as well. There will also be issues once JDK 8 hits EOL, if there's ever a serious CVE in it (or anything written on top of it that doesn't support anything newer), then lots and lots of people will need to scramble.\u003c/p\u003e\n\u003cp\u003eAt the same time, people still use languages that result in various vulnerabilities and unsafe memory access all the time, regularly mess up with public S3 buckets, implement everything from data validations to authn/authz poorly, write convoluted code that often is underdocumented, or happily write badly optimized code that results in N+1 problems all over the place, just waving their hands at the end and saying that optimization is on the TODO list. The very same people who'll look at a list of requirements for some Jira work item and say that anything more than 8 hours of work to get it done is unrealistic, whereas in practice the amount of time you \u003cem\u003eshould\u003c/em\u003e spend on the thing is anywhere between 8 and 40 hours.\u003c/p\u003e\n\u003cp\u003eYou might say that it's just a particular company or a particular project, but when both the tax system in my country falls over when it's time to submit your yearly tax statements, when the e-health system in my country falls over pretty regularly and doesn't have proper ETL with whatever the local hospitals are using, when the vaccination system used during COVID was an unoptimized mess, you start seeing a pattern. Even Windows 11 that I have on my computer right now just refuses to install updates, due to complaining about their web view components not being present - after I've checked that they are, in the OS that they themselves developed. Imagine not being able to just patch your OS because the people developing it messed up that badly.\u003c/p\u003e\n\u003cp\u003eBecause of all of that, these two are both simultaneously true in my mind:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003ewe're moving ahead at a rapid pace\u003c/li\u003e\n\u003cli\u003ewithout ever having gotten the basics down\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eSoftware engineering shouldn't be an artisanal craft that reminds us of the Wild West - a lawless place that sometimes looks like a wasteland of mismatched incentives and padded CVs. It should be actual engineering, with proper regulation, certifications, best practices, down to the very last line of code, down to the methods used, the programming languages themselves.\u003c/p\u003e\n\u003cp\u003eWe will never live in that world.\u003c/p\u003e\n\u003cp\u003eIn such a world, you'd have deeply researched and tested, as well as formally proven frameworks and languages for developing your webapp. You'd learn one front end framework in school and that's what all the professionals would be using. The same for the back end frameworks and databases. The fewer options, the better methods around using them you can introduce. At the very least, no database would be in use without something like \u003ca href=\"https://links.kronis.dev/9hef3\"\u003ethorough Jepsen testing\u003c/a\u003e. Every line of code that you write would undergo a lot of analysis, every line would need tests written to cover it, those tests would then also be inspected. The operating systems in use would be more like the \u003ca href=\"https://links.kronis.dev/dfg7l\"\u003eversions of BSD\u003c/a\u003e instead of Linux - engineered instead of grown. Every cloud technology would be certified with no less scrutiny than the databases.\u003c/p\u003e\n\u003cp\u003eThe pace of development would be slower, the tooling would (eventually) be better, the confidence in the code written would be higher, there would be fewer outages and less uncertainty and \u003ca href=\"https://links.kronis.dev/70x92\"\u003eimpostor syndrome\u003c/a\u003e - because you could actually master a single stack from bottom up and use it for decades, more than any T shaped individual of the current age. Innovation would be slower, but often that is directly opposed to reliability and stability, at least when new technologies and practices are used before they're ready.\u003c/p\u003e\n\u003cp\u003eNobody really wants that, unless they're building an airplane.\u003c/p\u003e\n\u003ch3\u003eProgramming languages and ecosystems\u003c/h3\u003e\n\u003cp\u003ePeople want code that is quick to write and don't care about the fact that their expectations of reliability in the current landscape are unreasonable.\u003c/p\u003e\n\u003cp\u003eThat's also how we got to the \u0026quot;cattle not pets\u0026quot; attitude towards servers and accepted that they'll just fail every now and then, and everyone moved onwards as if that's a normal take in what should be a highly deterministic system. That's the unfortunate reality of computers and operating systems being far beyond the capability of any individual to fully understand, when compared to the computers of the olden days.\u003c/p\u003e\n\u003cp\u003eThe good news is that you can pick any stack and gain a pretty deep understanding of it anyways, in a decade or two.\u003c/p\u003e\n\u003cp\u003eThe bad news is that the suggestion for one very powerful language above would only lead to something like C++, where you very much have a \u003ca href=\"https://links.kronis.dev/zbr3a\"\u003edesign by committee\u003c/a\u003e situation with every feature under the sun being crammed into the language and making it much harder to fully learn, understand and use... versus just doing less.\u003c/p\u003e\n\u003cp\u003eI look at the software landscape and it feels like there are too many languages out there that do too many things with meaningless differences between them. For example, if I have a collection of some sort, which one of these will be the correct way to get its length: \u003ccode\u003elen\u003c/code\u003e, \u003ccode\u003elen()\u003c/code\u003e, \u003ccode\u003e.size\u003c/code\u003e, \u003ccode\u003e.size()\u003c/code\u003e, \u003ccode\u003e.count\u003c/code\u003e, \u003ccode\u003e.count()\u003c/code\u003e, \u003ccode\u003e.length\u003c/code\u003e, \u003ccode\u003e.length()\u003c/code\u003e. Of course, the answer depends on the language in question, but maybe it shouldn't? Maybe we should have certain conventions that transcend the languages we use, for example, common ways of working with data types, validations, testing and so on, across all languages.\u003c/p\u003e\n\u003cp\u003eIf in a given week I have to work with Bash, Java, .NET, Python, Ruby, Go and JavaScript, then by the end of it things tend to get a bit fuzzy and even language constructs like loops (or \u003ca href=\"https://links.kronis.dev/69f27\"\u003eStream API\u003c/a\u003e and \u003ca href=\"https://links.kronis.dev/o056h\"\u003eLINQ\u003c/a\u003e) will seem to fade into the background as mostly cruft that stands in the way of me just wanting to accomplish a particular task. The same very much applies to all the web frameworks out there, or even how every stack has a different view on how an ORM should function, even if at the end of the day they all still talk SQL.\u003c/p\u003e\n\u003cp\u003eIt might seem silly, but even in regards to front end, \u003ca href=\"https://links.kronis.dev/3a9o5\"\u003ePrimeVue\u003c/a\u003e, \u003ca href=\"https://links.kronis.dev/uylyq\"\u003ePrimeReact\u003c/a\u003e and \u003ca href=\"https://links.kronis.dev/xvuy4\"\u003ePrimeNG\u003c/a\u003e are trying to do something very similar to what I'm suggesting: components that work across Vue, React and Angular respectively, with knowledge transfer that's better than any other methods of writing front ends in those, even if it's not perfect. Maybe then it wouldn't feel like some of those are different just for the sake of being different.\u003c/p\u003e\n\u003cp\u003eTo me, it feels like every problem should have one proper solution that you wouldn't stray from that far. And yet, everything outside of RFCs like how HTTP(S) works is still fragmented chaos. Let me give you an example of how getting some CSS ready worked in a project that I once was on:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eyou'd have \u003ca href=\"https://links.kronis.dev/n5gby\"\u003eSASS\u003c/a\u003e with its SCSS files\u003c/li\u003e\n\u003cli\u003ethose would need to be transpiled to CSS, but for that you'd need some tooling\u003c/li\u003e\n\u003cli\u003ethat particular project used \u003ca href=\"https://links.kronis.dev/aawr9\"\u003eGulp\u003c/a\u003e, because that's what was trendy at the time\u003c/li\u003e\n\u003cli\u003ebecause npm was slow at the time, the project also used \u003ca href=\"https://links.kronis.dev/b41w8\"\u003eYarn\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003ebecause the devs didn't want to manage Node.js versions manually, that was installed through \u003ca href=\"https://links.kronis.dev/0tu1h\"\u003ea Java plugin with Maven\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003efor whatever reason, I think \u003ca href=\"https://links.kronis.dev/buk9d\"\u003enode-gyp\u003c/a\u003e was also needed there\u003c/li\u003e\n\u003cli\u003ethat, in turn, needed Python 2 and anything newer would break\u003c/li\u003e\n\u003cli\u003ealso throw in \u003ca href=\"https://links.kronis.dev/1hq63\"\u003eGrunt\u003c/a\u003e for some tasks like watching for file changes, because of course it was there as well\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eImagine having to maintain that setup for years. Do you understand better why I'm not a fan of overcomplicated toolchains that start looking like Rube Goldberg machines after a while?\u003c/p\u003e\n\u003cp\u003eBut it's not all bad:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eI like languages like Go, they're easier to reason about than something like Java with codebases that have dynamic class loading and reflection.\u003c/li\u003e\n\u003cli\u003eI like SQL databases like PostgreSQL and MySQL/MariaDB (somewhat) and SQLite, because they've been tried and tested and seem like a good fit for the problem space.\u003c/li\u003e\n\u003cli\u003eI like web servers like Apache2, Nginx and even Caddy 2, because they're good at what they do and take some of the tasks (like TLS) away from the back end components.\u003c/li\u003e\n\u003cli\u003eI like front end solutions like Vue and PrimeVue, because they actually seem simpler than React in some cases (compare Pinia with Redux) without sacrificing too much functionality, even though maybe I'd be happier with Alpine.js, or HTMX, or something of that variety.\u003c/li\u003e\n\u003cli\u003eI like technologies like Docker because they allow providing pretty much everything your application needs to be deployed, in a consistent manner.\u003c/li\u003e\n\u003cli\u003eI like Linux when used with containers, because it's mostly a good base operating system, especially when you opt for distributions like Ubuntu LTS due to them being predictable (though they have their quirks).\u003c/li\u003e\n\u003cli\u003eI enjoy tools like RabbitMQ, Redis, S3 compatible object stores and so on - tools that solve a particular problem and do it well, without meaningless complexity.\u003c/li\u003e\n\u003cli\u003eI like that we have web standards like HTTP(S) and also conventions around developing APIs such as REST (even if people don't care much about HATEOAS), as well as the likes of OpenAPI for some common ground.\u003c/li\u003e\n\u003cli\u003eSame for the likes of Git for version control and CI/CD tools built around it, like Drone CI, Woodpecker CI, or even GitLab CI, especially when they get out of your way and work well.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eWhat I also like are tools that make using any and all of the above easier and give me confidence that things will work, like I think they should. If some of those tools happen to be AI because we don't have anything better, I won't pass judgement on that.\u003c/p\u003e\n\u003ch3\u003eHumans in the loop and drowning in agile\u003c/h3\u003e\n\u003cp\u003eSomething my friend said also made me pause and think:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eAI will give you solutions that look correct, but aren't.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eHearing that made me not want to nod my head and agree, but rather question why we even can have solutions that look correct in the first place, but aren't? Your IDE should be screaming at you if that's the case, so should your language server, your code analysis tools, or maybe your AI tools that you can also feed the feature requirements/specifications into, because there's no way people are going to ever write those in a format that a formalized tool understands.\u003c/p\u003e\n\u003cp\u003eThe bottom line is that humans have written and will keep writing code - and the track record so far is that while there are a few geniuses out there that get things right and advance entire industries (like \u003ca href=\"https://links.kronis.dev/a7hqv\"\u003eJohn Carmack\u003c/a\u003e, for example), the rest of us absolutely need any and all help that we can get. Would you want your average developer to attempt to validate data pertaining to money transfers with RegEx, for example, without using any sort of a search engine to find what others have used in the past? I wouldn't.\u003c/p\u003e\n\u003cp\u003eNow, obviously, the problem is that while AI tools will make life easier, the brainrot will still set in - less exploration will be done on our own and more reliance will happen on those tools, to the point of just copying and pasting in code snippets and expecting the magic machine to do everything for you. While the technology has the potential to make me waste less time writing JPA mappings and give me more time to work on an interesting project like a voxel renderer, the reality is that I might as well spend that newly acquired free time just scrolling memes online instead of studying anything.\u003c/p\u003e\n\u003cp\u003eAt the same time, if that applies to a lot of people out there, many of whom would rather talk to a lady that they're interested in rather than spend a weekend learning what monads are or how to use Haskell for some problems instead of Python, then it would be perhaps a bit silly to attempt to hold everyone to such a high standard, where they have to know every language that they use by heart, as well as be able to recall details about the frameworks in those languages minutes after being woken up.\u003c/p\u003e\n\u003cp\u003eIf we say that every single engineer that writes code should be an absolute master at their craft, then there are almost no true engineers out there. Though the few that exist, you probably want more of those people writing the software for your plane.\u003c/p\u003e\n\u003ch3\u003eSummary\u003c/h3\u003e\n\u003cp\u003eSo where did the conversation go? Not that far, since it felt too argumentative to get much utility out of it - especially since it won't exactly change the world, or even present that much of a better alternative to AI tools, short of just not using them and getting better on our own, gaining no utility out of it in the process, albeit avoiding its shortcomings.\u003c/p\u003e\n\u003cp\u003eI will admit that my conversational approach could be better. When considering implementing something or introducing some practice, I tend to focus on the potential drawbacks and issues that might arise with adopting a certain position without questioning them. In practice, that can clash with the strongly held views of others - it seems like I'm questioning their very views with a somewhat long string of gotchas, which can come across like \u0026quot;Here's exactly why your views aren't valid.\u0026quot;\u003c/p\u003e\n\u003cp\u003eObviously I don't use that sort of phrasing and don't believe that, but the important lesson here is not to treat conversations with people like engineering problems, the outcomes of which matter to a degree where I NEED to get down to some base truth. \u0026quot;Winning\u0026quot; an argument shouldn't be the goal, even in a somewhat factual discussion, if it ends up making someone frustrated. If someone says something that I feel is incomplete or flawed in some way, or ignores various real world intricacies, but they care about it strongly, then the correct thing for me, unless not exploring these would get them hurt (e.g. food safety and such), is to consider really well whether I want to pick that topic up.\u003c/p\u003e\n\u003cp\u003eAs for the topic itself, I guess only the future will show what happens to AI and our brains as a consequence of it. Personally, I feel that they're just tools and both the utility and risks depend on how they're used - though that it's impossible to ignore the fact that most people might very well misuse them. I think my friend is definitely right about being distrustful of AI and I have to admit that going in blind when learning something is probably more conductive to remembering it long term, unless the docs suck or are hard to navigate like is the case with Vagrant, as far as I'm concerned.\u003c/p\u003e\n"
        },
        {
            "title": "More PC shenanigans: my setup until 2030",
            "date_published": "2025-06-13",
            "id": "https://blog.kronis.dev/blog/more-pc-shenanigans-my-setup-until-2030",
            "url": "https://blog.kronis.dev/blog/more-pc-shenanigans-my-setup-until-2030",
            "content_html": "\u003cp\u003ePreviously, I explored why having two Intel Arc cards in the same PC is a bit of a mess: \u003ca href=\"https://links.kronis.dev/58jcj\"\u003eTwo Intel Arc GPUs in one PC: worse than expected\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eA little bit later, I discovered that Windows 11 fixes some of the issues and that most games and programs, but not all, mostly work: \u003ca href=\"https://links.kronis.dev/gpble\"\u003eWhat is ruining dual GPU setups\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eNow that I had an OS that's a little bit more cooperative, I decided to go from my setup with the Intel Arc B580 and AMD Radeon RX 580 to a dual Intel Arc setup, with both the A580 and B580 in the same computer, and see if I can fix the problematic software myself:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-intel-arcs.jpg\" alt=\"01-intel-arcs\" title=\"01-intel-arcs\"\u003e\u003c/p\u003e\n\u003cp\u003eSo, without further ado, let's get into it!\u003c/p\u003e\n\u003ch3\u003eA slightly cursed setup\u003c/h3\u003e\n\u003cp\u003eYou might have noticed in the previous examples that the inside of my case was quite messy. While I did attend to it later and managed to mostly improve everything, for running two GPUs I needed to make sure that all of the cables at the bottom of the case stay there, otherwise they risk hitting the fans and both creating noise, as well as preventing them from working:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02-using-tape.jpg\" alt=\"02-using-tape\" title=\"02-using-tape\"\u003e\u003c/p\u003e\n\u003cp\u003eAt that point I just looked at some of the other places where I have cables that are too long and just decided to tape them down as well, since it's quite unlikely that the tape would melt and since it doesn't conduct electricity either, there are very few risks related to doing that:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03-using-more-tape.jpg\" alt=\"03-using-more-tape\" title=\"03-using-more-tape\"\u003e\u003c/p\u003e\n\u003cp\u003eAs for the GPU itself, you can see how little clearance there is and why using tape might not have been the worst solution:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04-gpu-clearance.jpg\" alt=\"04-gpu-clearance\" title=\"04-gpu-clearance\"\u003e\u003c/p\u003e\n\u003cp\u003eOf course, when two of those are added in the same case, things do get pretty crowded, which makes me think that perhaps I'd need a bigger case for this setup to truly work:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"05-new-setup.jpg\" alt=\"05-new-setup\" title=\"05-new-setup\"\u003e\u003c/p\u003e\n\u003cp\u003eThe good news is that the setup itself works and a little bit better than before: Windows 11 lets me choose which games and programs should run on what GPU and since there is no AMD/Nvidia GPU in there, games like Delta Force also cannot decide on their own whim that they don't want to run on an Intel GPU, since those are the only ones that are available!\u003c/p\u003e\n\u003cp\u003eAside from that, you will notice that I got rid of the top case fans, because those were pretty messy and it turns out that the CPU cooler is not good enough either way:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"06-working-so-far.jpg\" alt=\"06-working-so-far\" title=\"06-working-so-far\"\u003e\u003c/p\u003e\n\u003cp\u003eI would have actually stuck with this setup, if not for some rather interesting problems down the road...\u003c/p\u003e\n\u003ch3\u003eThere is some quite cursed software out there\u003c/h3\u003e\n\u003cp\u003eThe first issue, was that even when all of the monitors are plugged into the main card, the framerate of videos and such (on YouTube, for example) is really bad when the browser is running with hardware acceleration on and is using the secondary GPU, my old A580. The fact that this doesn't work well kind of defeats the whole point of the setup: if I can't play a game on the B580 while watching a video that has the decoding done by the A580, why even bother?\u003c/p\u003e\n\u003cp\u003eSecondly, I noticed intermittent corruption of what's on the screen, for example when resizing windows, when their contents are redrawn and even once on the lock screen:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"07-visual-artifacts.jpg\" alt=\"07-visual-artifacts\" title=\"07-visual-artifacts\"\u003e\u003c/p\u003e\n\u003cp\u003eThirdly, I decided to look into whether I could fix OBS and make it allow me to choose which GPU it uses for encoding, since the setup already works in the form of Intel iGPUs and Intel Arc cards, right? Wrong.\u003c/p\u003e\n\u003cp\u003eAdding the UI controls was admittedly pretty easy, as would be retrieving the value from the OBS UI, similarly to how Nvidia does that:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003e// default value\nobs_data_set_default_int(settings, \u0026quot;gpu\u0026quot;, 0);\n// UI element\nobs_properties_add_int(props, \u0026quot;gpu\u0026quot;, obs_module_text(\u0026quot;GPU\u0026quot;), 0, 8, 1);\n// retrieving the value later\nint gpu = (int)obs_data_get_int(settings, \u0026quot;gpu\u0026quot;);\n// passing it to the encoder, Nvidia specific, but you get the idea\nav_opt_set_int(enc-\u0026gt;ffve.context-\u0026gt;priv_data, \u0026quot;gpu\u0026quot;, gpu, 0);\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eWith a bit of vibe coding, that part worked fine. Now initially I thought that all I'd have to do is provide a specific device ID to the QSV encoder internals, since by default it seems to be hardcoded:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003ests = MFXCreateSession(loader, 0, \u0026amp;m_session);\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eHere's the docs for the corresponding bit of code, if anyone is curious: \u003ca href=\"https://links.kronis.dev/9wkv1\"\u003eMFXCreateSession\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eHere's a GitHub issue talking about how it seems to be the correct way to do this: \u003ca href=\"https://links.kronis.dev/h6dnk\"\u003eHow to select different devices with MFXLoad and MFXCreateSession API\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eAnd yet, it didn't seem to work. If a game is running on the first GPU and I tried encoding with the second GPU, then I'd get the whole of OBS just freezing. With a little bit of additional error handling added to the code, I'd just get encoder errors, but never the desired output, a bit like how there are also issues with capturing games if you run OBS itself on the secondary GPU, not just the encoder.\u003c/p\u003e\n\u003cp\u003eSince I don't really work with C/C++ or other low level languages, I did try a bunch of vibe coding (since the task itself didn't seem like it should be that difficult, so an LLM that's trained on a bunch of that code did seem to be a good bet), but even its attempts to copy the texture through the CPU didn't succeed, because despite using a pretty basic setup of NV12 color format, the stuff that came out of the GPU was not recognized.\u003c/p\u003e\n\u003cp\u003eErgo, I abandoned that idea, because frankly looking at the codebase and also the error messages didn't fill me with the feeling that with a few more days/weeks of work I could get something working out of it - a bit like how seeing old enterprise projects in an outdated version of Java with outdated dependencies and no documentation makes you want to look elsewhere. If anything, that just made me have more respect for the people who maintain OBS and work on it, albeit I did see some \u003ca href=\"https://links.kronis.dev/ktq1y\"\u003ecurious historical code\u003c/a\u003e, like this for example:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003e/* this function is a hack for the annoying intel igpu + dgpu situation. I guess. I don't care anymore. */\nEXPORT void obs_internal_set_adapter_idx_this_is_dumb(uint32_t adapter_idx)\n{\n\tif (!obs)\n\t\treturn;\n\tobs-\u0026gt;video.adapter_index = adapter_idx;\n}\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eI feel like my expectations for a lot of the production code out there are a bit too high - often times it working is probably good enough and there's not that much going on behind the scenes, albeit I'm starting to think that the particular bit of code under \u003ccode\u003elibobs-d3d11/d3d11-subsystem.hpp\u003c/code\u003e could have been worth a look as well, if not for all of the other issues that made me look away from using two Intel Arc GPUs in the same system.\u003c/p\u003e\n\u003ch3\u003eLet's just solve what we can and look past what we can't\u003c/h3\u003e\n\u003cp\u003eA lot of people told me that dual GPU setups aren't really good. In some ways, they were right, even if the reasons for that are quite unfortunate. It's like a \u003ca href=\"https://links.kronis.dev/4yqpx\"\u003eCatch-22\u003c/a\u003e: dual GPU setups suck because nobody writes code with them in mind, because nobody uses them, because when they try, the setups don't work, because the code is not written with them in mind. Unfortunate status quo, but I can't exactly do much about this myself.\u003c/p\u003e\n\u003cp\u003eErgo, I decided to look into a slightly different direction that would still let me do recordings/streams, without the encoder getting overloaded, but first... fixing other issues with the PC! You see, for the most part, the setup has been in a state of \u0026quot;good enough for now\u0026quot;, however clearly the Ryzen 7 5800X is also struggling a bunch with it hitting 90C under full load, which probably won't exactly melt the CPU, but does mean that its clock frequency gets dialed back quite a lot.\u003c/p\u003e\n\u003cp\u003eBehold, an AIO:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"08-new-aio.jpg\" alt=\"08-new-aio\" title=\"08-new-aio\"\u003e\u003c/p\u003e\n\u003cp\u003eThis particular one is from \u003ca href=\"https://links.kronis.dev/on3cc\"\u003eAigo\u003c/a\u003e and I got it from AliExpress for about 50 EUR, which is about as good as the prices get, even on discount. That said, it should provide both better thermals, easier installation (no bulky heatsink on a CPU air cooler), as well as reduce the total amount of fans, because I can replace two case fans and the CPU fan with it, hopefully meaning a bit less noise and better thermals.\u003c/p\u003e\n\u003cp\u003eI don't really do sponsored content and this is no exception, I just went with this particular model because it was about 10-20 EUR cheaper than the stuff available in the online retailers in my country, in addition to me having had decent experiences with other Aigo coolers in the past (you can see that the case fans also come from them, even if they're not quite as beefy as the AIO ones). Another nice thing about the overall setup is that I can also replace my old PSU that has a bit of a rattle to it - since I don't intend to see what happens when its cooling gives out due to bad ball bearings.\u003c/p\u003e\n\u003cp\u003eLong story short, installing the AIO wasn't all that bad, I swapped out the PSU and could clean up at least some of the cabling along the way, albeit the back of the case is still somewhat messy, mostly thanks to me having 6 SATA drives connected in a case that definitely can't handle that many:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"09-no-front-panel.jpg\" alt=\"09-no-front-panel\" title=\"09-no-front-panel\"\u003e\u003c/p\u003e\n\u003cp\u003eI did end up disabling the front RGB strip on the case itself because it's quite annoying and did install the radiator in the front of the case instead of the top, which honestly might be a bit of a mistake from the cooling and airflow perspective (the front panel of the case doesn't have that many air vents, while the top has a mesh panel with a dust filter), though it would get a bit too crowded if I installed the AIO on the top, so this is good enough for now:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"10-aio-working.jpg\" alt=\"10-aio-working\" title=\"10-aio-working\"\u003e\u003c/p\u003e\n\u003cp\u003eSome quick tests without the panels installed also reveal that the temperatures are way better too, even under full load. I will admit that the AIO fans are a bit more beefy than the previous case fans I had, so while the pump itself is pretty quiet all things considered, under full load things can get pretty noticeable with my fan curves, but the thing is that most of the time I have headphones on anyways and the CPU not throttling itself into oblivion is the preferable thing here:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"11-benchmark-no-panel.jpg\" alt=\"11-benchmark-no-panel\" title=\"11-benchmark-no-panel\"\u003e\u003c/p\u003e\n\u003cp\u003eAfter that, I messed around a little bit more with \u003ca href=\"https://links.kronis.dev/jmbe0\"\u003eRyzen Master\u003c/a\u003e to have the PPT, TDC and EDC values at stock so the chip can pull enough power, as well as dialed back the Curve Optimizer value from \u003ccode\u003e-15\u003c/code\u003e back to \u003ccode\u003e-10\u003c/code\u003e, also hoping that this could prevent any instability issues, since at this point I should have had a bit more thermal headroom, right? I also made the memory run a bit faster, from the previous value of \u003ccode\u003e1333\u003c/code\u003e to \u003ccode\u003e1500\u003c/code\u003e, since those particular RAM sticks seem to be good enough for that.\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"12-ryzen-master.jpg\" alt=\"12-ryzen-master\" title=\"12-ryzen-master\"\u003e\u003c/p\u003e\n\u003cp\u003eUnfortunately, putting the side and front panels back on, coupled with the CO value that's closer to stock and slightly less aggressive fan curves means that under synthetic load it's still very much possible to make the CPU thermal throttle, albeit it's a little bit better than before:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"12-benchmark-with-panel.jpg\" alt=\"12-benchmark-with-panel\" title=\"12-benchmark-with-panel\"\u003e\u003c/p\u003e\n\u003cp\u003eThis still wouldn't usually happen, even when compiling software the CPU won't typically be pinned at 100% load all the time, plus I could probably mess around with PPT/TDC/EDC/CO values a bit more if I wanted to, but honestly having an all core sustained clock speed of 4.6 GHz under full load is good enough for me, it doesn't throttle as much as before and under normal loads the temperatures are at least 5-10C lower than before.\u003c/p\u003e\n\u003cp\u003eWith all of that out of the way, what's my secret weapon for preventing the GPU from getting utterly overwhelmed and stuttering, either the games themselves, or the encoder? A program that hooks into \u003ca href=\"https://links.kronis.dev/0itri\"\u003eRTSS\u003c/a\u003e and lets me limit the framerates in games and programs based on the GPU load: \u003ca href=\"https://links.kronis.dev/1k7r4\"\u003eDynamic FPS Limiter\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eWhile my earlier comment on a lot of software out there being a bit jank if not outright broken stands, it's also immensely cool when people write something like this, to build upon good software that already exists and add new features that more or less solve my exact problem:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"dynamic-fps-limiter.jpg\" alt=\"dynamic-fps-limiter\" title=\"dynamic-fps-limiter\"\u003e\u003c/p\u003e\n\u003cp\u003eYou see, the encoding stutters when the GPU load hits 100% for prolonged periods of time, so instead of the GPU dedicating all of its power to the game with no way for me to control that (process priority setting in Windows basically does nothing), now I can limit the maximum framerate that the game can run at when it's becoming too demanding, meaning that for example in intense sections I can drop from 60 FPS to 30 FPS until the situation improves, which is way better than either the framerate being very inconsistent or the recorded video stuttering altogether.\u003c/p\u003e\n\u003ch3\u003eSo now what?\u003c/h3\u003e\n\u003cp\u003eI have to admit that AIO coolers have a reputation for having a bit more of a limited life span compared to air coolers, but it will probably be fine: if everything goes okay, it should last me around 5 years, at which point I can probably look into other options, or just refill/clean it myself, since at that point I would no longer need to worry about a warranty of any sort. Heck, I might consider putting some car coolant in there, to see how that performs, since at that point the system will probably have run its course and won't matter that much.\u003c/p\u003e\n\u003cp\u003eSpeaking of which, I intend to run this computer setup until 2030 at least, maybe swapping out some drives along the way: the CPU itself is good enough and I am quite sick of the modern trend of UE5 games coming out very, very unoptimized which feels like an attempt by the Big GPU to sell more GPUs. Humor aside, I think it's easier for me to just choose not to play those titles and not give those developers money, if they can't release a product that runs well on a plethora of hardware.\u003c/p\u003e\n\u003cp\u003eThere basically is no good excuse not to do that, because most of the game engines out there in current use can scale all the way back to a Nintendo Switch and most of the graphical effects they try to push on people should also be toggleable off if the player so chooses. I will admit that I really enjoy upscaling: FSR/DLSS/XeSS and even the kind that \u003ca href=\"https://links.kronis.dev/nib51\"\u003eLossless Scaling\u003c/a\u003e has, all of which alongside frame generation, in theory, should make it so that you don't have to buy a new GPU for the upcoming decade but can just gradually dial those settings back for more demanding (but optimized) games and still get some enjoyment out of it.\u003c/p\u003e\n\u003cp\u003eClearly, that's not what's actually happening in the gaming industry though (or much anywhere else, seeing how demanding even basic software now is, shipping full browser instances just to render a UI, in part because cross platform solutions for UI aren't very good or easy to work with, compared to webdev technology), where you buy a modern AAA game and need both upscaling and framegen as a crutch just to get to 30 FPS and not even that often times in beloved franchises that are otherwise great.\u003c/p\u003e\n\u003cp\u003eFor example, have a look at this video: \u003ca href=\"https://links.kronis.dev/51apn\"\u003eGTX 1050 Ti: Oblivion Remastered LAG FEST! (Far Cry 4 Sanity Check) \u003c/a\u003e\u003c/p\u003e\n\u003cp\u003e\u003ca href=\"https://links.kronis.dev/51apn\"\u003e\u003cimg loading=\"lazy\" src=\"13-say-it-like-it-is.jpg\" alt=\"13-say-it-like-it-is\" title=\"13-say-it-like-it-is\"\u003e\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003e(just an image if you don't want to look at the video right now, I'm not doing full embeds)\u003c/p\u003e\n\u003cp\u003eIt's similar to how some people like \u003ca href=\"https://links.kronis.dev/9c4p7\"\u003eThreat Interactive\u003c/a\u003e are speaking the truth about modern game development, regardless of the status quo.\u003c/p\u003e\n\u003cp\u003eAt the same time, even on my slightly older A580 some other games that don't use UE5, like \u003ca href=\"https://links.kronis.dev/6ldby\"\u003eArma Reforger\u003c/a\u003e, can both run really well with a lot of the settings turned off on comparatively budget builds, in addition to letting you turn on whatever you want to get most out of your hardware if you want to go in the other direction. Here's a few screenshots, when I did that comparison myself:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"14-low-settings.jpg\" alt=\"14-low-settings\" title=\"14-low-settings\"\u003e\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"15-high-settings.jpg\" alt=\"15-high-settings\" title=\"15-high-settings\"\u003e\u003c/p\u003e\n\u003cp\u003eAs you can see, the game looks pretty good in most cases, except you can feasibly make it run really well even on a somewhat sub-optimal setup (I think I had the A580 combined with a Ryzen 5 4500 back then), which is what I think the future of gaming should be like. What's more, the lower graphical settings do not make the game any less amazing and wonderful than it would otherwise be, you can still have plenty of enjoyment out of it! Whereas if you can't turn the settings down, the game running really badly definitely would ruin it for anyone!\u003c/p\u003e\n\u003cp\u003eBasically, think more along the lines of Doom 2016, instead of Doom: Dark Ages, which also forces ray tracing to be on with no way to turn it off. It's not that ray tracing and similar expensive methods in of themselves are the issue, but just the fact that you don't have the ability to disable it.\u003c/p\u003e\n\u003cp\u003eAnyways, I digress.\u003c/p\u003e\n\u003ch3\u003eSummary\u003c/h3\u003e\n\u003cp\u003eIt's unfortunate that dual GPU setups come with so many issues, even though I still think that they should be the default, to avoid e-waste with old GPUs and offload a whole bunch of tasks to secondary cards. Yet, currently, trying to do that with two Intel Arc cards leads to even video playback not really working that well, whereas my old RX 580 actually had stability issues and some games would pick the AMD card instead of the Intel card, probably thinking that it's an iGPU, instead of following the Windows 11 settings.\u003c/p\u003e\n\u003cp\u003eEither way, I'm happy that I could clean the case up a little bit, the AIO should also help the CPU work better long term, as well as the dynamic framerate limiter and RTSS should give the setup slightly more lifespan, hopefully with more games in the future also supporting XeSS, although Lossless Scaling will surely help and step in when the games don't. I've no idea what the future of gaming looks like, but I'll definitely try to support the developers that care about optimizing their games and making good use of the hardware.\u003c/p\u003e\n\u003cp\u003eIf all goes well, this setup should last me until around 2030 before I need a new significant update, maybe aside from replacing some of the HDDs with less bulky SSDs (those are still a bit expensive over here, at least 1 TB and larger versions) or just replacing drives when they do fail, I still have a backup HDD sitting in a closet, so that I may replace one of the failed ones with it and restore any data from backups.\u003c/p\u003e\n\u003cp\u003eI have to admit that people who said that multi GPU setups don't work well were ultimately right, but I'm still happy that I looked into it all, even if it took a bunch of exploration and blog posts to get here. This will probably be the last hardware focused post for a little bit, I'll most likely get back to writing more about programming and software development after sorting all of this out.\u003c/p\u003e\n"
        },
        {
            "title": "Stop killing games and the industry response",
            "date_published": "2025-06-06",
            "id": "https://blog.kronis.dev/blog/stop-killing-games",
            "url": "https://blog.kronis.dev/blog/stop-killing-games",
            "content_html": "\u003cp\u003eRecently, there's been a \u003ca href=\"https://links.kronis.dev/3d9d6\"\u003eEuropean Citizens' Initiative called \u0026quot;Stop Destroying Videogames\u0026quot;\u003c/a\u003e which by now has hit the milestone of 1'000'000 signatures. You're still encouraged to sign it if you care about its goals and are a EU citizen, since not all of those are likely to be valid signatures, but overall this is a pretty positive trend:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-stop-killing-games.jpg\" alt=\"01-stop-killing-games\" title=\"01-stop-killing-games.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eIf it does indeed get enough valid signatures, it will \u003ca href=\"https://links.kronis.dev/y7l88\"\u003eget passed on to the European Commission\u003c/a\u003e and new laws might get passed as a consequence. However, there has been some opposition to it, so today I'd like to briefly describe what it's about, as well as why some of the people disagree with it, and why they might be quite wrong in doing so.\u003c/p\u003e\n\u003ch3\u003eWhy are video games being killed\u003c/h3\u003e\n\u003cp\u003eFirst up, the actual objectives are pretty concise, here they are:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThis initiative calls to require publishers that sell or license videogames to consumers in the European Union (or related features and assets sold for videogames they operate) to leave said videogames in a functional (playable) state.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eSpecifically, the initiative seeks to prevent the remote disabling of videogames by the publishers, before providing reasonable means to continue functioning of said videogames without the involvement from the side of the publisher.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThe initiative does not seek to acquire ownership of said videogames, associated intellectual rights or monetization rights, neither does it expect the publisher to provide resources for the said videogame once they discontinue it while leaving it in a reasonably functional (playable) state.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eIt was all more or less kicked off by \u003ca href=\"https://links.kronis.dev/8hrw0\"\u003ea YouTube creator named Accursed Farms\u003c/a\u003e, who illustrated why this matters with the example of the game \u003ca href=\"https://links.kronis.dev/5wsjm\"\u003eThe Crew\u003c/a\u003e:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02-videos.jpg\" alt=\"02-videos\" title=\"02-videos.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eSo what does this mean on a practical level?\u003c/p\u003e\n\u003cp\u003eSuppose you bought a game like The Crew back when it came out, in 2014. The game is about racing around in cars and can be played both in singleplayer modes and multiplayer modes, with other players. However, the game is made in such a way, that the account management functionality depends on servers, hosted by Ubisoft. This means that once some time passes and supporting those servers is no longer viable (once that AWS bill starts racking up and there are no new sales), Ubisoft is going to turn them off, just like they did in 2024, making the game unplayable.\u003c/p\u003e\n\u003cp\u003eEven the singleplayer components: you just wanting to race around cars in a world with you and other NPCs in it, is no longer viable. Essentially, you didn't \u0026quot;buy\u0026quot; the game, but in a sense were \u0026quot;renting\u0026quot; it for an indeterminate amount of time, a lease that expired due to the publishers and developers no longer wanting to provide that service for you. Yet, it wasn't marketed to you as a subscription, you thought that you were buying something, but were instead just being misled.\u003c/p\u003e\n\u003cp\u003eI don't think that's acceptable, not even remotely. Objectively, there are also no good reasons (there are reasons, just not good ones) for things to be that way: singleplayer games in general have been around for decades. If Need for Speed 2: SE could do that back in 1996 (which was my first racing game that I played years and years after its initial release on my dad's old work laptop), then surely there's no good reason for the comparably huge corporations of the current day and age to be unable to do so either:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"need-for-speed-2-se.jpg\" alt=\"need-for-speed-2-se\" title=\"need-for-speed-2-se.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eTo sum up what I think should happen, is being upfront when a game is online only and calling it a subscription, something that would be relatively easy to implement (and is often already supported) in all online stores. You don't have to re-architect your entire game if you don't want to say that you are letting people buy it. You can just call it what it is, a subscription.\u003c/p\u003e\n\u003cp\u003eSimilarly, you can sell a singleplayer game with optional multiplayer components: if I have a PC with no Internet connection and if it stayed that way for a decade, the singleplayer components should still work. On the other hand, if you want to sell a game that would otherwise be online only, you need to make it so that once you end your support for it, there must be a way for people to keep playing what they bought. I'll get into some of the ways how to do that later.\u003c/p\u003e\n\u003cp\u003eThat's it. But clearly there's lots of pushback.\u003c/p\u003e\n\u003ch3\u003eThe industry response\u003c/h3\u003e\n\u003cp\u003eOnce the initiative reached its initial signature goal, there was \u003ca href=\"https://links.kronis.dev/i8k98\"\u003ea statement released by Video Games Europe\u003c/a\u003e, an association of a bunch of video game publishers in Europe. First up, it's cool that organizations like that exist, that there's a decent video game industry in the EU as well and some organization around the craft. Secondly, I don't quite agree with the statement, but understand their concerns.\u003c/p\u003e\n\u003cp\u003eSo here it is:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eWe appreciate the passion of our community; however, the decision to discontinue online services is multi-faceted, never taken lightly and must be an option for companies when an online experience is no longer commercially viable. We understand that it can be disappointing for players but, when it does happen, the industry ensures that players are given fair notice of the prospective changes in compliance with local consumer protection laws.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003ePrivate servers are not always a viable alternative option for players as the protections we put in place to secure players’ data, remove illegal content, and combat unsafe community content would not exist and would leave rights holders liable. In addition, many titles are designed from the ground-up to be online-only; in effect, these proposals would curtail developer choice by making these video games prohibitively expensive to create.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eWe welcome the opportunity to discuss our position with policy makers and those who have led the European Citizens Initiative in the coming months.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eRemember how I said that there are no good reasons? Well, they list off some of the bad reasons for being opposed to what the initiative stands for. Let's cut out the fluff, we don't care about how carefully they deliberate before making us unable to play a game that we paid for, while thinking that we were buying something. I don't care about what they believe is a \u0026quot;fair notice\u0026quot; because I'm opposed to the practice in general. The consumer protection laws need to be amended, that's the whole point.\u003c/p\u003e\n\u003cp\u003ePrivate servers are ALWAYS a viable alternative option for anything that needs to talk to a server in an otherwise singleplayer experience.\u003c/p\u003e\n\u003cp\u003eIf a bunch of nerds could sneakily get together and create \u003ca href=\"https://links.kronis.dev/ucf09\"\u003ea private server implementation\u003c/a\u003e for a huge game like Genshin Impact, purely be reverse engineering the network traffic, then surely the first party developers should have no issue doing that either:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03-grasscutter.jpg\" alt=\"03-grasscutter\" title=\"03-grasscutter.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eCuriously, they later stopped supporting it, hopefully not due to under the board legal threats, but in the case of \u0026quot;dead\u0026quot; games like Battlefield: Bad Company 2 there have also been initiatives like \u003ca href=\"https://links.kronis.dev/f24ao\"\u003eProject Rome\u003c/a\u003e and similar ones. When the game developers act like scumbags and claim supporting the game is no longer possible, the community proves them wrong with simple emulation and redirecting network traffic:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04-it-is-that-simple.jpg\" alt=\"04-it-is-that-simple\" title=\"04-it-is-that-simple.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eSo, why the opposition? It's because their development practices are crap. They complain that there's content moderation systems and middleware that they cannot expose. Make those optional, then. For example, user created skins don't need to be in the singleplayer version of the games. Something like leaving behind items in the world like in Death Stranding doesn't also need to be there in a fully singleplayer game. You just need to let the players have the core experience, or alternatively, export whatever data you can at the end of the game's lifecycle and provide that as a static dump (e.g. messages left by players in something like Dark Souls).\u003c/p\u003e\n\u003cp\u003eFurthermore, the claim that many games are designed from the bottom up to be online only: I'm not responsible for your poor architecture decisions. You follow the laws we make, not the other way around. If you complained that it's too hard to hash the passwords entered by your users or to encrypt medical data, or that you don't feel like complying with GDPR, you'd get rightfully fined out of existence. Furthermore, the fact that pretty much most software can now be packaged and run in a Docker container is kind of embarrassing for you. You think the developers really can't be bothered to build their server software and even give the players a distribution ZIP file (or even the source code) and just provide a way to make your local game talk to something that's running locally?\u003c/p\u003e\n\u003cp\u003eNo, the thing is that they just don't want to. But it doesn't matter what they want. Remember the principles of good software development: making your software modular? Degrading service gracefully when not all services are available (e.g. not being able to connect to chat shouldn't mean that you can't play a round in a game altogether)? Do more of that, ship whatever components you can to ensure the continuity of the core gameplay and let everything else fail gracefully:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"05-it-is-that-simple.jpg\" alt=\"05-it-is-that-simple\" title=\"05-it-is-that-simple.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eNow in practice, obviously these can be more or less difficult depending on the game in question - sometimes developers go crazy with the amount of the services that a product needs, but just because you've made a mockery of the big ball of mud architectures of the olden day by implementing a big mess of services doesn't mean that publishing some of those after you end the support for a game is any less viable, your incompetence doesn't excuse that.\u003c/p\u003e\n\u003cp\u003eAnd if things are indeed that bad, that some of the core software you use is proprietary and that you cannot release it due to licensing issues, then either provide a simplified shim (e.g. we don't need some hyper-scalable account management solution, the same API that's backed by SQLite is more than enough), or just stop building products based on proprietary crap in the first place.\u003c/p\u003e\n\u003cp\u003eIt's the same with various art assets, just look at \u003ca href=\"https://links.kronis.dev/xfomt\"\u003eUFO2000\u003c/a\u003e, a remake of \u003ca href=\"https://links.kronis.dev/mvgfi\"\u003ethe older X-COM game\u003c/a\u003e, that has its own (arguably placeholder) assets but can also import the official data files and swap out the placeholders as needed. If you as a developer have the license for some of your music or other assets expire, just replace them with blank MP3s or checkerboard pattern textures (or, you know, draw up your own BMWn't logo and texture in an afternoon for all I care) and let the community figure everything else out. Boxes on wheels with the gameplay that you love, that someone can fix later in Blender and have proper models for, beats having a dead game.\u003c/p\u003e\n\u003cp\u003eI don't quite enjoy Richard Stallman's personality due to all of the controversy that he's had over the years, but when it comes to talking about \u003ca href=\"https://links.kronis.dev/r53ph\"\u003ethe ethics of software\u003c/a\u003e, his free software movement is right time and time again. Honestly at this point I'd even take source available, or just some stripped down binary release that keeps a game or other piece of software I need going, a bit like how I used the old \u003ca href=\"https://links.kronis.dev/giz4x\"\u003eBvckup\u003c/a\u003e tool for the longest time ever, even after the end of support.\u003c/p\u003e\n\u003ch3\u003eThe Pirate Software controversy\u003c/h3\u003e\n\u003cp\u003eThere was also \u003ca href=\"https://links.kronis.dev/a0x0e\"\u003ea video released by a popular creator named Pirate Software\u003c/a\u003e, which was quite critical of the initiative:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"09-pirate-software-video.jpg\" alt=\"09-pirate-software-video\" title=\"09-pirate-software-video.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eSince then, a lot of other people came out and critiqued both the arguments that he made in the video, alongside more personal attacks (one's ego shouldn't get in the way and you should admit when you're wrong, instead of doubling down; then again, we are all human). The video itself critiques the technical feasibility of whether this can be done and also blames it for being quite vague.\u003c/p\u003e\n\u003cp\u003eI will admit that the text of the initiative is indeed somewhat vague. Then again, it's not a law by itself, but just puts forth the bad industry practices for consideration. I have quite the negative take of the legal systems in general, where the letter of the law is supposed to be followed and enforced instead of the spirit of the law, which leads to a lot of issues when it's impossible to ever get the laws to be written down perfectly and without loopholes. That said, it doesn't mean that we shouldn't try to make things better at all! Nor does it mean that laws can't be improved down the road.\u003c/p\u003e\n\u003cp\u003eI will also say that he's spot on about the part where he suggests that the language around acquiring a game is a large part of the problem and that it should be addressed directly! That is also my problem with the state of the industry as I explained and something that should have definitely been a paragraph in the initiative, one of the main points of focus, as a matter of fact!\u003c/p\u003e\n\u003cp\u003eThe rest of the critique isn't valid - you don't need to move from a client-server model, you just need to make a big enough subset of the server to be runnable either locally, or on a server that you control yourself. This is already the case with pretty much every Minecraft server out there and has worked as an approach for decades.\u003c/p\u003e\n\u003cp\u003eWe don't need to be apologetic for the companies, or to claim that games having a low playerbase somehow justifies screwing over the few people who'd like to enjoy the game for a bit longer, even a decade later. The game preservation movement is largely against that sort of practice in the first place. It also doesn't mean that companies need to run their current live service games indefinitely, just do a proper job of sunsetting those games, instead of just pulling the plug.\u003c/p\u003e\n\u003cp\u003eI'm not sure whether the person is doing that for the sake of stirring up drama and views, whether they either misunderstand things or just focus on all the potential negatives (like I myself am guilty of doing, just look at \u003ca href=\"https://links.kronis.dev/ez224\"\u003emy blog post about AI brainrot\u003c/a\u003e), but either way they're wrong. I do suspect foul play because someone with years of industry experience should know better, not spread misinformation about the technical feasibility of it.\u003c/p\u003e\n\u003cp\u003eI will concede that this might take additional months of development work, but that's just the cost of doing business, if you let your devs roam freely (or rather forced them into a particular architecture and coupling) and didn't, for example, put interfaces in front of your implementations. None of that would even be a problem if you just had a few meetings before the initial implementation even starts, since \u003ca href=\"https://links.kronis.dev/mdtg6\"\u003ethe curve of the costs of software changes\u003c/a\u003e has been known in the software development circles for decades.\u003c/p\u003e\n\u003ch3\u003eThe ugly truth\u003c/h3\u003e\n\u003cp\u003eLet's suppose the initiative goes through and some well spirited EU law gets passed. You know what happens next?\u003c/p\u003e\n\u003cp\u003eThe same thing that happened with GDPR, with cookie consent in particular, have a look at \u003ca href=\"https://links.kronis.dev/rnfzc\"\u003eDark Patterns after the GDPR: Scraping Consent Pop-ups and Demonstrating their Influence\u003c/a\u003e or maybe have a look at some of \u003ca href=\"https://links.kronis.dev/9zig4\"\u003ethe dark patterns\u003c/a\u003e that were often knowingly employed by companies to coax users into submitting to whatever they want:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eWe scraped the designs of the five most popular CMPs on the top 10,000 websites in the UK (n=680). We found that dark patterns and implied consent are ubiquitous; only 11.8% meet the minimal requirements that we set based on European law.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eYou'd probably see the exact same trend in regards to games: packages that are unreasonably hard to compile, modes that are technically there but very much broken without supposedly optional online components, sometimes adding trash singleplayer modes for the sake of being able to say that you are selling a game instead of providing a subscription BUT having that mode be incredibly bad - similar to how in War Thunder there is the \u003ca href=\"https://links.kronis.dev/pxo77\"\u003eassault mode\u003c/a\u003e which just spawns dumb bots in waves that nobody really likes to play (despite being an otherwise great multiplayer game), compared to more competently done PvE in something like Armored Warfare, which has both PvE modes with better AI, as well as \u003ca href=\"https://links.kronis.dev/1jxw4\"\u003eco-op campaigns\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003eIt's not just about the difficulty in supporting games long term or the complexities of having to build modular software: it's very much about control.\u003c/p\u003e\n\u003cp\u003eIf a game or any piece of software is easy to configure to run locally or swap out components in, then suddenly if you as a company do too many disagreeable things, then users have a choice not to do business with you. All of those microtransactions and lootboxes etc., suddenly you'd actually need to price them in ways where people are okay with paying for them, instead of having a more or less addicted playerbase that you hold hostage.\u003c/p\u003e\n\u003cp\u003eI actually use War Thunder as a silly example because it's a free to play game and you don't really buy it, but if they hypothetically decided to sell it to you for a one time payment, suddenly further premium vehicles that can \u003ca href=\"https://links.kronis.dev/tze7q\"\u003esometimes cost close to 100 EUR\u003c/a\u003e would be a tough sell, when the community could come together and sneak in their own custom server implementations:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"06-war-thunder-pricing.jpg\" alt=\"06-war-thunder-pricing\" title=\"06-war-thunder-pricing.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eWhy do I construct such an example? Because it already happened, with Escape From Tarkov. It is a live service game that you buy, with \u003ca href=\"https://links.kronis.dev/x2ikb\"\u003ethe Unheard Edition costing around 200 EUR\u003c/a\u003e and having PvE as one of the selling points, something that did get quite a bit of criticism online:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"07-escape-from-tarkov.jpg\" alt=\"07-escape-from-tarkov\" title=\"07-escape-from-tarkov.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eThe thing is, the community wouldn't stand for this and there are projects right now that give you the ability to play Tarkov in singleplayer with a locally run and highly customizable server: \u003ca href=\"https://links.kronis.dev/whmvj\"\u003eSPT-AKI\u003c/a\u003e and not only that, but there's also \u003ca href=\"https://links.kronis.dev/xezbu\"\u003ea lively modding scene\u003c/a\u003e, which makes the game truly yours to play as you please. Want to turn the sweaty and difficult shooter into something where both you and bots can magdump each other from cover for a minute before anyone goes down, something closer to how those sorts of things actually work in reality? Sure thing!\u003c/p\u003e\n\u003cp\u003eThe sudden problem there is that of course people made it reach its logical conclusion and there's also a mod that everyone's very hushed about, that actually allows you to play Tarkov in co-op with the custom server: \u003ca href=\"https://links.kronis.dev/s7f5e\"\u003eProject Fika\u003c/a\u003e and suddenly, you don't need to pay around 200 EUR to have co-op, nor even buy the game at all to play it however you please. Yes, as it turns out, there are pirated versions (that I shall not link, as they're both illegal and not \u003cem\u003edirectly\u003c/em\u003e related to the aforementioned projects) where you can basically play a version of EFT that's better than the live service game that you don't control, in pretty much every way.\u003c/p\u003e\n\u003cp\u003eThis is what happens when you give the ability to the players to control their experience: they will optimize it for their own enjoyment of the gameplay, instead of the company's need for monetization. Admittedly, some companies might even care about viewing games as both an enjoyable hobby and sometimes an art form, but they absolutely will use every trick in the book to fight against anything that hurts their bottom line - their ability to make money.\u003c/p\u003e\n\u003cp\u003eThus, you'll see both the dark patterns mentioned above, as well as the (unfortunately true) allusions to how giving up any control might result in increased piracy. At the same time, I'm pretty sure that most people who actually play games do realize how harmful DRM is and why \u003ca href=\"https://links.kronis.dev/01wet\"\u003eplatforms like GOG\u003c/a\u003e are really nice in that regard, with their official DRM free releases.\u003c/p\u003e\n\u003ch3\u003eSummary\u003c/h3\u003e\n\u003cp\u003eI really wish people could do things in good faith. More games should have self-hostable servers. More games should have the source code released (yes, to the whole game and whatever assets the developers can release, with placeholders for everything else) once they are no longer supported. But this goes directly against the desires of publishers and developers - why would anyone give you money for a new release that's made arguably badly and mostly in the name of making fat stacks of money, versus an older version of the game that's lovingly kept alive by the community?\u003c/p\u003e\n\u003cp\u003eWe end up with the disconnect between games as a product vs something closer to an art form, or a niche interest that builds a rich and passionate community around it, where the latter actually has the potential to both extend the life of a game and do great good for the developers: just look at \u003ca href=\"https://links.kronis.dev/b875y\"\u003ehow many mods Skyrim has\u003c/a\u003e, many of which make the game way more enjoyable even a decade after release (it was actually released in 2011, I feel old now):\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"08-skyrim-mods.jpg\" alt=\"08-skyrim-mods\" title=\"08-skyrim-mods.jpg\"\u003e\u003c/p\u003e\n\u003cp\u003eSometimes there are fixes for games that were rushed to release, \u003ca href=\"https://links.kronis.dev/jgaud\"\u003elike what happened to Vampire: The Masquerade - Bloodlines\u003c/a\u003e or even \u003ca href=\"https://links.kronis.dev/xx3rf\"\u003emods like Deus Ex: Revision\u003c/a\u003e which also expand on what could have been the original vision for the game, as well as restorinng content that was unfinished and cut from the original release.\u003c/p\u003e\n\u003cp\u003eAt the same time, some of this definitely can hurt the business behind those games. That's why I think the mismatched incentives are quite unfortunate, but at the end of the day, consumers shouldn't be misled - if that means calling something a subscription when it clearly is one instead of a purchase, then so be it. Even if it's for an indeterminate amount of time (for how long the developers/publishers can and want to support it).\u003c/p\u003e\n\u003cp\u003eWhen it comes to the actual development, I think that the GNU people were once again correct: \u003ca href=\"https://links.kronis.dev/w5ffm\"\u003eService as a Software Substitute\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eIf I have to integrate with your software (regardless of whether in a web development context, or something else) and you can't give me a test environment, a locally runnable Docker container, or even a simple statically compiled executable (Go is really great for this) that can run against SQLite or an embedded key-value store of your choice, I'm not going to be eager about being your friend.\u003c/p\u003e\n\u003cp\u003eLastly, I think that the buy vs subscribe distinction should apply to everything, even your IoT devices and all other software, not just games.\u003c/p\u003e\n"
        },
        {
            "title": "It works on my Docker",
            "date_published": "2025-05-16",
            "id": "https://blog.kronis.dev/blog/it-works-on-my-docker",
            "url": "https://blog.kronis.dev/blog/it-works-on-my-docker",
            "content_html": "\u003cp\u003eI like the concept of containers and I like \u003ca href=\"https://links.kronis.dev/ysakp\"\u003eDocker\u003c/a\u003e. Despite jails and LXC also having been around for a while, Docker has really nice DX and no wonder that it's pretty widely used nowadays (as well as the other OCI compatible solutions).\u003c/p\u003e\n\u003cp\u003eIt's a pretty good way to get rid of some of the environment specific requirements and also have a consistent way of packaging applications, giving and limiting their resources, providing configuration and all that. I use it to run most of the software on my servers and some of the software locally nowadays, when it works, it's great.\u003c/p\u003e\n\u003cp\u003eHowever, they are lying to you. The core premise is that if you build a container and it runs, then the same container with the same configuration will run elsewhere, for example: locally, on your CI server and also on your environments.\u003c/p\u003e\n\u003cp\u003eThat is false.\u003c/p\u003e\n\u003ch3\u003eThere is no software reproducibility in the real world (your enterprise Java app at $DAYJOB)\u003c/h3\u003e\n\u003cp\u003eHave a look at this, I have an application that runs locally, in a container that I built:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-app-is-running.jpg\" alt=\"01-app-is-running\" title=\"01-app-is-running\"\u003e\u003c/p\u003e\n\u003cp\u003eThen, when the same \u003ccode\u003eDockerfile\u003c/code\u003e is used to build it on a CI server, when deployed on the environment, I get a circular dependency error:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02-circular-dependency-error.jpg\" alt=\"02-circular-dependency-error\" title=\"02-circular-dependency-error\"\u003e\u003c/p\u003e\n\u003cp\u003eYou might think that there are configuration changes or something else that is causing this issue. Nope, because if I change the configuration:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03-configuration-changes.jpg\" alt=\"03-configuration-changes\" title=\"03-configuration-changes\"\u003e\u003c/p\u003e\n\u003cp\u003eThen the error also changes:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04-another-error.jpg\" alt=\"04-another-error\" title=\"04-another-error\"\u003e\u003c/p\u003e\n\u003cp\u003eThe error itself is cause by working on a legacy project that has circular dependencies, but Spring Boot gives you a way around that until you can fix the code (that time hopefully not being never), where you can \u003ca href=\"https://links.kronis.dev/kj6f6\"\u003eenable that configuration\u003c/a\u003e, as mentioned above:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003espring.main.allow-circular-references=true\nspring.main.lazy-initialization=true\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eOf course, that does nothing for explaining why it works locally in a container, but not in the same sort of container if it's built on a CI server.\u003c/p\u003e\n\u003ch3\u003eWhat a mess\u003c/h3\u003e\n\u003cp\u003eAnd honestly? I don't care.\u003c/p\u003e\n\u003cp\u003eIf it doesn't work, then it doesn't work and no amount of appeal to complexity of modern software will make me not hate this.\u003c/p\u003e\n\u003cp\u003eThat said, I don't hate Docker. I hate \u003ca href=\"https://links.kronis.dev/02pbt\"\u003eSpring Boot\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003eMaybe not for creating a viable alternative to the likes of \u003ca href=\"https://links.kronis.dev/96ipg\"\u003eASP.NET\u003c/a\u003e but in the JVM ecosystem, that part is actually really nice, especially because of all the integrations and how productive it can make you when things work. Maybe not even for the performance impact it has compared to \u003ca href=\"https://links.kronis.dev/rdscc\"\u003esome of the other options\u003c/a\u003e since the likes of \u003ca href=\"https://links.kronis.dev/tzw5o\"\u003eDropwizard\u003c/a\u003e can be slower and I still enjoy those, since productivity and ease of use do often trump raw performance.\u003c/p\u003e\n\u003cp\u003eBut definitely for their approach to \u003ca href=\"https://links.kronis.dev/j6pv8\"\u003edependency injection\u003c/a\u003e, where a lot of it is done at runtime, as well as all of the proxying garbage that goes on under the hood, often ruining not just your stack traces, not only your transactional code, but also plenty of your sanity along the way. If you have a framework that throws runtime errors for code that you could have checked at runtime, you have largely failed to create a good DI system.\u003c/p\u003e\n\u003cp\u003eOh, and before you say:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eJust fix the code\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eI'd love to do that, but I have no idea how much time I'd need to do that across the whole codebase that is approaching a million lines of code. Plus, guess what, Spring Boot throws one error at a time, once you compile and launch the app, so that would be a lot of iteration and time, which I don't exactly have in ample amounts.\u003c/p\u003e\n\u003ch3\u003eSummary\u003c/h3\u003e\n\u003cp\u003eI'm not even sure why the likes of \u003ca href=\"https://links.kronis.dev/qvhrf\"\u003eDagger\u003c/a\u003e didn't catch on, but Spring DI is the one thing I absolutely despise, to the point where it makes me think more poorly of DI in general. Never believe when DI frameworks promise you that they'll make your life easier, because sooner or later their dynamic nature will burn you.\u003c/p\u003e\n\u003cp\u003eMaybe we still should ship my laptop, since it works there.\u003c/p\u003e\n"
        },
        {
            "title": "Windows bootloader: it just works",
            "date_published": "2025-05-11",
            "id": "https://blog.kronis.dev/blog/windows-bootloader-it-just-works",
            "url": "https://blog.kronis.dev/blog/windows-bootloader-it-just-works",
            "content_html": "\u003cp\u003eHere's a blog post of something positive that I recently experienced: the Windows bootloader is actually kinda cool!\u003c/p\u003e\n\u003cp\u003eI did move from Windows 10 to Windows 11 by installing it on a new 1 TB SSD, which will give me way more space than my old ~240 GB SSD had, in addition to future proofing it a bit more, as well as \u003ca href=\"https://links.kronis.dev/gpble\"\u003eallowing my dual GPU setup to mostly work\u003c/a\u003e (even despite the interesting title of that blog post, it turned out to be mostly fine).\u003c/p\u003e\n\u003cp\u003eHowever, there were also some arguably stupid things that the Windows 11 installer did, which ended up with me having to move the bootloader over to another drive, in addition to shrinking the partitions to accommodate it. But let's not get ahead of ourselves here. Let's start by illustrating that the typical partition layout on a drive for a Windows 11 install might look like the following:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eEFI partition (500 MB) - this contains the bootloader for your UEFI setup\u003c/li\u003e\n\u003cli\u003eMSR partition (16 MB) - this doesn't have user accessible data but is needed for Windows to work properly\u003c/li\u003e\n\u003cli\u003edata partitions for your drives (however much space you have) - where your \u0026quot;C: drive\u0026quot; would reside\u003c/li\u003e\n\u003cli\u003erecovery partition (650 MB) - this contains some tools for recovering from situations where the OS isn't bootable (\u003ca href=\"https://links.kronis.dev/os9uq\"\u003eWinRE\u003c/a\u003e)\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eWhen you install Windows 11 on a blank drive, typically all of these will be created.\u003c/p\u003e\n\u003ch3\u003eMy situation\u003c/h3\u003e\n\u003cp\u003eIn my case, while I did install the OS on a blank drive, I still had a drive with Windows 10 connected to it (my old SSD), so the installer basically saw that there already is an EFI partition on the old drive and reused it, instead of creating another one on the new drive. Basically, this saved about 500 MB of space (which I don't care about) but also means that I can't wipe the old drive, because then the system would no longer boot.\u003c/p\u003e\n\u003cp\u003eObviously, this was not acceptable, so I came up with a slightly wild plan: instead of reinstalling the whole thing again, I would just shrink the data partition on the new drive, move it a bit to the right, also move the MSR partition, and then squeeze the EFI partition from the old drive (straight up cloning it) in at the very start of the new drive. Then, all I'd have to do is wipe the old one clean and tell the BIOS to use the bootloader on the new drive:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"00-the-setup.jpg\" alt=\"00-the-setup\" title=\"00-the-setup\"\u003e\u003c/p\u003e\n\u003cp\u003eIn general, this is something that you'd regard as a \u0026quot;bad idea\u0026quot;, because there's a serious chance of things going wrong and experiencing data loss. Thankfully, I also had some Seagate 1 TB HDDs laying around as well as both some 2.5\u0026quot; and 3.5\u0026quot; enclosures, so I could just flash \u003ca href=\"https://links.kronis.dev/vf0gy\"\u003eRescuezilla\u003c/a\u003e on a USB drive and use it to backup the entire drive that I was about to mess around with:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-backups.jpg\" alt=\"01-backups\" title=\"01-backups\"\u003e\u003c/p\u003e\n\u003cp\u003eHonestly, it is great software! It's based on \u003ca href=\"https://links.kronis.dev/nwvq2\"\u003eClonezilla\u003c/a\u003e except also has a nice GUI and a desktop environment alongside tools like \u003ca href=\"https://links.kronis.dev/ulvtm\"\u003eGParted\u003c/a\u003e. It is simple to use and could do a sparse copy of the NTFS volume as well, so instead of 953 GB to copy instead it just needed to move the actual ~200 GB of things installed on the OS. In my eyes, this is definitely a piece of software that you should have flashed on a USB stick/SD card somewhere.\u003c/p\u003e\n\u003cp\u003eDefinitely one of those pieces of software that you should donate a bit of money to if you find it useful (I also donated to \u003ca href=\"https://links.kronis.dev/5lefa\"\u003eFreeFileSync\u003c/a\u003e, that one is great too), they even seem to have Patreon, although I couldn't find a way to donate to Clonezilla.\u003c/p\u003e\n\u003cp\u003eIf you do have a bit of a closer look at the admittedly low quality photo I took of my screen at about 1 AM when I was doing this process (since I couldn't be bothered to figure out where/how to persist screenshots in the LiveCD), you'll notice that the indicated speed is around 11.50 GB/minute or just under 200 MB/s. It later dropped a bit more, but that's how I realized that my new SSD isn't all that good.\u003c/p\u003e\n\u003ch3\u003eQuick tangent: SSDs are not made equal\u003c/h3\u003e\n\u003cp\u003eThere are various types of NAND flash used in the manufacture of SSDs: \u003ca href=\"https://links.kronis.dev/6n23z\"\u003eSLC, MLC, TLC and QLC\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003eEssentially, as the bits per cell of memory increase, you do get more storage, but both worse durability, as well as lower performance. To offset this, many of the SSDs out there use SLC cache to improve performance: meaning that under \u0026quot;normal\u0026quot; usage most of the time they'll appear to be reasonably fast, but in those cases where you try to copy a lot of data in one go, the cache will get exhausted and the write speeds will decrease.\u003c/p\u003e\n\u003cp\u003eThe problem is that very often none of this is advertised and while TLC is pretty good, if you end up with a QLC drive you're usually not gonna have a very good time if you want to use it for storing a lot of data or doing backups with it, which involve moving around a lot of data in one go. For example, look at this e-commerce listing for the drive I bought:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02-ssd-drive.jpg\" alt=\"02-ssd-drive\" title=\"02-ssd-drive\"\u003e\u003c/p\u003e\n\u003cp\u003eLook at \u003ca href=\"https://links.kronis.dev/dr0uw\"\u003ethe vendor's page\u003c/a\u003e which has a data sheet that isn't that much more illuminating:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03-drive-specs.jpg\" alt=\"03-drive-specs\" title=\"03-drive-specs\"\u003e\u003c/p\u003e\n\u003cp\u003eIt also isn't very clear about what sort of a memory it uses, in particular:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eAdvanced 3D-Nand technology with SLC data boost-cache\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003ewhat on earth is that even supposed to mean? I'm saying that you probably want to look up the drive you want to buy before committing to a purchase, because I experienced first hand how the performance wasn't exactly great. I'm not sure how withholding details like that is okay, as well as not listing the actual read and write speeds for large sequential transfers once the cache is exhausted, nor mentioning how big the SLC cache is in the first place, yet everyone seems to be doing that.\u003c/p\u003e\n\u003cp\u003eI did run some benchmarks and you can see the results below:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eupper left (H: drive): my old Patriot Blast drive, around ~240 GB in capacity\u003c/li\u003e\n\u003cli\u003eupper right (C: drive): my new OS boot drive, Intenso Top, 1 TB\u003c/li\u003e\n\u003cli\u003elower left (D: drive): one of my data drives, a Seagate 1 TB HDD\u003c/li\u003e\n\u003cli\u003elower right (F: drive): my new games drive, Intenso Top, 1 TB\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eHere's the read and write speeds:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04-drive-performance.jpg\" alt=\"04-drive-performance\" title=\"04-drive-performance\"\u003e\u003c/p\u003e\n\u003cp\u003eThere's certainly a bit of variance between the test runs but I didn't really have hours to spend on this, this is more or less illustrative, yet also quite revealing: you can see that for large sequential transfers my 1 TB Seagate HDDs do end up being better than the 1 TB, while for more read heavy workloads and ones that might involve a lot of smaller files being accessed randomly (such as for a boot drive for an OS), the SSDs still are a decent fit regardless of the type, which also applies for things like games that are primarily read heavy workloads.\u003c/p\u003e\n\u003cp\u003eEither way, I don't believe I can complain that much here aside from advising others to do some research, because the pricing itself was not that bad, although I could have probably used the money a bit better had I been more careful. It's still nice to not fear running out of space for my Windows 11 install though and to have a drive that is quite unlikely to have that many reliability issues anytime soon vs a drive that I've been using in my main PC for years:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"05-drive-status.jpg\" alt=\"05-drive-status\" title=\"05-drive-status\"\u003e\u003c/p\u003e\n\u003cp\u003eBack to the actual OS bootloader move process for now!\u003c/p\u003e\n\u003ch3\u003eThe data move\u003c/h3\u003e\n\u003cp\u003eAfter doing the backup with Rescuezilla, I then proceeded to use GParted for editing the partition layout and copying the EFI partition over. I will say that it actually had some bugs in it, such as the copied partition initially showing up as 499 MB and then trying to increase its size before the write operation to the proper size of 500 MB creating a separate step:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"06-partition-scheme.jpg\" alt=\"06-partition-scheme\" title=\"06-partition-scheme\"\u003e\u003c/p\u003e\n\u003cp\u003eThis would actually fail later and make me repeat the EFI partition move, but thankfully the more serious initial step of shrinking the NTFS partition without data loss and then proceeding to move it to the end of the freed up space to leave the start of the drive free (after also moving the MSR partition) worked just fine. It did take a bunch of time, more or less double of what was indicated here at the start, because it had to go over all of the partition and couldn't do what I'd call a \u0026quot;sparse move\u0026quot;, but at least it worked out fine in the end:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"07-partition-move-process.jpg\" alt=\"07-partition-move-process\" title=\"07-partition-move-process\"\u003e\u003c/p\u003e\n\u003cp\u003eI do quite also enjoy software like \u003ca href=\"https://links.kronis.dev/r9h6k\"\u003eMiniTool Partition Wizard\u003c/a\u003e, although their \u003ca href=\"https://links.kronis.dev/oeafg\"\u003ebootable version\u003c/a\u003e seems to be way out of my price point. There's also the \u003ca href=\"https://links.kronis.dev/l8b9u\"\u003eAOMEI Partition Assistant\u003c/a\u003e but honestly a Linux distro LiveCD with GParted is fine for bootable stuff and the built in Windows partition editor is fine for messing around with regular data drives, especially for something I use very infrequently.\u003c/p\u003e\n\u003cp\u003eNow, here's the good news: it worked. I moved the EFI partition over to the start of the new drive (even if I had to do the cloning \u003cem\u003eafter\u003c/em\u003e moving the old partitions, only then the size would show up correctly as 500 MB from the start in GParted), wiped the old drive and when I started up my computer, the BIOS correctly showed that there is a bootloader on the new drive. Furthermore, even if the actual partitions had been moved around a bunch, I didn't even need to do boot repair, it literally just decided to boot up the OS fine. I'm not quite sure how they managed to make it work like that (probably references by UUID or something), but that's pretty great!\u003c/p\u003e\n\u003cp\u003eOf course, initially, I did try to do a boot repair anyways, thinking that moving partitions around would necessitate, which on the other hand was pretty useless and just gave me a generic error:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"08-startup-repair-useless.jpg\" alt=\"08-startup-repair-useless\" title=\"08-startup-repair-useless\"\u003e\u003c/p\u003e\n\u003cp\u003e(this image is borrowed from the Internet, because I didn't take another phone picture of it)\u003c/p\u003e\n\u003cp\u003eI did later run into some silly situations with FreeFileSync when I decided to setup some backups for my new OS and drive setup. The old drive will also store my programming projects, so loading them and working with them won't slow down the main OS install on the boot drive, basically software can go wild with node_modules or whatever. However, when copying files over, it seems like the underlying filenames were updated correctly, yet somehow the display names of the folders on my backup drive were the same:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"09-windows-are-you-okay.jpg\" alt=\"09-windows-are-you-okay\" title=\"09-windows-are-you-okay\"\u003e\u003c/p\u003e\n\u003cp\u003eIt wasn't a bit issue because those two folders are pretty small and I could see the actual values in Command Prompt, PowerShell and also Git Bash, so fixing things wasn't that difficult. Basically, the actual folder names were, for example, \u003ccode\u003eDownloads\u003c/code\u003e and \u003ccode\u003emyuser Downloads\u003c/code\u003e, yet for whatever reason Explorer got confused.\u003c/p\u003e\n\u003ch3\u003eSummary\u003c/h3\u003e\n\u003cp\u003eRegardless of the few challenges along the way, now I have a working OS install with a bootloader on the same drive, which works great. If I ever want to replace the old SSD with something else, then I'll also be able to do so, ripping it out will no longer risk my OS becoming unbootable. I don't quite understand why the Windows 11 installer didn't give me an option to put the bootloader on the new drive as well, or even do it by default, cause if people ever want to upgrade from their new OS install to a new one on a new drive, they would probably expect to be able to pull the old drive with no issues, yet this would make the OS unbootable.\u003c/p\u003e\n\u003cp\u003eThe bootloader itself is pretty good though and it was pleasant to see that I can just clone a partition to another drive without even having to regenerate the UUID or rebuild anything, the Windows bootloader just worked: who knows, maybe Todd Howard helped them work on it in secret. I was also considering maybe going for a Linux dual boot setup, but if I ever have to do that, I'll probably move some of my data drives over to my homelab servers (maybe even get Samba working instead of having to rely on RaiDrive) and do that on an entirely new SSD, since currently all of the SATA slots are filled out (and I had to ditch my M.2 with Linux due to needing the PCIe lanes for the second GPU):\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"10-current-storage-setup.jpg\" alt=\"10-current-storage-setup\" title=\"10-current-storage-setup\"\u003e\u003c/p\u003e\n\u003cp\u003eI do also have to say that I don't really feel bad about having a bunch of 1 TB drives: for example, E is the backup drive that gets select data from C, D, F, G and H, then F is the dedicated game drive, whereas D and G is there for all sorts of files. It wasn't super expensive to buy them (there was a time when those Seagate 1 TB HDDs were about 40 EUR a pop), I still have some backups in a cupboard in packages if I ever need to pull one, plus I can also easily backup the contents of an entire drive with WinSCP to one attached to my homelab servers if I feel like it.\u003c/p\u003e\n\u003cp\u003eCould I have done most of this with fewer but beefier drives? Sure, but those drives might end up being more expensive overall, since I can't just buy one drive but also need a backup (or multiple) in case it fails. Drives don't seem to fail that often, but I've definitely had 2-3 die over the years, so in closing - definitely make sure that you have backups of your data as well!\u003c/p\u003e\n"
        },
        {
            "title": "What is ruining dual GPU setups",
            "date_published": "2025-05-09",
            "id": "https://blog.kronis.dev/blog/what-is-ruining-dual-gpu-setups",
            "url": "https://blog.kronis.dev/blog/what-is-ruining-dual-gpu-setups",
            "content_html": "\u003cp\u003eRecently I posted about the woes of \u003ca href=\"https://links.kronis.dev/58jcj\"\u003etrying to run two Intel Arc GPUs in the same system\u003c/a\u003e and how I couldn't easily use Windows 10 because it doesn't let me split tasks between them properly, nor use one of the cards for encoding in OBS, because the support for choosing which GPU to use for encoding just isn't there for Intel Arc cards (I'd need an Nvidia card for that, or wait until someone writes that feature).\u003c/p\u003e\n\u003cp\u003eSince then, I ended up moving to Windows 11 and some things got better, but others not so much. Since I have more information to share, I figured that I might as well make another blog post. For what it's worth, Windows 11 actually seems more stable than Windows 10 for me: I haven't had a GPU crash or system freeze since installing it, the resource usage is okay and with \u003ca href=\"https://links.kronis.dev/bcpvt\"\u003eRufus\u003c/a\u003e you still can make a local user account easily.\u003c/p\u003e\n\u003cp\u003eI'm surprised to say this, but it genuinely might replace Linux Mint as my daily driver distro, especially because it has compatibility with all of my games and a lot of the software I use anyways. Or rather, it should, even in my dual GPU setup, though things aren't always as simple. You see, there is a menu under \u003ccode\u003eSystem \u0026gt; Display \u0026gt; Graphics\u003c/code\u003e which lets you easily choose, which program will be handled by which of your GPUs:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"gpu-profile-set.jpg\" alt=\"gpu-profile-set\" title=\"gpu-profile-set\"\u003e\u003c/p\u003e\n\u003cp\u003eI love that menu. Most of the time Windows is smart enough to decide which GPU to use for what (though they totally should have backported that feature to Windows 10), which typically would be used to make games or other demanding software run on your dedicated card, leaving things like your browser or video players etc. running on the integrated GPU, if your CPU has one in it. At the same time, it works perfectly fine for my dual GPU setup, where my Intel Arc B580 is the gaming card and my old RX 580 is the secondary card for browsers and things like video encoding.\u003c/p\u003e\n\u003cp\u003eThere are some exceptions, such as you can run the OBS window on whatever GPU you want, but configure which GPU to use for the encoding inside of the program. Similarly, for video editing, I might run DaVinci Resolve on the RX 580 but use the encoder in the B580 because at that point I don't need its computing power for anything else, such as when trying to play/stream/record a game. For most software and games, however, it's far simpler: the OS tells the process which GPU to use and it does so.\u003c/p\u003e\n\u003cp\u003eUnder the hood, the Windows preferences are stored in the registry, under \u003ccode\u003eComputer\\HKEY_CURRENT_USER\\Software\\Microsoft\\DirectX\\UserGpuPreferences\u003c/code\u003e, with the following data format:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ccode\u003eName\u003c/code\u003e - path to the executable, like: \u003ccode\u003eC:\\Program Files\\Mozilla Firefox\\firefox.exe\u003c/code\u003e\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003eType\u003c/code\u003e - string value, so: \u003ccode\u003eREG_SZ\u003c/code\u003e\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003eData\u003c/code\u003e - preference value (see below)\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eImplied default value for \u003ccode\u003eLet Windows decide\u003c/code\u003e:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003eGpuPreference=0;\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eValue for default power saving GPU:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003eGpuPreference=1;\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eValue for default high power GPU:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003eGpuPreference=2;\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eIt can also refer to a specific GPU, not just the role that can be filled by different GPUs:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003eSpecificAdapter=\u0026lt;some_id\u0026gt;;GpuPreference=\u0026lt;some_id\u0026gt;;\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eHere's how it looks in the Registry Editor:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"windows-gpu-preferences.jpg\" alt=\"windows-gpu-preferences\" title=\"windows-gpu-preferences\"\u003e\u003c/p\u003e\n\u003cp\u003eIt's not exactly perfect, but it's simple enough that you can write some software to handle more advanced use cases yourself. For example, some programs out there (like Microsoft Edge) have the executable change based on the installed version, with different paths for each version:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ccode\u003eC:\\Program Files (x86)\\Microsoft\\EdgeCore\\136.0.3240.50\\msedge.exe\u003c/code\u003e\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003eC:\\Program Files (x86)\\Microsoft\\EdgeCore\\136.0.3240.51\\msedge.exe\u003c/code\u003e\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003eC:\\Program Files (x86)\\Microsoft\\EdgeCore\\136.0.3240.52\\msedge.exe\u003c/code\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eand so on.\u003c/p\u003e\n\u003cp\u003eI did write a program for myself to automatically set preferences for any globbed paths, like:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003eWindowsGPUPreferenceSetter.exe energy \u0026quot;C:\\Program Files\\Mozilla Firefox\\firefox.exe\u0026quot;\nWindowsGPUPreferenceSetter.exe power  \u0026quot;C:\\Program Files\\VideoLAN\\VLC\\vlc.exe\u0026quot;\nWindowsGPUPreferenceSetter.exe energy \u0026quot;C:\\Program Files (x86)\\Microsoft\\EdgeCore\\*\\msedge.exe\u0026quot;\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eI'm not really releasing it to the public yet, but it took me about an evening to write (.NET is actually quite lovely), so I'm sure that it's within the reach of most people that need something like it:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"windows-gpu-preference-setter.jpg\" alt=\"windows-gpu-preference-setter\" title=\"windows-gpu-preference-setter\"\u003e\u003c/p\u003e\n\u003cp\u003eFor the most part, it does work. My browsers use the secondary card and don't take up a lot of resources, I can get VLC to run on the B580 because it has better video decoding, whereas at least in theory I should also be able to run all of my games on the B580 too as the gaming card, leaving the other one up for other background software and streaming/recording.\u003c/p\u003e\n\u003cp\u003eExcept things aren't so nice...\u003c/p\u003e\n\u003ch3\u003eWhere it all goes wrong\u003c/h3\u003e\n\u003cp\u003eOne of the first games that I booted up to test this out was \u003ca href=\"https://links.kronis.dev/924m5\"\u003eDelta Force\u003c/a\u003e. It's a free shooter game that plays a little bit like Battlefield, is free, is made on Unreal Engine 4 (there's also a co-op campaign with Unreal Engine 5 but last I checked it was really bad, so we don't talk about that) and looks great.\u003c/p\u003e\n\u003cp\u003eSadly, when I booted it up, I was unpleasantly surprised. Here you can see the \u003ca href=\"https://links.kronis.dev/0itri\"\u003eRivaTuner Statistics Server\u003c/a\u003e output which I use both for framerate limiting sometimes (when games run away with high FPS whereas my monitor can only do 60 FPS) and also monitoring the available resources:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"bad-in-game-performance.jpg\" alt=\"bad-in-game-performance\" title=\"bad-in-game-performance\"\u003e\u003c/p\u003e\n\u003cp\u003eSo, the framerate doesn't hit 60 FPS, the CPU isn't overloaded at all and the GPU utilization sits around 20%, which is surprising. But remember, I'm monitoring the main GPU here, because that's the one that the games should be running on. Looking at the Task Manager quite quickly reveals where the issues lie:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"wrong-gpu-in-use-2.jpg\" alt=\"wrong-gpu-in-use-2\" title=\"wrong-gpu-in-use-2\"\u003e\u003c/p\u003e\n\u003cp\u003eGPU 0 is running at 100%, not 20%, which explains why the framerates are so bad! Looking at Task Manager in more detail, we can see that the game is using the RX 580, not the B580 as needed:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"wrong-gpu-in-use-1.jpg\" alt=\"wrong-gpu-in-use-1\" title=\"wrong-gpu-in-use-1\"\u003e\u003c/p\u003e\n\u003cp\u003eIt should have been simple to fix, but in the end it turned out to be impossible. Delta Force (and as it later turned out, some other games, too) just straight up ignores the GPU preference settings and picks whatever GPU it wants to run on by itself, something that you cannot change in any way:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eadding every single .exe in the game directory to the GPU preferences does nothing\u003c/li\u003e\n\u003cli\u003etrying parameters such as \u003ccode\u003e-adapter\u003c/code\u003e or \u003ccode\u003er.GraphicsAdapter\u003c/code\u003e for Unreal Engine also does nothing, it also sucks beyond belief when you \u003ca href=\"https://links.kronis.dev/wt036\"\u003ecan't even get proper answers\u003c/a\u003e and it's more like there's some mysticism around how the engine is supposed to work, in lieu of actual documentation\u003c/li\u003e\n\u003cli\u003ethere is no software out there that can hide a GPU from a program, the closest is \u003ca href=\"https://links.kronis.dev/a9k5h\"\u003eProcess Lasso\u003c/a\u003e but that is CPU only\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003ein other words, unless I want to disconnect/disable the RX 580 when I want to play Delta Force, then there's no way for me to play it.\u003c/p\u003e\n\u003cp\u003eUnderstandably, I uninstalled the game and won't touch it until this is patched out, which I assure you, is quite unlikely - because unfortunately dual GPU setups are so rare that most likely nobody on the dev team has such a setup and if they do, then it's probably in the form of Intel iGPU, which they will try to avoid ever selecting for running the game, not even considering that Intel also makes discrete GPUs.\u003c/p\u003e\n\u003cp\u003eThat got me thinking - just how screwed am I?\u003c/p\u003e\n\u003ch3\u003eSome of the games\u003c/h3\u003e\n\u003cp\u003eI spent the evening installing and booting up quite a few of the games in my Steam, Epic, GOG, EA, Xbox and other game libraries. I don't really care about which are supposed to be the most popular titles, merely some of the ones that I might play at some point in time. I was also unable to try out some of the more modern games like S.T.A.L.K.E.R. 2 because quite frankly the install sizes for most of them are quite crazy - there's no way I'm downloading 100 GB just to test how a game runs, that alone makes me want to not play it too much.\u003c/p\u003e\n\u003cp\u003eEither way, I looked at a total of 60 games, the oldest ones being from the late 90s (Half Life) and the latest being from just a few years ago, across a number of game engines. I did make a summary table a bit later on in the article, but first, let's look at some of the things I saw along the way.\u003c/p\u003e\n\u003ch4\u003eDeus Ex (2000)\u003c/h4\u003e\n\u003cp\u003eI did wonder how older games would run and since Deus Ex is a bit of a classic, I gave it a fair shot. Honestly, I was very pleasantly surprised, it detected the GPU well enough and gave me some additional options as well (such as choosing between a Direct3D and OpenGL renderer):\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"supported-gpu-detected.jpg\" alt=\"supported-gpu-detected\" title=\"supported-gpu-detected\"\u003e\u003c/p\u003e\n\u003cp\u003eThe game itself also seemed to run okay, except for one specific issue, which was the fact that it felt sped up, no doubt due to the developers coding it for a specific framerate back in the day and it getting quite confused on a modern system. That's not really a GPU related issue though and thankfully, since the game has a bit of a cult following, it's possible to get \u003ca href=\"https://links.kronis.dev/x10os\"\u003ea launcher like Kentie's\u003c/a\u003e and that would normally fix the issue.\u003c/p\u003e\n\u003cp\u003eEither way, look at 25 year old graphics, a few clever tricks here and there and it almost feels like you have raytracing:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"totally-raytracing.jpg\" alt=\"totally-raytracing\" title=\"totally-raytracing\"\u003e\u003c/p\u003e\n\u003cp\u003eOverall, it's quite nice that even such old titles can run on GPUs that came out relatively recently, I'm definitely happy that Intel is in the GPU market and is providing some competition to AMD and Nvidia in the budget segment (to where I would go for a B580 over the regular RTX 5060 8 GB)\u003c/p\u003e\n\u003ch4\u003eBrothers in Arms: Road to Hill 30 (2005)\u003c/h4\u003e\n\u003cp\u003eHere's another old game. While Deus Ex runs on the Unreal Engine 1, I also wanted to test something on Unreal Engine 2, 3, 4 and even 5. I haven't actually played a lot of BiA in the past, but it does feel a little bit like the first Call of Duty game and ran pretty nicely from the get go:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"brothers-in-arms.jpg\" alt=\"brothers-in-arms\" title=\"brothers-in-arms\"\u003e\u003c/p\u003e\n\u003cp\u003eYou can see that the field of vision would feel uncharacteristically narrow for a modern widescreen monitor and setup, as well as all of the UI elements feel much too big, but remember that back when this game came out, a \u003ccode\u003e1024x768\u003c/code\u003e was considered a good resolution. Admittedly, for some Unreal Engine 5 games, that isn't too far off from the internal resolution before upscaling, because badly optimized games just keep getting released.\u003c/p\u003e\n\u003cp\u003eEither way, no issues here.\u003c/p\u003e\n\u003ch4\u003eDoom 3: BFG Edition (2012 remaster of 2004 game)\u003c/h4\u003e\n\u003cp\u003eI do recall the BFG Edition of Doom 3 getting some critique when it came out over the changes to the flashlight mechanics and atmosphere of the game, but I'm just happy that there's a way for me to play the game on a decent resolution on a modern system. It also has an immensely cool option, where you can choose which monitor you want to run it on:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"doom-3-resolution.jpg\" alt=\"doom-3-resolution\" title=\"doom-3-resolution\"\u003e\u003c/p\u003e\n\u003cp\u003eEvery game should have this option (though at least \u003cem\u003emost\u003c/em\u003e of them start on the primary monitor, looking at you, Ashen) but either way, here's some Doom 3 on the id Tech 4 engine, running with no issues:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"doom-3.jpg\" alt=\"doom-3\" title=\"doom-3\"\u003e\u003c/p\u003e\n\u003cp\u003eI might just have to go back sometime later and replay it, such an iconic game from back in the day, it also ran fine on the B580.\u003c/p\u003e\n\u003ch4\u003eBioshock Infinite (2013)\u003c/h4\u003e\n\u003cp\u003eOn the list of iconic games, Bioshock Infinite definitely also takes a high spot! It's both a well made game, narratively and gameplay wise, but also pretty to look at:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"bioshock-infinite.jpg\" alt=\"bioshock-infinite\" title=\"bioshock-infinite\"\u003e\u003c/p\u003e\n\u003cp\u003eDon't be mislead, it is using Unreal Engine 3 under the hood and it does so really well. If nothing else, it proves both that artistic direction and skilled developers matter more than just raw graphics horsepower, especially because of how well it runs.\u003c/p\u003e\n\u003ch4\u003eDishonored (2015 remaster of 2012 game)\u003c/h4\u003e\n\u003cp\u003eThere's also the Dishonored game, which got a remaster, still running on Unreal 3, no issues in sight:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"dishonored.jpg\" alt=\"dishonored\" title=\"dishonored\"\u003e\u003c/p\u003e\n\u003cp\u003eI might prefer the gameplay more to than the graphics in the first game (the second one is really pretty, but also a really big download), but either way, it worked fine.\u003c/p\u003e\n\u003ch4\u003eDoom (2016)\u003c/h4\u003e\n\u003cp\u003eI also did test out the Doom game from 2016 and I have to say that I was even more pleasantly surprised.\u003c/p\u003e\n\u003cp\u003eHonestly, I have \u003cem\u003enever\u003c/em\u003e had a game work so well out of the box and be as optimized as this one was. If you asked me what the future of gaming should look like back when Crysis was all the rage, something like this game would definitely fit the bill. I'm not just saying that for the sake of it, but rather as a subtle dig at some of the other modern engines: if the big studios threw a bunch of money at the more recent version of id Tech and actually put some effort into optimizing their games, we'd be at much better of a spot when it comes to new releases.\u003c/p\u003e\n\u003cp\u003eFor this one, I specifically disabled the framerate limiter and increased the size of the graphs, just to demonstrate how insanely well it runs. Everything is more or less turned up to the maximum, is running at 200 FPS (that is an artificial cap though, seems like their physics engine would bug out if it ran at anything higher, though that would normally be separate from the rendering, either way people with 300 FPS monitors will complain) and the computer isn't breaking a sweat:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"doom-2016-1.jpg\" alt=\"doom-2016-1\" title=\"doom-2016-1\"\u003e\u003c/p\u003e\n\u003cp\u003eThe GPU usage on a B580 remains pretty low, the framerate is more stable than any other game I've seen even in the action scenes and the temperatures are pretty manageable. I will say that my Ryzen 7 5800X does run a bit hot under load, though I could probably undervolt it a bit to help with the cooling further. Either way, it's not thermally throttling and it runs the game just fine:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"doom-2016-2.jpg\" alt=\"doom-2016-2\" title=\"doom-2016-2\"\u003e\u003c/p\u003e\n\u003cp\u003eThe only thing I didn't quite enjoy was the fact that Vulkan seems to have had issues both on Windows 10 and Windows 11, where capturing Vulkan games both through OBS and even something as simple as just hitting Print Screen doesn't seem to work and returns a black screen. I took these screenshots through Steam and also had to run the game itself windowed initially (Print Screen would work then), that's why the resolution shows up a bit goofy:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"doom-2016-3.jpg\" alt=\"doom-2016-3\" title=\"doom-2016-3\"\u003e\u003c/p\u003e\n\u003cp\u003eEither way, here's some output from the Intel Graphics Software. There are a few spikes here and there while I was taking the screenshots, but I'm surprised at how well everything ran, the GPU rarely even utilized all of its power in a game that looks great, runs great and has the graphics more or less maxed out:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"doom-2016-intel.jpg\" alt=\"doom-2016-intel\" title=\"doom-2016-intel\"\u003e\u003c/p\u003e\n\u003cp\u003eMaybe I should also have a look at Wolfenstein II: The New Colossus, Doom Eternal and Indiana Jones and the Great Circle sometime, because those also use the engine. I'm surprised that more companies don't, none of these games are known for bad launches or technical issues (maybe outside of some poor managerial decisions, such as forcing raytracing to always be on).\u003c/p\u003e\n\u003ch4\u003eHunt: Showdown (2019)\u003c/h4\u003e\n\u003cp\u003eHere's a CryEngine game as well, I've heard a lot of good about it but haven't really played it much before myself. Either way, I just booted it up to see how well it works and it was another example of a well optimized game that runs and looks great:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"hunt-showdown.jpg\" alt=\"hunt-showdown\" title=\"hunt-showdown\"\u003e\u003c/p\u003e\n\u003cp\u003eCryEngine is an interesting option: there were previously forks like \u003ca href=\"https://links.kronis.dev/455n4\"\u003eAmazon Lumberyard\u003c/a\u003e that never took off, but now we basically get the tech \u003ca href=\"https://links.kronis.dev/qvbf6\"\u003erepackaged as O3DE\u003c/a\u003e with its own improvements as well, a project that happens to be backed by The Linux Foundation of all people and is Apache 2.0 licensed.\u003c/p\u003e\n\u003cp\u003eWhile I very much share hopes for a bright future for engines like \u003ca href=\"https://links.kronis.dev/t7cfs\"\u003eGodot\u003c/a\u003e (which is amazing for 2D and is getting better at 3D), if you need a powerful 3D engine, then O3DE might at the very least be worth a look, rather than just jumping into Unreal Engine 5 and realizing that you can't make a game that runs well to save your life.\u003c/p\u003e\n\u003ch4\u003eIsonzo (2022)\u003c/h4\u003e\n\u003cp\u003eHere's also a modern Unity game, that does sit somewhere between Battlefield 1 and the now retired Heroes \u0026amp; Generals game in how it plays, focused on the infantry portion. Again, a game that I want to play in the future when I'll actually have some free time, but it ran just fine:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"isonzo.jpg\" alt=\"isonzo\" title=\"isonzo\"\u003e\u003c/p\u003e\n\u003cp\u003eUnity is also a pretty decent engine, even if their \u003ca href=\"https://links.kronis.dev/ny7kj\"\u003eHDRP\u003c/a\u003e can be somewhat demanding. It's still cool to see an engine that has historically had more success with indie and mobile games be more than capable of pulling its weight both in complex projects, as well as pretty modern graphics.\u003c/p\u003e\n\u003ch4\u003eRoad to Vostok (2022, but not released yet)\u003c/h4\u003e\n\u003cp\u003eSpeaking of Godot, there is precisely one notable project attempting to make a modern looking 3D game in it, which is Road to Vostok:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"road-to-vostok.jpg\" alt=\"road-to-vostok\" title=\"road-to-vostok\"\u003e\u003c/p\u003e\n\u003cp\u003eThe project initially started out in Unity, but because of \u003ca href=\"https://links.kronis.dev/jxcjx\"\u003etheir runtime fee debacle\u003c/a\u003e a lot of developers looked elsewhere. I'm not sure how far the project will get, but it really seems like the developer has done a lot of good work and is proving that Godot is viable for more serious games too.\u003c/p\u003e\n\u003ch3\u003eOther interesting stuff\u003c/h3\u003e\n\u003cp\u003eAlong the way, I also ran into something interesting with Enlisted and War Thunder. Both of them complained that I'm running on an integrated GPU, which is a bit silly, because my CPU literally doesn't have one. They're probably just checking if the GPU vendor is listed as \u0026quot;Intel\u0026quot; because back when the engine and games were made, Intel wasn't in the discrete GPU market yet:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"enlisted.jpg\" alt=\"enlisted\" title=\"enlisted\"\u003e\u003c/p\u003e\n\u003cp\u003eThe good news is that the games just run after you click OK and the rest works just fine. Furthermore, War Thunder even supports XeSS upscaling, so it's a bit silly that the warning is there in the first place, since you kind of need to acknowledge that Intel GPUs exist to offer the upscaling integration.\u003c/p\u003e\n\u003cp\u003eCuriously, Need for Speed: Heat also complains about my driver version and suggests that there might be bugs in it:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"need-for-speed-heat.jpg\" alt=\"need-for-speed-heat\" title=\"need-for-speed-heat\"\u003e\u003c/p\u003e\n\u003cp\u003eThis is a bit silly, because I'm running the latest GPU drivers for both my Intel Arc and AMD card at the time of the writing, so it's probably just bad version checking logic. The reason why none of this is a problem though, is the fact that the games still run and it's essentially a win-win situation:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eif the game doesn't run because of an actual driver issue, at least they've provided me with a disclaimer (and instructions that might fix the problem in that particular case)\u003c/li\u003e\n\u003cli\u003eif the game runs well and it's just their version check logic that's bad, then I can dismiss the warning and enjoy the game\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eIt's very much the same as with the resolution selection dialog that also lets you pick the exact monitor you want to run the game on: it gives more control to the person running the software, which is how things should be, same as how you should have a good options menu where you can toggle whatever settings you like or dislike on or off.\u003c/p\u003e\n\u003ch3\u003eThe results\u003c/h3\u003e\n\u003cp\u003eThe above might have seemed a bit rambly, but I did want to show off some of the games and if nothing else, then prove that Intel Arc GPUs can handle most of the games out there with relatively few issues along the way. Actually, after testing out 60 games in total, across various genres (there were a few RTS games in there as well, racing games, even 2D games like ZERO Sievert and Stardew Valley), I found that most of them just work:\u003c/p\u003e\n\u003ctable\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth\u003eGame\u003c/th\u003e\n\u003cth\u003eEngine\u003c/th\u003e\n\u003cth\u003eGPU Selection\u003c/th\u003e\n\u003cth\u003eNotes\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n\u003ctr\u003e\n\u003ctd\u003eNexus: The Jupiter Incident\u003c/td\u003e\n\u003ctd\u003eBlack Sun Engine\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eHearts of Iron IV\u003c/td\u003e\n\u003ctd\u003eClausewitz\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eSkyrim (Special Edition)\u003c/td\u003e\n\u003ctd\u003eCreation\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eHunt: Showdown 1896\u003c/td\u003e\n\u003ctd\u003eCryEngine\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eCarrier Command 2\u003c/td\u003e\n\u003ctd\u003eCustom\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eHomeworld: Remastered Collection\u003c/td\u003e\n\u003ctd\u003eCustom\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eSleeping Dogs\u003c/td\u003e\n\u003ctd\u003eCustom\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eStarpoint Gemini Warlords\u003c/td\u003e\n\u003ctd\u003eCustom\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eTransport Fever 2\u003c/td\u003e\n\u003ctd\u003eCustom\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eEnlisted\u003c/td\u003e\n\u003ctd\u003eDagor\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003eComplains about iGPU, runs fine\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eWar Thunder\u003c/td\u003e\n\u003ctd\u003eDagor\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003eComplains about iGPU, runs fine\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eArma Reforger\u003c/td\u003e\n\u003ctd\u003eEnfusion\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eNeed for Speed: Heat\u003c/td\u003e\n\u003ctd\u003eFrostbite 3\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eStar Wars Squadrons\u003c/td\u003e\n\u003ctd\u003eFrostbite 3\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eFallout: New Vegas\u003c/td\u003e\n\u003ctd\u003eGamebryo\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eOblivion (original)\u003c/td\u003e\n\u003ctd\u003eGamebryo\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eZERO Sievert\u003c/td\u003e\n\u003ctd\u003eGameMaker\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003edeltaV: Rings of Saturn\u003c/td\u003e\n\u003ctd\u003eGodot\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eRoad to Vostok (Demo)\u003c/td\u003e\n\u003ctd\u003eGodot\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003eLaunches on the wrong monitor\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eHalf Life\u003c/td\u003e\n\u003ctd\u003eGoldSrc\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eDOOM 3: BFG Edition\u003c/td\u003e\n\u003ctd\u003eid Tech 4\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eDOOM (2016)\u003c/td\u003e\n\u003ctd\u003eid Tech 6\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eWargame: Red Dragon\u003c/td\u003e\n\u003ctd\u003eIRISZOOM V4\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eSid Meier’s Civilization V\u003c/td\u003e\n\u003ctd\u003eLORE\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eBarotrauma\u003c/td\u003e\n\u003ctd\u003eMonoGame\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eStardew Valley\u003c/td\u003e\n\u003ctd\u003eMonoGame\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eMorrowind\u003c/td\u003e\n\u003ctd\u003eNetImmerse\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eBurnout Paradise Remastered\u003c/td\u003e\n\u003ctd\u003eRenderWare\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eJagged Alliance 3\u003c/td\u003e\n\u003ctd\u003eSol\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eBlack Mesa\u003c/td\u003e\n\u003ctd\u003eSource\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eHalf Life 2\u003c/td\u003e\n\u003ctd\u003eSource\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eBeamNG.drive\u003c/td\u003e\n\u003ctd\u003eTorque 3D\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eCaptain of Industry\u003c/td\u003e\n\u003ctd\u003eUnity\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eCorpus Edax\u003c/td\u003e\n\u003ctd\u003eUnity\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eDue Process\u003c/td\u003e\n\u003ctd\u003eUnity\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eHardspace: Shipbreaker\u003c/td\u003e\n\u003ctd\u003eUnity\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eIsonzo\u003c/td\u003e\n\u003ctd\u003eUnity\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eNuclear Option\u003c/td\u003e\n\u003ctd\u003eUnity\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003ePeripeteia\u003c/td\u003e\n\u003ctd\u003eUnity\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eRegiments\u003c/td\u003e\n\u003ctd\u003eUnity\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eTrain World\u003c/td\u003e\n\u003ctd\u003eUnity\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eDeus Ex: Game of the Year Edition\u003c/td\u003e\n\u003ctd\u003eUnreal 1\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003eGame runs too fast\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eBrothers in Arms: Road to Hill 30\u003c/td\u003e\n\u003ctd\u003eUnreal 2\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eDeus Ex: Invisible War\u003c/td\u003e\n\u003ctd\u003eUnreal 2\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003eResolution doesn’t go past 1280x1024\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eBioshock: Infinite\u003c/td\u003e\n\u003ctd\u003eUnreal 3\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eDishonored\u003c/td\u003e\n\u003ctd\u003eUnreal 3\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eThe Bureau: Declassified\u003c/td\u003e\n\u003ctd\u003eUnreal 3\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eThe Bureau: XCOM Declassified\u003c/td\u003e\n\u003ctd\u003eUnreal 3\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eADACA\u003c/td\u003e\n\u003ctd\u003eUnreal 4\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eAshen\u003c/td\u003e\n\u003ctd\u003eUnreal 4\u003c/td\u003e\n\u003ctd\u003eNOT OK\u003c/td\u003e\n\u003ctd\u003eLaunches on the wrong monitor\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eBattlefleet Gothic: Armada\u003c/td\u003e\n\u003ctd\u003eUnreal 4\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eDakar Desert Rally\u003c/td\u003e\n\u003ctd\u003eUnreal 4\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eDeep Rock Galactic\u003c/td\u003e\n\u003ctd\u003eUnreal 4\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eDelta Force\u003c/td\u003e\n\u003ctd\u003eUnreal 4\u003c/td\u003e\n\u003ctd\u003eNOT OK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eVladik Brutal\u003c/td\u003e\n\u003ctd\u003eUnreal 4\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eAbiotic Factor\u003c/td\u003e\n\u003ctd\u003eUnreal 5\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eDark and Darker\u003c/td\u003e\n\u003ctd\u003eUnreal 5\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eEverspace 2\u003c/td\u003e\n\u003ctd\u003eUnreal 5\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eMotor Town: Behind The Wheel\u003c/td\u003e\n\u003ctd\u003eUnreal 5\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eSatisfactory\u003c/td\u003e\n\u003ctd\u003eUnreal 5\u003c/td\u003e\n\u003ctd\u003eOK\u003c/td\u003e\n\u003ctd\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eThe only ones that downright didn't work were the following:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eDelta Force - uses the RX 580 no matter what I do, ignores Windows preferences\u003c/li\u003e\n\u003cli\u003eAshen - uses the RX 580 no matter what I do, ignores Windows preferences, launches on the wrong monitor\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eBoth of those are Unreal Engine 4 games, but at the same time neither other engine versions had that particular problem, nor did other Unreal Engine 4 games that I tested: ADACA, Battlefleet Gothic: Armada, Dakar Desert Rally, Deep Rock Galactic, Vladik Brutal. Most of these are a bit niche because I couldn't be bothered to download games that are like 50 GB in size, but it might also prove that even smaller teams are capable of making the engine work just fine.\u003c/p\u003e\n\u003cp\u003eEven if the rest of the post and the overall situation proved to be pretty positive, then the cause of the issue is now pretty much clear in my eyes...\u003c/p\u003e\n\u003ch3\u003eSummary\u003c/h3\u003e\n\u003cp\u003eWho's to blame for all this? Developers. Much like Steve Ballmer shouted that from a stage years ago, I can now point in the direction of the people who made those games and repeat who is to blame, loud and clear.\u003c/p\u003e\n\u003cp\u003eI'm under no illusion that those games will be patched out or even that there's an easy solution for the issues, because dual GPU setups are uncommon in of themselves, to the point where nobody is going to spend a lot of time testing for them and debugging (which is also how War Thunder and Enlisted complain about the Arc GPUs supposedly being integrated ones, though the games themselves thankfully work). However, there is no good reason for the games to be broken like this. The operating system would definitely benefit from a mechanism to hide some GPUs from a specific process, the same way how you can set the process affinity for specific CPU cores, but at least the engines and games that play nice with the OS have no issues.\u003c/p\u003e\n\u003cp\u003eThis still does mean that if you have 100 games then maybe 3-4 of those might not work, or if you have 500 games, then that number might jump to 16-17 games. It's just me extrapolating from my little experiment here, but while that number is pretty low, it's still there. Imagine not being able to listen to a song that you enjoy, just because it refuses to play on your device. Or alternatively, having to use a specific device to play the song, which might be a bad analogy because some people definitely enjoy playing vinyl records, whereas me having to disable my GPU in the Device Manager wouldn't be too hard to do, but annoying nonetheless. Plus, if I wanted to stream/record any of those games without my RX 580 connected to the PC, I'd have to reconfigure OBS which is a pain.\u003c/p\u003e\n\u003cp\u003eIt's not all bad, though. Modern game engines give me plenty of hope, or at least this somber sense of knowing the truth: there are plenty of games made in CryEngine, Godot, id Tech, Unity and various versions of Unreal Engine, all of which work well. Sometimes it's because the engine pushes you towards good methods during development, other times it's because the games are a bit on the older side by now and thankfully even budget hardware can run them perfectly fine, but it's possible to make games that run well and look great.\u003c/p\u003e\n\u003cp\u003eSince I do have to do a little rant about that here, I will admit that it just isn't happening to a big part of the market - for whatever reason many use Unreal Engine 5 in a way that they try to push graphics above all else, oftentimes releasing broken, buggy and really badly optimized products: look at how S.T.A.L.K.E.R. 2 was on release and how it still is.\u003c/p\u003e\n\u003cp\u003eLook at how any of the following run:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eAlan Wake 2\u003c/li\u003e\n\u003cli\u003eArk: Survival Ascended\u003c/li\u003e\n\u003cli\u003eBlack Myth: Wukong\u003c/li\u003e\n\u003cli\u003eImmortals of Aveum\u003c/li\u003e\n\u003cli\u003eMechWarrior 5: Clans\u003c/li\u003e\n\u003cli\u003eRedfall\u003c/li\u003e\n\u003cli\u003eRemnant 2\u003c/li\u003e\n\u003cli\u003eSilent Hill 2\u003c/li\u003e\n\u003cli\u003eThe Matrix Awakens (yes, even the UE5 demo)\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eGames like Satisfactory that actually get optimized properly are a rarity in comparison. You can try to say that it's just about how the developers use the engine, but it's only a valid argument up to the point where we have a widespread industry issue with most modern releases on the engine that come out running really badly. It's the same as saying that developers who use C++ are just bad at memory management, yet time after time after time there are CVEs that are caused by this. At one point you just have to draw a line and say that enough is enough and look at alternatives (in the case of programming, it'd be memory safe languages, in the case of game engines - literally any other engine)\u003c/p\u003e\n\u003cp\u003eUpscaling and frame generation should make games more accessible even on lower end hardware, not get used as a crutch for bad performance because the developers couldn't be bothered to fix it. In part, that is probably driven by stupid industry trends, in part because the people running the companies care more about deadlines than they do about the experience that gamers will get on release day - since apparently that's the status quo now.\u003c/p\u003e\n\u003cp\u003eI do believe that if they could fix No Man's Sky, if they could fix Cyberpunk 2077, then games like S.T.A.L.K.E.R. 2 will also get fixed eventually, but the sad part is that many games will never get fixed and will be perpetually broken and impossible to enjoy on anything but unreasonably overpowered (and exorbitantly expensive) hardware. If anything, it's teaching me to never get a game on release day anymore and never do preorders, unless I care more about supporting the developers than I care about getting a game to play.\u003c/p\u003e\n\u003cp\u003eIf I wanted to have a bit more fun with this, I'd probably conjure up a scene in my imagination, where a bunch of shady Nvidia execs are giving employees of Epic briefcases full of money, for them to subtly push graphics technology on the industry that will absolutely get misused and necessitate more GPU purchases, all the while actually delivering on the promise of making iterating on assets and game scenes that much faster and more convenient, meaning that numerous studios will ditch their own proprietary engines in favor of Unreal, sometimes to good effect, but also sometimes with outstanding failures.\u003c/p\u003e\n\u003cp\u003eEither way, Intel Arc GPUs are actually pretty cool and there's lots of nice games out there to enjoy, I wonder if any of them would run better on Linux, though in the case of Delta Force I guess it's a matter of not being able to play it at all due to \u003ca href=\"https://links.kronis.dev/u204j\"\u003ethe intrusive anti-cheat solution\u003c/a\u003e they have, another case of a part of the community falling through the cracks because the business people controlling its development don't care (albeit from a business perspective I have to say that it probably makes sense), even if Steam and \u003ca href=\"https://links.kronis.dev/zrdrj\"\u003eProton\u003c/a\u003e are also finally making proper gaming on Linux a thing.\u003c/p\u003e\n\u003cp\u003eAside from that, I yearn for the day when we all have dual GPU setups: where when you get a new GPU, you can use the old one as a secondary for browsing and all of the background tasks. Why aren't we doing that? Windows 11 is finally a decent enough OS to take advantage of it and as I just proved, even most games are finally passable in that regard. You don't need a fancy SLI or CrossFire setup to benefit from two GPUs in your system!\u003c/p\u003e\n"
        },
        {
            "title": "Software licenses and hyperscalers",
            "date_published": "2025-05-03",
            "id": "https://blog.kronis.dev/blog/software-licenses-and-hyperscalers",
            "url": "https://blog.kronis.dev/blog/software-licenses-and-hyperscalers",
            "content_html": "\u003cp\u003eSoftware licensing is a bit of a mess.\u003c/p\u003e\n\u003cp\u003eThere's many different licenses out there that are meant to protect either the users of the project, the developers, or any corporate interests. Some licenses are proprietary and geared explicitly towards commercial interests, though others are meant to ensure the viability of open source projects, or just free software as a movement in general.\u003c/p\u003e\n\u003cp\u003eGenerally, the open source ones fall into one of two camps:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003epermissive licenses: \u003ca href=\"https://links.kronis.dev/xli09\"\u003eMIT\u003c/a\u003e, \u003ca href=\"https://links.kronis.dev/qwx9g\"\u003eBSD\u003c/a\u003e, \u003ca href=\"https://links.kronis.dev/qq5c3\"\u003eApache 2.0\u003c/a\u003e, which have minimal restrictions and are the closest to \u0026quot;do whatever you want\u0026quot; you can get\u003c/li\u003e\n\u003cli\u003ecopyleft licenses: \u003ca href=\"https://links.kronis.dev/ift62\"\u003eGPL\u003c/a\u003e, \u003ca href=\"https://links.kronis.dev/6ykah\"\u003eLGPL\u003c/a\u003e, \u003ca href=\"https://links.kronis.dev/9nk7j\"\u003eAGPL\u003c/a\u003e, which require the derivative works to remain under the same license, which cannot be changed\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eIn most cases (except for maybe LGPL), the copyleft licenses more or less \u0026quot;infect\u0026quot; the rest of your project and affect what you can do with the rest of it, which does matter a lot. There's also many others out there, but for now let's just focus on the well known ones. Because most of the examples out there kinda suck due to how non-specific they are, let's imagine some concrete situations.\u003c/p\u003e\n\u003ch3\u003eWhat do the software licenses mean\u003c/h3\u003e\n\u003cp\u003eSuppose I want to run an online file converter of some sort as a business: you subscribe to it with PayPal and in exchange can use an API where you upload files and get outputs. Let's see how the different licenses can alter my experience as a developer.\u003c/p\u003e\n\u003cp\u003eLet's say that I have a PDF file processing library that I do not change in any other way, merely include in my project, which is the most common case when it comes to dependencies (imagine your typical \u003ccode\u003enpm install\u003c/code\u003e, Maven, NuGet or any other dependency management solution).\u003c/p\u003e\n\u003cp\u003eHow does this affect the license of my software:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eMIT: no changes, I can license it however I want\u003c/li\u003e\n\u003cli\u003eBSD: no changes, I can license it however I want\u003c/li\u003e\n\u003cli\u003eApache 2.0: no changes, I can license it however I want\u003c/li\u003e\n\u003cli\u003eLGPL: dynamic linking is okay, but static linking would have obligations for the linked code; network use doesn't count\u003c/li\u003e\n\u003cli\u003eGPL: if I distribute the service (e.g. executables), the code that uses it must be GPL licensed; network use doesn't count\u003c/li\u003e\n\u003cli\u003eAGPL: much like the above GPL, except network use also counts, all of my service becomes AGPL\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eDo I have to publish ALL of my source:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eMIT: no, only need to include the original license/notice\u003c/li\u003e\n\u003cli\u003eBSD: no, only need to include the original license/notice\u003c/li\u003e\n\u003cli\u003eApache 2.0: no, only need to include the original license/notice\u003c/li\u003e\n\u003cli\u003eLGPL: no, only would have to distribute any changed LGPL code, but not applicable here, since we just include that library\u003c/li\u003e\n\u003cli\u003eGPL: if I distribute the service (e.g. executables), then I have to release the service source code; network use doesn't count\u003c/li\u003e\n\u003cli\u003eAGPL: much like the above GPL, except that offering the service to anyone else also means the source must be distributed\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eNow, let's say that I also have a DOCX file processing library, that I do change - maybe there are performance issues that I solve, or some usability problems that I want to address, data formats that need tweaking, anything really. I take their code off of GitHub, import in my own source management solution and do some changes to it.\u003c/p\u003e\n\u003cp\u003eHow does this affect the license of my software:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eMIT: no changes, I can license it however I want\u003c/li\u003e\n\u003cli\u003eBSD: no changes, I can license it however I want\u003c/li\u003e\n\u003cli\u003eApache 2.0: no changes, I can still license my software however I want\u003c/li\u003e\n\u003cli\u003eLGPL: my changes to the LGPL code must be LGPL, but the rest of my software can remain proprietary\u003c/li\u003e\n\u003cli\u003eGPL: if I distribute the service, the code that uses it must be GPL licensed; network use doesn't count\u003c/li\u003e\n\u003cli\u003eAGPL: much like the above GPL, except network use also counts, all I write becomes AGPL\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eDo I have to publish the changed library code:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eMIT: no, only need to include the original license/notice\u003c/li\u003e\n\u003cli\u003eBSD: no, only need to include the original license/notice\u003c/li\u003e\n\u003cli\u003eApache 2.0: no, need to include the original license/notice, need to document any significant changes I made\u003c/li\u003e\n\u003cli\u003eLGPL: yes, the modified library source code must be published, my code can remain with me; network use doesn't count\u003c/li\u003e\n\u003cli\u003eGPL: yes, the modified source code, alongside with whatever is calling it must be published; network use doesn't count\u003c/li\u003e\n\u003cli\u003eAGPL: much like the above GPL, except that offering the service to anyone else also means the source must be distributed\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eDo I have to publish ALL of my source:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eMIT: no, only need to include the original license/notice\u003c/li\u003e\n\u003cli\u003eBSD: no, only need to include the original license/notice\u003c/li\u003e\n\u003cli\u003eApache 2.0: no, only the original license/notice and documentation of changes\u003c/li\u003e\n\u003cli\u003eLGPL: no, only the modified library code (and any build scripts) have to be distributed\u003c/li\u003e\n\u003cli\u003eGPL: yes, the modified source code, alongside with whatever is calling it must be published; network use doesn't count\u003c/li\u003e\n\u003cli\u003eAGPL: much like the above GPL, except that offering the service to anyone else also means the source must be distributed\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThat turned out to be a lot of text, come to think of it, this would have probably been better served by a table:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"software-license-table.jpg\" alt=\"software-license-table\" title=\"software-license-table\"\u003e\u003c/p\u003e\n\u003cp\u003eTo sum it all up, essentially:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003ethe permissive licenses are, well, permissive and give you a lot of freedom in regards to what you do with the code\u003c/li\u003e\n\u003cli\u003eLGPL lets you incorporate the code in your apps and you only need to publish any of the code that you change, somewhat of a sweet spot\u003c/li\u003e\n\u003cli\u003eGPL makes any of the code that's integrated with it also GPL, but only if my service is distributed like an executable that you can run locally\u003c/li\u003e\n\u003cli\u003eAGPL also patches the loophole where the original GPL didn't really apply to software that's used over the network, honestly it feels more \u0026quot;in the spirit\u0026quot; of GPL, because that's how a lot of the current software is running out there\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eSo what are the problems with this?\u003c/p\u003e\n\u003ch3\u003eSome personal gripes with the licenses\u003c/h3\u003e\n\u003cp\u003eFirstly, the fact that I needed a table to even explain how these things work and even then I cannot be 100% sure that I haven't missed some nuance.\u003c/p\u003e\n\u003cp\u003eSecondly, while the idea of LGPL is nice, it essentially has legalese dig into the technical implementation details: suddenly, if I want to distribute a statically compiled executable, I need to think about also distributing the object files to comply with the LGPL's relinking requirements:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eConvey the Minimal Corresponding Source under the terms of this License, and the Corresponding Application Code in a form suitable for, and under terms that permit, the user to recombine or relink the Application with a modified version of the Linked Version to produce a modified Combined Work, in the manner specified by section 6 of the GNU GPL for conveying Corresponding Source.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eand\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThe “Minimal Corresponding Source” for a Combined Work means the Corresponding Source for the Combined Work, excluding any source code for portions of the Combined Work that, considered in isolation, are based on the Application, and not on the Linked Version.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eFor some languages \u003ca href=\"https://links.kronis.dev/o6719\"\u003elike Go\u003c/a\u003e that isn't status quo. Do I even know how to get object files out of it, so relinking can be done? It might not be in the happy path of what most tutorials and tooling out there is geared towards, in some languages and stacks not that much consideration might have been given to the use case at all, it might not be possible altogether!\u003c/p\u003e\n\u003cp\u003eDoes the user need to know the exact version and type of compiler used to be able to produce something that will work on a technical level, information that I now also have to give them not to be in breach of the license? You are also more or less obligated to share the actual instructions for building the app as well, giving anyone more insight into your build process.\u003c/p\u003e\n\u003cp\u003eThe Apache license also demands that changes of what you've done be summarized in NOTICE files:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eRedistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions:\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003ea. You must give any other recipients of the Work or Derivative Works a copy of this License; and\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eb. You must cause any modified files to carry prominent notices stating that You changed the files; and\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003ec. You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003ed. If the Work includes a \u0026quot;NOTICE\u0026quot; text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eThat means that you also need to keep track of everything you change, in addition to distributing that information, a bit more process that's required of you by the license. Aside from that, just look at the above - lawyers might be used to that sort of language, but it feels straight up jarring to your average developer. It reads like language that's the equivalent of a line of code with a bunch of chained ternaries in it, utterly unpleasant.\u003c/p\u003e\n\u003cp\u003eThat said, those are just personal gripes. The real issues are with the fact that depending on who you are, the license is hostile to you.\u003c/p\u003e\n\u003ch3\u003eWhy most licenses end up being hostile to you\u003c/h3\u003e\n\u003cp\u003eYou see, the licenses can be written to protect the users, or the authors of the software, maybe even to protect against corporate interests and such. The issue is that, depending on what you do, you might fit in any number of those roles at different points in time, meaning that software that might be great in your private life might be horrible at work (can't use it), or vice versa, some proprietary code that you could benefit greatly from work you could not use in any open source work that you do.\u003c/p\u003e\n\u003ch4\u003eProprietary code hurts users\u003c/h4\u003e\n\u003cp\u003eSuppose that there's a lot of proprietary, closed source software out there. You cannot get access to the source code to fix the bugs, nor if some SaaS gets retired, can you keep using it - instead, it's gone. I might even use some drivers for printers or maybe, as a more modern example, a GPU as an example here - if they're just a blob for the computer to interpret, you aren't fixing bugs in that yourself, nor can anyone else do that. Alternatively, if we're talking about a web service of some sort, they might just decide to raise prices a bunch, or move from a traditional software model to a subscription based one, which you might not really consent to, but the end result is the same - you lose.\u003c/p\u003e\n\u003cp\u003eThis is essentially what the free software movement is fighting against, to give more control to the users of the software, provided that the licenses are enforced in your country, instead of a company sneakily using a bunch of GPL/AGPL code without telling anyone. Even if not done maliciously, how much effort do you think actually goes into ensuring compliance in most companies, about having proper \u003ca href=\"https://links.kronis.dev/47s5q\"\u003eSBOM\u003c/a\u003e reporting, or even making sure that your developers aren't just straight up using pirated software? While I can say that the tooling is there thankfully, I seriously doubt that caring about intellectual property is a constant across all cultures (have a look \u003ca href=\"https://links.kronis.dev/1wt7d\"\u003ehere\u003c/a\u003e or \u003ca href=\"https://links.kronis.dev/rgyy8\"\u003ehere\u003c/a\u003e, though I don't think anyone is trying to document all the cases out there), but I digress.\u003c/p\u003e\n\u003ch4\u003eViral copyleft licenses hurt professionals and companies\u003c/h4\u003e\n\u003cp\u003eNow, imagine that you're a typical 9-5 developer who writes software for a company. You really need that PDF or DOCX library because there's no way you're writing those from scratch, your abilities are just human and you have deadlines. Yet, there's also no way that you can use GPL or AGPL code in your project, because that would mean that you'd have to publish everything - which is completely unacceptable in most projects and circumstances. In other words, the license can also end up making you fail, both yourself professionally and your entire team/company that needs some code that solves the issue, but which you just can't touch. The same applies with consulting or similar use cases.\u003c/p\u003e\n\u003cp\u003eSuppose that some government, even my own, decided to use AGPL licensed code in their new tax system. It would be developed under your typical government oversight with public sector salaries and the average developers those can attract. Suppose they'd have to publish all of their source code and would do so.\u003c/p\u003e\n\u003cp\u003eMany will shout from the rooftops that \u003ca href=\"https://links.kronis.dev/7putc\"\u003e\u0026quot;given enough eyeballs, all bugs are shallow\u0026quot;\u003c/a\u003e or that \u003ca href=\"https://links.kronis.dev/k8ovk\"\u003esecurity through obscurity\u003c/a\u003e is bad, but the ideology misses the reality: consider the percentage of capable developers that will be interested in solving the bugs and vulnerabilities in some random software system that nobody cares about (and going through the process to merge the changes with a bunch of underfunded devs on the other side, maybe even without proper contributing guidelines) vs the percentage of malicious individuals or organizations that might earn some actual money by pwning the system and selling whatever data or exploits they find.\u003c/p\u003e\n\u003cp\u003eEssentially, these viral licenses would make a lot of the software out there less secure on the account of the insecurity now being visible and there being nobody who cares enough or has enough time or resources to fix everything: a very clear case of resource assymetry, where you would have a few defenders and plenty of motivated attackers.\u003c/p\u003e\n\u003ch4\u003ePermissive licenses will make cloud companies freeload\u003c/h4\u003e\n\u003cp\u003eAside from that, many companies might actually not be opposed to shipping MIT, BSD and Apache 2.0 licensed code, except imagine that you're a popular database vendor that has just one such product. It gains a lot of attention, you're well liked and things seem to be going well. But then a cloud vendor that is 10x your size decides to offer a hosted version of your service, attracting a lot of your customers to their platform instead, probably earning millions in the process and you not seeing a single dime of that. In other words, they can take the code that you've written without giving anything back and benefit from that greatly.\u003c/p\u003e\n\u003cp\u003eIn a perfect world, people would get along. Companies would be ethical - the cloud vendor would give you money for your troubles and for continued development, maybe a certain cut of the profits and you could also just give your software away because people would donate due to benefitting from it. It would be easy to do (which \u003ca href=\"https://links.kronis.dev/hdna4\"\u003eat least some people are working on\u003c/a\u003e and \u003ca href=\"https://links.kronis.dev/t92dz\"\u003ealso this exists\u003c/a\u003e) and it would be as ingrained in the culture, as the tipping culture is in the US, maybe even a comfy little popup in your OS letting you know all of the cool software running on it and being able to just send an aggregated donation through PayPal that then gets distributed or what have you - not unlike in principle to how you can work on your environmental impact, though something that's offered to you and that you don't have to seek out.\u003c/p\u003e\n\u003cp\u003eBut a perfect world would also have UBI or something like it, or ample government programs to support developing useful things for the society (which, again, \u003ca href=\"https://links.kronis.dev/narww\"\u003eat least some people are working on\u003c/a\u003e). Sadly, for the most part, that isn't quite the case - a lot of the open source software out there is developed more or less \u003ca href=\"https://links.kronis.dev/i0m6a\"\u003eunder the poverty line\u003c/a\u003e and can be a thankless job, which is how you get memes like this:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"modern-infrastructure.jpg\" alt=\"modern-infrastructure\" title=\"modern-infrastructure\"\u003e\u003c/p\u003e\n\u003cp\u003e(from \u003ca href=\"https://links.kronis.dev/5rrcy\"\u003eXKCD 2347: Dependency\u003c/a\u003e)\u003c/p\u003e\n\u003cp\u003eI'm not even slightly exaggerating here.\u003c/p\u003e\n\u003cp\u003eGo read this, it has some more nice examples: \u003ca href=\"https://links.kronis.dev/y9fuf\"\u003eProfessional maintainers: a wake-up call\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eFurthermore, since companies seem to attempt to maximize profits at the expense of all else, they seem to have no qualms about capitalizing on someone else's labor, or even stealing ideas outright. The reality is that the licenses can't be written with a best faith interpretation and wishful thinking. Sadly, this also meant that we ended up with new licenses that make them be hostile to us in more cases than just the above.\u003c/p\u003e\n\u003ch3\u003eBut wait, it gets worse\u003c/h3\u003e\n\u003cp\u003eThe companies writing software very much saw the trend of these cloud companies, these hyperscalers, taking their software and earning the big money, arguably some of which should be theirs instead, because they're putting in all of the effort into creating the software. Lacking any other means of actually getting the money that might be rightfully theirs, some of them moved to a new class of software licenses - often no longer considered to be \u0026quot;open source\u0026quot;, but rather source available and with other restrictions in place.\u003c/p\u003e\n\u003cp\u003eI will offer up only the relevant fragments from blog posts regarding them or license text to prevent this from getting overly long.\u003c/p\u003e\n\u003cp\u003eEither way, here's a few...\u003c/p\u003e\n\u003ch4\u003eServer Side Public License\u003c/h4\u003e\n\u003cp\u003eMongoDB moved from AGPL \u003ca href=\"https://links.kronis.dev/sgmvx\"\u003eto their own license in 2018\u003c/a\u003e, claiming that even the boundaries of AGPL were being tested by other companies:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThe SSPL builds on the spirit of the AGPL, but clarifies the condition for providing open source software as a service. The license retains all of the same freedoms that the open source community had with MongoDB under the AGPL: freedom to use, review, modify and redistribute the software. The only substantive change is an explicit condition that any organization attempting to exploit MongoDB as a service must open source the software that it uses to offer such service. This license change will not impact customers who have purchased a commercial license from MongoDB.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eIf you read the license, it pretty much reads like the nuclear option:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eIf you make the functionality of the Program or a modified version available to third parties as a service, you must make the Service Source Code available via network download to everyone at no charge, under the terms of this License. Making the functionality of the Program or modified version available to third parties as a service includes, without limitation, enabling third parties to interact with the functionality of the Program or modified version remotely through a computer network, offering a service the value of which entirely or primarily derives from the value of the Program or modified version, or offering a service that accomplishes for users the primary purpose of the Program or modified version.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003e“Service Source Code” means the Corresponding Source for the Program or the modified version, and the Corresponding Source for all programs that you use to make the Program or modified version available as a service, including, without limitation, management software, user interfaces, application program interfaces, automation software, monitoring software, backup software, storage software and hosting software, all such that a user could run an instance of the service using the Service Source Code you make available.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eEssentially, we're not even talking about open sourcing whatever interacts with the MongoDB instance directly anymore.\u003c/p\u003e\n\u003cp\u003eYou have an analytics or a reporting solution? You have to give up the source. Invoicing platform? You have to give up the source. Something for backups, or some DevOps automation? You have to give up the source.\u003c/p\u003e\n\u003cp\u003eEssentially, everything that runs your business and your MongoDB SaaS, which is kind of hypocritical, because I'm not seeing the MongoDB organization do the same thing, which, of course, is permissible, because they're the copyright holders. I'm not even sure how everyone is not saying that this is completely outrageous, because it more or less is - if you complied with the license and published everything, suddenly your company would have to deal with a lot of malicious actors, which would hack your systems with a dangerously high degree of success, at least in the case of most companies out there. Well, either that, or every single one of your marketing funnels or user acquisition mechanisms, analytics, or anything else would be put under a lot of scrutiny by the media that seeks flashy headlines for the sake of clicks. In most cases, SSPL is a non-starter, maybe that's the whole point.\u003c/p\u003e\n\u003cp\u003eThe message here is clear - if you want to benefit from the software, time to buy their commercial license.\u003c/p\u003e\n\u003ch4\u003eElastic License\u003c/h4\u003e\n\u003cp\u003eElastic moved from a pure Apache 2.0 license \u003ca href=\"https://links.kronis.dev/kteah\"\u003eto their own license in 2021\u003c/a\u003e and with \u003ca href=\"https://links.kronis.dev/mzfcf\"\u003esome further clarifications\u003c/a\u003e alongside SSPL, directly addressing what happened with AWS offering their software as a service:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eSo why the change? AWS and Amazon Elasticsearch Service. They have been doing things that we think are just NOT OK since 2015 and it has only gotten worse. If we don’t stand up to them now, as a successful company and leader in the market, who will? Our license change is aimed at preventing companies from taking our Elasticsearch and Kibana products and providing them directly as a service without collaborating with us.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eOver the last three years, the market has evolved and the community has come to appreciate that open source companies need to better protect their software in order to maintain a high level of investment and innovation. With the shift to SaaS as a delivery model, some cloud service providers have taken advantage of open source products by providing them as a service, without contributing back. This diverts funds that would have been reinvested into the product and hurts users and the community.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eWe chose this path because it gives us the opportunity to be as open as possible, while protecting our community and company. In some ways, this change allows us to be even more open. As a follow-up to this change, we will begin moving our free proprietary features from the Elastic License to be dual-licensed under the SSPL as well, which is more permissive and better aligned with our goals of making our products as free and open as possible.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThe Elastic License 2.0 applies to our distribution and the source code of all of the free and paid features of Elasticsearch and Kibana. Our goal with ELv2 is to be as permissive as possible, while protecting against abuse. The license allows the free right to use, modify, create derivative works, and redistribute, with three simple limitations:\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cul\u003e\n\u003cli\u003eYou may not provide the products to others as a managed service.\u003c/li\u003e\n\u003cli\u003eYou may not circumvent the license key functionality or remove/obscure features protected by license keys.\u003c/li\u003e\n\u003cli\u003eYou may not remove or obscure any licensing, copyright, or other notices.\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eand their FAQ cleared up a point of confusion:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u0026quot;I am building an application on top of Elasticsearch, how does ELv2 work for me?\u0026quot;\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eYou may freely use Elasticsearch inside your SaaS or self-managed application, and redistribute it with your application, provided you follow the three limitations outlined above.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eEventually it also got AGPL floating in there somewhere, which makes things even more complicated, but it's a very clear move against hyperscalers, to protect their software. It is quite interesting that as time goes on, more and more licenses like these seem to pop up, from quite prominent companies and widely used software, clearly it's a widespread issue.\u003c/p\u003e\n\u003ch4\u003eBusiness Source License\u003c/h4\u003e\n\u003cp\u003eThis one is \u003ca href=\"https://links.kronis.dev/vh2wk\"\u003ea license used by MariaDB\u003c/a\u003e for their \u003ca href=\"https://links.kronis.dev/txl8x\"\u003eMaxScale product\u003c/a\u003e, which eventually converts to an open source license after some time has passed, usually something like 4 years:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eBSL is a new alternative to closed source or open core licensing models. Under BSL, the source code is always publicly available. Non-production use of the code is always free, and the licensor can also make an Additional Use Grant allowing limited production use. Source code is guaranteed to become Open Source at a certain point in time. On the Change Date, or the fourth anniversary of the first publicly available distribution of the code under the BSL, whichever comes first, the code automatically becomes available under the Change License. The Change License is mandated to be GPL Version 2.0 or later, or a compatible license (i.e., the Change License is always an Open Source license that enables use of the software in a GPL project).\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eI don't actually have that much to say about this, except that I think the license itself is interesting, because it ensures that the software will eventually be open source, while at the same time protecting whatever competitive edge those initial years might give you. Of course, I don't really imagine that very many companies out there would want to start their commercial venture on 4 year old software, but since the term is something that the copyright holder can decide on, I don't think 1-2 years would be out of the question either.\u003c/p\u003e\n\u003ch3\u003eWhat are the consequence of these licenses\u003c/h3\u003e\n\u003cp\u003eNo matter how you look at it, SSPL is the biggest case here and what essentially started the movement of protectionism against the hyperscalers, with quite a few prominent pieces of software adopting the same license:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eMongoDB\u003c/li\u003e\n\u003cli\u003eElasticsearch\u003c/li\u003e\n\u003cli\u003eRedis\u003c/li\u003e\n\u003cli\u003eGraylog\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eFor many companies out there, that's quite a large part of their software development stack and in many cases it takes that software off the table altogether.\u003c/p\u003e\n\u003cp\u003eBut surely the community and organizations in question realized the perspective of the original developers and were accepting of this decision, right? Well, no. If MongoDB had fallen down and needed a bit of help getting back up, the reaction was more like getting kicked in the face again, it was very much the case for all of the software above: because in many cases, the maintainers felt that they had been contributing to an open piece of software (because they were) and then suddenly someone took their work away from them, or imposed restrictions to it that they no longer agreed with, something that was more or less forced upon them. In other words, they felt like they were just responding to being kicked in the face first, in kind.\u003c/p\u003e\n\u003cp\u003eSomeone might say: \u0026quot;But hey, the old versions are still permissively licensed, you can just use those.\u0026quot; That is patently false, because the statement is nonsensical: you don't get to just use old versions of software, versions that will no longer get updates and eventually will drown in CVEs to the degree where you won't be able to run them without ending up with a \u0026quot;surprise backup\u0026quot;. Just ask anyone who wanted to keep using Windows XP how long their machine can be connected to the Internet before it ends up as a part of some botnet. Ask people who couldn't keep up with the updates to \u003ca href=\"https://links.kronis.dev/mdw2u\"\u003eLog4j\u003c/a\u003e, just how well it worked out for them.\u003c/p\u003e\n\u003ch4\u003eThe forks\u003c/h4\u003e\n\u003cp\u003eSo what did the community and those aforementioned large companies do? Well, they did what nerds do when they get angry: they grabbed their keyboards and forked whatever they could fork, when there were no viable alternatives for the problem that they want to solve with the software.\u003c/p\u003e\n\u003cp\u003eIt wasn't unlike what happened to \u003ca href=\"https://links.kronis.dev/vrqal\"\u003eGogs and Gitea\u003c/a\u003e, or later \u003ca href=\"https://links.kronis.dev/d5lft\"\u003eGitea and Forgejo\u003c/a\u003e. The irony of the same thing happening twice doesn't surprise me, but in the open source world, at least it's an option - if the userbase puts their foot down and decide that they want to govern and develop the project differently, they can go ahead and do just that.\u003c/p\u003e\n\u003cp\u003eThe unfortunate part is that it might as well kill the company just trying to protect its own software from these large tech companies, which once more is the case of them not aligning with the ideas behind free software, because they weren't conceived in a world \u003ca href=\"https://links.kronis.dev/svl4f\"\u003ewhere your ideas will be blatantly stolen\u003c/a\u003e, or even the software you've spent years on, trying to build business upon, which is now just a click away in some large platform that will easily outcompete you.\u003c/p\u003e\n\u003cp\u003eWhen it happened initially with \u003ca href=\"https://links.kronis.dev/3kwsu\"\u003eMySQL and MariaDB\u003c/a\u003e it seemed pretty good - it was a smaller group of developers and community members forking a project that now belonged to a large enterprise, but this time it went in the opposite direction. Of course that was before \u003ca href=\"https://links.kronis.dev/l6ndq\"\u003ethe SPAC\u003c/a\u003e and the mess that followed, but oh well.\u003c/p\u003e\n\u003ch4\u003eMongoDB\u003c/h4\u003e\n\u003cp\u003eCuriously, no big alternatives to MongoDB emerged, there was \u003ca href=\"https://links.kronis.dev/cl56u\"\u003ePercona Server for MongoDB\u003c/a\u003e but it never caught on that much, whereas AWS went with \u003ca href=\"https://links.kronis.dev/ahamz\"\u003eDocumentDB\u003c/a\u003e, maybe on the account of the popularity of MongoDB already declining, after the NoSQL boom more or less dissipated.\u003c/p\u003e\n\u003ch4\u003eElasticsearch and OpenSearch\u003c/h4\u003e\n\u003cp\u003eElasticsearch was also forked by AWS and became \u003ca href=\"https://links.kronis.dev/nh1ae\"\u003eOpenSearch\u003c/a\u003e, licensed under Apache License 2.0. As with other forks, the functionality diverged slightly over time, but a good chunk of the community was definitely fragmented and the attempt by Elastic to reign in the competition didn't really work. Essentially, the previously free software didn't have any sort of a moat or any inherent difficulties that would prevent a fork from taking off. Furthermore, the \u003ca href=\"https://links.kronis.dev/rfkqf\"\u003eLinux Foundation also got involved\u003c/a\u003e:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eToday the OpenSearch project has significant momentum, with more than 700 million software downloads and participation from thousands of contributors and more than 200 project maintainers. The OpenSearch Software Foundation launches with support from premier members AWS, SAP, and Uber and general members Aiven, Aryn, Atlassian, Canonical, Digital Ocean, Eliatra, Graylog, NetApp® Instaclustr, and Portal26.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eI guess is good for the users, but not so much the company that just wanted to exist, I can't help but to feel a bit sorry for them.\u003c/p\u003e\n\u003ch4\u003eRedis and the forks\u003c/h4\u003e\n\u003cp\u003eWith Redis, a prominent key-value store, the consequences were even more significant, there were more numerous forks and alternatives:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"https://links.kronis.dev/4lj8i\"\u003eValkey\u003c/a\u003e - one of the more popular forks, also supported by the Linux Foundation and gaining a lot of popularity\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://links.kronis.dev/k666j\"\u003eKeyDB\u003c/a\u003e - another fork that also attempts to improve the performance somewhat, albeit not as popular\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://links.kronis.dev/u5hbe\"\u003eDragonfly DB\u003c/a\u003e - this is an interesting case, because while it's not a direct fork, it attempts to be a multi-threaded drop in replacement that maintains compatibility with Redis\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThe thing here is that if your software or product is simple enough (relatively so) to be replicated by others, even from zero, then you can't really be sure that you have that much survivability to it. That'd be more or less like me trying to create a social media platform and even if I have some good ideas, they'd probably quickly be \u0026quot;borrowed\u0026quot; by others relatively quickly - at the same time, I think that Redis is great software and have nothing against them.\u003c/p\u003e\n\u003ch4\u003eGraylog and the alternatives\u003c/h4\u003e\n\u003cp\u003eWith Graylog, however, it seems that nobody really cared that much?\u003c/p\u003e\n\u003cp\u003eIt appears to be a case of a fairly populated market with lots of options available for log management:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eOpenSearch and \u003ca href=\"https://links.kronis.dev/35rh0\"\u003eOpenSearch Dashboards\u003c/a\u003e - essentially the aforementioned fork of the Elastic Stack\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://links.kronis.dev/7p24i\"\u003eFluentd\u003c/a\u003e - one of the better options if you need something that just works, a log collector that lets you use any number of storage outputs, a project under CNCF and is Apache 2.0 licensed\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://links.kronis.dev/35rh0\"\u003eGrafana Loki\u003c/a\u003e - this one is licensed under AGPL, but is still somewhat prominent (as is Grafana itself)\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eIn other words, it's a bit like the above case with Redis, except nobody really needs to replicate your product verbatim, just something that is easy enough to switch to, especially with logging in particular having all sorts of facade libraries or generic implementations in front of it, given that the domain of shipping a bunch of information from your application to some storage and analysis solution isn't rocket science, even if it takes a good amount of work to get right.\u003c/p\u003e\n\u003ch3\u003eWhat to do against hyperscalers\u003c/h3\u003e\n\u003cp\u003eI'm seeing a pattern here.\u003c/p\u003e\n\u003cp\u003eEven if you have pretty popular software, if you try to pull anything funny with the license, the following will usually happen:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eif your software is popular enough and you change it to a non open source license, at least one prominent fork will be created of your open source version before the change\u003c/li\u003e\n\u003cli\u003eover time, this will create a fragmentation in the userbase and you won't really stop the hyperscalers either, since they have a lot of money to throw at the problem\u003c/li\u003e\n\u003cli\u003eyou will definitely end up losing a lot of the community goodwill and along with that, some prominent contributors\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eIn other words, SSPL and the alternatives don't really work towards the stated goals and are more or less like shooting yourself in the foot. The correct thing to do here is to take the gloves off, but only towards the parties that you feel will directly harm your business, not just catch a big part of the community in the blast range of something like SSPL as collateral damage.\u003c/p\u003e\n\u003cp\u003eMeet the Anti-Hyperscaler Software License:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eTake whatever license you desire as a base, this would work even with the more permissively licensed ones like BSD or MIT, or Apache 2.0.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eAllow pretty much everyone use the software however they choose, EXCEPT for specific companies, to which a different set of terms will apply.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eIn the more permissive version of the license, allow the software to be used for internal projects within said companies BUT not for managed services.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eIn the more restrictive version of the license, disallow the company or any of its subsidiaries to use the software in ANY capacity.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eDisallow any forks to change the license, for as long as your organization exists and has an interest in the project, or maybe take a page from BSL's book.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eNow here's the interesting bit: name the exact companies. For example: \u0026quot;These restriction clauses apply to the following companies and their subsidiaries: Amazon, Google, Microsoft, Alibaba Group, IBM, Oracle, Tencent, ...\u0026quot;\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThat way, you choose exactly which companies are restricted and you can even write that the list may be updated in the future and that they must acquire a commercial license from you within X months of being added to the list.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eThat's it. The community members get to use the software how they please (outside of modifying the license), smaller companies can use the software and have a healthy ecosystem and roughly competitive managed offerings to whatever you might have, whereas the large hyperscalers are not allowed to even touch the software, should you choose to, unless they fork over whatever amount of money you demand.\u003c/p\u003e\n\u003cp\u003eI did offer up as much as an idea in a comment on a related HackerNews discussion, where it promptly got downvoted and didn't get much traction, the critiques I got being:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e#1: No legal department will ever approve using software under a license like that. Who wants to risk being the next addition to that list?\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003e#2: You could, but it still wouldn't be open source.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eI think both of those arguments aren't very good.\u003c/p\u003e\n\u003cp\u003eCorrect me if I'm wrong, but I'm not Amazon Web Services, nor have billions in revenue under my couch cushion. Statistically, neither are you. Nor will you ever be. Most businesses out there, I'd reckon that around 90-99% of them are far smaller in their scale and there's no reason for you to fear ending up on that list. And if you do? Just pay up. If you have enough money not to worry about software licenses, like those of the Oracle database, or any other piece of enterprise hardware bankrupting you, then you have no reason not to pay your fair share.\u003c/p\u003e\n\u003cp\u003eSecondly, it's not open source. Yes. That's correct. For most folks, it won't matter. It's been proven with plenty of the cases above, that open source doesn't really work in our current world and society, outside of rare exceptions. Even Linux gets most of its contributions from large corporations that have people be paid to work on the project. For example, let's look at \u003ca href=\"https://links.kronis.dev/c0upz\"\u003ethe statistics for the 6.0 release of the kernel\u003c/a\u003e, helpfully summarized by LWN:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"linux-kernel-employer-statistics.jpg\" alt=\"linux-kernel-employer-statistics\" title=\"linux-kernel-employer-statistics\"\u003e\u003c/p\u003e\n\u003cp\u003eEven if you summarize just the top 20 most active employers, then you still get 90% of the total lines changed coming from them. Even in such a widely used open source project, at the end of the day, money still plays a role. If those big companies could move to a permissively licensed alternative and it'd give them a competitive edge, be sure that they would. In other words, practicality matters more than ideological stances and if source available licenses will prevent your company from dying, then don't try to go full Stallman and just do what works.\u003c/p\u003e\n\u003cp\u003eFurthermore, even a switch from something like Apache 2.0 license to this hypothetical anti-hyperscaler license would be a really tough sell, since if the users feel like some freedoms are taken away from them (even if not really, because they are not and will never be on that list themselves), they will still rebel and reject your software. Ergo, the license to protect yourself as the author of the software must be there from day 1.\u003c/p\u003e\n\u003cp\u003eAlternatively, you can do what \u003ca href=\"https://links.kronis.dev/r97mb\"\u003eDrone CI\u003c/a\u003e and lots of other software does: have an open core that lets most of the smaller organizations still find plenty of value in the software, but require additional enterprise features to be licensed. Drone's case is particularly interesting, because their Enterprise license has the following in it:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003col\u003e\n\u003cli\u003eYou must limit use of this software in any manner primarily intended for or directed toward commercial advantage or private monetary compensation to a trial period of 32 consecutive calendar days. This limit does not apply to use in developing feedback, modifications, or extensions that you contribute back to those giving this license.\u003c/li\u003e\n\u003c/ol\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003e...\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eWaiver: Individual and Small Business\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eContributor waives the terms of rule 1 for companies meeting all the following criteria, counting all subsidiaries and affiliated entities as one:\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cul\u003e\n\u003cli\u003eworldwide annual gross revenue under $5 million US dollars, per generally accepted accounting principles\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cul\u003e\n\u003cli\u003eless than $5 million US dollars in all-time aggregate debt and equity financing\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eContributor will not revoke this waiver, but may change terms for future versions of the software.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eFor what it's worth, there was still the \u003ca href=\"https://links.kronis.dev/0jpg2\"\u003eWoodpecker CI fork\u003c/a\u003e, though I still use Drone for my personal needs, because they got no bad will from me for their licensing choices. If nothing else, those terms feel very much fair. And even \u003ca href=\"https://links.kronis.dev/uf9df\"\u003eas Drone is being retired in favor of Harness\u003c/a\u003e, I don't really feel like I've missed out on anything over the years I've been using the software, because even the free version had every feature I needed.\u003c/p\u003e\n\u003cp\u003eOn the other hand, there's software like \u003ca href=\"https://links.kronis.dev/4myzf\"\u003eMattermost\u003c/a\u003e, that more or less ruin the open core model: despite it being a great self-hostable alternative to Slack, you can't do more than 1:1 calls in the free version, nor can you have AD/LDAP sync of the users. That obviously sucks if you just want to have some software for your scrappy startup with like 10 people working in it, vs arguably the more sensible exemptions in other licenses for small businesses, maybe even a limit on the total user count, like \u003ca href=\"https://links.kronis.dev/e2786\"\u003eDocker has done with Docker Desktop\u003c/a\u003e:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eDocker Desktop is free for small businesses (fewer than 250 employees AND less than $10 million in annual revenue), personal use, education, and non-commercial open source projects.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eOtherwise, it requires a paid subscription for professional use.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eIn my eyes, there's nothing wrong with it, nor is there anything wrong with them imposing limitations for how often container images can be pulled from their Docker Hub, but there is definitely an issue when you cripple your software while claiming that it has a good free version.\u003c/p\u003e\n\u003ch3\u003eSummary\u003c/h3\u003e\n\u003cp\u003eI still very much believe that both software licensing and the whole situation around it is an absolute mess.\u003c/p\u003e\n\u003cp\u003eWhile these license changes do get a lot of negative attention, they still feel like something that is necessary to ensure the longevity of the companies actually developing the software.\u003c/p\u003e\n\u003cp\u003eOpen source and free software are both nice, but due to the prevalence of hyperscalers and none of the companies wanting to give anything back to the authors of the software, some sort of protectionism is needed. The problem there is that any license changes or most community members (that are not causing the financial hardships) being caught in the blast range will lose a lot of goodwill and often will lead to the death of your software by being repeatedly forked, alongside your community and ecosystem.\u003c/p\u003e\n\u003cp\u003eTo me, \u0026quot;source available\u0026quot; as a term seems to minimize all of the nuance, but licenses like BSL or even ones that have company size and revenue limits feel sensible. Not only that, but if you want to protect your software against hyperscalers while not harming the little guy (e.g. small to medium sized companies and competition that feels more fair), then I think that the gloves should come off and those large companies should be named directly within the licenses of the software you make, so only they get the more stringent restrictions on how they can use your software.\u003c/p\u003e\n\u003cp\u003eIf AWS wants to swallow the entirety of your userbase with their managed offering of your software, they shouldn't get neither free beer, nor free speech, unless that speech is telling you that they'd like to license out a commercial version of your software and give you plenty of money for using it in their cloud offering, so it becomes fair.\u003c/p\u003e\n\u003cp\u003eI'm surprised that this stance isn't embraced more and even in regards to software development, open source devs do it for close to no money. As someone in a country where neither the salaries nor inflation are that good, it makes me wonder - are they just swimming in money to the degree where they can afford to do that? For example, when opening a GitHub issue on some project, I wonder why I don't get a bot comment along the lines of:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThe current bug/improvement bounty for this issue is: 50 USD (link to contribute more)\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eAny committed bounties will be returned to you within 30 days if the issue isn't solved, whereas upon it being solved you will be able to vote alongside the other patrons about releasing the funds.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eAny money held in the escrow doesn't contractually obligate the developers to solve the issue the exact way you desire to, but rather indicates to them the interest and commitment by the community - that the issue and its resolution matters to them.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eBut hey, maybe it's easy for me to say that, when the principles of free software never quite landed, given the state of Nvidia drivers and the state that trying to run a free software only OS or even IDE leaves my computer in: with the mouse and audio and Wi-Fi not working, as well as not being able to install plugins or even have features like the commercial IDEs have.\u003c/p\u003e\n\u003cp\u003eThat is why I pay for the JetBrains tools and also don't shy away from using both Windows and the likes of Linux Mint with a bunch of proprietary software running on them when needed, alongside paying for the likes of \u003ca href=\"https://links.kronis.dev/scqld\"\u003eMobaXTerm\u003c/a\u003e and donating to the likes of \u003ca href=\"https://links.kronis.dev/5lefa\"\u003eFreeFileSync\u003c/a\u003e. If they provide value to me, I will pay for them. That's how it should work.\u003c/p\u003e\n\u003cp\u003eI have donated money to open source projects in the past and see myself doing so in the future, but I won't lie in saying that trying to work on open source in most cases will often doom you to freeloaders, rude feature requests and poverty; unless you have enough time, energy and motivation to work on it after work hours pro bono, which I sadly do not.\u003c/p\u003e\n\u003cp\u003eAt the end of the day, choose whatever license you think works best for you. As always, I just offer my thoughts and honest look on the state of the world, from my perspective.\u003c/p\u003e\n"
        },
        {
            "title": "Less free will is good, actually",
            "date_published": "2025-04-12",
            "id": "https://blog.kronis.dev/blog/less-free-will-is-good-actually",
            "url": "https://blog.kronis.dev/blog/less-free-will-is-good-actually",
            "content_html": "\u003cp\u003eThere's this one sci-fi short story called \u003ca href=\"https://links.kronis.dev/9j6lg\"\u003eWhat's Expected of Us\u003c/a\u003e, that I quite enjoyed. It was \u003ca href=\"https://links.kronis.dev/0bvg5\"\u003epublished back in 2005 by Nature\u003c/a\u003e, though if you want, you can also read \u003ca href=\"https://links.kronis.dev/0bk56\"\u003ean archived copy\u003c/a\u003e of it.\u003c/p\u003e\n\u003cp\u003eThe premise of the story is quite simple, there's a device called a Predictor:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eBy now you've probably seen a Predictor; millions of them have been sold by the time you're reading this. For those who haven't seen one, it's a small device, like a remote for opening your car door. Its only features are a button and a big green LED. The light flashes if you press the button. Specifically, the light flashes one second before you press the button.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eIt has a fictional circuit with a negative time delay, essentially it sends a signal back in time. From the perspective of the user, it is a device that predicts the future and which cannot be fooled: if you are going to press the button one second in the future, then the light will come on now, whereas if you won't press the button, there will be no light. Therefore, every time the light comes on, you will press the button shortly afterwards.\u003c/p\u003e\n\u003cp\u003eThe focus in the story was not as much on the time travel aspect, but rather the implications of this: that there is a set future in which things go a particular way and from your present perspective, nothing you do can change it because every decision of yours, even ones to attempt changing things, are predetermined and participate in that future, you just don't know that you're going to make them until you do.\u003c/p\u003e\n\u003cp\u003eThat is more or less the school of thought of \u003ca href=\"https://links.kronis.dev/em60v\"\u003edeterminism\u003c/a\u003e, which posits that there is no indeterminable force that's driving the universe, but rather that what happens is one long string of cause and effect. There is a bit more nuance to it and attempts at making determinism be compatible with the idea of free will, but the story simplifies things a bit.\u003c/p\u003e\n\u003cp\u003eIn the story, this implied that there isn't free will, which sent a lot of people into a deep depressive state:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eTypically, a person plays with a Predictor compulsively for several days, showing it to friends, trying various schemes to outwit the device. The person may appear to lose interest in it, but no one can forget what it means — over the following weeks, the implications of an immutable future sink in. Some people, realizing that their choices don't matter, refuse to make any choices at all. Like a legion of Bartleby the Scriveners, they no longer engage in spontaneous action. Eventually, a third of those who play with a Predictor must be hospitalized because they won't feed themselves. The end state is akinetic mutism, a kind of waking coma. They'll track motion with their eyes, and change position occasionally, but nothing more. The ability to move remains, but the motivation is gone.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eFrom my perspective, the concept of not having free will doesn't sound that bad.\u003c/p\u003e\n\u003cp\u003eIf you can calculate how thick the walls and the floors of a building need to be, for it to be safe, if you can calculate how much food of a certain kind you should eat, for you to be healthy, or even if it's possible to calculate how to get rockets into space, why wouldn't we be able to figure out how people react to certain stimuli, how they make decisions and what the larger implications of all that are for an entire society, or humanity as a whole?\u003c/p\u003e\n\u003cp\u003eThe problem there, of course, is the fact that nobody has enough data to do all of that. Yet, if we assume that the entirety of a human being is contained within the body and we know that we're more or less very complex chemical computers, that we still obey the laws of physics fully, then it's not extreme to suggest that one's free will is at best an unsolved mathematical equation. Whether we can or we will ever be able to solve it doesn't even matter, just that there is one.\u003c/p\u003e\n\u003cp\u003eThe thing is, that even if that was true, if all of the beliefs about determinism were true and the future was more or less predetermined, that doesn't really make me sad.\u003c/p\u003e\n\u003ch3\u003eIf nothing matters, then you're absolved of burden\u003c/h3\u003e\n\u003cp\u003eRight now, I live in a time period of uncertainty. A period of illnesses sweeping the world, wars and armed conflict, the rise of authoritarian governments, countless types of hateful rhetoric, a world where the way how people engage with information screams \u0026quot;post-factual society\u0026quot; and probably the looming threat of a mass extinction of all sorts of both flora and fauna.\u003c/p\u003e\n\u003cp\u003eYou can't really escape all of that, even if you don't listen to the news. For example, as yet another person that's employed in your typical 9-5 and therefore has to think about what will happen with the money I've earnt so far, the stock market feels very unstable due to various political factors and pulling my money out of it would just be choosing how much of it I want to lose vs leaving the investments alone - the value of which might recover years down the line, or not at all.\u003c/p\u003e\n\u003cp\u003eEven in regards to what happens there, I can't and won't be able to predict the future, at best I could make educated guesses, but even educating myself on all of the factors at play, over which I hold no sway, would just be an exercise in frustration and futility. Making oneself educated is okay, but attempting to intuit the future is like fighting against a windmill: with you only feeling that you're winning briefly, when the wind isn't blowing. In other words, that's just gambling with extra steps, but the same delusions behind it.\u003c/p\u003e\n\u003cp\u003eBut for a second, imagine that whatever I'm going to do and what is going to happen is predetermined. If I lose a large chunk of my life savings, then I would have always lost it, it's just a waiting game for that moment to arrive and become my present. It's the same if I don't end up losing money, or even turn a profit. It's also the same in regards to any and every choice that I could make - which country to live in, which companies I choose to work for or whether I try to do some successful/unsuccessful entrepreneurship, what crafts and technologies I choose to invest my time into and get educated in and so on.\u003c/p\u003e\n\u003cp\u003eThat doesn't mean that I need to be angsty when I say that \u0026quot;nothing matters\u0026quot;, but rather, that both my good and bad decisions are just the logical outcomes of the string of events that is my life, from the moment that I was born to the events of the current day. Making those choices or even having the ability to make them is just a reflection of the circumstances that I'm in.\u003c/p\u003e\n\u003cp\u003eI have a lowest point that I'll go to in my life, as well as a highest point, a ceiling of sorts - I'm probably never going to be an astronaut, or a president, or even one of those fancy millionaires who spend their lives with the ability to use some of their money to take away plenty of the issues they face. Some day I will get old and struggle with poor health, just like everyone else. If nothing else, that just means that I shouldn't consider myself as one of those \u0026quot;temporarily embarrassed millionaires\u0026quot;, John Steinbeck or Ronald Wright might have had a point.\u003c/p\u003e\n\u003cp\u003eAs for my decisions, sometimes I will make them impulsively, sometimes I'll make them based on whatever information is available to me and it doesn't matter whether I was right or not in any of those instances. This also applies to other people: those who are good and help others will always have done so, whereas those who are selfish also will have been predetermined to function that way. Wars will happen, betrayal will happen, it would have always happened and we just reached the point in time where it has, without knowing what's next.\u003c/p\u003e\n\u003cp\u003eWhy say all this? Because we shouldn't expect ourselves to be more than human, to live up to some contrived ideals.\u003c/p\u003e\n\u003cp\u003eI've seen people having so much anxiety about the seeming indeterminism in life and worrying about the choices that they need to make. I've been there myself, debating what sort of a degree I should get, what to do in regards to relationships, or even what technologies to try and explore in my career. Second guessing myself and also wondering about whether I should have done things differently in the past, to have fewer regrets. At the end of the day, some choices will be made anyways regardless of whether they're good or bad, sometimes you won't even get a choice, yet there's no reason to ruin one's own mental health by replaying it over and over again in your mind.\u003c/p\u003e\n\u003cp\u003eIt doesn't seem like we frame things that way, though.\u003c/p\u003e\n\u003ch3\u003eWe should live with less worries and overthinking\u003c/h3\u003e\n\u003cp\u003eInstead, we sometimes put the supposed human free will over almost any other factor, as opposed to viewing human beings as more predicable (and admittedly flawed) beings.\u003c/p\u003e\n\u003cp\u003eIt's so deeply ingrained into us that we believe that people can fundamentally \u0026quot;change\u0026quot; when relationships are going badly or when someone is closed minded - not to say that people can't change, but rather that someone completely changing their mind is unlikely to the degree of it bordering wishful thinking. That's how you get people deluding themselves with reasons to stay in abusive relationships, or tolerating narcissists just because they're relatives or something.\u003c/p\u003e\n\u003cp\u003eNot to say that I'm perfect myself, even if in ways that are less harmful to others: I'm no stranger to having a bad sleep schedule, eating junk food, not really taking care of my body as well as I should, or even getting nitpicky and picking arguments that don't really matter and shouldn't have happened. I am, of course, working on self-improvement, but if I'd start having sleepless nights where I judge myself over my failures, you'd know that I've gotten lost in those unrealistic expectations towards success.\u003c/p\u003e\n\u003cp\u003ePeople shouldn't obsess over the idea of having to make the right choices to the degree of becoming sick with worry. I think what they should do instead is reframe things and put them in perspective: while we are each the main protagonist of our individual stories, in the grand scheme of things, most of the things we do are quite inconsequential. Why should I obsess over the investments if it does me no good? I'll just make a more or less educated decision when I feel that it's the right time and whatever happens, happens.\u003c/p\u003e\n\u003cp\u003eI'm also helping out a friend of mine that's in a difficult financial situation after going through chemo and not being able to work for a while. I still very much have those same concerns and worries about how much I should help them, but the more correct thing here is to shut that stupid little anxious voice in my mind and just help the person as best as I can - because it's quite literally a matter of either them being able to pay rent or ending up homeless.\u003c/p\u003e\n\u003cp\u003eI've also helped other friends and will have to keep doing that in the future, even with things like trying to donate towards improving the climate conditions, donating to those who are less fortunate and so on - regardless of exactly how much that matters in the grand scheme of things, because I'm destined to at least try, the same way how someone else might be destined to be way better at this than I am, or on the contrary, someone else is destined to do bad.\u003c/p\u003e\n\u003cp\u003eOn the other hand, I've also cut people out of my life, when I felt that they don't embody the values that I want to associate with. People who are needlessly mean, the ones who are homophobic, transphobic, xenophobic, whatever. Maybe I give a bit more leeway in regards to being cautious of other nations, given the history of mine and our Eastern neighbors, the government of which doesn't particularly want our country or culture to exist. Either way, you don't have to take the high road or try to appease everyone or even be the best person you can be.\u003c/p\u003e\n\u003cp\u003eOn the same note, why should someone obsess over what degree to get? They already have a certain trajectory in life (e.g. Ivy League vs any number of other options, or no options at all) and even if they choose wrong or fail a few times, a few years of their life here or there won't radically change anything. They can always go back and get a different degree and if they can't, they might want to have a word or two with whatever higher deity might exist (or not).\u003c/p\u003e\n\u003cp\u003eYou can't predict what is the \u0026quot;right\u0026quot; choice, so why hold yourself to that standard? It's the same in regards to employers, technologies, entire careers - the human life is thankfully so long that as long as you try your best, you'll probably be okay and if not, then chances are that it's outside of your control (e.g. entire economies and job markets being messed up beyond belief, which you can't fix).\u003c/p\u003e\n\u003ch3\u003eSummary\u003c/h3\u003e\n\u003cp\u003eI'm not saying that you need to live each of your days as it's your last, I'm actually suggesting that you should make what seem to be good decisions in the service of your future self, but the world is huge and human lives are long, so maybe don't worry too much the next time when you are faced with a decision of some sort. Have that late night snack when you shouldn't, skip writing code tests when you need to ship in a few hours, but do call your parents every now and then if they're nice people. We're all human beings, nobody is perfect, be good when you can.\u003c/p\u003e\n\u003cp\u003eDon't expect yourself to always succeed and whether it's something like the kinds of beliefs in Buddhism, or trying to practice mindfulness, or even believing that the free will presented to you is an illusion, you will only benefit from having any sort of a way to deal with whatever life throws at you and working on changing the things that you can change, while at the same time making peace with the things that you can't change.\u003c/p\u003e\n\u003cp\u003eOr, as a more positive reading of the sci-fi story would put it: what you are going to do is already decided in the future, and it's just waiting for you to make that your present.\u003c/p\u003e\n"
        },
        {
            "title": "Two Intel Arc GPUs in one PC: worse than expected",
            "date_published": "2025-04-10",
            "id": "https://blog.kronis.dev/blog/two-intel-arc-gpus-in-one-pc-worse-than-expected",
            "url": "https://blog.kronis.dev/blog/two-intel-arc-gpus-in-one-pc-worse-than-expected",
            "content_html": "\u003cp\u003eOver the years, I've had quite a few different GPUs:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eGeForce 9800 GT (this one melted to the point where reflow soldering wouldn't help)\u003c/li\u003e\n\u003cli\u003eGeForce GTX 650 Ti (this one ended up being too weak for most games, I gave it away to a friend)\u003c/li\u003e\n\u003cli\u003eRadeon RX 570 (this one ended up not having enough VRAM for modern games)\u003c/li\u003e\n\u003cli\u003eRadeon RX 580 2048SP (this one was pretty good, but a bit unstable, especially in VR)\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eMore recently, however, I've moved my main PC over to some Intel GPUs, due to their nice amount of VRAM, XeSS being a bit better than FSR in my experience (even though the support for it isn't quite as widespread), but most of all, them also being affordable when Nvidia seems to be charging an arm and a leg for their GPUs. While I would have otherwise gotten a new AMD GPU, I wanted to support more market competition and the price felt just right.\u003c/p\u003e\n\u003cp\u003eInitially I started with the A580 and thought that it wasn't all that good, but it turns out that even after the driver issues had been mostly patched out, it was actually my CPU that was the problem: the \u003ca href=\"https://links.kronis.dev/bkgxl\"\u003eRyzen 5 4500\u003c/a\u003e (which I got for ReBAR support, not wanting to hack around with my old \u003ca href=\"https://links.kronis.dev/zdrvv\"\u003eRyzen 7 1700\u003c/a\u003e, which wasn't officially supported). Essentially, all of \u003ca href=\"https://links.kronis.dev/dhwmo\"\u003ethe synthetic benchmarks that I could find online\u003c/a\u003e were lying to me and it wasn't as good as my previous CPU, quite possibly due to only having 8MB of L3 cache, despite me being able to otherwise OC it pretty well.\u003c/p\u003e\n\u003cp\u003eI would get low framerates in games, with the causes for that not being entirely obvious: it would almost never hit 100% usage but after upgrading to the \u003ca href=\"https://links.kronis.dev/k7ppl\"\u003eRyzen 7 5800X\u003c/a\u003e all of the issues seemed to disappear even in combination with the A580. In other words, suddenly the framerates were way better, not just when it comes to the B-series card where people suggested that there's driver overhead for the older CPUs, but also on the A-series card as well.\u003c/p\u003e\n\u003cp\u003eBy then, I had already acquired the successor by Intel (somehow managing to get the limited edition for a pretty okay price), which means that I had two Intel GPUs:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"https://links.kronis.dev/qr43b\"\u003eASRock Arc A580 Challenger\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://links.kronis.dev/inyax\"\u003eIntel Arc B580 Limited Edition\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eNow, they are decently powerful on their own in combination with the better CPU, but I have noticed that if I try to play a demanding game and record/stream it at the same time, the encoder still seems to get overloaded with either of the GPUs and there's no real way for me to tell the OS:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u0026quot;Hey, I want you to prioritize the recording, so the game framerate would just dip more, but the overall recording/stream would remain stable.\u0026quot;\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eBecause a lot of people suggest that the Intel Arc cards are pretty good for encoding and that you could get a card just for that, it seemed like it'd be a pretty good setup: since the A580 that I have is essentially a spare, that also has the pretty nice \u003ca href=\"https://links.kronis.dev/gh9zd\"\u003eIntel Quick Sync Video\u003c/a\u003e encoders in it, meaning that I could use the B580 for games or anything else and the A580 for the encoding, or even driving the other monitors (I have 4 in total).\u003c/p\u003e\n\u003cp\u003eIt also seemed like a good idea, because I could make most of the hardware that I have: one of my friends has an \u003ca href=\"https://links.kronis.dev/vsix4\"\u003eRTX 4080 Super\u003c/a\u003e and they still get similar issues while trying to both play games and record/stream at the same time, the encoder just plainly gets overwhelmed, despite the card costing around 1000 USD, so about twice the total of what I paid for both of my cards. In my case, it seemed that using two separate GPUs would sidestep the whole issue entirely by giving each of them a task to do.\u003c/p\u003e\n\u003cp\u003ePretty simple, right?\u003c/p\u003e\n\u003ch3\u003eThe hardware issues of a two GPU setup\u003c/h3\u003e\n\u003cp\u003eWell, not really.\u003c/p\u003e\n\u003cp\u003eI should probably get this out of the way first and foremost: I hate computers. Not computing, but rather the hardware and setting it up. In my years of working on them, I have NEVER had a case where I put things together and everything just works off the bat.\u003c/p\u003e\n\u003cp\u003eInstead, I've had failing PSUs and HDDs, bent motherboards because whoever came up with the ATX 24-pin connector is evil, though the locking mechanisms being trash also pretty much applies to the 6/8 pin GPU connectors. Front panel connectors are a mess, for whatever reason half the time the USB 3 headers do nothing, CMOS resets are unpleasant to do because the manufacturers put the battery under the GPU, sometimes the GPU blocks off SATA connections AND also the PCIe slots where I could put a splitter card, the cases are often unergonomic, CPU coolers are hard to install. I'm not sure whether all of this is due to trying to do things on a budget, but I absolutely hate working with the hardware and it has only ever made me be miserable.\u003c/p\u003e\n\u003cp\u003eErgo, when you see me throwing both of the GPUs in and see an ominous bundle of wires, know that I've given up on being organized and it's all in the name of getting things working:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-attempted-pc-setup.jpg\" alt=\"01-attempted-pc-setup\" title=\"01-attempted-pc-setup\"\u003e\u003c/p\u003e\n\u003cp\u003eDust aside, the story here is that even with a better cooler, the Ryzen 7 5800X still puts out a bunch of heat, I managed to use the \u003ca href=\"https://links.kronis.dev/jmbe0\"\u003eRyzen Master\u003c/a\u003e utility to dial it back a little, to make this more bearable with the Curve Optimizer and PBO:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003ePPT: 170 (from 180)\u003c/li\u003e\n\u003cli\u003eTDC: 95 (from 100)\u003c/li\u003e\n\u003cli\u003eEDC: 115 (from 120)\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThe good news is that it works, the bad news is that I still needed more airflow. Because I can't really be bothered to go through the trouble of setting up an AIO right now, I just opted for a bunch of case fans, hooking them up to monitor the CPU temps and have a pretty aggressive curve near thermal throttling, such as hitting 100% across everything around 87C.\u003c/p\u003e\n\u003cp\u003eNeither the old nor the new motherboard had 5 case fan headers, so I just opted for getting a fan controller. The problem there was that it has a whole bunch of wires and there's not enough space for me to put on the back side of the case. Thus, there's the ominously hanging bundle of wires, contained only by a liberal application of duct tape. If we're going by Warhammer 40'000 logic, that is where the Machine Spirit lives.\u003c/p\u003e\n\u003cp\u003eThe problem of the actual dual GPU setup, of course, was the fact that the second GPU wasn't being detected at all. I dug around the BIOS and found some options for splitting up the PCIe lanes, which didn't seem like a bad idea, given that the B580 only has PCIe 4.0 x8 lanes (even if I only have a PCIe 3.0 motherboard), so giving up half of the x16 shouldn't be a big deal:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02-lane-setup.jpg\" alt=\"02-lane-setup\" title=\"02-lane-setup\"\u003e\u003c/p\u003e\n\u003cp\u003eOf course, that was a mistake, as I found rather quickly, because I went from 1 working GPU to 0 working GPUs and would get no video output at all:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03-no-video-signal.jpg\" alt=\"03-no-video-signal\" title=\"03-no-video-signal\"\u003e\u003c/p\u003e\n\u003cp\u003eI knew that I had selected the wrong setting, but I wasn't sure what the actual issue was, why would x8x8 cause problems, when x16 had worked just fine all this time? That was quite puzzling. At first I lamented the fact that the BIOS doesn't actually attempt to explain pretty much anything (the poster child of poorly documented software; I've almost always seen BIOSes that have dozens of oddly named options, but never anything remotely resembling a good explanation of what most of those do; it's like the manufacturers hate you), though thankfully I did notice a QR code which could bring me to a better source of information:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04-useless-qr-code.jpg\" alt=\"04-useless-qr-code\" title=\"04-useless-qr-code\"\u003e\u003c/p\u003e\n\u003cp\u003eThat, of course, didn't help me. It just links to the PDF of the motherboard documentation, which doesn't even explain the exact options, just that some of the lanes seem to be shared. I find it quite distasteful that the motherboard or the computer as a whole doesn't seem to have that much status output about what's going wrong, or even what might be the expected state of everything. For example, is it really too much to ask to put like 10-20 LEDs on the motherboard that would indicate the state of each of the connected components, as opposed to a black box with spinning fans but no other signs of life.\u003c/p\u003e\n\u003cp\u003eThis was one of those cases where I had to mess around until things finally started working (with a few times of pulling the CMOS battery in the middle of that), with me eventually settling on the x8x4x4 preset, still giving my main GPU 8 lanes, 4 lanes for the secondary GPU (no big deal for encoding) and 4 lanes for... something else:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"07-proper-lane-setup.jpg\" alt=\"07-proper-lane-setup\" title=\"07-proper-lane-setup\"\u003e\u003c/p\u003e\n\u003cp\u003eThe next problem was that suddenly like half of my drives were no longer being detected, even though this time around I wasn't using a PCIe extension card or anything of the sort, just the SATA ports on the motherboard itself. Yet, the drives were not there:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"05-drives-not-detected.jpg\" alt=\"05-drives-not-detected\" title=\"05-drives-not-detected\"\u003e\u003c/p\u003e\n\u003cp\u003eNor did they show up in the BIOS either, for whatever reason they were just not being detected:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"06-nothing-in-bios-either.jpg\" alt=\"06-nothing-in-bios-either\" title=\"06-nothing-in-bios-either\"\u003e\u003c/p\u003e\n\u003cp\u003eMany folks won't be affected by this, but I do have an interesting storage setup, where I just use a bunch of cheap Seagate Barracuda 1 TB HDDs that I got back when they were cheap for a little bit, around 40 EUR for each, for the storage of my files. For gaming and boot drives, however, I use SATA SSDs because they do seem to both be affordable and fast enough.\u003c/p\u003e\n\u003cp\u003eEventually, I dug through the manual and realized that it might be the M.2 NVME drive that I intended to use for dual booting Linux in the future (because my current setup of splitting the same SSD into partitions, some for Windows and some for Linux distros isn't entirely comfortable), because it might be eating up the PCIe lanes that would otherwise go for the other SATA drives.\u003c/p\u003e\n\u003cp\u003eSo, I removed it:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"08-the-culprit.jpg\" alt=\"08-the-culprit\" title=\"08-the-culprit\"\u003e\u003c/p\u003e\n\u003cp\u003eAnd suddenly my issues were resolved and all of the drives started showing up properly, both in the BIOS:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"09-drives-protected.jpg\" alt=\"09-drives-protected\" title=\"09-drives-protected\"\u003e\u003c/p\u003e\n\u003cp\u003eAs well as in the OS itself, it seems like I won't be able to use the NVME M.2 drive alongside everything else, but that's a problem for the future:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"10-drives-detected.jpg\" alt=\"10-drives-detected\" title=\"10-drives-detected\"\u003e\u003c/p\u003e\n\u003cp\u003eWith all of that frustration out of the way (and no actual answers, rather just toggling a few BIOS settings until things start properly working), the PC was finally brought back to life and now I had a setup with two Intel Arc GPUs, side by side.\u003c/p\u003e\n\u003cp\u003eHere's a curious detail along the way: I don't know why the older A580 needs two 8-pin connectors because it doesn't really draw that much power and it doesn't seem to be like my old RX 570 card that had an 8+6 setup, where you could just leave the 6-pin connector unplugged if you didn't intend to OC: with the A580, for whatever reason, only plugging in one of those wouldn't work.\u003c/p\u003e\n\u003cp\u003eFor now, I'm happy that the setup works, though if I did ever intend to fix this cable management, then I'd also probably want to look in the direction of an Y splitter cable for the A580:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"11-both-gpus-in.jpg\" alt=\"11-both-gpus-in\" title=\"11-both-gpus-in\"\u003e\u003c/p\u003e\n\u003cp\u003eDespite the shared lanes, it didn't actually seem like there was much of a negative effect on the I/O of the system, even when everything is loaded at the same time (sequential tests here, because the random ones are horrible for HDDs, thankfully that's not the workload that I need them for, more like storing all sorts of files and movies and whatnot), so that was nice to see:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"12-write-speeds-okay.jpg\" alt=\"12-write-speeds-okay\" title=\"12-write-speeds-okay\"\u003e\u003c/p\u003e\n\u003cp\u003eFinally, I would be able to discover why many of my friends recommended against a dual GPU setup and whether this would really be workable.\u003c/p\u003e\n\u003ch3\u003eThe software issues of a two GPU setup\u003c/h3\u003e\n\u003cp\u003eIn theory, there shouldn't actually be that much that's difficult about two GPUs running in the same system, as long as you don't expect graphical workloads to be split among them (like SLI), but rather just want to delegate some tasks to them. It was nice to see that I wouldn't even have to use DDU because both of them were detected and working out of the box, though I did run a regular GPU update with the clean install option some time later:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"13-both-cards-detected.jpg\" alt=\"13-both-cards-detected\" title=\"13-both-cards-detected\"\u003e\u003c/p\u003e\n\u003cp\u003eHere's the GPU-Z output, in case anyone is curious. The ASRock card is actually pretty good and I did OC it a bunch previously, but wanted to mostly keep it stock for now, just to not overload my PSU (for what it's worth, 650W seems to be mostly sufficient for a 190W + 150W GPU setup and a 100W CPU):\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"14-gpuz-two-intel-cards.jpg\" alt=\"14-gpuz-two-intel-cards\" title=\"14-gpuz-two-intel-cards\"\u003e\u003c/p\u003e\n\u003cp\u003eThere were also immediate problems on the software side though, notably the fact that Windows 10 gets pretty confused. Apparently in Windows 11 you get a nice dropdown for when you want to set which programs should use which GPU, but in Windows 10 it just tells you to go frick yourself. There are some \u003ca href=\"https://links.kronis.dev/0unv7\"\u003esupport questions\u003c/a\u003e about how to change it into something more sane, but no actual answers, leaving us with something like this:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"15-windows-10-is-kinda-dumb.jpg\" alt=\"15-windows-10-is-kinda-dumb\" title=\"15-windows-10-is-kinda-dumb\"\u003e\u003c/p\u003e\n\u003cp\u003eApparently an \u003ca href=\"https://links.kronis.dev/ky1sv\"\u003eInsider Build had such a feature\u003c/a\u003e, but sadly I'm an outsider, so no dice there.\u003c/p\u003e\n\u003cp\u003eFurthermore, I saw a really interesting type of BSOD while running some stress testing (that I couldn't replicate later), where the main GPU was loaded to the max with \u003ca href=\"https://links.kronis.dev/uv7kj\"\u003eFurmark\u003c/a\u003e and the other was used for encoding a video of me doing that, using \u003ca href=\"https://links.kronis.dev/qafch\"\u003eOpen Broadcaster Software\u003c/a\u003e, which lead to the monitors connected to the main GPU showing the typical BSOD screen but the others just freezing on whatever was the last visible picture:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"16-interesting-instability.jpg\" alt=\"16-interesting-instability\" title=\"16-interesting-instability\"\u003e\u003c/p\u003e\n\u003cp\u003eI've never really seen that before.\u003c/p\u003e\n\u003cp\u003eAlso, you might have noticed that only the main screen was connected to the B580. My idea was that I could perhaps lower the load on the main game GPU by letting the secondary one take over rendering whatever is on the other monitors, however this seems to have actually made things a lot worse for reasons that once more aren't entirely clear to me.\u003c/p\u003e\n\u003cp\u003eFor example, when I would have videos (like YouTube) playing in a browser that's on one of the other monitors, I'd get very high GPU load on the main GPU, the culprit being the Desktop Window Manager process:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"17-dwm-misbehaving.jpg\" alt=\"17-dwm-misbehaving\" title=\"17-dwm-misbehaving\"\u003e\u003c/p\u003e\n\u003cp\u003eThis wasn't just related to video playback, but anything that would cause a lot of things to be drawn often. For example, if I used a regular \u003ca href=\"https://links.kronis.dev/kaahd\"\u003eFPS test page\u003c/a\u003e, then everything would be fine with one of the monitors running the test:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"18-fps-main-monitor-video.jpg\" alt=\"18-fps-main-monitor-video\" title=\"18-fps-main-monitor-video\"\u003e\u003c/p\u003e\n\u003cp\u003eBut as soon as I'd launch multiple windows of the same test, one on each monitor, suddenly the framerates across everything would fall horribly. This would happen with games, OBS, or frankly any piece of software with any kind of visuals going on. As you can probably tell, a regular browser tab overloading the main GPU and making the FPS dip to \u0026lt;20 isn't remotely acceptable:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"19-fps-all-monitors-video.jpg\" alt=\"19-fps-all-monitors-video\" title=\"19-fps-all-monitors-video\"\u003e\u003c/p\u003e\n\u003cp\u003eAnother interesting issue was that after a restart and just trying to use the B580 for all of the monitors, suddenly one of the monitors only had lower resolutions available, as opposed to the regular 1920x1080 at 60 FPS, in contrast to all of the other monitors I have:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"20-monitor-resolution-weirdness.jpg\" alt=\"20-monitor-resolution-weirdness\" title=\"20-monitor-resolution-weirdness\"\u003e\u003c/p\u003e\n\u003cp\u003eIt was as if suddenly the 1920x1080 isn't even a supported resolution by the monitor itself:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"21-only-some-resolutions-available.jpg\" alt=\"21-only-some-resolutions-available\" title=\"21-only-some-resolutions-available\"\u003e\u003c/p\u003e\n\u003cp\u003eI also noticed an interesting thing, which is that the refresh rate would show up as 60.020 Hz, no idea where that particular figure comes from:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"22-weird-refresh-rate.jpg\" alt=\"22-weird-refresh-rate\" title=\"22-weird-refresh-rate\"\u003e\u003c/p\u003e\n\u003cp\u003eI did eventually narrow it down to probably having a bad adapter, which I need to do because only one of my monitors has an HDMI port, whereas all the others only have VGA/DVI available, meaning that the contents of my cabinets of computer supplies look more or less like this:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"29-adapter-hell.jpg\" alt=\"29-adapter-hell\" title=\"29-adapter-hell\"\u003e\u003c/p\u003e\n\u003cp\u003eEither way, I'm not spending 400 EUR to replace my current monitors because they work and the OS suddenly being weird is stupid, so I just ordered a new 10 EUR adapter and it should be okay eventually, for now I just grabbed another spare out of a package and it seems that it solved the issue.\u003c/p\u003e\n\u003cp\u003eI'd be surprised if you have gotten this far and aren't starting to at least slightly believe that every single attempt of mine to mess around with hardware inevitably leads to things breaking, even in ways that shouldn't be possible due to me not interacting with said hardware in any abnormal ways. Why on earth would the adapter decide to suddenly die, after having worked okay for months? I guess I need to make a sacrifice of an old HDD to make the Machine Spirit happy or something.\u003c/p\u003e\n\u003cp\u003eI was shortly going to hit even more issues, ones without immediate solutions.\u003c/p\u003e\n\u003ch3\u003eYou can't have two Intel Arc GPUs and choose which one to use for encoding\u003c/h3\u003e\n\u003cp\u003eThe biggest problem, however, was that if I have two Intel Arc GPUs in the same system, then I can't choose which one I want to use for encoding on Windows 10. Let me repeat that: I put in a second GPU in the system for the explicit purpose of doing video encoding with it (after realizing that I can't drive the other monitors with it without weird overloading issues) and I can't do that one exact thing.\u003c/p\u003e\n\u003cp\u003eWithin OBS you get a dialog to choose what sort of an encoder you want and \u003ca href=\"https://links.kronis.dev/tfy6l\"\u003efor Nvidia GPUs you do get a nice dropdown\u003c/a\u003e to indicate which one should handle the encoding (see under \u0026quot;Advanced Settings\u0026quot;), whereas for Intel Arc GPUs there is no such thing:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"22-no-gpu-obs-option.jpg\" alt=\"22-no-gpu-obs-option\" title=\"22-no-gpu-obs-option\"\u003e\u003c/p\u003e\n\u003cp\u003eIn other words, when you have two Intel Arc GPUs on a regular install of Windows 10 (I cannot test Windows 11 right now), it will use the main GPU, the same that you use for playing games, the very same where games that put it under a bunch of load make the encoder overloaded, making the setup with two GPUs completely useless.\u003c/p\u003e\n\u003cp\u003eThe sort of good news is that OBS is open source and if anyone was feeling brave, you could just \u003ca href=\"https://links.kronis.dev/83orl\"\u003ewrite the missing functionality yourself\u003c/a\u003e, compared to what is \u003ca href=\"https://links.kronis.dev/o0q2i\"\u003ealready present with Nvidia\u003c/a\u003e, but frankly that's a bit above my paygrade for now. In other words \u0026quot;just write the missing code\u0026quot; is doing a lot of heavy lifting here, especially when C isn't my main language and I'm very demotivated already.\u003c/p\u003e\n\u003ch3\u003eGiving up, two vendors, half the issues\u003c/h3\u003e\n\u003cp\u003eI decided to sidestep the whole issue in its entirety. While that RX 580 was a bit too weak for games and doesn't have support for AV1, it still is a GPU, still doesn't draw too much power and still should be good enough for encoding, so I pulled it out of storage:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"23-my-old-rx580.jpg\" alt=\"23-my-old-rx580\" title=\"23-my-old-rx580\"\u003e\u003c/p\u003e\n\u003cp\u003eCompared to the A580 it is a bit less advanced, a bit less power efficient (for what it is), but also will save me dealing with at least some of these weird issues:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"24-two-gpus-side-by-side.jpg\" alt=\"24-two-gpus-side-by-side\" title=\"24-two-gpus-side-by-side\"\u003e\u003c/p\u003e\n\u003cp\u003eI left the B580 in as my main GPU because I'm quite satisfied with it and just replace the A580 with the RX 580. Still a very snug fit, still need some tape to prevent the cables going to the front panel from hitting the fans and slowing things down, still can't be bothered to clear all of the dust, because at this point I'm quite sure that the computer would just burst into flames if I tried, but hey, it's a setup:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"25-the-current-pc-setup.jpg\" alt=\"25-the-current-pc-setup\" title=\"25-the-current-pc-setup\"\u003e\u003c/p\u003e\n\u003cp\u003eInstalling the \u003ca href=\"https://links.kronis.dev/sx2g8\"\u003eAMD Software\u003c/a\u003e wasn't difficult either and it just worked as expected in combination with the \u003ca href=\"https://links.kronis.dev/lm8hf\"\u003eIntel Graphics Software\u003c/a\u003e, didn't even need to use DDU for this either:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"26-driver-install.jpg\" alt=\"26-driver-install\" title=\"26-driver-install\"\u003e\u003c/p\u003e\n\u003cp\u003eCuriously, when I last had the RX 580 in as the main GPU for that system, I actually needed to use an older driver version, because the latest ones actually made VR quite unstable, though thankfully I won't have to use that GPU for VR anymore (while the B580 isn't officially supported, at least \u003ca href=\"https://links.kronis.dev/99g4w\"\u003eVirtual Desktop\u003c/a\u003e has my back there).\u003c/p\u003e\n\u003cp\u003eThere was a little bit of weirdness, such as for a bit Intel believing that I have no GPUs present, but that resolved itself after a restart:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"27-mostly-okay.jpg\" alt=\"27-mostly-okay\" title=\"27-mostly-okay\"\u003e\u003c/p\u003e\n\u003cp\u003eSo where does that leave me? Finally I can choose which GPU to do the encoding on, because now I have one type of encoder supported by each card and the RX 580 still does support H265 (HEVC) for recordings, which has pretty passable quality for most of the things I might want to do:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"28-encoders-are-mostly-okay.jpg\" alt=\"28-encoders-are-mostly-okay\" title=\"28-encoders-are-mostly-okay\"\u003e\u003c/p\u003e\n\u003cp\u003eI did test out how far I can push the older card and it become clear pretty quickly that I can't actually try to stream with H264 at 1080p 60 FPS, because it gets overloaded. H265 at the same resolution and framerate doesn't seem to cause much load upon the GPU at all, not sure why H264 at those settings makes it shoot up to 100% utilization and breaks everything.\u003c/p\u003e\n\u003cp\u003eThankfully, by dialing down the resolution, I figured out that I can still have a passable setup:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003estreaming: H264, 720p, 60 FPS, CBR at 6'000 Kbps\u003c/li\u003e\n\u003cli\u003erecording: H265, 1080p, 60 FPS, VBR at 14'000 Kbps\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003ewhich is a step up from having the A580/B580 as the sole GPU and not being able to do much more than 1080p at 30 FPS without them getting too overloaded, or in games like S.T.A.L.K.E.R. 2 them getting overloaded regardless of what settings I try to use. Now, if the main GPU struggles with a game or something else, at least the secondary one remains pretty stable with doing its encoding task.\u003c/p\u003e\n\u003cp\u003eI did need to setup my monitor layout all over again (this time plugging all of them into the B580 because I don't want another run in with the Desktop Window Manager), but thankfully that wasn't too hard and \u003ca href=\"https://links.kronis.dev/ewagg\"\u003eFancyZones\u003c/a\u003e also makes custom snapping configuration easy:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"30-fixing-monitor-layout.jpg\" alt=\"30-fixing-monitor-layout\" title=\"30-fixing-monitor-layout\"\u003e\u003c/p\u003e\n\u003cp\u003eWith that, I could finally rest. I got a dual GPU setup working, though definitely not in the way how I had hoped.\u003c/p\u003e\n\u003ch3\u003eSummary\u003c/h3\u003e\n\u003cp\u003eIn the end, the setup looks like this:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"31-gpuz-final-setup.jpg\" alt=\"31-gpuz-final-setup\" title=\"31-gpuz-final-setup\"\u003e\u003c/p\u003e\n\u003cp\u003eOne card from 2018, the other from 2024, working in the same system well enough. Dual GPU setups are definitely a bit of a mess and I cannot in good conscience recommend that most folks look into them, unless you:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003ethrow a bunch of money at Nvidia who seem to have good software support\u003c/li\u003e\n\u003cli\u003eare either okay with waiting for it to get better for your vendor of choice or can write a bunch of code yourself\u003c/li\u003e\n\u003cli\u003eor you choose to have two GPUs from different vendors\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eI'm happy that I resolved the issue that the friend of mine had with their RTX 4080 Super, especially because the RX 580 cost me around 120-140 EUR when I got it a few years ago, though for whatever reason it seems that the price has now climbed up to 190 EUR even for an RX 570, so you might as well get a newer card if you need something now, like an RX 3050 or RX 6600.\u003c/p\u003e\n\u003cp\u003eI would still recommend an \u003ca href=\"https://links.kronis.dev/n5vgb\"\u003eIntel Arc A310\u003c/a\u003e if you need a dedicated encoder card, because there are nice single slot versions that are very conservative on power, while Intel Quick Sync Video is quite lovely, though I can only make that recommendation if you don't already have an Intel Arc GPU as your main one.\u003c/p\u003e\n\u003cp\u003eI won't lie to myself though and will admit that it's a little bit sad that I couldn't use a setup with two Intel Arc GPUs and that good encoder functionality ends up not getting used in my case, but hey, I'll probably end up sending the A580 for the friend that has the RTX 4080 Super, because it'd probably solve their issues as well.\u003c/p\u003e\n\u003cp\u003eIn summary: when we're not held back by hardware weirdness, we are held back by software weirdness. With the state of how things are, I'm half surprised that my PC even boots up most of the time when I press the power button. This experience did absolutely nothing for my hatred of computer hardware and with every passing day I understand the folks who just get a Mac more and more (even though only like 25% of my Steam library would work on a Mac).\u003c/p\u003e\n"
        },
        {
            "title": "They took Skype from us, Teams sucks",
            "date_published": "2025-04-07",
            "id": "https://blog.kronis.dev/blog/they-took-skype-from-us-teams-sucks",
            "url": "https://blog.kronis.dev/blog/they-took-skype-from-us-teams-sucks",
            "content_html": "\u003cp\u003eToday, I'd like to complain about Teams, but to do that, we must first mention Skype!\u003c/p\u003e\n\u003ch3\u003eSkype, it just works\u003c/h3\u003e\n\u003cp\u003eSkype is one of the older communication apps and has been around for a while, it recently turned \u003ca href=\"https://links.kronis.dev/fj5x0\"\u003e20 years old\u003c/a\u003e. It's gotten a bunch of updates over the years, some of which were liked more or less than others, but overall a lot of people have been using it to keep in touch. To some degree, that proves that you don't need the rapid growth of the likes of WhatsApp to be successful - instead, sometimes gradual and stable growth is equally as good, as is the staying power to be able to have the product survive for decades.\u003c/p\u003e\n\u003cp\u003eWhile, for professional collaboration, options like Slack or Teams are pretty popular, for having group calls or even 1:1 calls, Skype is still pretty good - essentially, it did what Zoom did, just better and earlier. I'm surprised that during the events of 2020 and the subsequent years, Skype didn't see the similar sort of headlines, comparable to those of \u003ca href=\"https://links.kronis.dev/e9zk7\"\u003ethe explosive growth of Zoom\u003c/a\u003e. I can tell you for a fact that some of the teams that I collaborate with have been using Skype for years with no issues, given the limitations of the free version of Zoom and not even having group calls in the free version of Slack. Of course, \u003ca href=\"https://links.kronis.dev/oxb1h\"\u003eSkype for Business\u003c/a\u003e might be its own unique kind of a mess, but we're not really talking about that today. Regular Skype is just there for us and it works.\u003c/p\u003e\n\u003cp\u003eExcept, no, that's coming to an end. Recently, Microsoft decided that Skype will soon be shut off and people have to migrate over to Teams. Why? Because \u003ca href=\"https://links.kronis.dev/1ggwx\"\u003ethey said so\u003c/a\u003e. Instead of actually putting in the work into keeping the service alive or maybe even improving it, it seems like they've decided to throw in the towel and increase the user numbers of their Teams solution. On some level, sure, having to manage one platform is probably easier for them, but on another it kind of feels like neglecting Skype for years and then trying to sweep it under the rug - why haven't there been aggressive ads for Skype, the same way there have been for Teams? But alas, this is just the situation that we find ourselves in.\u003c/p\u003e\n\u003cp\u003eIt's on the front page of \u003ca href=\"https://links.kronis.dev/5dgv4\"\u003eSkype's homepage\u003c/a\u003e and everything:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-skype-being-retired.jpg\" alt=\"01-skype-being-retired\" title=\"01-skype-being-retired\"\u003e\u003c/p\u003e\n\u003cp\u003eSo where does that leave us?\u003c/p\u003e\n\u003ch3\u003eTeams, it just sucks\u003c/h3\u003e\n\u003cp\u003eFor what it's worth, migrating over from Skype to Teams actually isn't a difficult process or anything. You can use the same account, your message history is kept and for the most part, you can just keep using it.\u003c/p\u003e\n\u003cp\u003eExcept for the occasional weirdness, such as meetings that are started from Teams do not show up in Skype, which lead to an awkward situation where people were asking me why I'm not joining a call, but I had to share a screenshot in which there is no call - as if I was in a completely different universe.\u003c/p\u003e\n\u003cp\u003eMore serious problems arise, however, once you realize that the features that you used previously won't be available anymore. After this migration, I was presented with the fact that supposedly I have both a work and a personal account under the same e-mail address:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03-account-types.jpg\" alt=\"03-account-types\" title=\"03-account-types\"\u003e\u003c/p\u003e\n\u003cp\u003eDo I know who made these and when? No, I don't have the slightest idea, it's an e-mail I used for a regular Microsoft account and Skype from way back: The problem is, that neither option really works. For example, if I pick the work account, then I get a warning about not being able to access an organization (what organization?):\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03-no-work-account.jpg\" alt=\"03-no-work-account\" title=\"03-no-work-account\"\u003e\u003c/p\u003e\n\u003cp\u003eOn the other hand, with a personal account I can log into the web version of Teams, but not if I install their app locally (like I would with Slack or Skype or whatever), which just throws an error and wants to redirect me to the web based version:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03-teams-web-only.jpg\" alt=\"03-teams-web-only\" title=\"03-teams-web-only\"\u003e\u003c/p\u003e\n\u003cp\u003eCuriously, I couldn't reproduce this on a completely newly created Microsoft account that has never had Skype, or anything else connected to it. There, the Teams app would just work, which makes me think that my account could just be uniquely screwed up, which doesn't do much when it has a lot of contacts that I need on it. Also, this newly created test account got blocked within a day anyways and I had to verify my phone number to unblock it, how odd.\u003c/p\u003e\n\u003cp\u003eBefore you critique me for being a unique case of some weirdly messed up account structure, remember: Skype had literally no issue with whatever is going on there.\u003c/p\u003e\n\u003cp\u003eAs for Teams not letting me use the desktop app, it doesn't have a good technical reason to be that way. If their local app just wraps the website (maybe with some native integrations sprinkled in), there's no reason why they couldn't just embed it, even in a system native web view or what have you. If it already works for the workplace accounts and works for newly created accounts, then clearly the software itself is capable of running, this is just some weird artificial limitation.\u003c/p\u003e\n\u003cp\u003eMaybe they're trying to push their users to pay for \u003ca href=\"https://links.kronis.dev/cl3gb\"\u003ethe business licenses\u003c/a\u003e? Maybe not explicitly in this case, though I definitely did see a few of the people that I collaborate with having the same sorts of issues (so my account situation isn't entirely unique) and moving over to the business license would definitely fix whatever this is. That said, it's not like there'd be a good timeframe for being able to fix it and there's more or less nothing that I can do on my own end here.\u003c/p\u003e\n\u003cp\u003eThat was kind of the beauty of Skype: you weren't encumbered with a set number of seats and didn't have to pay for anything. You'd just have an account, would log in with it and that was that. No attempts to upsell you with bundled office products or whatever, just a communication tool you can use. But for now, it seems like the age of freeloading is more or less over:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02-teams-pricing.jpg\" alt=\"02-teams-pricing\" title=\"02-teams-pricing\"\u003e\u003c/p\u003e\n\u003cp\u003eRegardless, I felt like the limitation is artificial and stupid, so I decided to get some control back over what's running on my PC, since I don't want Teams to be just some browser tab, but rather a program that I can manage separately (such as making it start automatically on system startup on a work computer).\u003c/p\u003e\n\u003ch3\u003eTaking a bit of control back\u003c/h3\u003e\n\u003cp\u003eDoing it isn't actually that difficult. You see, you can basically use \u003ca href=\"https://links.kronis.dev/d2yyt\"\u003eElectron\u003c/a\u003e to wrap any website that you like and launch it as a separate process, basically an embedded version of Chromium, with a few optional bits of native functionality. Admittedly, I also quite like the idea behind \u003ca href=\"https://links.kronis.dev/cd1ij\"\u003eWails\u003c/a\u003e, which does something very similar, except uses the web views available on the system itself, as opposed to making you bundle a whole browser runtime.\u003c/p\u003e\n\u003cp\u003eEither way, the result is pretty effective, you can have what's essentially the web version on Teams running in a local app:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04-teams-workaround.jpg\" alt=\"04-teams-workaround\" title=\"04-teams-workaround\"\u003e\u003c/p\u003e\n\u003cp\u003eFurthermore, you can also set the user agent string to whatever you want and it's functionally identical to just running a website, also a bit like how PWAs are supposed to work. The code for it isn't even that complex either!\u003c/p\u003e\n\u003cp\u003eHere's a \u003ccode\u003econfig.json\u003c/code\u003e that I extracted:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003e{\n    \u0026quot;windowTitle\u0026quot;: \u0026quot;Microsoft Teams\u0026quot;,\n    \u0026quot;mainURL\u0026quot;: \u0026quot;https://teams.microsoft.com/\u0026quot;,\n    \u0026quot;zoomLevel\u0026quot;: 1.00\n}\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eAnd here's \u003ccode\u003emain.js\u003c/code\u003e for Electron:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003econst { app, BrowserWindow, Notification, screen, desktopCapturer } = require('electron');\nconst fs = require('fs');\nconst path = require('path');\n\n// load config.json\nconst configPath = path.join(__dirname, 'config.json');\nlet config;\ntry {\n    const data = fs.readFileSync(configPath);\n    config = JSON.parse(data);\n} catch (error) {\n    new Notification({ title: 'Error!', body: 'Error reading config.json: ' + error }).show();\n    app.quit();\n}\n\nfunction createWindow() {\n    const configOk = config \u0026amp;\u0026amp; config.mainURL;\n    if (!configOk) {\n        new Notification({ title: 'Error!', body: 'mainURL is missing in config.json' }).show();\n        app.quit();\n        return;\n    }\n\n    // *almost* full size window by default\n    const primaryDisplay = screen.getPrimaryDisplay();\n    const { width, height } = primaryDisplay.workAreaSize;\n    const screenSpaceToLeaveFree = 128; // pixels\n    const screenWidth = width - screenSpaceToLeaveFree;\n    const screenHeight = height - screenSpaceToLeaveFree;\n\n    // prepare the Electron window\n    let window = new BrowserWindow({\n        width: screenWidth,\n        height: screenHeight,\n        backgroundColor: '#000000',\n        webPreferences: {\n            nodeIntegration: true\n        },\n        title: config.windowTitle || 'Electron',\n        icon: path.join(__dirname, 'images/logo-128.png'),\n    });\n\n    // set Chrome user agent, fake regular web browser (otherwise tries to use IPC, breaks)\n    const userAgent = '...(put whatever you want here)...';\n    window.webContents.setUserAgent(userAgent);\n\n    // configure session for better compatibility, fake regular web browser (otherwise tries to use IPC, breaks)\n    const session = window.webContents.session;\n\n    // block Teams from detecting Electron, fake regular web browser (otherwise tries to use IPC, breaks)\n    session.webRequest.onBeforeSendHeaders((details, callback) =\u0026gt; {\n        const { requestHeaders } = details;\n        if (requestHeaders['User-Agent']) {\n            requestHeaders['User-Agent'] = userAgent;\n        }\n        callback({ requestHeaders });\n    });\n\n    // for screen sharing, allow the user to select their screen\n    ... (see below for an example)\n\n    // load the actual page we'll browse\n    console.log('Loading URL: ', config.mainURL);\n    window.loadURL(config.mainURL);\n\n    // if we need to change the zoom level\n    if (config.zoomLevel) {\n        window.webContents.on('did-finish-load', () =\u0026gt; {\n            window.webContents.setZoomFactor(config.zoomLevel);\n        });\n    }\n}\n\napp.whenReady().then(createWindow);\n\napp.on('window-all-closed', () =\u0026gt; {\n    if (process.platform !== 'darwin') {\n        app.quit();\n    }\n});\n\napp.on('activate', () =\u0026gt; {\n    if (BrowserWindow.getAllWindows().length === 0) {\n        createWindow();\n    }\n});\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eI'm not saying that this is 100% the correct way to do it (since I didn't care much for Electron IPC and just wanted to make the web version to stop trying to use it), but it just goes to show that it's neither super long or complex, nor something that vibe coding with an LLM in an afternoon couldn't do - because that's exactly what I used to make it, alongside a healthy dose of Googling and testing.\u003c/p\u003e\n\u003cp\u003eThere were some odd things along the way that seem new to Windows 11, such as needing to enable some developer features to be able to build the app:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"05-windows-developer-options.jpg\" alt=\"05-windows-developer-options\" title=\"05-windows-developer-options\"\u003e\u003c/p\u003e\n\u003cp\u003eRegardless, you just need Node.js and after that things work just fine:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"06-build-process.jpg\" alt=\"06-build-process\" title=\"06-build-process\"\u003e\u003c/p\u003e\n\u003cp\u003eAfter building the app, you get a nice installer (Windows is pictured here, other platforms are also supported) that you can either run directly, or send to someone:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"07-setup-files.jpg\" alt=\"07-setup-files\" title=\"07-setup-files\"\u003e\u003c/p\u003e\n\u003cp\u003eOnce it installs, it shows up under the installed apps like any other program, so removing it later is pretty trivial, should you need to do so:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"08-installed-apps.jpg\" alt=\"08-installed-apps\" title=\"08-installed-apps\"\u003e\u003c/p\u003e\n\u003cp\u003eIn addition to which, the same goes for launching it, it's just there in the start menu.\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"09-start-menu.jpg\" alt=\"09-start-menu\" title=\"09-start-menu\"\u003e\u003c/p\u003e\n\u003cp\u003eNow, since I am not a fan of getting a cease \u0026amp; desist in the mail, I'm not really sharing any binaries because there'd clearly be IP issues with using their trademarks, but nobody can prevent me from showing you a screenshot where I used whatever icon or name I please, as well as giving you the source code that I wrote myself, which you can honestly apply to any browser based app.\u003c/p\u003e\n\u003cp\u003eA while back, I actually used the same approach for getting a cross platform mail client that actually just opens the \u003ca href=\"https://links.kronis.dev/mt6l4\"\u003eNextcloud Mail\u003c/a\u003e app underneath.\u003c/p\u003e\n\u003cp\u003eWhile I do personally dislike how much Electron seems to be overused (especially given that projects like Wails or Tauri both already use the available system web view components, which most of the time should be used instead to not waste space), this sort of versatility is something that I very much do enjoy. Who knew that actually making apps with Electron (even across multiple platforms) would be a downright pleasant experience?\u003c/p\u003e\n\u003cp\u003eI feel like if we want to go back to most software being native, the native platforms (or some cross platform solution that transpiles down to native components, a bit like the \u003ca href=\"https://links.kronis.dev/bvfol\"\u003eLazarus Component Library\u003c/a\u003e) need to have the developer experience be equally easy or easier. That's more or less also why Docker became so popular vs something like FreeBSD jails or other methods of containerization. If you want people to use your technology, make it easy and pleasant to use it (or hold them hostage, like Jira does with the industry).\u003c/p\u003e\n\u003cp\u003eOf course, what I did is a workaround to the desktop app refusing to work, but there were still other issues that remained.\u003c/p\u003e\n\u003ch3\u003eThere are still issues\u003c/h3\u003e\n\u003cp\u003eFor starters, it seems like Electron is missing a few useful bits, once again for no good reason.\u003c/p\u003e\n\u003cp\u003eFor example, when I attempt to share my screen, there should be a window/screen picker that shows up and lets me choose, what exactly I want to share. Electron has APIs for this, but \u003ca href=\"https://links.kronis.dev/8qeff\"\u003enot the actual visual component\u003c/a\u003e which just seems weird. Why would you ship a half baked solution instead of some sensible default that could be replaced with something else for the more custom use cases? Lots of other people have also \u003ca href=\"https://links.kronis.dev/2th0x\"\u003erun into similar issues\u003c/a\u003e because of this deficiency.\u003c/p\u003e\n\u003cp\u003eEssentially, what I had to do in that very same evening, was to hack together a custom window picker:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"10-custom-window-picker.jpg\" alt=\"10-custom-window-picker\" title=\"10-custom-window-picker\"\u003e\u003c/p\u003e\n\u003cp\u003eI didn't want to overcomplicate the code and I didn't care about more specific use cases, like sharing specific windows instead of entire screens, because quite frankly I couldn't be bothered, but at the very least it was doable in the end, even though I really shouldn't have to do something like that in the first place.\u003c/p\u003e\n\u003cp\u003eHere's the corresponding JavaScript that mostly works:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003e// for screen sharing, allow the user to select their screen\nsession.setDisplayMediaRequestHandler((request, callback) =\u0026gt; {\n    // so we only do the callback once\n    let screenSelectionDone = false;\n    try {\n        desktopCapturer.getSources({ types: ['screen'] }).then((sources) =\u0026gt; {\n            try {\n                // create a selection window for the user to choose a screen\n                const selectionWindow = new BrowserWindow({\n                    width: 1024,\n                    height: 576,\n                    webPreferences: {\n                        nodeIntegration: true,\n                        contextIsolation: false,\n                    },\n                    icon: path.join(__dirname, 'images/logo-128.png'),\n                    modal: true,\n                    parent: BrowserWindow.getFocusedWindow()\n                });\n\n                // load a custom HTML page for screen selection\n                selectionWindow.loadFile(path.join(__dirname, 'screen-selection.html'));\n\n                // pass the sources to the selection window\n                selectionWindow.webContents.once('did-finish-load', () =\u0026gt; {\n                    selectionWindow.webContents.send('screen-sources', sources);\n                });\n\n                // handle the case where the user closes the window without selecting a screen\n                selectionWindow.on('close', () =\u0026gt; {\n                    if (!screenSelectionDone) {\n                        console.log('Selection window closed without selecting a screen.');\n                        new Notification({ title: 'Screen sharing', body: 'No screen selected (closed selection window).' });\n                        screenSelectionDone = true;\n                        callback(null); // no screen selected\n                    }\n                });\n\n                // listen for the user's selection\n                const { ipcMain } = require('electron');\n                ipcMain.once('screen-selected', (event, sourceId) =\u0026gt; {\n                    const selectedSource = sources.find((source) =\u0026gt; source.id === sourceId);\n                    if (selectedSource) {\n                        if (!screenSelectionDone) {\n                            console.log('Selected source: ', selectedSource.name);\n                            new Notification({ title: 'Screen sharing', body: 'Sharing screen: ' + selectedSource.name }).show();\n                            screenSelectionDone = true;\n                            callback({ video: selectedSource });\n                        }\n                    } else {\n                        if (!screenSelectionDone) {\n                            console.log('No screen selected.');\n                            new Notification({ title: 'Screen sharing', body: 'No screen selected.' });\n                            screenSelectionDone = true;\n                            callback(null); // no screen selected\n                        }\n                    }\n                    selectionWindow.close();\n                });\n            } catch (error) {\n                new Notification({ title: 'Error!', body: 'Error in getSources: ' + error }).show();\n                console.error('Error in getSources:', error);\n                if (!screenSelectionDone) {\n                    callback(null);\n                }\n            }\n        });\n    } catch (error) {\n        new Notification({ title: 'Error!', body: 'Error in setDisplayMediaRequestHandler: ' + error }).show();\n        console.error('Error in setDisplayMediaRequestHandler:', error);\n        if (!screenSelectionDone) {\n            callback(null);\n        }\n    }\n});\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eHere's how the HTML for the window looks like:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003e\u0026lt;!DOCTYPE html\u0026gt;\n\u0026lt;html lang=\u0026quot;en\u0026quot;\u0026gt;\n\u0026lt;head\u0026gt;\n    \u0026lt;meta charset=\u0026quot;UTF-8\u0026quot;\u0026gt;\n    \u0026lt;meta name=\u0026quot;viewport\u0026quot; content=\u0026quot;width=device-width, initial-scale=1.0\u0026quot;\u0026gt;\n    \u0026lt;title\u0026gt;Select a Screen\u0026lt;/title\u0026gt;\n    \u0026lt;style\u0026gt;\n        body {\n            font-family: Arial, sans-serif;\n            margin: 0;\n            padding: 0;\n            display: flex;\n            flex-wrap: wrap;\n            justify-content: center;\n            align-items: center;\n            background-color: #202020;\n            color: #d8d8d8;\n        }\n        .screen {\n            margin: 10px;\n            text-align: center;\n        }\n        img {\n            width: 320px;\n            height: 180px;\n            cursor: pointer;\n            border: 2px solid transparent;\n        }\n        img:hover {\n            border: 2px solid blue;\n        }\n    \u0026lt;/style\u0026gt;\n\u0026lt;/head\u0026gt;\n\u0026lt;body\u0026gt;\n    \u0026lt;script\u0026gt;\n        const { ipcRenderer } = require('electron');\n\n        ipcRenderer.on('screen-sources', (event, sources) =\u0026gt; {\n            const container = document.body;\n            sources.forEach((source) =\u0026gt; {\n                const div = document.createElement('div');\n                div.className = 'screen';\n\n                const img = document.createElement('img');\n                img.src = source.thumbnail.toDataURL();\n                img.alt = source.name;\n                img.onclick = () =\u0026gt; {\n                    ipcRenderer.send('screen-selected', source.id);\n                };\n\n                const label = document.createElement('p');\n                label.textContent = source.name;\n\n                div.appendChild(img);\n                div.appendChild(label);\n                container.appendChild(div);\n            });\n        });\n    \u0026lt;/script\u0026gt;\n\u0026lt;/body\u0026gt;\n\u0026lt;/html\u0026gt;\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eThe rest of the problems are with Teams itself.\u003c/p\u003e\n\u003cp\u003eFor example, if I create a few accounts and attempt to message myself, sometimes the messages won't arrive (nor will the typing notification show up) until I refresh the page:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"11-messages-being-dropped.jpg\" alt=\"11-messages-being-dropped\" title=\"11-messages-being-dropped\"\u003e\u003c/p\u003e\n\u003cp\u003eNow, initially I thought that it was an issue with the Electron wrapper after doing some further refactoring, since the screen sharing also didn't seem to work.\u003c/p\u003e\n\u003cp\u003eBut no, it was Teams just being plainly broken. You see, I tried using the web based version directly in Edge and it had this exact same issue: screen sharing wouldn't work at all and messages would just be randomly dropped. Then, 30 minutes later, it suddenly started working again in both. That is something that I've never had issues in with neither Skype nor Slack, they just worked.\u003c/p\u003e\n\u003cp\u003eHere's what I got from the logs in one of those cases:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003eERROR:sdp_offer_answer.cc(424)] A BUNDLE group contains a codec collision for payload_type='124. All codecs must share the same type, encoding name, clock rate and parameters. (INVALID_PARAMETER)\nERROR:sdp_offer_answer.cc(3669)] The order of m-lines in subsequent offer doesn't match order from previous offer/answer. (INVALID_PARAMETER)\nERROR:sdp_offer_answer.cc(955)] Failed to set remote offer sdp: The order of m-lines in subsequent offer doesn't match order from previous offer/answer.\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eThat's some great software right there.\u003c/p\u003e\n\u003ch3\u003eThere are not that many alternatives\u003c/h3\u003e\n\u003cp\u003eAt the end of the day, you might want to throw Teams away in favor of something else.\u003c/p\u003e\n\u003cp\u003eBut here's the thing: there are almost no other free options. For example, I rather like \u003ca href=\"https://links.kronis.dev/4myzf\"\u003eMattermost\u003c/a\u003e because you can self-host it, it has mobile and desktop apps, you can write bots (such as notifications for application uptime), there's even plugins for calls and whatnot, as well as AD/LDAP integration. Except no, not really, unless you \u003ca href=\"https://links.kronis.dev/6x9xh\"\u003epay per seat\u003c/a\u003e to actually unlock the functionality:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"12-mattermost-pricing.jpg\" alt=\"12-mattermost-pricing\" title=\"12-mattermost-pricing\"\u003e\u003c/p\u003e\n\u003cp\u003eThe software is so good that I could easily imagine replacing Skype for meetings and Slack for general collaboration with it (yaay, free software), even if that meant that I had to convince people at work to open the application to public access (unless people with mobile apps want to connect to the VPN), except the pricing once more gets in the way. I guess the more reasonable thing is to just give up and open our wallets and pay either for the business version of Teams, or just pay for Slack.\u003c/p\u003e\n\u003cp\u003eWhen it comes to other free options, I also looked into the likes of \u003ca href=\"https://links.kronis.dev/8hiha\"\u003eZulip\u003c/a\u003e which seems to be fully self-hostable, but you need to pay either for support or if you want \u003ca href=\"https://links.kronis.dev/m5sml\"\u003emobile notifications\u003c/a\u003e, because the way how support for those is coded into mobile OSes is just plain stupid (needing to use APN and FCM) and feels like yet another walled garden:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eZulip’s iOS and Android mobile apps support receiving push notifications from Zulip servers to notify users when new messages have arrived. This is an important feature for having a great mobile app experience.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eGoogle’s and Apple’s security model for mobile push notifications does not allow self-hosted Zulip servers to directly send mobile notifications to the Zulip mobile apps. The Zulip Mobile Push Notification Service solves this problem by forwarding mobile push notifications generated by your server to the Zulip mobile apps.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eFurthermore, Zulip doesn't do \u003ca href=\"https://links.kronis.dev/1r5e5\"\u003evideo calls\u003c/a\u003e out of the box either and for that something like self-hosting \u003ca href=\"https://links.kronis.dev/2bfup\"\u003eJitsi\u003c/a\u003e would be needed, which is also \u003ca href=\"https://links.kronis.dev/a84do\"\u003ea bit of a mess\u003c/a\u003e, despite being lovely software otherwise.\u003c/p\u003e\n\u003ch3\u003eSummary\u003c/h3\u003e\n\u003cp\u003eAs it stands, Skype is being killed off and an alternative is needed.\u003c/p\u003e\n\u003cp\u003eTeams kind of sucks unless you pay for it, then it sucks a bit less. Most decent software out there is paid, even Slack. The free options like WhatsApp or Telegram or whatever don't have support for workspaces and channels, like those others do. Even self-hostable software like Mattermost (ignoring the work needed to mitigate security risks) expect you to open your wallet for any remotely enterprise-like feature and other options like Zulip aren't actually free because you can't have proper mobile notifications and you'd also need to configure and host your own call solution etc.\u003c/p\u003e\n\u003cp\u003eHonestly, the software landscape when it comes to free apps for collaboration sucks when you're a freeloader. Maybe there's some Matrix based option out there, but I haven't looked into those yet. What I'd ideally want is something a bit like what \u003ca href=\"https://links.kronis.dev/trqz5\"\u003ePeerTube\u003c/a\u003e has done for hosting videos: a single application that you can deploy for smaller scale teams that does all of the essentials out of the box. No more and no less.\u003c/p\u003e\n\u003cp\u003eIf you're curious, here's my checklist, which is obviously a lot of development work for anyone to make:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eworkspaces, channels and threads, as well as DMs\u003c/li\u003e\n\u003cli\u003evoice and video calls, both 1:1 and group calls, screen sharing support\u003c/li\u003e\n\u003cli\u003esharing of media and links, inline preview of GIFs and videos\u003c/li\u003e\n\u003cli\u003ebot account support and an API for any degree of automation\u003c/li\u003e\n\u003cli\u003einvite links and AD/LDAP integration for those who need it (or honestly OIDC/OAuth2 support)\u003c/li\u003e\n\u003cli\u003edesktop app, web based app and mobile apps (with push notifications)\u003c/li\u003e\n\u003cli\u003edeployable as a single binary or single container, with a single data store (e.g. PostgreSQL/MariaDB/SQLite)\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eI guess I might keep waiting for a while, clearly the solutions that went anywhere needed to find a way to monetize their labor.\u003c/p\u003e\n\u003cp\u003eIn the mean time, software complexity will eat the world.\u003c/p\u003e\n"
        },
        {
            "title": "The SSH tunnel is dead, long live Tailscale",
            "date_published": "2025-03-03",
            "id": "https://blog.kronis.dev/blog/the-ssh-tunnel-is-dead-long-live-tailscale",
            "url": "https://blog.kronis.dev/blog/the-ssh-tunnel-is-dead-long-live-tailscale",
            "content_html": "\u003cp\u003eI ended up reworking how my hybrid cloud does networking.\u003c/p\u003e\n\u003cp\u003eIn the \u003ca href=\"https://links.kronis.dev/2ifwz\"\u003eprevious post\u003c/a\u003e, it became apparent that my web servers should work as a proper point of ingress: the kind that sees the IP addresses of incoming traffic, so it's easier for me to drop anything I don't like. When it comes to achieving that in a simplistic way, running the web servers directly on the nodes that receive the traffic makes more sense, than trying to use SSH tunnels or having me mess around with \u003ccode\u003eiptables\u003c/code\u003e rules.\u003c/p\u003e\n\u003cp\u003eFurthermore, if I don't want to write a bunch of manual rules of what traffic should be forwarded where or mess with IP addresses directly in my web server configuration, I can just use my \u003ca href=\"https://links.kronis.dev/pahgc\"\u003eDocker Swarm\u003c/a\u003e cluster for that. I can use it to tell the web server that's running on the public VPS that it needs to act as a reverse proxy for a container that's running on my homelab, behind NAT. While I could previously forward traffic to ports 80 and 443 over an SSH tunnel, the limitations of that quickly became apparent.\u003c/p\u003e\n\u003cp\u003eRemember this image from the previous post?\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-the-architecture.jpg\" alt=\"01-the-architecture\" title=\"01-the-architecture\"\u003e\u003c/p\u003e\n\u003cp\u003eTrying to use Docker overlay networks doesn't work when you can't make the nodes communicate with one another over UDP ports as well, not just TCP. That could be achievable with SSH tunnels, but would involve some unpleasant hacks. In addition, you need a bunch of ports, so the configuration for that quickly becomes unwieldy.\u003c/p\u003e\n\u003cp\u003eNot only that, but with how Docker Swarm works, it expects an IP address and port combination for each node, instead of just random ports exposed on \u003ccode\u003e127.0.0.1\u003c/code\u003e. In other words, I needed a proper networking setup to allow my servers to talk with one another, otherwise the containers are somewhat useless:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02-all-containers-up.jpg\" alt=\"02-all-containers-up\" title=\"02-all-containers-up\"\u003e\u003c/p\u003e\n\u003cp\u003eAt this point, it's probably useful to point out that some of the details in the post will be a bit blurry, because I finished doing all of this around 3 AM and even now, as I write this, the sleep deprivation still hasn't quite worn off. The joys of trying to improve things and running into more and more and more issues, I guess.\u003c/p\u003e\n\u003cp\u003eRegardless, if I didn't want to stay up until 5 AM instead of 3 AM, there was one solution that made everything easier!\u003c/p\u003e\n\u003ch3\u003eTailscale\u003c/h3\u003e\n\u003cp\u003eEnter \u003ca href=\"https://links.kronis.dev/spfn3\"\u003eTailscale\u003c/a\u003e, a piece of software that promises to make establishing private networks easy and has client software for Linux, Windows, Mac and a bunch of other platforms. There's even a Docker container, but it did seem a tad unwieldy. I hadn't actually needed to use Tailscale before, because for the most part \u003ca href=\"https://links.kronis.dev/0ni1e\"\u003eWireGuard\u003c/a\u003e had worked for getting around NAT as well, when I had just a few nodes.\u003c/p\u003e\n\u003cp\u003eYet now, I wanted a virtual network where every single node could reach out to every other one. Not just for containers, but also things like monitoring with Zabbix or some backup software, to make things just a little bit more safe, in addition to any other auth mechanisms that the software has.\u003c/p\u003e\n\u003cp\u003eWith WireGuard, I'd need to manage the private and public keys for each of the servers, for example, servers 1-6 would need to know the public key of server 7. Same for servers 1-5 and 7 for server 6. You can imagine how the configuration would become a little bit unwieldy. Tailscale solves all of that for me. I just install their software, join the nodes to a network and don't have to worry too much about it:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03-tailscale-is-nice.jpg\" alt=\"03-tailscale-is-nice\" title=\"03-tailscale-is-nice\"\u003e\u003c/p\u003e\n\u003cp\u003e(notice how I try to blur some information here, sometimes I'll get a bit overzealous with that, but that's because right now everything is a blur; also, to any of the state actors or motivated people that actually have the motivation to recover the information from some pixelation and knowing what the font in question is, please don't steal my very valuable meme stash on the servers, adding black bars would break up the text too much)\u003c/p\u003e\n\u003cp\u003eTailscale also has \u003ca href=\"https://links.kronis.dev/vixmq\"\u003epretty decent pricing\u003c/a\u003e, where the free tier, at the time of writing, gives you the ability to connect up to a 100 devices. I'm actually surprised that they're not a bigger corporation, because their offering is simple to use and very useful. Not a paid ad or anything, I don't really do those, I just like what they have. Of course, it's a bit naughty to give data about my internal network to some corporation, but then again, if they ever became truly evil, I could just drop them and do the grunt work to switch to WireGuard directly.\u003c/p\u003e\n\u003cp\u003eBut alas, every node is now connected, there is transparent \u003ca href=\"https://links.kronis.dev/oq6ne\"\u003eNAT traversal\u003c/a\u003e and life is good. All that's left is to make the Swarm cluster use it.\u003c/p\u003e\n\u003cp\u003eFirst, I make the worker nodes leave the Swarm, with:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003edocker swarm leave\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003ethen followed by the manager node. This does bring everything down, but thankfully I don't care that much about the downtime in my case.\u003c/p\u003e\n\u003cp\u003eI then re-create the cluster, specifying the virtual network addresses to use, for example:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003edocker swarm init --advertise-addr \u0026lt;manager-tailnet-ip\u0026gt; --listen-addr \u0026lt;manager-tailnet-ip\u0026gt; --task-history-limit=3\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eand then join the nodes to the cluster, like:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003edocker swarm join --advertise-addr \u0026lt;worker-tailnet-ip\u0026gt; --listen-addr \u0026lt;worker-tailnet-ip\u0026gt; --token \u0026lt;swarm-worker-join-token\u0026gt; \u0026lt;manager-tailnet-ip\u0026gt;:2377\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eThe added bonus here is that all Swarm traffic will be routed through this network, meaning that it's also a little bit more secure. After a few minutes of work, the cluster seems to be working okay. I can then redeploy my stacks and it all should just work, right?\u003c/p\u003e\n\u003ch3\u003eWe can't have nice things\u003c/h3\u003e\n\u003cp\u003eNo, not really. Initially, things seem okay. But as I redeploy a bunch of web servers and a bunch of containers, suddenly I'm getting networking issues.\u003c/p\u003e\n\u003cp\u003eFor example, there are errors along the lines of:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003e[proxy_http:error] ... (20014)Internal error (specific information not available): [remote ...] AH01102: error reading status line from remote server ...\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003ein the web server logs when I do requests that cross server boundaries:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03-error-over-tailscale.jpg\" alt=\"03-error-over-tailscale\" title=\"03-error-over-tailscale\"\u003e\u003c/p\u003e\n\u003cp\u003eand\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04-error-over-tailscale-2.jpg\" alt=\"04-error-over-tailscale-2\" title=\"04-error-over-tailscale-2\"\u003e\u003c/p\u003e\n\u003cp\u003eAt first, I thought that maybe it's due to issues with the web servers themselves or the containers running behind them, but it got even weirder.\u003c/p\u003e\n\u003cp\u003eFor example, Mattermost in the native client doesn't work:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"05-mattermost-wont-load.jpg\" alt=\"05-mattermost-wont-load\" title=\"05-mattermost-wont-load\"\u003e\u003c/p\u003e\n\u003cp\u003eMattermost when opened in Chromium based browsers like Edge doesn't work.\u003c/p\u003e\n\u003cp\u003eMattermost when opened in Firefox seems to kind of work, some files loading but not others.\u003c/p\u003e\n\u003cp\u003e\u003ccode\u003ecurl\u003c/code\u003e seems to always return correct responses, at least for the files that I checked.\u003c/p\u003e\n\u003cp\u003eThere's also nothing wrong with the TLS certificates either:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"06-direct-connection-okay.jpg\" alt=\"06-direct-connection-okay\" title=\"06-direct-connection-okay\"\u003e\u003c/p\u003e\n\u003cp\u003eIt's just your average flaky connection situation, except that it doesn't work like 90% of the time and works 10% of the time. Highly confusing and hard to track down. At this point I was wondering about whether this was related to the amount of traffic going through the network and whether the issue was on Tailscale's end or my end, since there's no way their service would be that bad.\u003c/p\u003e\n\u003cp\u003eMy suspicions were somewhat confirmed, because initially Portainer had worked pretty fast, but now I couldn't even load the list of deployed stacks:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"07-portainer-messed-up.jpg\" alt=\"07-portainer-messed-up\" title=\"07-portainer-messed-up\"\u003e\u003c/p\u003e\n\u003cp\u003eIt's even weirder, though. Sometimes I could deploy new stacks, but trying to get their status immediately afterwards would fail, leading to things like this in the web interface:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"08-what.jpg\" alt=\"08-what\" title=\"08-what\"\u003e\u003c/p\u003e\n\u003cp\u003eThe error itself would be pretty consistent:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003eUnable to find an agent on any manager node\nThe agent was unable to contact any other agent located on a manager node\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003ewhich I could see it both in the browser:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"09-networking-issues.jpg\" alt=\"09-networking-issues\" title=\"09-networking-issues\"\u003e\u003c/p\u003e\n\u003cp\u003eand in the logs for Portainer itself:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"10-workers-cant-access-server.jpg\" alt=\"10-workers-cant-access-server\" title=\"10-workers-cant-access-server\"\u003e\u003c/p\u003e\n\u003cp\u003eThis definitely means that something was broken with how things are networked at a lower level, because to even get that error in the browser, the web server needs to work correctly, at least for some requests, whereas if I see that message in the Portainer logs, then the issue is present with traffic that doesn't go through the web server.\u003c/p\u003e\n\u003cp\u003eIt could have still been something else, like a Portainer misconfiguration and someone actually had \u003ca href=\"https://links.kronis.dev/2jzr7\"\u003ea temporary workaround\u003c/a\u003e for something like that:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"11-temporary-fix.jpg\" alt=\"11-temporary-fix\" title=\"11-temporary-fix\"\u003e\u003c/p\u003e\n\u003cp\u003eSadly, that's not an actual fix and \u003ca href=\"https://links.kronis.dev/2bg97\"\u003epeople complained\u003c/a\u003e that such issues have been around for a long time:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"12-lots-of-issues.jpg\" alt=\"12-lots-of-issues\" title=\"12-lots-of-issues\"\u003e\u003c/p\u003e\n\u003cp\u003eUnfortunately, that didn't help me. I wrote a script to do the workaround more easily, even do full redeploys of the Portainer stack. It would run successfully, all of the nodes would still show up as reachable:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"13-but-nodes-are-available.jpg\" alt=\"13-but-nodes-are-available\" title=\"13-but-nodes-are-available\"\u003e\u003c/p\u003e\n\u003cp\u003eAfter that, things would be great for about a minute and the network condition would then deteriorate severely once more. This lead me to believe that the network itself is definitely to blame, especially when its hit by more traffic. This would explain why everything worked with a bare cluster, but once I'd have ~50 containers running across the nodes, that it'd get much worse.\u003c/p\u003e\n\u003ch3\u003eThe actual issue\u003c/h3\u003e\n\u003cp\u003eI stumbled upon someone talking about how the \u003ca href=\"https://links.kronis.dev/pfjo1\"\u003emaximum transmission unit (MTU)\u003c/a\u003e configuration had been a problem for them, specifically when trying to run Tailscale and that lowering that value resolved their issues.\u003c/p\u003e\n\u003cp\u003eAgain, I'm not a networking engineer, so all I can do is spitball along the lines of the network doing the equivalent of trying to put large luggage cases into containers that doesn't quite fit them and when they're sometimes fully packed, that causes them to fall off and make a mess. The linked Wikipedia page is actually both highly technical and quite vague, but you get the idea:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"14-info-about-mtu.jpg\" alt=\"14-info-about-mtu\" title=\"14-info-about-mtu\"\u003e\u003c/p\u003e\n\u003cp\u003eUpon digging a bit deeper, even \u003ca href=\"https://links.kronis.dev/dxdlc\"\u003ePortainer has similar suggestions\u003c/a\u003e, so it would appear that I wasn't the only person who has run into such issues:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"15-suggested-solution.jpg\" alt=\"15-suggested-solution\" title=\"15-suggested-solution\"\u003e\u003c/p\u003e\n\u003cp\u003eThe command to actually check the MTU sizes for each interface is pretty simple, though not the one that you'll commonly see mentioned:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003eifconfig -s\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eThis shows the problematic 1500 value, whereas 1280 was suggested, \u003ca href=\"https://links.kronis.dev/cdmjt\"\u003ethe same MTU that Tailscale uses\u003c/a\u003e for their networking:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"16-mtu-might-be-the-issue.jpg\" alt=\"16-mtu-might-be-the-issue\" title=\"16-mtu-might-be-the-issue\"\u003e\u003c/p\u003e\n\u003cp\u003eThere's a bit of a problem with all of that, though. You see, if I want to change that value, then according to the \u003ca href=\"https://links.kronis.dev/4fe0v\"\u003eDocker documentation\u003c/a\u003e I need to:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003edestroy the whole Docker Swarm (including all of the deployed stacks, just like when initially leaving it)\u003c/li\u003e\n\u003cli\u003einstall the additional \u003ccode\u003ebridge-utils\u003c/code\u003e package\u003c/li\u003e\n\u003cli\u003estop Docker, delete the resource and recreate it\u003c/li\u003e\n\u003cli\u003ethen start Docker, re-create the whole cluster, re-deploy the stacks and hope it all works\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThose instructions kind of suck! If it's needed for technical reasons, I guess I have no other options, but even if the tasks are simple, they're still quite time consuming and having an iteration of experimenting with a new configuration taking 15 minutes is never good.\u003c/p\u003e\n\u003cp\u003eHave a look for yourself:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"17-the-instructions-suck.jpg\" alt=\"17-the-instructions-suck\" title=\"17-the-instructions-suck\"\u003e\u003c/p\u003e\n\u003cp\u003eWhat's worse, the instructions don't actually work, I just get an error along the lines of:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003ebridge docker_gwbridge is still up; can't delete it\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eeven when Docker is stopped, which is quite unfortunate:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"18-bridge-still-up.jpg\" alt=\"18-bridge-still-up\" title=\"18-bridge-still-up\"\u003e\u003c/p\u003e\n\u003cp\u003eAt least I knew approximately what I'd need to do and thankfully I was on the precipice of finding the solution I needed.\u003c/p\u003e\n\u003ch3\u003eThe light at the end of the tailnet\u003c/h3\u003e\n\u003cp\u003eI ended up doing a few more things along the way, such as putting the following in \u003ccode\u003e/etc/docker/daemon.json\u003c/code\u003e:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003e{\n  \u0026quot;mtu\u0026quot;: 1280,\n  \u0026quot;ipv6\u0026quot;: false,\n  \u0026quot;experimental\u0026quot;: true\n}\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eThe instructions for handling \u003ccode\u003edocker_gwbridge\u003c/code\u003e ended up looking like this for me:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003e# Check the status\nifconfig -s\nbrctl show docker_gwbridge\n# Leave the swarm (apparently didn't even need to shut off Docker)\ndocker swarm leave\n# Remove the interface and then re-create it, with the default configuration BUT with the smaller MTU of 1280\ndocker network rm docker_gwbridge\ndocker network create --subnet 172.18.0.0/16 --opt com.docker.network.bridge.name=docker_gwbridge --opt com.docker.network.bridge.enable_icc=false --opt com.docker.network.bridge.enable_ip_masquerade=true docker_gwbridge --opt com.docker.network.driver.mtu=1280\n# Restart Docker anyways, just in case\nsudo service docker restart\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eI have to say that at least the options you can use for the networks are \u003ca href=\"https://links.kronis.dev/f12b7\"\u003edocumented pretty well\u003c/a\u003e, which gave me at least a bit more confidence. Finally, it seemed that the MTU values for both the Docker and the bridge interfaces matched that of the Tailscale one:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"19-finally-fixed.jpg\" alt=\"19-finally-fixed\" title=\"19-finally-fixed\"\u003e\u003c/p\u003e\n\u003cp\u003eThe other command that I mentioned previously also seemed to confirm this:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"20-eureka.jpg\" alt=\"20-eureka\" title=\"20-eureka\"\u003e\u003c/p\u003e\n\u003cp\u003eAnd just like that, Portainer started working nicely again, there were no more weird traffic issues, the web servers also properly did their job and I can finally forward traffic from the cloud VPSes to containers running on my homelab, without any custom \u003ccode\u003eiptables\u003c/code\u003e rules, nor do I need to manage a WireGuard install manually at the present time.\u003c/p\u003e\n\u003cp\u003eIt just works:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"21-finally-working.jpg\" alt=\"21-finally-working\" title=\"21-finally-working\"\u003e\u003c/p\u003e\n\u003cp\u003eFurthermore, the aforementioned benefits of having a virtual network that's private for the most part are also apparent. For example, I can now run Zabbix traffic over it and don't have to risk a misconfiguration exposing too much information to the world, because none of this traffic is publicly routed either. I don't have to worry about NAT. I can also use this same setup for some backup software as well:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"22-zabbix-also-works.jpg\" alt=\"22-zabbix-also-works\" title=\"22-zabbix-also-works\"\u003e\u003c/p\u003e\n\u003cp\u003eI will still keep using mTLS and a custom CA for some of the sites that I'd prefer to keep more private and sit behind my reverse proxy, but it's yet another useful layer in my security model, which already looks like Swiss cheese, so it's nice to at least pretend that any of this is remotely secure:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"23-swiss-cheems.jpg\" alt=\"23-swiss-cheems\" title=\"23-swiss-cheems\"\u003e\u003c/p\u003e\n\u003cp\u003eNotice how I keep layering on technological solutions, even though I'll probably accidentally push an SSH key somewhere where it shouldn't be some day, the kind that doesn't have a passphrase protecting it. Oh, also, the image is more or less borrowed from \u003ca href=\"https://links.kronis.dev/ltini\"\u003ethis blog post\u003c/a\u003e, which talks about cybersecurity a little bit more.\u003c/p\u003e\n\u003ch3\u003eSummary\u003c/h3\u003e\n\u003cp\u003eI do have a lot of appreciation for the likes of WireGuard and Tailscale, even SSH tunnels are useful for specific use cases. Networking still sucks, that's the opinion that hasn't changed though.\u003c/p\u003e\n\u003cp\u003eFor example:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003ewhy doesn't the MTU drop to 1280 automatically if that's all that's supported over the Tailscale interface, isn't that the whole point of exposing that value?\u003c/li\u003e\n\u003cli\u003ewhy are there no better mechanisms for detecting MTU related issues, instead of my web server just throwing \u003ccode\u003eInternal error (specific information not available)\u003c/code\u003e\u003c/li\u003e\n\u003cli\u003ewhy do the official instructions say that I have to wipe my entire container cluster, just to change one networking value? You're all about automation, you do it.\u003c/li\u003e\n\u003cli\u003ewhy does Tailscale support 1280 in particular, instead of the more standard 1500? If it's so messy, why doesn't Linux also just opt for 1280 out of the box?\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eTo me, it feels like there's something missing. There's tooling for working with most of this stuff, but not enough automation to avoid making us do the work. For example, Docker Swarm or Tailscale or whatever just doing a quick connection test, figuring out what MTUs are supported and at what values things start failing, output the results in the logs and do some auto configuration for me. Fin.\u003c/p\u003e\n\u003cp\u003eThat said, now my web servers see the proper IP addresses and it's become easier to ban whoever tries to spam me with requests for that dead domain from the previous post:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003e#!/bin/bash\nINPUT_FILE=\u0026quot;blocklist.txt\u0026quot;\n\nwhile IFS= read -r ip; do\n    echo \u0026quot;Blocking IP: $ip\u0026quot;\n    iptables -A INPUT -p tcp --dport 80 -s \u0026quot;$ip\u0026quot; -j DROP\ndone \u0026lt; \u0026quot;$INPUT_FILE\u0026quot;\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eOr maybe I should just put fail2ban inside of my Apache2 container and setup some additional rules, go figure.\u003c/p\u003e\n"
        },
        {
            "title": "Zombie software and haunted tunnels",
            "date_published": "2025-03-02",
            "id": "https://blog.kronis.dev/blog/zombie-software-and-haunted-tunnels",
            "url": "https://blog.kronis.dev/blog/zombie-software-and-haunted-tunnels",
            "content_html": "\u003cp\u003eSo here's a fun one, and by fun, I mean absolutely frustrating. To start off, there is very little support for IPv6 in my country, though it seems to be the case for a lot of the world, meaning that I can't really use it at all:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"00-only-ipv4.jpg\" alt=\"00-only-ipv4\" title=\"00-only-ipv4\"\u003e\u003c/p\u003e\n\u003cp\u003eIPv6 promised to be a great technology, that would solve the shortcomings of IPv4 and would simplify networking, compared to what it had become: a bunch of workarounds for the limited amount of addresses available. IPv6 seemed to be a pretty sound design and made a bunch of promises... which, judging by the above, are nowhere to actually be seen.\u003c/p\u003e\n\u003cp\u003eThat leaves us with IPv4. Because there's not enough addresses, it's not like anyone will give every device its own publicly accessible (infrastructure in the middle willing) address, so having a bunch of servers in my room isn't easily doable. I remember that it wasn't the case in the university that I went to, where I could connect a few boxes running Debian to the network and shortly after get addresses for them that I could reach from anywhere. That was a pretty nice setup, while it lasted!\u003c/p\u003e\n\u003cp\u003eNow, before I continue, let me make something clear: I suck at networking. With every passing year, I suck at it a bit less and perhaps some day I'll consider myself okay at it. Yet, as someone whose day job is primarily writing software and doing your run of the mill web development, my comfort zone currently only reaches so far. Configuring TLS? Sure. Configuring a load balancer? Okay. Making sure that the firewall plays nicely with things. No problem. Troubleshooting simple network related issues? Doable. Having a tunnel here or there? Fine. Split VPN or split DNS? Sure thing.\u003c/p\u003e\n\u003cp\u003eYet, the lower the level I need to look at, the murkier things get and often the entrenched technologies feel like design that isn't good but also not so bad that we'd be forced to move on to something new, so things just trudge along. In my eyes, in the perfect world, every piece of hardware should be uniquely addressable. For a second pretend that the OSI model doesn't exist and that every NIC has a singular address, like we already have MAC addresses. Essentially a UUID that you can use as a publicly routable address:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"00-simple-addressing.jpg\" alt=\"00-simple-addressing\" title=\"00-simple-addressing\"\u003e\u003c/p\u003e\n\u003cp\u003eNow, whether your PC, laptop, phone, fridge, tea kettle or light bulb would pick up when something is trying to reach out to it, that's a different question. Whether they even need to be connected to any kind of a network, that's also not something that I'll get into here. But if the world was around such a concept, things could be really simple. Services like VPNs surely would exist to tunnel your traffic through for privacy reasons, but if you'd want to reach out to your homelab server or your public cloud server, that'd all be one address and some network hops away. I wouldn't hate networking then.\u003c/p\u003e\n\u003cp\u003eBut that's not the world that we live in.\u003c/p\u003e\n\u003ch3\u003eHaunted tunnels\u003c/h3\u003e\n\u003cp\u003eIn my case, my personal devices sit behind NAT. Even in the best case where my router is reachable from the outside, that still gives me one IP address that I can't exactly split between two of my homelab servers (e.g. two separate web servers) if I want to use standard ports and don't want any trickery.\u003c/p\u003e\n\u003cp\u003eI could probably pay for an additional static IP address, but that's outside of the scope here, because the companies love to charge you a bunch of money for that. Come to think of it, I can't even find any pricing of static IP addresses on their homepage, but I know that the previous ISP wanted 6 EUR per month per address, I can get a whole VPS for that amount, with and address instead.\u003c/p\u003e\n\u003cp\u003eTo cope with that, previously, I made my homelab accessible through a \u003ca href=\"https://links.kronis.dev/l5hnc\"\u003eWireGuard network\u003c/a\u003e, where I forwarded the traffic through a publicly accessible VPS. So, even if my homelab can't be reached directly, it can reach out to the cloud server instead and the cloud server can forward the traffic to it. Plus, if I ever move or change ISPs, I just need any sort of an Internet connection and the homelab servers can still connect to the same network with the same IP addresses that are configured, so it's a pretty stable setup.\u003c/p\u003e\n\u003cp\u003eThe actual configuration is also pretty simple, even if the blog post is a bit dated at this point. The problems started with actually routing the traffic, since I needed to mess around with \u003ccode\u003eiptables\u003c/code\u003e, which I really dislike. It's essentially software that's not very approachable and gives you ample opportunities to mess up. It's actually surprising that I haven't locked myself out of any servers while poking around in the dark. Either way, the setup was a bit of an interplay between WireGuard and some additional routing. It worked, but was perhaps a bit too complex.\u003c/p\u003e\n\u003cp\u003eSo, wanting to keep my setup simple, I looked in another direction: SSH tunnels.\u003c/p\u003e\n\u003cp\u003eSince I already run Linux distros and use SSH, it sounded pretty much perfect! I could use the software I already have to expose specific ports from my local software (e.g. Apache2) to be available on the remote server, with no additional software. Just a bit of configuration, along the lines of:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003eAllowTcpForwarding yes\nGatewayPorts yes\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003ein \u003ccode\u003e/etc/ssh/sshd_config\u003c/code\u003e and then either:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003eHost my-server.com\n    HostName my-server.com\n    User myuser\n    # HTTP\n    RemoteForward 80 127.0.0.1:80\n    # HTTPS\n    RemoteForward 443 127.0.0.1:443\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003ein \u003ccode\u003e~/.ssh/config\u003c/code\u003e or just using the following syntax directly:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003essh -i ~/.ssh/whatever -R 0.0.0.0:80:127.0.0.1:80 -R 0.0.0.0:443:127.0.0.1:443 myuser@my-server.com\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eThere's actually a pretty nice article that goes into \u003ca href=\"https://links.kronis.dev/0qb10\"\u003emore detail\u003c/a\u003e and could be useful to anyone to learn a bit more.\u003c/p\u003e\n\u003cp\u003eOf course, there are a few caveats:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eyou can only tunnel TCP, not UDP, at least without involving a bunch of other hacks\u003c/li\u003e\n\u003cli\u003eit sometimes doesn't work, good luck figuring out why\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eIt might not be super descriptive, but guess what: I still don't know why every now and then the connection seemed to drop. Enter the next piece of software in the puzzle: \u003ca href=\"https://links.kronis.dev/8ojry\"\u003eautossh\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eAt this point, I still hadn't lost my faith in getting this to work, it still seemed to be simpler than going back to WireGuard and shuffling traffic around \u003ccode\u003eiptables\u003c/code\u003e (I hate it that much). After all, I just needed the following to get things working:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003eautossh -M 0 -o \u0026quot;ServerAliveInterval 20\u0026quot; -o \u0026quot;ServerAliveCountMax 3\u0026quot; -o \u0026quot;ExitOnForwardFailure=yes\u0026quot; -N \u0026quot;$SERVER_URL\u0026quot; \u0026gt; \u0026quot;$LOG_FILE\u0026quot;\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eExcept that also didn't work, same issue. No amount of messing around with the monitoring ports, or timeout values or whatever seemed to help. Eventually, I decided to get a hacky solution working, along the lines of:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003e#!/bin/bash\n\n# Define paths and variables\nPID_FILE=\u0026quot;/some/path/autossh.pid\u0026quot;\nLOG_FILE=\u0026quot;/some/path/autossh.log\u0026quot;\nSERVER_URL=\u0026quot;my-server.com\u0026quot;\n\n# Check if the PID file exists\nif [ -f \u0026quot;$PID_FILE\u0026quot; ]; then\n  # Read the PID from the file\n  PID=$(cat \u0026quot;$PID_FILE\u0026quot;)\n\n  # Check if the process is running and kill it\n  if ps -p \u0026quot;$PID\u0026quot; \u0026gt; /dev/null 2\u0026gt;\u0026amp;1; then\n    echo \u0026quot;Killing existing process with PID $PID\u0026quot;\n    kill \u0026quot;$PID\u0026quot; || { echo \u0026quot;Failed to kill process $PID\u0026quot;; exit 1; }\n  fi\n\n  # Remove the PID file\n  rm -f \u0026quot;$PID_FILE\u0026quot;\nfi\n\n# Start autossh and save the PID\n# add -vvvv for more output\nautossh -v -M 0 -o \u0026quot;ServerAliveInterval 20\u0026quot; -o \u0026quot;ServerAliveCountMax 3\u0026quot; -o \u0026quot;ExitOnForwardFailure=yes\u0026quot; -N \u0026quot;$SERVER_URL\u0026quot; \u0026gt; \u0026quot;$LOG_FILE\u0026quot; 2\u0026gt;\u0026amp;1 \u0026amp;\n\n# Save the PID of the background process\necho $! \u0026gt; \u0026quot;$PID_FILE\u0026quot;\n\necho \u0026quot;autossh started with PID $(cat \u0026quot;$PID_FILE\u0026quot;) for server $SERVER_URL\u0026quot;\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eIt still fits within one page. I can understand what the script does. I am totally not sinking deeper into the \u003ca href=\"https://links.kronis.dev/72xja\"\u003esunk cost fallacy\u003c/a\u003e. Not at all. Especially because running the script is just one file, and I can use cron to make it execute upon every server restart, or even make it execute periodically, if I accept that there's going to be occasional interruptions but still can't resolve the issue of the connection just dying:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003e@reboot sleep 30 \u0026amp;\u0026amp; /root/ssh-tunnel-to-cloud-server.sh\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eThis, if course, isn't ideal. What if I decide to use some backup solution that needs an S3 bucket that's on my homelab, but it fails to upload backup files because the connection resets every now and then? I decided to look a bit more deeply into what was going on. After a cursory look at the logs, it seemed that the server that was getting issues was actually getting way more traffic than the other one (I have a few homelab servers).\u003c/p\u003e\n\u003cp\u003eSo what's the cause?\u003c/p\u003e\n\u003ch3\u003eZombie software\u003c/h3\u003e\n\u003cp\u003eAt first, I thought that maybe it was Mattermost or Nextcloud acting up and requesting data more often than it should. But nope, those were actually external requests coming in from across the world, requesting things that they shouldn't. At first, I thought that it was an attempt at finding vulnerabilities, just how like if you have anything reachable by SSH without disabling password auth, within minutes you'll probably start getting (hopefully futile) attempts at cracking your password.\u003c/p\u003e\n\u003cp\u003eYet, it wasn't really an attack, the requests were periodic and accessed the same endpoint: \u003ccode\u003e/api\u003c/code\u003e, under the default virtualhost, even without TLS (no HTTPS), meaning that they weren't even attempting to hack a particular piece of software on my servers. Instead, they'd see this placeholder page baked into my Apache2 container images, which I left there for debugging purposes:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-a-bit-of-a-web-developer.jpg\" alt=\"01-a-bit-of-a-web-developer\" title=\"01-a-bit-of-a-web-developer\"\u003e\u003c/p\u003e\n\u003cp\u003eI'm a bit of a web developer myself, with a passion for graphic design.\u003c/p\u003e\n\u003cp\u003eAs far as I'm concerned, though, if it was an attempt at hacking, it was a pretty bad one. Since I had recently migrated over to Contabo for cost reasons, my best guess was that someone still had their domain pointing at the IP address that was now given to my server. Because my server doesn't actually have any of the content that they were looking for, they were fruitlessly hitting the default page, along the lines of:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-what-is-going-on.jpg\" alt=\"01-what-is-going-on\" title=\"01-what-is-going-on\"\u003e\u003c/p\u003e\n\u003cp\u003eIt's odd that the software wouldn't give up, I thought that maybe returning 403 (forbidden) or 410 (gone) might change its mind, so I did a quick change to remove the default page, along the lines of:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003e\u0026lt;VirtualHost *:80\u0026gt;\n    Deny from all\n\u0026lt;/VirtualHost\u0026gt;\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eWhich would return the following to anyone attempting to access the resources:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-forbidden.jpg\" alt=\"01-forbidden\" title=\"01-forbidden\"\u003e\u003c/p\u003e\n\u003cp\u003eI thought that it might prevent similar issues in the future, but of course, nothing is ever so easy and oftentimes the software out there is rather badly behaved, so the requests still kept coming in from a handful of IP addresses:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02-countless-requests.jpg\" alt=\"02-countless-requests\" title=\"02-countless-requests\"\u003e\u003c/p\u003e\n\u003cp\u003eYou know what the worst part is? If I run the web server on my local server and the cloud server pushes the traffic to it through the SSH tunnel, then there's no way for me to tell the real IP address. I can't really ban the badly behaving IP addresses outright, because the web server only sees the address of my cloud server as the source. This is also pretty bad for anything you might want to call a serious deployment from a security standpoint, yet perhaps isn't such a big deal for me at the moment.\u003c/p\u003e\n\u003cp\u003eNormally, you'd have an upstream load balancer just pass you \u003ccode\u003eX-Forwarded-For\u003c/code\u003e \u003ca href=\"https://links.kronis.dev/6s55k\"\u003erequest header\u003c/a\u003e or something like that, but my whole setup was built with the idea that I wouldn't have to run two separate servers and coordinate between them - that'd be wholly counterproductive to my goal of decreasing the amount of involved components into the mix.\u003c/p\u003e\n\u003cp\u003eLooking at the exact contents of the requests with \u003ccode\u003engrep\u003c/code\u003e also didn't reveal that much useful information, aside from the \u003ccode\u003eHost\u003c/code\u003e header and the domain that they were trying to use. You'll notice that I haven't really attempted to clear the data so much, because it is using HTTP in the first place anyways and is sending data to a server that they don't own. Oh, and they attempted to implement something like SOAP in a JSON API:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03-request-contents.jpg\" alt=\"03-request-contents\" title=\"03-request-contents\"\u003e\u003c/p\u003e\n\u003cp\u003eHeresy. Looking at the domain itself, it seems that it will expire at the end of this year, unless they decide to renew it. If they do, I guess I'll have to reach out to them, but because they're located \u003ca href=\"https://links.kronis.dev/wqyy3\"\u003esomewhere in the Caribbean\u003c/a\u003e I didn't really feel like trying to contact them just yet and instead reached for a technical solution. Well, maybe because it's also 3 AM and I doubt I could compose a polite e-mail right now:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04-the-domain.jpg\" alt=\"04-the-domain\" title=\"04-the-domain\"\u003e\u003c/p\u003e\n\u003cp\u003eNow, I did have an idea about how I could fix this...\u003c/p\u003e\n\u003ch3\u003eLosing my sanity\u003c/h3\u003e\n\u003cp\u003eWhat if I just ran the web server on the cloud server directly and then just let the Docker Swarm \u003ca href=\"https://links.kronis.dev/cf3gl\"\u003eoverlay network\u003c/a\u003e talk to my homelab node and the containers running on it? It might mean putting some of my TLS certificates and other information on the cloud server instead of keeping it within the confines of my homelab and complicate things further, but on the surface level it seemed like it could work:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"05-docker-swarm-overlay-network.jpg\" alt=\"05-docker-swarm-overlay-network\" title=\"05-docker-swarm-overlay-network\"\u003e\u003c/p\u003e\n\u003cp\u003eIt would also mean that the servers would both take the majority of the incoming traffic and if anything happened to them, the impact on my homelab would be a bit lessened. Not only that, but I could also see the real IP addresses of the incoming requests and block whatever I didn't like, integrations with software like \u003ccode\u003efail2ban\u003c/code\u003e should also work nicely, as would any rate limits and whatnot.\u003c/p\u003e\n\u003cp\u003eThe first step was shutting down the Apache2 containers on my homelab and disabling the SSH tunnels. That was pretty easy to do.\u003c/p\u003e\n\u003cp\u003eThe next step was connecting the public servers to the Docker Swarm cluster, so that I could schedule services (container instances) to be run on them and so that they could partake in the overlay networks. Done.\u003c/p\u003e\n\u003cp\u003eThen, I'd just need to launch the same Apache2 instances, after carrying over the necessary configuration. Took me a few minutes to drag a few \u003ccode\u003e.tar.gz\u003c/code\u003e archives around, but nothing too bad.\u003c/p\u003e\n\u003cp\u003eExcept it didn't work. You see, the nodes showed up in the Docker Swarm cluster. The containers were running. Doing \u003ccode\u003ecurl http://localhost:80\u003c/code\u003e returned the contents of the web server container, just as expected. However, when I did the request from any external server or even my own web browser, it would just show \u003ccode\u003eConnection refused\u003c/code\u003e. The port was open, but the connection would just be refused.\u003c/p\u003e\n\u003cp\u003eI checked that server against a few others that had no such issue. Same container. Same type of configuration. Same \u003ccode\u003ehost\u003c/code\u003e binding against the ports on the server and the same \u003ccode\u003eoverlay\u003c/code\u003e networks for communication with the other containers, even ones on the same node. The same \u003ccode\u003eiptables\u003c/code\u003e rules. The same configuration for SSH, no SSH tunnel ports left open. The same privileges and user running the containers, even launching a container with \u003ccode\u003edocker run --rm -p 81:80 ...\u003c/code\u003e so on another low numbered port right next to the main one would work.\u003c/p\u003e\n\u003cp\u003eWhat on earth was wrong with it!?\u003c/p\u003e\n\u003cp\u003eI tried using \u003ccode\u003etcpdump\u003c/code\u003e, I tried digging around in the every possible log that I could think of, I tried every type of configuration I could, I inspected every Docker container and network configuration and the node configuration. Everything seemed the same and should have worked, but didn't. I felt like I was going more or less insane, so in the end I did the only thing that made sense.\u003c/p\u003e\n\u003ch3\u003eWiping everything\u003c/h3\u003e\n\u003cp\u003eMost of the data on my servers is pretty well isolated and I'm working on getting a new backup system working, so it's pretty safe to wipe servers whenever the need arises. And now was definitely the time to do so:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"05-the-reinstall.jpg\" alt=\"05-the-reinstall\" title=\"05-the-reinstall\"\u003e\u003c/p\u003e\n\u003cp\u003eThis also meant that I'd need to do a bit of cleanup on the Docker Swarm side, to remove the networks and containers in question, which I would later recreate as necessary, just to avoid any dangling stuff left over, at this point I was taking no risks:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"06-wiping-stacks.jpg\" alt=\"06-wiping-stacks\" title=\"06-wiping-stacks\"\u003e\u003c/p\u003e\n\u003cp\u003eAbout 30 minutes of setting everything up later (I could spend a day or two automating everything with Ansible, not really worth it yet), I had the servers back up and running and there were no longer issues! It was all looking pretty good, even though I felt a bit sour about not being able to pinpoint the cause of the issues. I suspect that it might have been some rogue \u003ccode\u003eiptables\u003c/code\u003e rule, but it would have been very odd when the rules just wouldn't have showed up, that'd make no sense!\u003c/p\u003e\n\u003cp\u003eSo, with the new web servers in place, everything finally started working:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"07-dns-errors.jpg\" alt=\"07-dns-errors\" title=\"07-dns-errors\"\u003e\u003c/p\u003e\n\u003cp\u003eOh wait, it didn't. The web server itself was up and running, and accepting traffic. The problem was that it couldn't reach any of my homelab servers. Now, I was thinking that I could just update the SSH tunnel configuration to include whatever Docker Swarm needs... except that \u003ca href=\"https://links.kronis.dev/loyrj\"\u003ewhat Docker Swarm needs is UDP ports\u003c/a\u003e:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"08-docker-swarm-ports.jpg\" alt=\"08-docker-swarm-ports\" title=\"08-docker-swarm-ports\"\u003e\u003c/p\u003e\n\u003cp\u003eWhich I cannot tunnel easily. Some of these ports can't even be remapped to others and there are pretty strongly worded reminders about not exposing these in the wrong places. In other words, it became painfully apparent to me that I should have stuck with my WireGuard setup and perhaps even just join all of my nodes to a WireGuard VPN and let them talk through that, for the added security. Trying to simplify things just made me get burnt pretty badly on all of the above.\u003c/p\u003e\n\u003cp\u003eA learning experience, sure, but an unwelcome one.\u003c/p\u003e\n\u003ch3\u003eGiving up\u003c/h3\u003e\n\u003cp\u003eSo what did I do in the end? I settled on letting the zombie software just do its thing, but at least tell it that it's unwelcome here:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003e\u0026lt;VirtualHost *:80\u0026gt;\n    ServerName dead-domain.com\n    RewriteEngine On\n    RewriteRule \u0026quot;.*\u0026quot; \u0026quot;-\u0026quot; [G,NC]\n\u0026lt;/VirtualHost\u0026gt;\n\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eEnough for me to sleep at night, not at all enough to actually solve the damn problem:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"09-giving-up.jpg\" alt=\"09-giving-up\" title=\"09-giving-up\"\u003e\u003c/p\u003e\n\u003cp\u003eIt feels like in the future I'll just need to go back to using WireGuard and hopefully figure out a more manageable solution for managing traffic past that.\u003c/p\u003e\n\u003ch3\u003eSummary\u003c/h3\u003e\n\u003cp\u003eI fully accept that I might still have years to go until I'm good at networking, but literally none of this makes me feel good about the state of configuring anything or trying to make anything work, when nothing does, certainly not for a lack of trying.\u003c/p\u003e\n\u003cp\u003eHere's a list of grudges:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eWireGuard not having its own simple syntax for routing traffic, something like SSH tunnels would be great\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003eiptables\u003c/code\u003e, all of it, though I guess \u003ccode\u003enftables\u003c/code\u003e is a bit nicer (not really using it myself for now)\u003c/li\u003e\n\u003cli\u003eneeding \u003ccode\u003eautossh\u003c/code\u003e in the first place, not being able to figure out much from the logs (what is \u003ccode\u003edebug3: send packet: type 96\u003c/code\u003e and hundreds of lines like that, use your words)\u003c/li\u003e\n\u003cli\u003edebugging anything being hard, even with \u003ccode\u003etcpdump\u003c/code\u003e and \u003ccode\u003engrep\u003c/code\u003e, though I'm happy that at least they exist\u003c/li\u003e\n\u003cli\u003eagain, I'm happy that Docker Swarm is as usable as it is (compared to something like Kubernetes), but it's failure mode being \u0026quot;It sort of works on the same node, but suddenly will throw DNS resolution issues at you without clearly telling you that the nodes can't communicate to use the overlay network\u0026quot; is really bad. Even worse because there's no good way to diagnose that. If they figured out something closer to the WireGuard configuration where everything is pretty obvious (e.g. if you could choose who makes the connection and in which direction) then it'd be great, maybe aside from missing the ability to share volumes over the network (though I guess NFS exists, though that has its own issues).\u003c/li\u003e\n\u003cli\u003ethis whole situation, why can't I just use a single command to route traffic from 80 and 443 of my cloud server to my homelab server, while preserving the real IP addresses; it's not that niche of a use case!\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eAnd a few takeaways. Unless you hate getting things done in a timely manner or if you haven't been doing this for years, don't try doing all of this yourself. Just use a \u003ca href=\"https://links.kronis.dev/e395s\"\u003eCloudflare Tunnel\u003c/a\u003e or look at \u003ca href=\"https://links.kronis.dev/yj72g\"\u003esome of the tools here\u003c/a\u003e if you really need something that runs on your servers like sshuttle or frp or even WireGuard, but for the most part seriously just look into giving in to the centralized nature of professionally developed services in exchange for control over how you run things, that will save you a lot of time.\u003c/p\u003e\n\u003cp\u003eOtherwise, it's going to feel a lot like that one meme from Rick \u0026amp; Morty:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"10-rick-and-morty.jpg\" alt=\"10-rick-and-morty\" title=\"10-rick-and-morty\"\u003e\u003c/p\u003e\n"
        },
        {
            "title": "America In 2030: The Hateful Ones",
            "date_published": "2025-02-25",
            "id": "https://blog.kronis.dev/blog/america-in-2030-the-hateful-ones",
            "url": "https://blog.kronis.dev/blog/america-in-2030-the-hateful-ones",
            "content_html": "\u003cp\u003eSo here's something a bit different: a political post. I made this blog to primarily talk about technology related topics, but the state of the world is quite the mess and I'm long overdue for a good rant, so here we are.\u003c/p\u003e\n\u003cp\u003eHowever, this time I won't attempt to substantiate what I say with links or sources. I won't try to claim that what I say is true: as far as you're concerned, read everything below the following line as fiction. If you want, have a look at how much of it coincides with the reality that we live in, but that's on you.\u003c/p\u003e\n\u003c!-- raw HTML omitted --\u003e\n\u003ch3\u003eWhere we are in 2025\u003c/h3\u003e\n\u003cp\u003eAs a European, the state of the USA and the recent political movements in Europe is concerning, to say the least. America is not in a good state and I reckon that it will get worse, I think it wouldn't be out of place to say that the American democracy is somewhere between being in the process of dying or already being dead and that fact just needing acknowledgement.\u003c/p\u003e\n\u003cp\u003eTrump claimed no relation to Project 2025 and now it's coming true and they're implementing all of it, step by step. The American political system is supposed to be a democracy but there's also been more or less a full power grab by the political right: they have the majority in the Supreme Court, the Congress and they have the president. Even among the republicans, anyone who doesn't follow the party line most likely will be ousted, nobody is going to vote on laws on a per-issue basis. They have control.\u003c/p\u003e\n\u003cp\u003eThere was some right winger from the Heritage Foundation that said that there's a revolution in progress and that it will be bloodless, if the political left lets it be. It seems that the left did nothing of consequence and that it happened.\u003c/p\u003e\n\u003cp\u003eNow, blatantly illegal things will follow. What if the blue states will refuse to follow undemocratic orders by the government, or refuse to spread their rhetoric? Attempts to cut their federal funding will be made. You think that the Democrats can fight back with obstructionism and trying to draw out hearings when laws are made for as long as possible? I'd say that it's only a matter of time until such attempts lead to the people in question being escorted out.\u003c/p\u003e\n\u003cp\u003eThe idea of following procedure and any notion of decorum will probably only be for show and taken into account when it benefits them. \u0026quot;Us vs them\u0026quot; rhetoric will prosper and they won't even attempt to reach across the aisle and work together, or whatever the now-dead phrase was supposed to be. They have no need to do that. It's far easier to abuse their power and even lower themselves to using slurs against people that they don't like, because while they were supposed to represent their constituents, it seems that a large amount of them would be cheering them on.\u003c/p\u003e\n\u003cp\u003eOnce they've grabbed the power, it's unlikely that they'll exactly want to give it up. I would say that it's not out of the question that Trump might try for a third term, in a way that also excludes relatively well liked politicians like Obama from running again. So many changes will be made to the structure of the government and its agencies, that even if the democrats ever regain control of it, they'll be paralyzed and won't be able to do anything, leading to the public opinion of them falling lower.\u003c/p\u003e\n\u003cp\u003eThis administration will be blatantly two faced: they and their followers were against things such as student debt forgiveness (\u0026quot;I had to pay my full share of student debt, why would these kids be free of it? That's unfair to me!\u0026quot;), but now will throw around the idea of abolishing property tax or anything else that would benefit them, after their sponsors no doubt profiting greatly from acquiring as much real estate and resources as possible.\u003c/p\u003e\n\u003cp\u003eThe modern day America will most likely end up somewhere between being a dictatorship and kleptocracy, or maybe just an oligarchy.\u003c/p\u003e\n\u003ch3\u003eThe economy and corruption\u003c/h3\u003e\n\u003cp\u003eLargely, the population of the country will be robbed under the pretense of cost cutting, their social services will be gutted, not that the people in charge care about any of those.\u003c/p\u003e\n\u003cp\u003eAn uneducated populace is easier to manipulate and they're more likely to be hateful, so Department of Education will be ruined. Workers' rights and regulations hurt their bottom line, so the likes of OSHA will be dismantled. They will even attempt to fire a lot of people in the Federal Aviation Administration, even if it causes more aviation accidents to happen. Once that crisis arises, they'll attempt to blame it on the former administration's practices and will replace the former federal employees with those of their own, as an expensive private sector contract.\u003c/p\u003e\n\u003cp\u003eYou wanted tax cuts? They will cut taxes for the rich. They won't care about your healthcare or any benefits that you might eventually need, but probably will increase the funding for the military and the police, attempting to build even more of a police state. The right wants a small government and they will succeed at that - the government getting so small that you will either end up with a King, or all of the power just being in the hands of a few oligarchs.\u003c/p\u003e\n\u003cp\u003eWhat's worse, they can make you thank them for it. They are in control of the media and anyone attempting to truthfully report on news, or even not bend a knee to whatever agenda they are pushing, will be swiftly barred from reporting on them. Just look at Associated Press and what happened to them. The oligarchs also own a lot of the tech companies and Big Tech as a whole, so they will manipulate the public opinion freely. They are crafting terms of service that specifically permit hate speech against certain groups of people that they don't like, but not vice versa.\u003c/p\u003e\n\u003cp\u003eWe live in a time where the line between opinion and fact has been more or less destroyed. Channels like Fox will spew propaganda all the time and people will listen to it as much as the citizens of Russia do. They will praise the government that's stomping all over the values that the country was founded upon and they will celebrate it. And what about the courts? What has happened so far shows that they will either do nothing, Trump being a convicted criminal that is seeing no punishment, or that attempts to do anything will be completely ineffectual.\u003c/p\u003e\n\u003cp\u003eWhat do you think happens when the administration gets a court order to follow the law... and they choose not to? There is no integrity or respect at play here. It's blatant to the point where Trump launched his own crypto coin and scammed people out of billions of dollars (though it wouldn't be surprising if it was a thinly veiled bribe) and yet there is no outcry about this. There is no liability.\u003c/p\u003e\n\u003ch3\u003eThe rise of authoritarianism and demise of democracy\u003c/h3\u003e\n\u003cp\u003eThere are also no checks and balances in place anymore. You have the likes of Musk, an unelected non-government employee digging around the personal data of millions of Americans. That's illegal, you say? Well, it will be made legal, first with claims that it is, then with procuring whatever documents are needed to keep up the theater of the rule of law.\u003c/p\u003e\n\u003cp\u003eEven if it means having a bunch of college kids with no oversight digging around systems that sometimes contain classified information. Even if it means them being unqualified enough to inadvertently leak it on social media, when attempting to make an unrelated point. There are no laws, if there is no enforcement of them. There are no laws, if the people who should enforce them choose not to. At that point, they're just words on a piece of paper, a historical record of some intent. Nothing more.\u003c/p\u003e\n\u003cp\u003eThey will also freely remove any information that might contradict them. Decades of research will be removed and even if not, then just outright disregarded. Bad science will be praised, whenever it backs up whatever points they want to make, be it something like the Cass Review, or something that tells them that clearly they should use more fossil fuels. They don't care for the nerds who respect the scientific method. Science is just a tool to take advantage of the Appeal to Authority fallacy, nothing more.\u003c/p\u003e\n\u003cp\u003eWhen people resign from their offices in protest, normally this would be a warning sign - if qualified specialists with a track record of years of good discourse are leaving, that means that something is severely wrong. Yet, that is exactly what they want. To them, it is a political win, to be shouted from the rooftops, that they have gotten rid of the old administration and will replace everyone with \u0026quot;yes men\u0026quot;.\u003c/p\u003e\n\u003cp\u003eIt's also not just the \u0026quot;us vs them\u0026quot; rhetoric and the claims about an enemy within, which draws historical parallels. They will also blatantly forge documents and change numbers around to say whatever they want, there are no watchdogs in place and any fact checking is also being quickly dismantled. That's how you get claims made by the likes of DOGE, that they're saving money on an 8 billion USD contract that they've gotten rid of, even when the actual sum is 8 million USD instead, because they just added a few zeroes in their information systems and called it a day.\u003c/p\u003e\n\u003cp\u003eThere are also blatant claims that soon there \u0026quot;won't be blue states\u0026quot;, as well as bragging about how they \u0026quot;know the voting machines\u0026quot; and that soon people \u0026quot;won't have to vote\u0026quot;. That's about as clear of a signal that you can get, that they'll both try to destabilize any of the states that don't share their political affiliations, nor that whatever elections might happen in the future will be remotely fair.\u003c/p\u003e\n\u003ch3\u003eAttacks on women's rights, the theocracy\u003c/h3\u003e\n\u003cp\u003eAll of this started a while before the current election, as their compatriots were installed in high government seats, such as the Supreme Court, where the justices have lifetime tenure.\u003c/p\u003e\n\u003cp\u003eThat's how you the likes of Dobbs v. Jackson - meaning that states in the country can restrict the rights of the people to access abortion, already done in great deal in the deeply religious and conservative states. To date, dozens of states have shamelessly done that and celebrate it as some sort of a victory, even in the cases where it is necessary to save the life of the mother. It shouldn't be surprising that many people on the political right have a disdain for women and their rights.\u003c/p\u003e\n\u003cp\u003eIt's not the party of \u0026quot;pro choice\u0026quot;, it's the party of pro birth. They don't care what happens to the kids afterwards, something as simple as looking at how many voted against free school lunches should be telling. They pretend to not understand or just don't care that if abortions are forbidden, then children will be brought into this world in relationships where the parents cannot properly start a family due to financial issues and weren't planning or doing so, or in cases that are much worse. This ruling will ruin lives, if not outright end many, just because the religious people believe that a human being in the making should be prioritized over the people that are living now.\u003c/p\u003e\n\u003cp\u003eI wouldn't be surprised if the logical reaction is something like the 4B movement in Korea, where women choose not to associate with the men who hold such beliefs - no dating them and no marriage, presenting a more extremist feminist view, but in this case one where plenty of women would be pushed in that direction by these rulings, since they need to stay safe themselves.\u003c/p\u003e\n\u003cp\u003eThe thing is, that the government shouldn't have a say in a woman's bodily autonomy. That is none of their business and it definitely shouldn't be the business of some religious extremists that don't even view women as equals. I'm not sure whether Americans were too misogynist to vote for a woman and had too much internalized misogyny for that (\u0026quot;I don't think a woman could do the job.\u0026quot;) but the end result is that it hurts countless women across the country.\u003c/p\u003e\n\u003cp\u003eAll of this is happening, while they are suggesting that there should be religious classes in school and while many states are staunchly against science, like wanting to teach Intelligent Design instead of the Theory of Evolution, while they don't want to teach sex-ed which also talks about things like contraception and the idea of consent, but want to leave people unprepared for the realities of adulthood.\u003c/p\u003e\n\u003cp\u003eFreedom of religion should also mean freedom from religion, to whoever wants that. In their minds, instead, everyone should follow their teachings, not atheism, not agnosticism, not even other religions. Not even their own religion, not reading the overall intent of books like the Bible, but instead praising a selective interpretation that lets them be more hateful and manipulate people to their heart's content.\u003c/p\u003e\n\u003cp\u003eDon't you think that it might be within their sights to target women's voting rights soon after? Or even something like no-fault divorce? The Republican party cares so much about women and children that they replaced a \u0026quot;DEI hire\u0026quot; with a \u0026quot;DUI hire\u0026quot; and support a president that's received sexual misconduct allegations.\u003c/p\u003e\n\u003ch3\u003eAttacks on diversity and LGBTQ+\u003c/h3\u003e\n\u003cp\u003eIt doesn't end there, there's also an attack for numerous programs of diversity, equity and inclusion. It's curious that the right wingers seldom say the full thing out loud, but usually just prefer to shorten it to DEI, because that's easier to rally against without even giving what you're standing against some thought. I believe that equality of opportunity is more important than equality of outcome, but it seems that these people just use the concept as some thinly veiled excuse for racism and bigotry, which they now celebrate.\u003c/p\u003e\n\u003cp\u003eThey will try to remove anyone who doesn't look or act like them from any position they can, claiming that they were a \u0026quot;DEI hire\u0026quot;, which was there in the first place to compensate for the biases in the hiring processes due to the status quo. It's quite absurd, how direct they are being. Honestly, nowadays you can't even enjoy a video game, a movie or a show, because these right wing grifters also attempt to wage their culture war and attempt to belittle a work of art, just because the creators fit any number of social groups that they don't like.\u003c/p\u003e\n\u003cp\u003eOf course, it's much more serious than that. Not only do they spread their hateful rhetoric which might translate into the violence, but the likes of Tate, Peterson and Rogan corrupt the minds for the generations to come. Peterson and Rogan don't seem to have started out this way, but shifted their views to the right when it became profitable to do so, very much having no ethics themselves. It's the same as Trump posing with a rainbow flag that one time, but dropping support for any LGBTQ+ groups when he decided that hate earns more attention and support.\u003c/p\u003e\n\u003cp\u003eSocial programs will be cut. Attempts to rewrite history will be made, like the history of the Stonewall riots. Science which contradicts the right wing culture war claims will be censored and replaced with faux science that does, such as studies that show that the trans people who receive gender affirming care have low regret rates and experience increased quality of life. Out of the ones that don't, the majority of bad outcomes are due to social pressures - probably these very same conservative people not having it in their hearts to respect others and be nice to them. They will even claim that surgery is done on minors, when it plainly isn't. They will exaggerate the effects of puberty blockers, when the people that take them for other medical reasons don't seem to have horrible complications.\u003c/p\u003e\n\u003cp\u003eThey are going to use words like \u0026quot;mutilation\u0026quot; and force trans people to get passports which don't match who they are. They don't care about the difference between sex and gender. They will claim that an Olympic boxer was trans, when in actuality she was just a cis woman, to support their own arguments. There is no regard for truth here. The cruelty is the point. I'd be surprised if they didn't try to prosecute LGBTQ+ people more, like is already done against people of color due to institutionalized racism. I'd be surprised if they didn't crack down on the groups, much like China has also done in the past, as well as Russia describing something as simple as gender identity as \u0026quot;radical extremism\u0026quot;.\u003c/p\u003e\n\u003cp\u003eIt's the same as with the \u0026quot;groomer rhetoric\u0026quot; or trying to claim that drag shows or whatever are somehow an affront, all while the majority of the crimes committed against other people are due to those right wingers (including priests and religious figures, even). It's odd to see the gay community figuratively throwing trans people under a bus, when they themselves would have been the target of similar rhetoric a few decades ago. Come to think of it, don't you think that gay marriage will also be under attack now?\u003c/p\u003e\n\u003cp\u003eIt's hard to listen to the inauguration speech and the newly elected president speaking about how they will have a golden age and that they will build great things together... all while tossing so many people aside. Since when are the LGBTQ+ people not Americans? Do they not deserve your golden age? Sounds like hypocrisy and doublespeak to me, or rather just ignorant populism.\u003c/p\u003e\n\u003ch3\u003eAttacks on the climate and nature\u003c/h3\u003e\n\u003cp\u003eIt doesn't end there either. How about eco terrorism against his own countrymen? Deploying the military to California to release water from dams, from their water reserves. Close to 10 billion liters of water released downstream, with it being wasted in the winter season, and because it would have never made its way down to Los Angeles in the first place. Claiming that it would have helped with extinguishing wildfires, while the Southern California already has historically high water levels and it wouldn't have helped at all.\u003c/p\u003e\n\u003cp\u003eWater, which, mind you, was there in the first place as a reserve due to the risk of having a hot and dry summer, now actually putting farmers in California at risk. They are either stupid or malicious beyond belief. Neither option looks good, their actions are exactly the kind of thing that makes you think about the Aral sea.\u003c/p\u003e\n\u003cp\u003eThis is the same sort of ignorance that lead to them dropping out of the Paris Agreement and without a doubt they'll do whatever they can to extract as much natural resources from their land as possible, the environment and even things like the quality of drinking water be damned. They couldn't care less about that, about biodiversity, about sustainability, anything.\u003c/p\u003e\n\u003cp\u003eDo you not think that climate science will more or less either cease to be a thing, or will get lots of falsified records that say that everything is okay, leaving the coming generations to deal with the consequences?\u003c/p\u003e\n\u003ch3\u003eIsrael and Gaza\u003c/h3\u003e\n\u003cp\u003eWhat about the people who voted against Harris due to spite over the situation in Gaza? The new administration is even more militant and intent on giving Israel whatever they need to do whatever they want.\u003c/p\u003e\n\u003cp\u003eThe situation in the Middle East isn't inconceivably difficult. Let me break it down:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eHAMAS is a terrorist organization and should be treated as such, not as a legitimate governing force\u003c/li\u003e\n\u003cli\u003ePalestinians are not inherently HAMAS and deserve the same human rights as anyone else, even if their dwelling is occupied\u003c/li\u003e\n\u003cli\u003eIsrael has a right to defend itself from HAMAS, but it doesn't give them the right to harm Palestinians in the process\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThe people suffering in any conflict the most are the civilians, both on the side of Israel and Palestine. You don't get to crash a musical concert and commit untold atrocities. You don't get to take hostages or build bases under hospitals. You don't get to call groups of people \u0026quot;animals\u0026quot;. You don't get to starve people. You don't get to level city blocks where civilians live.\u003c/p\u003e\n\u003cp\u003eDefend yourself, but if you claim that you've done no war crimes, then clearly you should stand in front of ICC with no worries. If you ask me who are the \u0026quot;good guys\u0026quot; in that conflict, I will simply say: \u0026quot;Israel is clearly culturally closer to the West, but right now neither of the military forces in the area are good in an absolute sense, not even close.\u0026quot;\u003c/p\u003e\n\u003cp\u003eYet, the voters think that choosing the more warmongering candidate as a protest against less-than-ideal handling of the situation was a good choice. A candidate that says that the conflict would have never happened with him in power, a claim that's also clearly impossible to prove or disprove, but sounds nice to his supporters. Do you think that the current administration will try to get Israel to show restraint, or will they try to instead profiteer from the conflict and its consequences as much as possible?\u003c/p\u003e\n\u003ch3\u003eUkraine and Russia\u003c/h3\u003e\n\u003cp\u003eSpeaking of badly handled situations: Ukraine.\u003c/p\u003e\n\u003cp\u003eRussia occupied a part of the country in 2014. Then, in 2022 they attempted to take the whole country. Russia is the aggressor in the conflict, that much is clear. Ukraine deserves to defend its borders, that much is obvious. The way they have treated the conflict and those captured in it, even Israel could learn from them. If the US wanted to support countries that need to buy a lot of military equipment, then selling arms to Ukraine would earn them a lot of good will with Europe. Even if it was just donating the Cold War equipment that would need to be decommissioned anyways.\u003c/p\u003e\n\u003cp\u003eI'm not saying that Ukraine is perfect, of course it isn't, but when they signed the Budapest Memorandum and gave up their nuclear arms, imagine how they must feel when one of the countries that were supposed to \u0026quot;assure\u0026quot; their safety (the specific wording actually being quite useless if you read into it) ends up starting an invasion of them, all while refusing to even call it a war, which is what it is.\u003c/p\u003e\n\u003cp\u003eI will say that I don't hate the Russian people, or someone with Russian heritage. They have a rich history and art. At the same time, it would be delusional to listen to a single word coming from their government in the past decades sincerely, the same government that is stealing their future and suppressing anyone who wants to live in a democratic country. Demagogues like Lavrov should be evidence enough.\u003c/p\u003e\n\u003cp\u003eAnd now, when Trump is asking Ukraine to give up hundreds of billions in natural resources, all just to have the privilege of giving up parts of their country to the aggressor, while not being allowed to join NATO, knowing that the captured territories and those resources would also be stolen... am I the only one that can't help but to think of Molotov and Ribbentrop? It's so obvious that they're even threatening to shut off Starlink if Ukraine doesn't cave to the American demands.\u003c/p\u003e\n\u003cp\u003eThe list of Russia's red lines that have been crossed is long, lines that they draw in blood on soil that they have no business on being, yet somehow we're not all living in a nuclear winter right now. So clearly America now thinks that the correct option is to blame Poland, excuse me, Ukraine for being invaded and to chop it up and give a significant part of the country to the aggressor... while not even intending to let European powers to participate in these fake peace talks.\u003c/p\u003e\n\u003cp\u003eIt's actually quite clear that Trump views Putin as more of an ally than those who are in charge of actual democratic powers, perhaps because he finds Putin closer to how he himself wants to govern. If birds of a feather flock together, then I guess that must also apply to dictators. Imagine selling out an independent nation that is aligning itself with Europe to a dictatorial regime that your country spent the entire Cold War preparing to defend against.\u003c/p\u003e\n\u003cp\u003eAt this point you might as well say that the \u0026quot;MAGA\u0026quot; movement is patently un-American because it stands for Russia, with such statements as \u0026quot;I'd rather be Russian than Democrat.\u0026quot; Or worse yet, maybe that's exactly what America stands for nowadays. They are selling Ukraine out. They will lie however they want, claim that the Ukrainian government doesn't have the support of its people and how much of a tragedy all of the people dying is, without confronting the fact that nobody asked Russia to invade the country. They will blame Biden for the actions of Putin, the blame will never be put where it should be.\u003c/p\u003e\n\u003cp\u003eFear mongering about WW3 will soon follow. Someone will probably ask whether you would be willing to end the world if your neighboring country just took one of your towns. How about a city? A region? What these people are missing is that placating dictators by giving them more land didn't exactly work out great the last time.\u003c/p\u003e\n\u003ch3\u003eOther foreign policy\u003c/h3\u003e\n\u003cp\u003eHis poor treatment doesn't extend to just Ukraine or Europe as a whole, but just look at their neighbors - Canada and Mexico. How much good will can you ruin in a few weeks, how many tariffs can you threaten, which will just end up hurting the people and businesses in your own country, because they will pay them? How much can you vilify those countries and make fake claims about drugs, or trade deficit, claims in which you are the harmless party that everyone else tries to exploit, the victim?\u003c/p\u003e\n\u003cp\u003eIt's not even the tariffs or petty things like renaming the Gulf of Mexico (and promptly banning the journalists who dare mention the old name), but there's also plans of mass camps and deportations. Sold to the public as a measure against illegal immigration and that it'd extend to violent criminals, bolstered by false claims of entire prisons being emptied into the country, later extended as a form of racism, eventually tearing families apart.\u003c/p\u003e\n\u003cp\u003eEven if you don't have a modicum of human decency, who do you think will do all of the jobs that your own citizens don't want to do? Who is going to harvest your crops? How do you think your food supply will be impacted by attempting to do this so rapidly, even to the point where you will grab some of your own citizens in those raids, military veterans, no less. Do you think that the powers that be wouldn't enjoy the idea of also deporting any of the people that they find undesirable or that speak out against them? What about building camps that aren't on American soil, where large concentrations of people will be crammed into a small area? If only there was a term for camps like that.\u003c/p\u003e\n\u003ch3\u003eHow we got here\u003c/h3\u003e\n\u003cp\u003eNone of it looks good. The Republicans lied about not being related to Project 2025 and now are dismantling whatever they can from the former democratic government. If anyone is wondering how things got so wrong in the last century, here is a pretty good replay of all of it in real time.\u003c/p\u003e\n\u003cp\u003eHistorically, authoritarian regimes like that have been toppled by copious amounts of violence, like the French Revolution. My argument isn't that violence is necessary in such a situation, because it would likely result in instability not seen before and more or less end the country then and there. Rather, in the modern day world there won't and cannot be violence, it's also possibly already too late for any sort of action, even proper and ethical law enforcement.\u003c/p\u003e\n\u003cp\u003eIf Trump didn't go to jail after the events of January 6th, then the law enforcement failed. If he was convicted of crimes but not punished, how could you expect anything but that the rioters would be freed and a coup attempt dismissed? If all of the institutions that were supposed to prevent an actual coup like Project 2025 taking place did nothing, then how can you be surprised that America is no longer a democracy? Curiously, sending Napoleon off to a vacation home on some island didn't exactly do much historically either.\u003c/p\u003e\n\u003cp\u003eThere are two options: the election was stolen or the election was chosen. Neither is good.\u003c/p\u003e\n\u003cp\u003eIf it was stolen, then nothing has been done to set the record straight, even if there are communities online that are gathering around the observation that the voting behavior was quite irregular and different from the past elections, as well as if there were bomb threats, attempts to purge voter records and make many ineligible to vote, primarily against the left leaning communities. Even if none of that is true and plenty of people just didn't turn up for the election, what's up with that?\u003c/p\u003e\n\u003cp\u003eDo they not care about their civic duty, especially after hearing about the risks posed by Project 2025? After the election, there was plentiful critique of the campaigning done by Harris. It might not have been a perfect campaign, but in a world where people would use their common sense and wouldn't be plain evil, she did nothing wrong. The choice between a criminal and a prosecutor for a president should be clear to everyone, it's not even a question worthy of that much consideration and no amount of criticism to cope will make it not so.\u003c/p\u003e\n\u003cp\u003eIf the election was chosen, then it doesn't get much better either. It's as if one third of the country is okay with standing idly by and watching another third of the country ruin things for everyone. Maybe people indeed did choose the extremist candidate. No doubt, lots of angry men as well, historically that's been quite the issue - liking charlatans that promise you the world while feeling disfranchised by the status quo. In many cases, they also won't be the ones getting oppressed either, so it works out for them.\u003c/p\u003e\n\u003cp\u003eIt's enough to go on any Republican community and you'll see sentiments like \u0026quot;I'm not getting tired of winning.\u0026quot; which only fits the definition when you personally like all of the above happening, in which case you are a horrible human being. But there's something greater at play here as well: a democratic system ending, because enough people wanted it to. They chose this, even if it harms others in the process.\u003c/p\u003e\n\u003cp\u003eI don't even think you need enough electoral college votes in practice, or over 50% of the votes in places where that is the system at play. All you need is sufficient support where you can project power, even if your actual numbers might be lower. If you have enough power and you cheat in the elections, nobody will be able or willing to properly challenge you. Imagine the case of 25-30% of the country voting for you, with plenty of radical extremists in their midst, that's millions of people. Who's going to go against that?\u003c/p\u003e\n\u003cp\u003eFrom where I stand, it seems that nobody will. There very much is the paradox of tolerance at play here: you believe that when others are being ethically bankrupt you still need to treat them with respect and hear them out, the \u0026quot;when they go low, we go high\u0026quot; rhetoric and then are suddenly surprised when they use a plethora of dirty tactics to win, that suddenly the public discourse is being horrible to each other and that extremist viewpoints are getting normalized.\u003c/p\u003e\n\u003cp\u003eIt didn't always used to be this. The other day, I looked at a recording of a political debate between Obama and Romney. They still had opposing views from each other back then, but they were more respectful. There was less polarization going on, less hatred and no calling each other wholly incompetent. Nowadays, looking at America, it looks like they're at each other's throats, however the Democrats have no teeth. So what's to blame for all that?\u003c/p\u003e\n\u003ch3\u003eWhere the problem lies\u003c/h3\u003e\n\u003cp\u003eHuman nature.\u003c/p\u003e\n\u003cp\u003eLet me get a bit more abstract for a moment. I'd argue that these behaviors and outcomes are independent of any given political party, national identity, or any other number of factors. You'll see the same sorts of behaviors in the past century, as well as this century, all across the world, in different cultures that speak different languages. You can't ban or outlaw it and oppressive tendencies will almost inevitably fester.\u003c/p\u003e\n\u003cp\u003eWhy do I need to point this out? Because on paper, there's a lot of political and economical systems that could work, yet they don't when put in practice. Communism could work, but never really has. Capitalism could work, but almost always goes wrong (late stage capitalism). Socialism could work and generally does work in Europe, though with caveats. Democracy also seems to work, though it can end up being short lived. Even authoritarian forms of government can work, yet usually end up being quite oppressive instead. Nationalism could be done in ways that are beneficial to lots of people... but it just isn't.\u003c/p\u003e\n\u003cp\u003eConsider this example of what nationalism could look like:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u0026quot;America is the best country on the Earth, so we will have good diplomatic relationships with other countries to be respected.\u0026quot;\u003c/li\u003e\n\u003cli\u003e\u0026quot;Americans are the best people on the Earth, so we will maintain ourselves to a high standard and be excellent to others.\u0026quot;\u003c/li\u003e\n\u003cli\u003e\u0026quot;We will always stay true to our trade deals and other agreements and support other allied powers, as they do to us.\u0026quot;\u003c/li\u003e\n\u003cli\u003e\u0026quot;We will have good border control to only let good people in, but at the same time, welcome those who can do the jobs that we don't want to.\u0026quot;\u003c/li\u003e\n\u003cli\u003e\u0026quot;We will have a good military to defend ourselves, but also prepare those in it for success in any number of industries, educate them etc.\u0026quot;\u003c/li\u003e\n\u003cli\u003e\u0026quot;We will have the best healthcare and education for our people, so that they can work towards making the nation great.\u0026quot;\u003c/li\u003e\n\u003cli\u003e\u0026quot;We will support small and medium businesses to avoid monopolies/oligopolies and profit off of trade and technology, not tariffs.\u0026quot;\u003c/li\u003e\n\u003cli\u003e\u0026quot;We will also respect men, women, the religious and not alike, as well LGBTQ+ and other minority groups, because they are Americans too.\u0026quot;\u003c/li\u003e\n\u003cli\u003e\u0026quot;We will fight against any rhetoric that seeks to weaken us, to divide us, like the rhetoric that Yuri Bezmenov laid out.\u0026quot;\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eIt might sound a bit silly, because every nationalist government in the history of the world is not that. Instead, they sow hateful rhetoric and mobilize people to hate each other so that they can solidify their power and rob people blind, just look at what happened to Russia and its oligarchs or what's happening to America right now.\u003c/p\u003e\n\u003cp\u003eIt's the same with attempts at having communism, they never work because you inevitably end up having an inner circle of the Party that are \u0026quot;more equal\u0026quot; than everyone else who is supposed to be equal and if you speak out against them, you'll get taken out in a black KGB car to the Corner House where you'll be forced to sign a paper saying you've done crimes against the Party and deserve punishment. Even if not that, you'll still live in poverty while the Party officials have separate stores where they lead a lavish lifestyle, all things considered.\u003c/p\u003e\n\u003cp\u003eSometimes people also get too caught up in ideology and what they should do, instead of what's the reality that they live in. Look at the Great Leap Forward, which lead to one of the greatest famines in the history of humanity. Or, on a smaller scale, countries welcoming in immigrants, yet not being able to make them integrate within the populace and then suffering because on one hand you have groups of people that are incompatible with your culture and on the other you have voters that are going to parties that promise radical solutions.\u003c/p\u003e\n\u003cp\u003eI guess you could make an attempt at grouping people in how much they care about their own success vs that of the people around them, as well as how much they actually believe in what they're saying. Have a look at the following sketch:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"collectivism-individualism-sincerity-insincerity.jpg\" alt=\"collectivism-individualism-sincerity-insincerity\" title=\"collectivism-individualism-sincerity-insincerity\"\u003e\u003c/p\u003e\n\u003cp\u003eOn the axes you have:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003ecollectivism vs individualism - how much someone focuses on the outcomes for everyone vs themselves\u003c/li\u003e\n\u003cli\u003esincerity vs insincerity - how much someone believes in their publicly expressed views and positions\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eFor example, Sanders has a history of political activism on the side of human rights and regularly is on the side of the citizens, the workers, all of the people who deserve fair treatment, whereas Harris often takes a more moderate stance while still holding somewhat similar views.\u003c/p\u003e\n\u003cp\u003eOn the other hand, the likes of Trump and Musk say whatever wins them favor - Trump taking pictures with the rainbow flag that had \u0026quot;LGBTs for Trump\u0026quot; written on it, before doing a 180 when it was convenient to do so. Musk having a somewhat left leaning persona before it became more profitable to lean to the far right. The same goes for podcasters such as Rogan, who might have been more neutral in the past but became more extremist in their views.\u003c/p\u003e\n\u003cp\u003eThat's not to say that extremes are good either: someone who is 100% sincere in their collectivist beliefs will probably listen to propaganda and might rat you out to KGB or your oppressive communist agency of choice for trying to make ends meet, while at the same time the Party will probably be in the insincere group and enjoy their separate stores. Someone on the far side of individualism will most likely have radical and hateful beliefs, or will be a bit of a sociopath and say whatever leads to more personal success.\u003c/p\u003e\n\u003cp\u003eIn truth, I'm quite sure that the good people are those who care about good outcomes for most people sincerely, yet don't lose track of reality and don't give in to slogans and such. I know for a fact that I'm not the best person in the world and that I have shortcomings of my own, but when you see those genuinely good people out there, help them succeed. I've had the pleasure of meeting some really kind and nice people and they do give me hope in the world, even if right now it feels like the hateful ones are winning.\u003c/p\u003e\n\u003ch3\u003eSo now what?\u003c/h3\u003e\n\u003cp\u003eI hold a few truths self evident: the Universal Declaration of Human Rights should be a starting point, human beings deserve respect and to pursue happiness and self-fulfillment, to express themselves and live their best lives, as long as it doesn't cause others harm. At the same time, you shouldn't try to be tolerant of those who are mean spirited or just evil.\u003c/p\u003e\n\u003cp\u003eDemocracy doesn't really work when people are uneducated or increasingly angry and both of those have been brought on by powers that try to subvert democracies all over the world, especially when they can fault to their political enemies and blame them for everything, no matter how much they need to lie to do that. We don't live in a world of facts anymore, but rather one of opinions.\u003c/p\u003e\n\u003cp\u003eYou can't really separate the concept of a political (or financial) system from its implementations: you can't claim that communism is good or the best system and we just haven't had the right nation implement it, when what's keeping it from being made a reality is human nature itself. A system of governing human beings that doesn't work with human nature is no system at all. Ergo, most of our current systems are at least somewhat dysfunctional, that does also apply to late stage capitalism which ends up exploitative and democracies that can't help but to devolve in authoritarianism of some description.\u003c/p\u003e\n\u003cp\u003eThis also means that systems need people upholding them, to prevent them from being corrupted. Americans like to believe that freedom of speech is some inalienable right that's granted to them by God, whereas more realistically it ends the moment you're forcibly picked up from a protest and thrown in the back of a police van. Ask the people of Russia their experiences with protesting and OMON.\u003c/p\u003e\n\u003cp\u003eThe same goes for systems that seem to follow the rule of law, except the laws aren't made to benefit you: what are you going to do when you need to buy insulin to continue living, but every vendor in the market is doing price fixing and also lobbying to the point where it costs way more to buy than it does to produce, yet literally nobody cares about challenging the status quo?\u003c/p\u003e\n\u003cp\u003eI can't help but to wonder whether America will even have a different administration by the time 2030 has arrived. If it would have changed by then, then at least I'd hope it's with people like AOC at the forefront. America could have also been great to all people, with someone like Sanders in a position of power as well, or even just okay with someone like Harris, certainly better than the circus that is happening right now.\u003c/p\u003e\n\u003cp\u003ePersonally, I don't mind conservative values on a surface level (e.g. traditional family structure, believing in God and attending church) and haven't once been bothered by them, as long as it's not built around the idea of restricting the freedoms of others to live life the way they want, the same way how you choose how to live your own. Then again, what is conservative in the EU and what is conservative in the US is worlds apart in many cases, so perhaps I'm not quite as sure about that anymore.\u003c/p\u003e\n\u003cp\u003eThe reality is that you just can't please everyone and in trying to reach across the aisle, you can also lose sight of what you were supposed to protect, especially when the other side doesn't want to show you the same respect and instead wants to tear down your entire world. And it's quite messed up that from where we stand, that sentence holds true both from the subjective POV of the Democrats and the Republicans, regardless of how accurate their perceptions are. Of course, with the Republicans already being in the process of tearing down a lot of social progress, or at the very least being complicit in the process.\u003c/p\u003e\n\u003cp\u003eI don't necessarily believe that America is ruined beyond belief, at least not yet, but I also don't see a clear path for them to become a proper democracy, or to have a social cooling effect of sorts, where cooler heads prevail. Plus, with the rise of extremist political parties in Europe and how things went down in the Middle East, and with how Ukraine isn't getting the support it should be getting to defend its borders, things are looking quite grim.\u003c/p\u003e\n\u003cp\u003eObviously, I won't do any calls to action since all of this is supposed to be fiction anyways (I'm not the biggest fan of falling out of any windows), but the best you can probably do is support those around you who are good people, and live one day at a time. If nothing else, then this was a pretty good way to rant about the state of the world.\u003c/p\u003e\n"
        },
        {
            "title": "You might not need font subsetting",
            "date_published": "2025-02-19",
            "id": "https://blog.kronis.dev/blog/you-might-not-need-font-subsetting",
            "url": "https://blog.kronis.dev/blog/you-might-not-need-font-subsetting",
            "content_html": "\u003cp\u003eAs you've probably noticed, this blog uses a few custom fonts, because I rather liked how the \u003ca href=\"https://links.kronis.dev/iid8r\"\u003ePT Fonts\u003c/a\u003e look. They support my own language, a bunch of extended latin characters, offer both sans serif, serif and monospaced variations and have an open license, meaning that I can use them wherever I want, from my websites to any desktop apps or even games I might make.\u003c/p\u003e\n\u003cp\u003eHowever, the obvious issue with this approach is that anyone who comes to the site needs to make some additional web requests and, even \u003ca href=\"https://links.kronis.dev/r5xhp\"\u003ewith how efficient WOFF2 is\u003c/a\u003e, that is still a bit of bandwidth:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"00-custom-fonts.jpg\" alt=\"00-custom-fonts\" title=\"00-custom-fonts\"\u003e\u003c/p\u003e\n\u003cp\u003eI will acknowledge that nowadays additional 200 KB of downloaded data probably isn't the end of the world, but admittedly it isn't ideal either. The good news is that the fonts only need to be downloaded once and can be cached for a while, depending on the web server configuration:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"00-font-caching.jpg\" alt=\"00-font-caching\" title=\"00-font-caching\"\u003e\u003c/p\u003e\n\u003cp\u003eBut there's also another approach that you can use, which could help quite a bit.\u003c/p\u003e\n\u003ch3\u003eFont subsetting\u003c/h3\u003e\n\u003cp\u003eYou have the option of taking a font which contains all of the glyphs and \u003ca href=\"https://links.kronis.dev/b9liq\"\u003esplitting it up into multiple files\u003c/a\u003e, so that the browser only downloads the font files that are actually used (the glyphs are present in the page).\u003c/p\u003e\n\u003cp\u003eFor example, the PT Fonts have support for cyrillic characters but I don't really think many (any?) of my blog posts need those, since I write in English. Therefore, I could create a separate font file and people wouldn't need to download unneeded characters in most pages. Google Fonts does that by default across various sets of languages, and there are even \u003ca href=\"https://links.kronis.dev/45eq0\"\u003eweb based tools\u003c/a\u003e for doing that:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-font-subsetter.jpg\" alt=\"01-font-subsetter\" title=\"01-font-subsetter\"\u003e\u003c/p\u003e\n\u003cp\u003eNow, for \u003ca href=\"https://links.kronis.dev/2nszi\"\u003emy own font site\u003c/a\u003e, I might want to use a local solution, that I can run from the confines of my OS, commit some new font files to the Git repository and have it be live in a little bit. The same probably goes for my blog (where I could use those prepared font files). Luckily, there exist a few packages to do that, for example, \u003ca href=\"https://links.kronis.dev/39ncy\"\u003eFont Ranger\u003c/a\u003e:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-font-ranger.jpg\" alt=\"01-font-ranger\" title=\"01-font-ranger\"\u003e\u003c/p\u003e\n\u003cp\u003eIt's not a super active project (the last commit being years ago), but thankfully it still seems to work and it's not like the workload itself has changed much over the years. The usage is pretty simple and we do get the support for a variety of font sets out of the box:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02-script-example.jpg\" alt=\"02-script-example\" title=\"02-script-example\"\u003e\u003c/p\u003e\n\u003cp\u003eHere's the full list:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003elatin\u003c/li\u003e\n\u003cli\u003elatin-ext\u003c/li\u003e\n\u003cli\u003ecyrillic\u003c/li\u003e\n\u003cli\u003ecyrillic-ext\u003c/li\u003e\n\u003cli\u003egreek\u003c/li\u003e\n\u003cli\u003egreek-ext\u003c/li\u003e\n\u003cli\u003evietnamese\u003c/li\u003e\n\u003cli\u003esinhala\u003c/li\u003e\n\u003cli\u003ehebrew\u003c/li\u003e\n\u003cli\u003eoriya\u003c/li\u003e\n\u003cli\u003emalayalam\u003c/li\u003e\n\u003cli\u003egurmukhi\u003c/li\u003e\n\u003cli\u003ekannada\u003c/li\u003e\n\u003cli\u003earabic\u003c/li\u003e\n\u003cli\u003etamil\u003c/li\u003e\n\u003cli\u003ekhmer\u003c/li\u003e\n\u003cli\u003etelugu\u003c/li\u003e\n\u003cli\u003ebengali\u003c/li\u003e\n\u003cli\u003ethai\u003c/li\u003e\n\u003cli\u003edevanagari\u003c/li\u003e\n\u003cli\u003emyanmar\u003c/li\u003e\n\u003cli\u003egujarati\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eAdmittedly, I haven't even heard of all the languages, but I'm sure that they're separated from the larger sets due to having interesting and unique characters, that would still increase the file sizes quite a bit. Just for the fun of it, I decided to retrieve every individual font set into a separate file. For comparison's sake, here's how the regular WOFF2 fonts look like.\u003c/p\u003e\n\u003cp\u003eYou can see that the \u003ccode\u003ePTSans-Regular.woff2\u003c/code\u003e font is \u003ccode\u003e111 KB\u003c/code\u003e in size. It was generated with a different tool, but since the compression is more or less the same, any other output should be comparable:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03-regular-font-contents.jpg\" alt=\"03-regular-font-contents\" title=\"03-regular-font-contents\"\u003e\u003c/p\u003e\n\u003cp\u003eAfter running the script, we get a folder with a whole bunch of \u003ccode\u003ePTSans-Regular.*\u003c/code\u003e subsets, for example, the latin variety is only \u003ccode\u003e44 KB\u003c/code\u003e in size, whereas the latin-ext variety is an additional \u003ccode\u003e26 KB\u003c/code\u003e, meaning that if we only need the characters contained within those two, then we could save on not downloading a bunch of cyrillic characters.\u003c/p\u003e\n\u003cp\u003eThis also accidentally ends up being a great tool for exploring what character sets are truly represented by a font, because we can see that aside from the latin and cyrillic characters, there aren't many more sets that have a large number of characters in them:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04-subset-font-contents.jpg\" alt=\"04-subset-font-contents\" title=\"04-subset-font-contents\"\u003e\u003c/p\u003e\n\u003cp\u003eHowever, things are sadly not that simple.\u003c/p\u003e\n\u003ch3\u003eThere are drawbacks\u003c/h3\u003e\n\u003cp\u003eThe most immediate drawback is that the total sum of all the font sizes in that folder with the subsets is now \u003ccode\u003e167 KB\u003c/code\u003e instead of the self contained font, which was just \u003ccode\u003e111 KB\u003c/code\u003e. In other words, you only benefit from those savings when the initial font has support for a lot of languages and symbols which you won't need, so this will depend on what exactly you're dealing with, whether the added overhead won't become a problem.\u003c/p\u003e\n\u003cp\u003eSecondly, there's the CSS. Originally, I had a pretty simple font file, which just defined all of the font families that I want to support: \u003ccode\u003ePT Sans\u003c/code\u003e, \u003ccode\u003ePT Serif\u003c/code\u003e, \u003ccode\u003ePT Mono\u003c/code\u003e and their various versions, with the styles and weights. That was pretty simple and the total CSS file was about \u003ccode\u003e3 KB\u003c/code\u003e in size, which might get some additional space savings from further compression, because it's just a text file:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"05-regular-font.jpg\" alt=\"05-regular-font\" title=\"05-regular-font\"\u003e\u003c/p\u003e\n\u003cp\u003eHowever, with the subset font, now we have a \u003ccode\u003e5 KB\u003c/code\u003e file just for the regular variety of \u003ccode\u003ePT Sans Regular\u003c/code\u003e, which also doesn't have the same configuration as my own CSS file did previously. Either way, the trend here is clear: if I had 15 different font files then with \u003ccode\u003e5 KB\u003c/code\u003e per file, we'd end up with the total size of \u003ccode\u003e75 KB\u003c/code\u003e and a lot of contents:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"06-subset-font.jpg\" alt=\"06-subset-font\" title=\"06-subset-font\"\u003e\u003c/p\u003e\n\u003cp\u003eFurthermore, those CSS files would always need to be loaded, to tell the browser that we even have our custom font subsets available. Now, it wouldn't be super bad, because there would usually still be compression for the CSS, but each of the files would get compressed separately and realistically we'd still need to deal with about \u003ccode\u003e20 KB\u003c/code\u003e or so of additional downloads, as well as the fact that this would mean more HTTP requests and round trips to the server.\u003c/p\u003e\n\u003ch3\u003eSummary\u003c/h3\u003e\n\u003cp\u003eI didn't actually go onwards to test the numbers, to automate generating every variety, which seems like it would probably be around 345 font files (even is most of those wouldn't have many/any characters in them), because at this point it was quite clear that for my particular use case, I don't need font subsets.\u003c/p\u003e\n\u003cp\u003eIt definitely makes sense for the scales that Google Fonts operates at and while I might consider separating the cyrillic glyphs from the rest of them, for now the simplicity of having a manageable amount of files and CSS that I can fit within a single file without issues beats the operational complexity font subsetting would introduce, for marginal at best bandwidth savings.\u003c/p\u003e\n\u003cp\u003eI do appreciate that font subsetting exists, but I'd probably need to look into optimizing the images first and foremost, there's about \u003ccode\u003e512 KB\u003c/code\u003e of them in this page alone. For now, I just convert everything to JPG because it's widely supported and is a lossy format that lets me slightly decrease the quality while having better filesizes than PNG, but \u003ca href=\"https://links.kronis.dev/e12o9\"\u003eWebP or even AVIF\u003c/a\u003e might be worth a look at some point.\u003c/p\u003e\n"
        },
        {
            "title": "Gitea isn't immune to issues either",
            "date_published": "2025-02-18",
            "id": "https://blog.kronis.dev/blog/gitea-isnt-immune-to-issues-either",
            "url": "https://blog.kronis.dev/blog/gitea-isnt-immune-to-issues-either",
            "content_html": "\u003cp\u003eIt's a little bit amusing that some of my first posts on the new blog setup are about how software is broken, but here we are. Regardless of whether I'm just quite unlucky, or just plain cursed, let's explore the fact that \u003ca href=\"https://links.kronis.dev/gjvgz\"\u003eGitea\u003c/a\u003e also has some pretty unfortunate issues.\u003c/p\u003e\n\u003cp\u003eI previously moved to self-hosting Gitea from my older GitLab setup, because it's much easier to update and administer, in addition to needing fewer resources to run well. Overall, I'm quite happy with it, but it's not bulletproof either. When I was migrating the blog, I decided to check how much storage \u003ca href=\"https://links.kronis.dev/4cabt\"\u003eGit LFS\u003c/a\u003e was using. Much to my surprise, that figure was 0 KB, which made no sense.\u003c/p\u003e\n\u003cp\u003eTrying to see why that was, I noticed a bunch of notices in the Self Check section of the admin pages:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-manual-changes-needed.jpg\" alt=\"01-manual-changes-needed\" title=\"01-manual-changes-needed\"\u003e\u003c/p\u003e\n\u003cp\u003eThis is where the first issues became apparent: there was a bunch of supposedly deprecated configuration being used, which might break things, but this was only visible in the admin dashboard. I wanted to verify whether this is really the issue, so I tried cloning the repository fresh and it did indeed have problems with binary files that are managed by Git LFS:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02-clone-failed.jpg\" alt=\"02-clone-failed\" title=\"02-clone-failed\"\u003e\u003c/p\u003e\n\u003cp\u003eIt's extremely alarming, especially when you consider that I got no issues when trying to initially push the resources, neither from the Git client, nor in Gitea. In other words, I might have been pushing code that would never work properly due to an incomplete set of changes being put on the server, Gitea failing silently. But it's not just a matter of the configuration, rather, the way it's managed is bad.\u003c/p\u003e\n\u003ch3\u003eBad ways to handle configuration\u003c/h3\u003e\n\u003cp\u003eSuppose that you have an application that needs some configuration to be changed, which, if not done, can have catastrophic circumstances like the above. If you want to use version control, then it makes sense to expect that things would be versioned correctly and that not happening is as bad as it gets.\u003c/p\u003e\n\u003cp\u003eNow, which of the following do you think is the best approach for managing such circumstances:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003edescribing the new configuration in your changelog or docs and letting things silently break for like 90% of your userbase\u003c/li\u003e\n\u003cli\u003ealso adding some sanity checks and making the old bad configuration be logged in the app itself somewhere, bringing that number down to maybe 70%\u003c/li\u003e\n\u003cli\u003eshowing the configuration warnings and the possible impact of those in the application itself (or even a generic message like \u0026quot;There is deprecated configuration in use, please see the admin Self Check section for more information\u0026quot;), making that number a more manageable 40%\u003c/li\u003e\n\u003cli\u003ealso sending e-mails out to the administrative users on startup or other important events, so that even if they don't look at the Web UI for a bit, they'll still be notified of what they need to do, making that number maybe 20%\u003c/li\u003e\n\u003cli\u003edeciding to reject Git pushes when the broken configuration is used, so that things cannot fail silently and so that users see exactly what's going on, essentially making that figure 0%\u003c/li\u003e\n\u003cli\u003eor perhaps, if the configuration values are pretty straightforward and what you've done is essentially some reorganization, then migrate the old values on startup, failing fast in the case that can't be done, also making that figure be 0%\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eIn other words, it's possible to go from approx. 90% of your userbase having issues due to the actions that you've done, down to that figure essentially being 0%, by progressively making it more apparent, regardless of whether it's done just for the repositories that use Git LFS or all of them. Instead, letting the code silently fail is the worst possible thing that you can do. It doesn't end there, though, the rabbit hole goes deeper.\u003c/p\u003e\n\u003cp\u003eYou see, I run my instance of Gitea in a Docker container, they even have \u003ca href=\"https://links.kronis.dev/kzrk8\"\u003eofficial instructions on how to set it up\u003c/a\u003e. If most of the configuration can and is done through the \u003ca href=\"https://links.kronis.dev/s09p6\"\u003eDocker environment variables\u003c/a\u003e then it'd make sense for all of the critical parameters to be correctly set there, or at least the underlying files to be updated to a good state, right? Well, no, because I didn't have many of the deprecated values anywhere in the description of my environment:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03-no-actual-configuration.jpg\" alt=\"03-no-actual-configuration\" title=\"03-no-actual-configuration\"\u003e\u003c/p\u003e\n\u003cp\u003eThis is actually a mistake that a lot of software makes, I'm not just being critical of Gitea here in particular, but rather the overall trend. If you're going to containerize your software, you more or less owe it to your users to have a \u003ccode\u003edocker-entrypoint.sh\u003c/code\u003e script that maps ALL of the allowed \u003ca href=\"https://links.kronis.dev/5qfix\"\u003econfiguration values\u003c/a\u003e to the files upon every startup and also has sane defaults for the values that aren't set. Defaults, which could technically change or be migrated across releases, all done in a way where the user doesn't have to think about them.\u003c/p\u003e\n\u003cp\u003eI know why it doesn't happen - people often use Docker as just a packaging format of their pre-existing software and don't really want to invest in the tooling around it too much, such as having a formal schema of all of the allowed configuration values and iterating through it and checking against all of the incoming values and the ones stored in the configuration files. It's difficult and I mostly only do that for my private software, because getting the buy in for that at work is slow, although there is some promise there finally.\u003c/p\u003e\n\u003cp\u003eTo Gitea's credit, they have at least done some of the things to inform you about what you need to do, even if you're still on the hook yourself for checking the self check dashboard after bumping a version, or even know that it exists. They're setting you up for failure, because human beings are fallible, but at least they \u003ca href=\"https://links.kronis.dev/j7qxr\"\u003edocument\u003c/a\u003e what changes you might need to do:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04-docs-mention-breaking-changes.jpg\" alt=\"04-docs-mention-breaking-changes\" title=\"04-docs-mention-breaking-changes\"\u003e\u003c/p\u003e\n\u003cp\u003eOddly enough, the page mentions that the app shouldn't start with bad config, yet it does and just trudges along. It's also quite unfortunate, because the actual configuration that I needed to change was very trivial in pretty much all of the cases, to the point where you could easily migrate it over with a Bash script or a Python script (or even in Go), just iterating over the old config and fixing it:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"05-fixing-configuration-manually.jpg\" alt=\"05-fixing-configuration-manually\" title=\"05-fixing-configuration-manually\"\u003e\u003c/p\u003e\n\u003cp\u003eFor example, couldn't you just take \u003ccode\u003eserver.LFS_CONTENT_PATH\u003c/code\u003e and transform that into \u003ccode\u003elfs.PATH\u003c/code\u003e? And if there is no \u003ccode\u003eSTORAGE_TYPE\u003c/code\u003e set, then just set it to \u003ccode\u003elocal\u003c/code\u003e, which corresponds to the previous configuration. You can be doubly sure, if you see that the actual path value is the default value of \u003ccode\u003e/data/git/lfs\u003c/code\u003e, meaning that you're unlikely to ever need to consider whether migrating it could cause further issues:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"06-lfs-configuration.jpg\" alt=\"06-lfs-configuration\" title=\"06-lfs-configuration\"\u003e\u003c/p\u003e\n\u003cp\u003eOf course, there were some other issues, that refused to go away, even after me setting the correct parameters. Apparently, my mail configuration was all wrong, even if the values mentioned here were set NEITHER in the Docker environment variables, nor in the actual configuration file:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"08-won't-go-away.jpg\" alt=\"08-won't-go-away\" title=\"08-won't-go-away\"\u003e\u003c/p\u003e\n\u003cp\u003eThat's a bit silly, especially given that the actual e-mail functionality kept working with no issues:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"07-emails-work.jpg\" alt=\"07-emails-work\" title=\"07-emails-work\"\u003e\u003c/p\u003e\n\u003cp\u003eOf course, fixing the configuration wasn't enough, because by now the damage was already done.\u003c/p\u003e\n\u003ch3\u003eSoftware silently failing is the worst\u003c/h3\u003e\n\u003cp\u003eIf any software fails silently, such as \u003ca href=\"https://links.kronis.dev/xihmr\"\u003eMySQL quietly truncating values\u003c/a\u003e in the older versions, or even something like running non-ECC memory and getting corruption issues in your backup archives, then by the time you realize that something is wrong, it might already be too late. I guess in this case I got pretty lucky, because I caught it all early on, but this more or less meant that I had to go through my local repositories and push all of Git LFS files again, to make sure that they're actually on the server, since I did bump the server version a while back.\u003c/p\u003e\n\u003cp\u003eLuckily, Git LFS actually has \u003ca href=\"https://links.kronis.dev/iu545\"\u003ea command\u003c/a\u003e for that: \u003ccode\u003egit lfs push [remote] --all\u003c/code\u003e\u003c/p\u003e\n\u003cp\u003eIt actually works pretty well, which is unlike 90% of my experiences with Git LFS:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"09-fixing-the-lfs-files-manually.jpg\" alt=\"09-fixing-the-lfs-files-manually\" title=\"09-fixing-the-lfs-files-manually\"\u003e\u003c/p\u003e\n\u003cp\u003eAfter doing that, I can clone the repository correctly, meaning that the data is on the server:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"10-finally-fixed.jpg\" alt=\"10-finally-fixed\" title=\"10-finally-fixed\"\u003e\u003c/p\u003e\n\u003cp\u003eThe actual problem is knowing which repositories need to be pushed again. The first step is probably to find out which ones use Git LFS in the first place. I actually ended up just turning to ChatGPT for this, to give me a script to iterate through each of my project folders and tell me how widespread the damage is. I do recall using Git LFS for some binary assets like images (supposedly a best practice) and some GameDev projects, but I really don't want this to be a case of data loss due to me forgetting something:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"11-detect-git-lfs.jpg\" alt=\"11-detect-git-lfs\" title=\"11-detect-git-lfs\"\u003e\u003c/p\u003e\n\u003cp\u003eUnsurprisingly, the first iteration actually didn't work, so I had to ask for some more specific changes, such as checking for whether the output is empty. Since the task itself is simple, but some of the valid solutions, such as \u003ccode\u003egit lfs track | grep -q '.'\u003c/code\u003e (due to it printing the tracked file types) feel not \u003ca href=\"https://links.kronis.dev/bpz2o\"\u003eimmediately obvious\u003c/a\u003e, this was actually a really good use case for an LLM:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"12-ai-needed-checking.jpg\" alt=\"12-ai-needed-checking\" title=\"12-ai-needed-checking\"\u003e\u003c/p\u003e\n\u003cp\u003eIn other words, we pick up on the output in the following format:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003e$ git lfs track\nListing tracked patterns\n    *.png (.gitattributes)\n    *.jpg (.gitattributes)\n    *.jpeg (.gitattributes)\n    *.gif (.gitattributes)\n    *.mp4 (.gitattributes)\n    *.ico (.gitattributes)\n    *.xcf (.gitattributes)\n    *.phar (.gitattributes)\n    grav\\bin\\* (.gitattributes)\n    *.jpg (.gitattributes)\n    *.jpeg (.gitattributes)\n    *.png (.gitattributes)\n    *.gif (.gitattributes)\n    *.mp4 (.gitattributes)\n    *.ico (.gitattributes)\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eThe script worked pretty nicely and soon I knew that none of the other projects in that directory needed further actions, but I did find a few in the GameDev project directory:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"13-checked-local-scripts.jpg\" alt=\"13-checked-local-scripts\" title=\"13-checked-local-scripts\"\u003e\u003c/p\u003e\n\u003cp\u003eThat was all I needed to do, to solve this issue.\u003c/p\u003e\n\u003ch3\u003eSummary\u003c/h3\u003e\n\u003cp\u003eOverall, I'm pretty happy that I could figure this out, instead of realizing that something is very wrong like a year down the line, when the local files would no longer be present on my PC (especially because I do intend to reinstall the OS soon), but it's rather unfortunate that I had this issue in the first place, because I definitely shouldn't. I still like Gitea, but this just goes to show that almost no software will put that much care into the configuration management.\u003c/p\u003e\n\u003cp\u003eTruth be told, people won't always care about the details, which is inevitable, so you should just bite the bullet and make the machine care, put that care into the code that will be run regardless of who runs it and how.\u003c/p\u003e\n\u003cp\u003eAlso, I've had issues with software like Nextcloud in the past where it would refuse to update properly, but at least in those cases it failed fast and printed exactly what was wrong in the logs. Not that it wasn't annoying, because it was and it inspired the \u003ca href=\"https://links.kronis.dev/jlvte\"\u003eSoftware updates as clean wipes\u003c/a\u003e post, but I will take that instead of silent data corruption any day of the week.\u003c/p\u003e\n\u003cp\u003eI will keep using Gitea because it's pretty good for my needs, but this is definitely something to keep in mind, especially for the software that we write ourselves.\u003c/p\u003e\n"
        },
        {
            "title": "Docker error messages are pretty cryptic sometimes",
            "date_published": "2025-02-17",
            "id": "https://blog.kronis.dev/blog/docker-error-messages-are-pretty-cryptic-sometimes",
            "url": "https://blog.kronis.dev/blog/docker-error-messages-are-pretty-cryptic-sometimes",
            "content_html": "\u003cp\u003eWith the new version of the blog up and running, let's complain about technology again for a little bit!\u003c/p\u003e\n\u003cp\u003eI use Docker a lot for development. Nowadays, most of the software I make and want to deploy on servers, whether for personal projects, or at work, is containerized. I take a Git repository, build a container image from that and then can launch it wherever I need, with the exact versions of dependencies that it was built with. There are quite a few benefits to this:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eno more issues with slightly different runtime versions, I've had \u003ca href=\"https://links.kronis.dev/9amb1\"\u003eproblems due to that in the past\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003eno more human factor related risks, when it comes to setting up the app instance\u003c/li\u003e\n\u003cli\u003ethe ability to set resource limits (e.g. CPU, RAM), so that the app plays nicely with others and can't bring the whole server down\u003c/li\u003e\n\u003cli\u003ea common way of configuring the environment variables, storage, networking and a lot of other things\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eYou don't always need containers per se (and I shipped some stuff running directly in the OS as a systemd service last week), but when you do, the DX (developer experience) is generally quite pleasant. I would also say that this was one of the selling points of Docker in particular, even if \u003ca href=\"https://links.kronis.dev/uvoe3\"\u003econtainers existed long before it\u003c/a\u003e. Even when technologies like Kubernetes are sometimes more trouble than they are worth.\u003c/p\u003e\n\u003cp\u003eBut this isn't about Kubernetes, not at all. It's about something a bit more fundamental breaking down, since yesterday I started randomly seeing some errors in the logs, when building some containers:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"01-no-such-file-or-directory.jpg\" alt=\"01-no-such-file-or-directory\" title=\"01-no-such-file-or-directory\"\u003e\u003c/p\u003e\n\u003cp\u003eSo, I basically got:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003eno such file or directory\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eThat'd suggest that I've messed up something in the Dockerfile, right? Well, not really, not in this case, because the very same pipeline would other times throw this:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02-failed-to-compute-cache-key.jpg\" alt=\"02-failed-to-compute-cache-key\" title=\"02-failed-to-compute-cache-key\"\u003e\u003c/p\u003e\n\u003cp\u003eHere's the other error (hey SEO):\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003efailed to compute cache key: unlazy requires an applier\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eNow, the obvious question is:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eWhat am I even looking at?\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eI have no idea what unlazy is or why it'd break the Docker caching mechanism. Even the Dockerfile was very simple, there's literally not a lot that could go wrong there:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003eFROM ...\n\n# Disable time zone prompts etc.\nARG DEBIAN_FRONTEND=noninteractive\n\n# Time zone\nENV TZ=Europe/Riga\n\n# Use Bash as our shell\nSHELL [\u0026quot;/bin/bash\u0026quot;, \u0026quot;-c\u0026quot;]\n\n# We don't need the default files, just the directory\nRUN rm -rf /var/www/html \u0026amp;\u0026amp; mkdir -p /var/www/html\n\n# Copy over Grav files\nCOPY --chown=33:33 ./grav /var/www/html\n\n# Copy over config that we will use for Grav by default\nCOPY ./apache/etc/apache2/sites-enabled/000-default.conf /etc/apache2/sites-enabled/000-default.conf\n\n# Setup entrypoint\nCOPY ./apache/docker-entrypoint.sh /docker-entrypoint.sh\nRUN chmod +x /docker-entrypoint.sh\nCMD \u0026quot;/docker-entrypoint.sh\u0026quot;\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eThe base image that I use does have some files under \u003ccode\u003e/var/www/html\u003c/code\u003e for the purposes of testing out whether the configuration has been setup correctly. Having to remove those for production builds isn't very efficient because it creates a spare Docker layer that we don't really need, but isn't necessarily the worst thing ever, either:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"02-layers.jpg\" alt=\"02-layers.jpg\" title=\"02-layers\"\u003e\u003c/p\u003e\n\u003cp\u003eBut why would that break, especially when building everything locally is perfectly okay and similar setups have worked with no issues in the past? My first idea was to take the Dockerfile and temporarily remove the cleanup and see if things start working better without it:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003e# We don't need the default files, just the directory\nRUN rm -rf /var/www/html \u0026amp;\u0026amp; mkdir -p /var/www/html\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eIf that's the part that's causing the issues, then surely this should fix it? Well, no, it didn't:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"03-still-get-the-error.jpg\" alt=\"03-still-get-the-error\" title=\"03-still-get-the-error\"\u003e\u003c/p\u003e\n\u003cp\u003eSo after the changes, we get the same issue but in a different location. The same happens if I remove further instructions, which is really odd. There simply isn't a lot that could go wrong with the simple COPY instruction, which more or less confirms that the issue lies with the Docker environment itself (or perhaps the cache in particular), instead of what's in the file:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"04-other-commands-randomly-fail.jpg\" alt=\"04-other-commands-randomly-fail\" title=\"04-other-commands-randomly-fail\"\u003e\u003c/p\u003e\n\u003cp\u003eSome comments online suggested that it could be related to the \u003ccode\u003e.dockerignore\u003c/code\u003e file, but it still fails even after removing the file altogether:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"05-empty-dockerignore-doesn't-help.jpg\" alt=\"05-empty-dockerignore-doesn't-help\" title=\"05-empty-dockerignore-doesn't-help\"\u003e\u003c/p\u003e\n\u003cp\u003eWhat did seem to help, was not running multiple builds in parallel, but instead running them one at a time:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"06-one-at-a-time-fixes-it.jpg\" alt=\"06-one-at-a-time-fixes-it\" title=\"06-one-at-a-time-fixes-it\"\u003e\u003c/p\u003e\n\u003cp\u003eExcept that's also a lie, because the very same commit sometimes works and sometimes doesn't:\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"07-inconsistency.jpg\" alt=\"07-inconsistency\" title=\"07-inconsistency\"\u003e\u003c/p\u003e\n\u003cp\u003eFrankly, I don't really know what to do here.\u003c/p\u003e\n\u003ch3\u003eSummary\u003c/h3\u003e\n\u003cp\u003eI think the most annoying thing in cases like this, is that we don't really know exactly what the problem is. Some troubleshooting can help us narrow it down somewhat, but at the end of the day, it's still pretty much like the absolutely useless \u0026quot;Unspecified error\u0026quot; messages that you can get from Windows or other software, where the developers didn't really care that much about helpful error messages.\u003c/p\u003e\n\u003cp\u003eI fear that the real solution might just involve wiping the server and setting it up anew. Best case, I'll have to wipe the cache and that'll help it, but because that's done automatically and didn't seem to help either (the problem persists today as it did yesterday), I wouldn't count on it. There were also some networking changes that I did, but honestly those really shouldn't be at fault.\u003c/p\u003e\n\u003cp\u003eOtherwise, I have no idea because it's not like the software is trying to be helpful.\u003c/p\u003e\n"
        }
    ]
}