{"version":"https:\/\/jsonfeed.org\/version\/1","title":"blog.kronis.dev feed","home_page_url":"https:\/\/blog.kronis.dev\/tutorials","feed_url":"https:\/\/blog.kronis.dev\/tutorials.json","description":"A list of the latest posts on the blog at https:\/\/blog.kronis.dev","author":{"name":"Kristi\u0101ns Kronis"},"items":[{"title":"Self-hosting an AI (LLM) chatbot without going broke","date_published":"2023-06-06T18:20:00+03:00","id":"https:\/\/blog.kronis.dev\/tutorials\/self-hosting-an-ai-llm-chatbot-without-going-broke","url":"https:\/\/blog.kronis.dev\/tutorials\/self-hosting-an-ai-llm-chatbot-without-going-broke","content_html":"
In the recent years, the efforts of OpenAI<\/a> and other organizations haven't gone unnoticed. While there's definitely controversy regarding things like copyrights and intellectual property, at the same time the results speak for themselves - many out there are exploring the use cases for generative AI applications, to aid in anything from writing and doing programming, to creating art. <\/p>\n Sometimes, it works pretty decently, like how you can use something like GitHub Copilot<\/a> for boilerplate code with decent success, or something like Midjourney<\/a> for creating anything from sketches, placeholder art or even what some would describe as proper art pieces. Other times, it causes lots of issues, like when the Clarkesworld sci-fi\/fantasy magazine was spammed with AI generated submissions<\/a>, due to many utilizing the AI for their own personal benefit, and not really thinking about the potential consequences of such irresponsible behavior.<\/p>\n Regardless of how you feel about these forms of AI (or whether they should even be called that), I'm curious about something else: is it possible to run our own, on our local device, without going broke? <\/p>\n <\/p>\n You see, almost nobody out there actually runs their own LLMs locally, instead almost everyone is okay with building their entire business on a cloud API that they don't control. In other words, they are using Services as a Software Substitute<\/a>, and are fully at the mercy of whoever runs them - if OpenAI decides to increase their prices 2x tomorrow,...<\/p>","image":"\/user\/pages\/03.Tutorials\/self-hosting-an-ai-llm-chatbot-without-going-broke\/logo.jpg"},{"title":"Increase container build speeds when you use apt","date_published":"2023-02-22T11:46:00+02:00","id":"https:\/\/blog.kronis.dev\/tutorials\/increase-container-build-speeds-when-you-use-apt","url":"https:\/\/blog.kronis.dev\/tutorials\/increase-container-build-speeds-when-you-use-apt","content_html":" It's no secret that I use containers for most of my workloads, due to the reproducibility and standardization that they provide. I've also gone a step further than most folks, since I base all of my container builds on Ubuntu<\/a>, actually getting pretty close to the old joke of: \"If it works on your machine, we'll just have to ship it.\" In other words, if I run Ubuntu locally and build my containers by installing packages with apt instead of various \"hacks\" like adding pre-built binaries, then I can get pretty much the same development setup, even without using containers for local development (remote debugging is sometimes a pain).<\/p>\n However, while I avoid the issue of using something like Alpine Linux<\/a> (which I still think is great) for my containers and having to \"pretend\" that it's going to be close enough to a local dev setup, until I run into some issues<\/a>, there is something that still need a bit of talking about: performance. So, how do I keep my build times reasonable, even without optimized distros like Alpine, all in the name of simplicity, consistency and productivity?<\/p>\n <\/p>\n It's actually not that hard, so let's jump in.<\/p>\n For starters, here's a quick refresher on the setup. Instead of using ready made images with a mishmash of various distro versions and customizations from Docker Hub<\/a>, I need just the base Ubuntu image, which I then further customize...<\/p>","image":"\/user\/pages\/03.Tutorials\/increase-container-build-speeds-when-you-use-apt\/01_summary.JPG"},{"title":"How to add a wildcard SSL\/TLS certificate for your K3s cluster","date_published":"2023-01-27T14:44:00+02:00","id":"https:\/\/blog.kronis.dev\/tutorials\/how-to-add-a-wildcard-ssl-tls-certificate-for-your-k3s-cluster","url":"https:\/\/blog.kronis.dev\/tutorials\/how-to-add-a-wildcard-ssl-tls-certificate-for-your-k3s-cluster","content_html":" Here's a short tutorial that I felt could be useful for some people, but didn't get around to actually writing previously. Suppose that you have a project that uses Kubernetes and you'd like to secure the network traffic for the services that you expose through your ingress to your users:<\/p>\n <\/p>\n So essentially what you'd like to do is make it so that any of your services can securely communicate to your users, in this particular case by using a wildcard SSL\/TLS certificate, like But first, a few words about when and why someone might choose K3s in particular.<\/p>\n In this case, you're not running just any Kubernetes distro out there, suppose that you have a Kubernetes cluster that uses the excellent K3s distribution<\/a>. <\/p>\n It's certified and therefore will work with many of your workloads with few surprises, but at the same time is wonderfully lightweight and portable across different OSes. I actually benchmarked it against Docker Swarm<\/a> as a part of my Master's Degree thesis back in the day, and the resource usage was pretty close, which...<\/p>","image":"\/user\/pages\/03.Tutorials\/how-to-add-a-wildcard-ssl-tls-certificate-for-your-k3s-cluster\/diagram.jpg"},{"title":"What to do when a disk dies","date_published":"2022-11-13T16:35:00+02:00","id":"https:\/\/blog.kronis.dev\/tutorials\/what-to-do-when-a-disk-dies","url":"https:\/\/blog.kronis.dev\/tutorials\/what-to-do-when-a-disk-dies","content_html":" For the past few years, I've been running my own hybrid cloud: some of my software runs in the cloud in servers that I rent from Time4VPS<\/a>, however a lot of my other software (CI nodes, backups, chat applications) is running locally, in my homelab servers. <\/p>\n This is excellent both because it allows me to remain in control of my own data, as well as because it's more affordable when I might have to deal with multiple TB of data - buying a regular 1 TB HDD costs me around 40 euros, whereas getting that amount from the cloud might be far more expensive (even if I get multiple disks for redundancy myself). In addition, it also helps me avoid contributing to e-waste too much, because my homelab servers are running off of my old CPUs and other parts.<\/p>\n However, the downside of this approach is the fact that all of the sudden I'm responsible for keeping everything running. This is a story of one such occasion, when a HDD on one of the servers died. Initially, it looked like it was actually up, with the power LEDs being lit up normally and the PSU fans spinning and whatnot, however a glance at my Zabbix<\/a> monitoring system revealed the problem:<\/p>\n <\/p>\n I've scheduled the nodes to hibernate during the night when I sleep and don't need them, however it would appear that one of them didn't start up after one such occasion. It...<\/p>","image":"\/user\/pages\/03.Tutorials\/what-to-do-when-a-disk-dies\/current setup.jpg"},{"title":"(4\/4) Pidgeot: a system for millions of documents; Testing and Conclusions","date_published":"2022-10-21T10:16:23+03:00","id":"https:\/\/blog.kronis.dev\/tutorials\/4-4-pidgeot-a-system-for-millions-of-documents-testing-and-conclusions","url":"https:\/\/blog.kronis.dev\/tutorials\/4-4-pidgeot-a-system-for-millions-of-documents-testing-and-conclusions","content_html":" You can read the previous part here: (3\/4) Pidgeot: a system for millions of documents; Deploying and Scaling<\/a><\/p>\n Thanks to the work we did previously, we have everything we need for actually testing this system in place:<\/p>\n Figuring out how to test and what we want to test is actually the hard part here. Some might be tempted to just run load tests against something like this and it's not like I disagree with such an approach. I've successfully used K6s<\/a> in the past, for my Master's degree.<\/p>\n Yet this time, I care a bit more about how individual requests are served, how quickly document versions and such can be returned, how quickly data can be looked up,how it actually looks under the hood and whether partitions are actually useful or not.<\/p>\n For something like that, starting out with testing the web API seems like the reasonable thing to do, which also has the added benefit of being both realistic and easy.<\/p>\n For this, we'll use Postman<\/a>, a program for interacting with APIs. I...<\/p>","image":"\/user\/pages\/03.Tutorials\/4-4-pidgeot-a-system-for-millions-of-documents-testing-and-conclusions\/00_pidgeot_logo.jpg"},{"title":"(3\/4) Pidgeot: a system for millions of documents; Deploying and Scaling","date_published":"2022-10-21T10:01:50+03:00","id":"https:\/\/blog.kronis.dev\/tutorials\/3-4-pidgeot-a-system-for-millions-of-documents-deploying-and-scaling","url":"https:\/\/blog.kronis.dev\/tutorials\/3-4-pidgeot-a-system-for-millions-of-documents-deploying-and-scaling","content_html":" You can read the previous part here: (2\/4) Pidgeot: a system for millions of documents; The Application<\/a><\/p>\n In the previous parts, we setup PostgreSQL and MinIO for our document storage system, as well as wrote a Java API to glue them together. We also implemented some logic to generate test data, however now we actually need to get all of this up and running somewhere, so that I can generate a reasonable amount of data for testing!<\/p>\n The easiest way to get things to be portable by far, is to use containers, which will include the runtimes that any of my apps need, as well as allow for consistent and easy configuration, logging, resource management and so on. <\/p>\n I generally build my own containers when applicable and store them in my Nexus<\/a> instance, to lessen my reliance on Docker Hub<\/a> and also improve how consistent my containers are, to know how everything works inside of them.<\/p>\n In this case, I don't even need to build the PostgreSQL or MinIO containers separately, because they are already in my Nexus registry (which is mostly private for now), however you can use the Bitnami container images<\/a> for this. <\/p>\n All I need is to launch them in some container stack\/deployment and connect the application\/dbmate to them. For that, I already have a Docker Compose file:<\/p>\n You can read the previous part here: (1\/4) Pidgeot: a system for millions of documents; The Data<\/a><\/p>\n In the previous part, we settled on PostgreSQL and MinIO for our document storage and versioning solution, with the aim of creating something that's capable of storing millions of documents with very little in the way of issues. But now, let's crate an API that'd actually realistically interact with the database and blob storage solution!<\/p>\n Here, I think I'll go for Java with Dropwizard<\/a>, a pretty nice alternative to Spring<\/a> in my eyes: it's pretty lightweight and simple to get up and running with.<\/p>\n Many might reach for Spring Boot and while it's a good enterprise solution, it's also a bit of a heavyweight that's full of \"magic\", at least in my opinion. In contrast, Dropwizard runs on Jetty and lets you actually pick how much complexity you want in your project, based on your needs, although it might not cover 100% of them, unless you feel comfortable stitching together libraries.<\/p>\n There are more projects like that in the category of \"micro frameworks\", instead of \"platform frameworks\", around which you'd build your entire business or system. I'd say that in many cases the smaller options are good enough for both lightweight APIs, as well as entire ERP systems, though building systems out of many smaller (and simpler) components personally feels worthwhile.<\/p>\n Of course, I'm obligated to say that Java is...<\/p>","image":"\/user\/pages\/03.Tutorials\/2-4-pidgeot-a-system-for-millions-of-documents-the-application\/00_pidgeot_logo.jpg"},{"title":"(1\/4) Pidgeot: a system for millions of documents; The Data","date_published":"2022-10-21T02:21:29+03:00","id":"https:\/\/blog.kronis.dev\/tutorials\/1-4-pidgeot-a-system-for-millions-of-documents-the-data","url":"https:\/\/blog.kronis.dev\/tutorials\/1-4-pidgeot-a-system-for-millions-of-documents-the-data","content_html":" Let me set the scene: you need to process a large amount of documents and have some basic versioning for them, quite possibly for a project at work. The specifics aren't even as important as the overall domain, you can make up whatever feels suitable for such a scenario. There are numerous situations like that out there in the wild: be it working with generated PDF invoices, having a document management system, some sort of batch reporting, processing images and other media content that your users upload, perhaps something along the lines of an ETL pipeline or anything else, really.<\/p>\n Yet, I've seen many pitfalls with such solutions, both personally and through the experiences of others: sometimes there are issues with performance, other times the technologies are hard to work with, sometimes the whole system is brittle and can break.<\/p>\n That made me want to experiment with my own proof of concept, or rather something a bit like what I made for \"Apturi Covid\"<\/a> a while back. Yet, back then, the solution was simpler and was primarily used for the storage of static assets, whereas I want to test things at a larger scale. That's how I came up with the idea of Pidgeot, a system built just so I could play around with some of the technologies and have it act as a proof of concept:<\/p>\n <\/p>\n (the logo was generated with the DALL·E 2 AI<\/a>, the project is...<\/p>","image":"\/user\/pages\/03.Tutorials\/1-4-pidgeot-a-system-for-millions-of-documents-the-data\/00_pidgeot_logo.jpg"},{"title":"Let's run our own CA","date_published":"2022-10-10T10:44:05+03:00","id":"https:\/\/blog.kronis.dev\/tutorials\/lets-run-our-own-ca","url":"https:\/\/blog.kronis.dev\/tutorials\/lets-run-our-own-ca","content_html":" Having our own certificate authority would let us generate certificates for our applications and, upon importing them in our OS or browser, provide secure communications. It would also reduce our dependence on third parties as well as the overall technical complexity of everything, which is especially useful for private homelab setup and self-hosted applications that aren't meant for a wider audience.<\/p>\n Nowadays we often count on Let's Encrypt<\/a> to provide SSL\/TLS certificates for our public sites, so that we can use HTTPS, to make communication between our user's browsers and the web server secure. This eliminates a whole group of man-in-the-middle attacks, for example, someone changing certain page contents (such as executable files that you may download), as well as prevents your ISP from injecting tracking scripts or advertisements into the pages.<\/p>\n And yet, there are cases when you might want SSL\/TLS certificates for sites that aren't publicly available and thus the popular HTTP-01 challenge<\/a> type won't work - you won't be able to get certificates from Let's Encrypt. Now, you might be able to use the DNS-01 challenge<\/a> instead, but it's generally a bit harder to set up, because any automation that you might want to utilize will need to be able to setup the necessary DNS TXT record with your domain registrar.<\/p>\n For example, Apache recently added mod_md functionality<\/a> for getting certificates without the need for certbot<\/a>, which feels like a nice simplification as you...<\/p>","image":"\/user\/pages\/03.Tutorials\/lets-run-our-own-ca\/01 example of root certificates.JPG"},{"title":"How and why to use Apache httpd in 2022","date_published":"2022-07-24T18:51:27+03:00","id":"https:\/\/blog.kronis.dev\/tutorials\/how-and-why-to-use-apache-httpd-in-2022","url":"https:\/\/blog.kronis.dev\/tutorials\/how-and-why-to-use-apache-httpd-in-2022","content_html":" Suppose that you need to set up a web server to either serve some static assets or a web application front end, work as a reverse proxy, or maybe even act as the provider for SSL\/TLS certificates. In the current year, you might consider something like Nginx<\/a>, Caddy<\/a> or even Traefik<\/a> as your software of choice, given their popularity, performance or just how much of the mind share of the industry they've captured.<\/p>\n Nowadays, Nginx has almost become synonymous with reverse proxy or web server, especially given how many Kubernetes clusters out there use it as an ingress. Apache\/httpd<\/a>, on the other hand, has gotten no such love<\/a>. That said, it's kind of unfortunate, because the Apache web server project has been in active development for decades at this point and there are a few advantages to using it:<\/p>\n Let's actually talk about the performance for a second, because it's a common point for people wanting to choose Nginx over anything else (even Caddy).<\/p>\n There's this lovely site called OpenBenchmarking<\/a> that has a variety of benchmarks, including some for Apache and Nginx:...<\/p>","image":"\/user\/pages\/03.Tutorials\/how-and-why-to-use-apache-httpd-in-2022\/nature of results.JPG"}]}
Increase container build speeds when you use apt<\/h2>\n
My current setup<\/h3>\n
How to add a wildcard SSL\/TLS certificate for your K3s cluster<\/h2>\n
*.myapp.com<\/code>. The difference from many other tutorials here is that we are NOT going to be using the ACME approach<\/a> with the likes of Let's Encrypt<\/a>, but rather our own manually provisioned certificate. Personally, I think that Let's Encrypt is great, but in some enterprise settings you'll instead have commercially bought certificates without such ACME automation behind them.<\/p>\n
K3s<\/h3>\n
What to do when a disk dies<\/h2>\n
(4\/4) Pidgeot: a system for millions of documents; Testing and Conclusions<\/h2>\n
\n
How to test<\/h3>\n
Setting up for testing the web API<\/h3>\n
(3\/4) Pidgeot: a system for millions of documents; Deploying and Scaling<\/h2>\n
Technology choices<\/h3>\n
version: '3.7'\nservices:\n pidgeot_dbmate:\n image: docker-dev.registry.kronis.dev:443\/dbmate\n volumes:\n - .\/db\/migrations:\/workspace\/db\/migrations\n environment:\n - DATABASE_URL=postgres:\/\/pidgeot_pg_user:pidgeot_pg_password@pidgeot_postgres:5432\/pidgeot_pg_database?sslmode=disable\n deploy:\n resources:\n limits:\n memory:...<\/code><\/pre>","image":"\/user\/pages\/03.Tutorials\/3-4-pidgeot-a-system-for-millions-of-documents-deploying-and-scaling\/00_pidgeot_logo.jpg"},{"title":"(2\/4) Pidgeot: a system for millions of documents; The Application","date_published":"2022-10-21T02:21:30+03:00","id":"https:\/\/blog.kronis.dev\/tutorials\/2-4-pidgeot-a-system-for-millions-of-documents-the-application","url":"https:\/\/blog.kronis.dev\/tutorials\/2-4-pidgeot-a-system-for-millions-of-documents-the-application","content_html":"
(2\/4) Pidgeot: a system for millions of documents; The Application<\/h2>\n
Technology choices<\/h3>\n
(1\/4) Pidgeot: a system for millions of documents; The Data<\/h2>\n
Let's run our own CA<\/h2>\n
Before we begin, a bit of background<\/h3>\n
How and why to use Apache httpd in 2022<\/h2>\n
Apache, the alternatives and benchmarks<\/h3>\n
\n