Journal from Kotnik

I enjoy reading what other people are doing with their homelab setups, and inspired by recent blog I decided to sit down and write about the state of my own homelab at the start of 2026.

The whole homelab sits in my workshop, located at the basement of our house:

One thing that is obvious is that the setup has evolved. I started building it while we were living in an apartment and I didn't have a lot of space so I started with 10” mini rack. Now, there are two :).

I will not go into too much detail, since, of course, for each element shown here there's a lot of background, or it is performing multiple tasks. So, let us start with the equipment that is outside.

Eaton 3S 850 DIN

Everything is behind this UPS, you can see it down on the left, and with current idle usage of 180W it can keep it running for about 18 minutes. That is enough when I am dealing with electricity. My plan is to replace it with LiFePO4 based UPS once they became more available.

Telekom.de Speedport Smart 4 Plus

Internet router. Currently vDSL, but cabling for fiber-to-home is almost ready. Don't get me started about the state of German infrastructure :)

Nabu Casa Z-Wave Long Range ZWA-2

So Z-Wave devices can talk with my Home Assistant setup. It's the white stick with blue light on the top.

Sonoff Zigbee 3.0 USB Dongle Plus

For all Zigbee devices. Most of sensors in our home talk via Zigbee network and this little fellow you can see to the right of the Telekom.de router handles it all with grace.

Now let us peek what's inside the top rack, starting from the top. At the bottom of each rack are electricity plugs, those gray blocks. I like to keep them inverted. Each rack also has temperature/humidity sensor which are barely visible on the photos.

Mikrotik RB5009

This is the network backbone. This awesome little device routes all the traffic and gives power to 4 Mikrotik cAP ax access points, one for each floor, that are not shown here.

Mikrotik CSS610

Active switch. It is there because RB5009 doesn't have enough ports :) It tags certain traffic, for example from outside surveillance cameras, otherwise it would be replaced with very cheap one. It is very hard to buy crappy switch these days.

Blackview MP80 – Frigate

Mini PC that runs Frigate NVR, and only that. This Frigate setup has 8 active cameras and even though it has little Google Coral edge TPU accelerator for ML tasks it is constantly around 40-60% CPU usage.

Blackview MP80 – Home Assistant

I installed Home Assisant OS on it's own dedicated Mini PC. It pretty much runs the house, and more than 200 devices are reporting it. Still, it is usually around 2% CPU usage. I didn't expect that, and if I knew I'd have it as Proxmox VM. But it is what it is, for now.

Now we move to the bottom rack.

Synolgoy DiskStation DS115j

This was my first NAS. It has been running for more than 10 years with the same 2TB disk inside. I don't use it for anything important, but I like to have it around, it's family friend after all this time.

Synology DiskStation DS220j

This is the main NAS and it has 2 x 6TB disks inside. All the backups are kept here, from our computers, to network devices. Of course, we keep off-site backups, but only for the things I deemed important. I'll just write we use Hetzner's storage box, without going into too much details.

Lenovo ThinkCentre M72e

Mini PC that runs experiments and most of the Docker Compose based services. From Immich that holds all our photos, to my son's Minecraft and Luanti servers. I had two of these, but one died. It is over 10 years old, but still run really well.

HP Elitedesk 705 G4

Another cheap Mini PC I bought on Ebay. This one is running Proxmox, and in it a bunch of VMs, from our main DNS server to OpenBSD based PKI infrastructure and Zabbix that monitors all of this.

And that is it. I have a lot of plans for my homelab this year (hello LLMs!) and now I am happy I wrote this down so I can compare later. Happy hacking!

I've been pretty much driving the whole day today. I am using Google's Maps to help me with navigation and, for those that haven't been using it, it is getting worse and worse. So, since there's not much else to do I was thinking a lot about that fact and I have some half-baked conclusions that I'd like to share with you.

For me, Google Maps used to be rock-solid piece of software that I could rely on. An important part of the infrastructure of the world that most people have been using for free. Not me though, we as a family pay two subscriptions to Google, so I can assume we're actually also paying for Maps as well.

But lately, some few months ago, it's been really buggy. For example, this is what was displayed frequently during today's trip:

Sometimes road colors would change for no particular reason too. Or notifications from Waze would pop-up sooner than they should. The trend here is worrying. If you follow the news you might be aware of recent chaos Google Maps did in Germany, and while some people love to see malicious hackers in everything, I think this is just the part of the trend I've been observing.

And that trend is the idea I am trying to pass here: I think that the path for new companies, and start-ups, is opening rapidly and we will soon have opportunities we didn't imagine could exists.

Let me elaborate a bit.

Google is not a cool company it used to be. I know people that have quit their job 10 years ago to take a stab at getting hired at Google, and it used to be sign of exceptional engineering to be there, but that's not the case anymore. You see, software code is not main thing companies are built around. Not even close. The main thing are the people, the knowledge they share and the culture they build. And I think that nowdays people at Google just don't care. I really can't blame them.

It's the same with other big technology companies. With original founders retiring, leaving or simply going completely crazy, companies struggle to keep the same level of software quality. The same level of care.

The window for the stories similar to Microsoft vs. IBM is opening, I believe, and I am really happy about that.

I read books modestly, dedicating most of my time to technical documentation and contemporary short articles (a blog post of the best from those is incoming). And what I do read I give my best to pick up those books that would interest me the most, and quickly give up if it doesn't hook me in the first 50 pages or so. You may link to me on BookWyrm, or any other Fediverse hub using the same handle: @kotnik@bookwyrm.social.

Fiction

• R. F. Kuang, Babel. A story about a cohort of translation students with deep discussion on language, but also significantly overlapping antrophology, society and politics. One of the best character developments I have read in years, deep into the book I had built complete profiles of each character and I could actually see and hear them.
• Andy Weir, Project Hail Mary. Modern and realistic sci-fi that takes us with intergalactic mission world saving mission, but generously enriched with humor and a story of an unlikely friendship. I very much welcomed optimism of humanity: I think we all need more of that.

Nonfiction

• Benjamín Labatut, The Maniac. A story about János Lajos Neumann, better known as John von Neumann, who was probably, though that is very hard to measure, the most versatile and intelligent human being that ever existed. But also very controversial. It is incredible how much of modernity has been founded and established by this person alone and this book allows us an insight into his personality with lightly fictionalized events from his life.

I visited an apartment I used to live a long time ago and picked up some of my old stuff. Among them was a computer magazine Svet Kompjutera (World of Computers) I used to buy religiously. It's from April 1988, full 36 years ago. It was really fun to read it from this perspective.

The price was 1000 Yugoslavian dinars. That country does not exist anymore, and I have no idea what the price would look like today.

It featured computing lessons. This part was about programming and it was dry as you can imagine, going over boring algorithm schemas nobody uses anymore. I used to learn it old school like this, and I am happy to report learning resources are much more better nowdays. Progress!

IBM's dBase III was all the rage in 1988.

But now into fun stuff. They organized a competition of chess games, they weren't even called engines at the time, and then, as it is now, CPU power was everything so from this stage 8 bit computers were separated from 16 bit ones, since the latter were significantly more powerful. I had my horse in the game: Colossus Chess on C64. I think it's my most frequent opponent for my lifetime, and at tender age of 7, it sure did beat me most times.

Two biggest platforms at that time were ZX Spectrum 48 and Commodore 64. The magazine had assembly language programming tricks, and here you can see some scrolling effect for Spectrum. I never learned to program Z80 processor, but I can read and understand it's assembly thanks to this magazine.

Here you can see complete map of the game. These were immensely helpful and generating them required skill and hacking tricks of its own kind.

Now onto some serious PTSD! We used to type in assembly programs like this, a stream of seemingly endless DATA lines with POKE instructions that were checksumed at the end, and if you made the slightest mistake you'd have to re-type it from scratch again. Here they gave us 16KB big RAM disk on C64. Playing with these routines in monitor was source of countless fun weekends for me.

Here's classifieds section for your viewing pleasure. Software piracy was legal and you could order any game for a fraction of price. Not that we didn't want to buy originals, they simple weren't available on our socialist Yugoslavia market (we did have a market of sorts). So, games would be compressed and sold in compilations like this.

And here is the exact time I found out about Life algorithm. Some years after this I would become obsessed with it, figuring stuff on my own without having any guidance or knowing what the rest of the folks were doing since I didn't have access to any computer network.

Finally, the last page of the magazine. This is what peek PC was at the time. I am still amazed that it won computing. At that time we had multimedia power machines like Amiga and Atari that had multi-tasking operating systems and were able to display colorful images accompanied by nice music. And if you to go back in time and tell me they would lose, that they would be utterly and completely destroyed by a dull, gray, monochrome beeping box sold to accountants?! I would consider you mad.

Bonus

Finally, here's a treat for #RetroGaming fans: top games in Yugoslavia at the end of 1994. Remember, they were all almost free to us, and there was no marketing – so these lists were pure and trusted.

I've been a bit under the weather lately so, in order to distract myself a little bit, I took on finishing a few projects that have been laying dormant.

As chronic self-hoster who's been self-hosting stuff for decades, I need to keep up with what's running and what is not. For a long time I used Statping, but recently I switched to Uptime Kuma, mainly because it is more maintained and also faster. There was only one piece of the puzzle missing... So, here it is now, all published now: Uptime Kuma for Homer. In case you are using Homer dashboard, now you can display stats from your Uptime Kuma instance:

• On Github, or
• over here on my own personal Git server.

It's a small Python project that uses Flask framework to bridge two services and make them interact. I wrapped it up quickly – frankly, writing README.md file took more time – but during the build I was thinking how things are slowly getting out of hand. Even for a small project like this, one needs to have a lot of context and knowledge of many different technologies. Just to name a few:

• Python programming language,
• but also Markdown.
• Containers, especially the Docker implementation.
• How Git works, too. The project lives on two repos that sync.
• REST services, a bit of Prometheus.
• YAML too.

So, maybe, just maybe, all this hype regarding AI LLMs could be an attempt to handle this in the longer run?

I have always been an exceptionally geeky person and somewhat of a hardware hoarder. This post is documenting all the laptop computers I am currently using.

Lenovo Thinkpad X13. This is my current primary laptop. It runs on #Archlinux and I use it as my main driver. Everything is supported and just works. The computer itself is joy to use: the build feels great and I really like this keyboard as well.

Lenovo Thinkpad T430s. My fallback laptop. It's rather old and all scratched out but I really like it. It runs on #FreeBSD and all hardware is supported. Actually, I am right now using it to write this post. I use it for my #retro-computing programming as well (I am back coding on 6510 CPU).

Lenovo Thinkpad T480s. My work laptop running #Fedora. And the one that gave me the most headaches. When I purchased it I had to return it back two times since the keyboard was so awful it would make the computer unusable. Literally, using the keys around red nub (the TrackPoint) would stuck it somehow and the cursor would just run that way rendering mouse unusable. Eventually, support fixed the issue but the keyboard is awful and I dread having to use this computer. Luckily, it's only when I am traveling, otherwise I just keep it clean and up to date.

Lenovo Thinkpad X1 Carbon (3rd generation). My old primary laptop currently used for experimenting. I don't like it since the screen is glossy, so whenever I work on it during the day it is as if I am sitting in front of the mirror.

PineBook Pro. Like some people, I also believe that the future of computing is in RISC, and currently ARM architecture represents it the best. So I purchased PineBook Pro. It runs #Armbian. I don't do anything heavily on this computer, but I use it to play with ARM architecture, recompile stuff I worked on somewhere else, and generally as a playground.

MSI GL75. This is my gaming laptop. It runs Windows 10 and is used only for playing games. Actually, mostly by my kids, but since I have fallen with Counter Strike CS:GO, deleted forced Counter Strike 2, and decided to give up on that game, I have discovered that Enemy Territory community still thrives and came back to it.

And that is it.

While replacing living room computer for streaming I ended up with a box with pre-installed #Windows 11. It was my first time with this operating system and I was curious so I decided to see how would it work for this role: stream a few services to connected TV and access Samba shares from local NAS.

After spending almost 4 hours on this tasks I ended up disgusted and horrified, eventually installing #Fedora. What follows is what I found out in this short time.

First of all, default install is utterly bloated with crap you will never need, and some of it you can't even uninstall, like Microsoft Edge browser, Xbox integration, Microsoft's storage solution that loves to send notifications, etc. I know there are hacks for Edge, but system update always brings it back.

And don't even get me started on ads! For example, the first thing I saw when I started the computer was ugly face of the politician I really don't like staring me from news widget. I don't want to get slapped by news when I don't ask for it. Why, oh why, is this a default in any operating system? Mind you, some other photos from news feed are equally bad, and this is unacceptable for living room computer in a family with little kids. Luckily, one can easily disable that awful widget thing.

Finally, and this is the straw that broke the back, you really can't change Windows system language after the install. This was official and activated Windows 11 Pro installation, not a single language Home version, but yet all attempts to change the system language would fail. We are a family that uses four languages at home, and this operating system can simply not support us.

The photo above is result of trying to switch to English language using only available settings, not poking the registry or other hidden buttons. Good work on UI.

Things like this, from annoying ones to the utter stupidity, are the real reason people switch back to Windows 10. Maybe I would do that too in this case, but OEM licenses for it can not be purchased anymore, and I don't want to be stuck in this dead-end environment anymore.

Final goodbye to the Windows, this home is now (almost) Microsoft free.

Update: I had to revert back to Windows 10, since Ryzen 5 integrated graphics is relatively unsupported in Linux and requires frequent reboots in order for it to recognize it is attached to video output.

We have our #Renault Twingo E-Tech electric for almost a year now. We are using almost daily for chores around the town, and sometimes to drive on a highway to nearby city. Here are my impressions.

• This car is very small, but not uncomfortable inside. I can park anywhere, or do a quick u-turn even in smaller streets.
• Regardless of its size, it can quickly accelerate and I found that very useful for quick overtakes or just for plain fun. It's not super powerful, but very noticeable if you come from a gas powered vehicle. But it's not very fast, and it burns a lot of energy on highways because it's not aerodynamic. It's okay for me since I don't need to go faster than 100 km/h anyway.
• Trunk space is ridiculously small. I can fit in the groceries, or kids' school bags, but that's it. It doesn't have a frunk, there is simply no space for it.
• It's equipped with a digital radio, and I've been enjoying that immensely. The type of music I like is not very popular so radio stations playing it are not saturated with ads, and the radio is on most of the times. I can't stand the sound of FM anymore, as a side effect.
• Most of controls are physical buttons and dials. Even turning on the engine is done by turning the key! I really appreciate this and my brain was quickly trained to automatically do stuff without looking. Gear stick is also sensor-less and follows standard P-R-N-D system.
• The car is connected, it has a SIM card built-in and sends a ton of metrics to Renault. We can not access any of it. Yes, you can remotely turn on the climate control, or see the battery level, but that is it. I worry about privacy aspect since that data will most probably be sold, or end up leaking.
• Mainly due to its size, the car is very energy efficient. Judging by its own stats it spends 11.3 kWh per 100 km. That is really, really low. My gas powered vehicle spends 10 times more than that, mostly by sending the heat and pollution in the atmosphere.
• I was surprised by the price of electricity. I was expecting that using EV vehicle would be more affordable, but it really isn't that much. We spend around 60€ per month for electricity.
• Cruise controls and speed limiter are really awful, they are my biggest issue with this car. I use them frequently, especially the speed limiter for urban zones, and hate it every time. To turn it on you use seesaw switch next to the gear stick, but the rest of controls are on the wheel itself: on the left side are +/– and the right of the wheel pause and resume buttons. Just next to them is voice-assistant one, and I frequently hit it by accident. It's a usability nightmare.
• There are no front or side proximity sensors. Cars in these pay-range usually don't have them, but I would really appreciate the side ones. It does have a high quality parking camera and it is very helpful.
• Battery capacity is very small. I knew this before purchase but I still hate to have to recharge the car twice a week. It is designed to be plugged in during the night at home, but we can not do that. Also, forget about advertised autonomy, it's a complete fairy tale. During the summer one can go about 160 km until battery drops below 10%, but during the winter, when you have to have heating turned on, we can count on only 90 km of autonomy. It's a small city car, I understand that, and I happily recharge it frequently, but I would really appreciate battery upgrade option.
• I don't like automatic wipers, it works well if it's really raining, but where we live it usually drizzles and I have to nudge it manually all the time.
• I can control regenerative braking in three levels, and it is great when going downhill. I use the maximum level for deceleration generally, since it is very powerful and I like the feeling of regenerating energy, but a lot of times I worry about cars behind me since the maximum level does not turn on the stop lights even though it can bring a car to almost a halt fairly quickly.

Regardless of all the criticism above we are very happy with our little Twingo and use it all the time. Would we purchase it again? Definitely yes! I am sorry I didn't consider this option before.

I would love the option to upgrade the battery, maybe even after the purchase. I hope that will be possible with newest developments in battery tech. But, I strongly advise having a plug at home, even 220 V one works and car can be fully recharged over night using it. In the meantime we will continue using charging stations on the street and driving our car around the town.

In case you followed my previous posts you already know that ARM architecture should be avoided. It's in early development, things barely work. But, if you are like me: that is exactly what brought you here! So we can continue in the next episode of setting up my #PinebookPro.

But first, #opsec. You really want to fully encrypt your laptop storage, swap included. I think Pine64 is making big mistake by not having an option to do that easily. This post is all about that, if there was an option to secure system then I would happily continue using default Manjaro install.

These are the steps to install Armbian with full-disk (or in this case eMMC) encryption based on very helpful Armbian forum post. Installer does not support this, so you have to do it manually, so let's go.

1) Download Armbian for Pinebook Pro and dd it on microSD. Boot from it using Tow-Boot. I chose Desktop version, but do use CLI option if you want something else than Cinnamon desktop environment.

2) Once in Armbian set up cache directory:

export WORKDIR=/mnt

3) Update your system and install dependencies:

apt update && apt upgrade
apt install cryptsetup-bin gdisk

4) Start installer, when asked choose booting from eMMC and ext4 filesystem (or something else, of course), then exit at the end. Ideally, all this work should be part of installer itself, and eventually it will:

armbian-install || true

5) Now, we want to move installed files and replace the disk with encrypted volume. So, we need to take a copy of data:

mkdir -p ${WORKDIR}/emmcdata
mount /dev/mmcblk2p1 ${WORKDIR}/emmcdata
rsync -a --info=progress2 ${WORKDIR}/emmcdata/. ${WORKDIR}/backup
sync
umount /dev/mmcblk2p1
rmdir ${WORKDIR}/emmcdata

6) Create partition layout. We need two partitions: unencrypted /boot and the rest for encrypted data:

sgdisk -og /dev/mmcblk2
sgdisk -n 1:32768:+512M -t 0:8300 /dev/mmcblk2
sgdisk -n 0:0:0 -t 0:8300 /dev/mmcblk2

7) Create partitions. Simple /boot and encrypted rootfs volume. Notice how we use temporary key for encryption. Don't worry, we'll set up passphrase later. Again, I chose ext4, but you don't have to:

mkfs.ext4 -F -L bootfs /dev/mmcblk2p1
dd if=/dev/zero bs=$((512/8)) count=1 of=/dev/shm/keyfile
cryptsetup luksFormat --batch-mode --cipher=aes-xts-plain64 --key-size=512 --hash=sha512 /dev/mmcblk2p2 /dev/shm/keyfile
cryptsetup open /dev/mmcblk2p2 rootfs --key-file=/dev/shm/keyfile
mkfs.ext4 -L rootfs /dev/mapper/rootfs

8) Mount partitions for sync:

mkdir -p ${WORKDIR}/restore
mount /dev/mapper/rootfs ${WORKDIR}/restore
mkdir -p ${WORKDIR}/restore/boot
mount /dev/mmcblk2p1 ${WORKDIR}/restore/boot

9) Restore installer files from backup at step 5:

rsync -a --info=progress2 ${WORKDIR}/backup/. ${WORKDIR}/restore
sync

10) Tell Armbian not to try to be smart and attempt partition resizing at the first boot, or it will mess up your encrypted volume:

touch ${WORKDIR}/restore/root/.no_rootfs_resize

11) Prepare your new environment:

cd ${WORKDIR}/restore
mount -o rbind /dev dev
mount -t proc proc proc
mount -t sysfs sys sys
cat /etc/resolv.conf > etc/resolv.conf
cat /etc/hosts > etc/hosts
cat /etc/apt/sources.list > etc/apt/sources.list
cat /etc/apt/sources.list.d/armbian.list > etc/apt/sources.list.d/armbian.list

12) Make it aware of proper root filesystem volume:

sed -i '/^bootlogo=/s,=.*,=false,;/^rootdev=/s,=.*,=/dev/mapper/rootfs,' boot/armbianEnv.txt

13) Add active modules to initramfs:

lsmod | cut -d ' ' -f1 | tail -n+2 > etc/initramfs-tools/modules

14) Create crypttab:

echo "rootfs UUID=$(lsblk /dev/mmcblk2p2 --nodeps --noheadings -o UUID) none initramfs,luks" > etc/crypttab

15) Create fstab:

echo "/dev/mapper/rootfs / ext4 defaults,noatime,nodiratime,commit=600,errors=remount-ro 0 1" > etc/fstab
echo "UUID=$(lsblk /dev/mmcblk2p1 --noheadings -o UUID) /boot ext4 defaults,noatime,nodiratime,commit=600,errors=remount-ro 0 2" >> etc/fstab
echo "tmpfs /tmp tmpfs defaults,nosuid 0 0" >> etc/fstab

16) Chroot into your new system and do what installer should be doing in the first place (don't forget, you should be in ${WORKDIR}/restore in case you went to explore around in the meantime):

cat << EOF > config
#!/bin/sh -vx
apt update
echo 'force-confdef' > /root/.dpkg.cfg
apt --yes install cryptsetup-initramfs
rm /root/.dpkg.cfg
lsinitramfs /boot/initrd.img* | grep 'usr.*cryptsetup'
exit
EOF
chmod +x config
chroot . ./config
rm config

17) Set up your final passphrase that will make temporary key obsolete. Use something strong here:

cryptsetup luksChangeKey --key-file=/dev/shm/keyfile --cipher=aes-xts-plain64 --hash=sha512 /dev/mmcblk2p2

18) Umount all filesystems:

umount | awk '/restore/{print $3}' | sort -r | xargs umount 

19) And finally, power-off and remove Armbian installer's microSD card, then start your system. Your data is now protected at rest and you can finally start properly using your Pinebook Pro.

---

Posts in #PinebookPro series:

• My Pinebook Pro
• Installing Tow-Boot on Pinebook Pro
• Armbian on Pinebook Pro

This is the second post in my #PinebookPro series. I plan to document all the steps of making it usable.

The first step is installing an operating system. Your Pinebook will arrive with #Manjaro pre-installed on eMMC. It's a fine operating system, but the system is not encrypted and there is no way to do it post-install. Yes, you can do tricks to have encrypted /home directory, but my #opsec requires full disk encryption.

Pinebook does not support booting off an USB drive, and you can't just insert microSD card with operating system and expect it to work. Boot order is hard-coded: SPI chip, then eMMC, and finally microSD.

Side note: I still didn't figure out what does SPI mean, but basically it's a 16MB disk on chip. It's some ARM relic. It arrives empty, and you can flash it with custom bootloader, which is exactly what we will do here.

But first, you need to make sure pre-installed Manaro isn't booting up the first. In order to do that you need to open your Pinebook and disable the eMMC chip. It's easy, there's an on/off switch on the motherboard just next to it :) And opening your laptop is also very simple since it is designed to be opened and modified.

The chip is on the top right corner, between ARM SoC (CPU, GPU, etc) and the battery. Next to it is a little on/off switch, and I had to peel off a small protection cover in order to turn it off. So, it's off, put back the bottom cover and prepare the boot loader on microSD card.

There are other options, but I chose Tow-Boot. Get the latest release for Pinebook Pro, extract it, flash spi.installer.img onto microSD card, and boot your laptop with it. Select Flash firmware to SPI on the mini-menu, reboot, remove microSD card and you are done. Now you can switch your eMMC to on, and firmly assemble your laptop.

That's it. Now, when you boot your computer, you will be able to press Escape key and select the boot device. Even USB is supported! This will be essential to the next blog post: installing Armbian. But, more about that the next time.

PS. I really enjoy the hacking involved in making all this possible. Working directly with devices, sending commands and watching the verbose output during laptop initialization. Nowdays Linux on x86 just works, and I didn't know I was missing the excitement of doing things myself.

---

Posts in #PinebookPro series:

• My Pinebook Pro
• Installing Tow-Boot on Pinebook Pro
• Armbian on Pinebook Pro