Anyone else just sick of trying to follow guides that cover 95% of the process, or maybe slightly miss a step and then spend hours troubleshooting setups just to get it to work?
I think I just have too much going in my “lab” the point that when something breaks (and my wife and/or kids complain) it’s more of a hassle to try and remember how to fix or troubleshoot stuff. I lightly document myself cuz I feel like I can remember well enough. But then it’s a style to find the time to fix, or stuff is tested and 80%completed but never fully used because life is busy and I don’t have loads of free time to pour into this stuff anymore. I hate giving all that data to big tech, but I also hate trying to manage 15 different containers or VMs, or other services. Some stuff is fine/easy or requires little effort, but others just don’t seem worth it.
I miss GUIs with stuff where I could fumble through settings to fix it as is easier for me to look through all that vs read a bunch of commands.
Idk, do you get lab burnout? Maybe cuz I do IT for work too it just feels like it’s never ending…
I don’t run a service unless it has reasonably good documentation. I’ll go through it first and make sure I understand how it’s supposed to run, what port(s) are used, and if I have an actual, practical use case for it.
You’re absolutely correct in that sometimes the documentation glosses over or completely omits important details. One such service is Radicale. The documentation for running a Docker container is severely lacking.
My biggest problem is every docker image thinks they’re a unique snowflake and how would anyone else be using such a unique port number like 80?
I know I can change, believe me I know I have to change it, but I wish guides would acknowledge it and emphasize choosing a unique port.
Why expose any ports at all. Just use reverse proxy and expose that port and all the others just happen internally.
Still gotta configure ports for the reverse proxy to access.
Most put it on port 80 with the perfectly valid assumption that the user is sticking a reverse proxy in front of it. Container should expose 80 not port forward 80.
Containers are ment to be used with docker networks making it a non-issue, most of the time you want your services to forward 80/443 since thats the default port your reverse proxy is going to call
Yes, I get lab burnout. I do not want to be fiddling with stuff after my day job. You should give yourself a break and do something else after hours, my dude.
BUT
I do not miss GUIs. Containers are a massive win in terms because they are declarative, reproducible, and can be version controlled.
Yeah, since Christmas, I more it sounds silly, but I’ve been playing a ton of video games with my kids lol. But not like CoD, more like Grounded 2, Gang Beasts, and Stumble Guys lmao
You’re doing i right. Playing cool games with your kids sounds like a blast and some great memories :)
While I am gaining plentiful information from this comments section already, wanted to add that the IT brain drain is real and you are not alone.
Haha, thanks! It’s probably more problematic being a solo IT guy as it feels like I don’t always have did dedicated time to get projects done. Part of why my lab is overkill is because I want something at work, so I spend a little time at home figuring stuff out, but, you know, family time n all…
Its still fun mostly, but work keeps assuming I must’ve freed up a lot of time in automating or improving stability so I keep being rewarded with more work outside of IT.
If you’ll let me self promote for a second, this was part of the inspiration for my Ansible Homelab Orchestration project. After dealing with a lot of those projects that practically force you to read through the code to get a working environment, I wanted a way to reproducably spin up my entire homelab should I need to move computers or if my computer dies (both of which have happened, and having a setup like this helped tremendously). So far the ansible playbook supports 117 applications, most of which can be enabled with a single configuration line:
immich_enabled: true nextcloud_enabled: trueAnd it will orchestrate all the containers, networks, directories, etc for you with reasonable defaults. All of which can be overwritten, for example to enable extra features like hardware acceleration:
immich_hardware_acceleration: "-cuda"Or to automatically get a letsencrypt cert and expose the application on a subdomain to the outside world:
immich_available_externally: trueIt also comes with scripts and tests to help add your own applications and ensure they work properly
I also spent a lot of time writing the documentation so no one else had to suffer through some of the more complicated applications haha (link)
Edit: I am personally running 74 containers through this setup, complete with backups, automatic ssl cert renewal, and monitoring
That’s neat. I never gave ansible playbooks any thought because I thought it would just add a layer of abstraction and that containers couldn’t be easier but reading your post I think I have been wrong.
While it is true that Ansible is a different tool that you need to learn the basics of (if you want to edit/add applications), all of the docker stuff is pretty comparable. For example, this is the equivalent of a docker compose file for SilverBullet (note taking app): https://github.com/Dylancyclone/ansible-homelab-orchestration/blob/main/roles/silverbullet/tasks/main.yml
You can see it’s volumes, environment variables, ports, labels, etc just like a regular docker compose (just in a slightly different format, like environment variables are listed as
envinstead ofenvironment), but the most important thing is that everything is filled in with variables. So for SilverBullet, any of these variables can be overwritten, and you’d never have to look at/tweak the “docker compose.” Then, if any issue is found in the playbook, anyone can pull in the changes and get the fix without any work from themselves, and if manual intervention is needed (like an app updated and now requires a new key or something), the playbook can let you know to avoid breaking something: https://dylancyclone.github.io/ansible-homelab-orchestration/guides/updating/#handling-breaking-changes
Yeah, self promote away lol
@Dylancyclone @selfhosted This looks very useful. I will study your docs and see if it’s right for me. Thanks for sharing!
I reject a lot of apps that require a docker compose that contains a database and caching infrastructure etc. All I need is the process and they ought to use SQLite by default because my needs are not going to exceed its capabilities. A lot of these self hosted apps are being overbuilt and coming without defaults or poor defaults and causing a lot of extra work to deploy them.
Databases.
I ran PaperlessNGX for a while, everything is fine. Suddenly I realize its version of Postgresql is not supported anymore so the container won’t start.
Following some guides, trying to log into the container by itself, and then use a bunch of commands to attempt to migrate said database have not really worked.
This is one of those things that feels like a HUGE gotcha to somebody that doesn’t work with databases.
So the container’s kinda just sitting there, disabled. I’m considering just starting it all fresh with the same data volume and redoing all that information, or giving this thing another go…
…But yeah I’ve kinda learned to hate things that rely on database containers that can’t update themselves or have automated migration scripts.
I’m glad I didn’t rely on that service TOO much.
Its a big problem. I also dump projects that don’t automatically migrate their own SQLite scehema’s requiring manual intervention. That is a terrible way to treat the customer, just update the file. Separate databases always run into versioning issues at some point and require manual intervention and data migration and its a massive waste of the users time.
Some apps really go overboard, I tried out a bookmark collection app called Linkwarden some time ago and it needed 3 docker containers and 800MB RAM
Found an alternative solution to recommend?
No, but I’d like to hear it if anyone else finds one
You should take notes about how you set up each app. I have a directory for each self hosted app, and I include a README.md that includes stuff like links to repos and tutorials, lists of nuances of the setup, itemized lists of things that I’d like to do with it in the future, and any shortcomings it has for my purposes. Of course I also include build scripts so I can just “make bounce” and the software starts up without me having to remember all the app-specific commands and configs.
If a tutorial gets you 95% of the way, and you manage to get the other 5% on your own, write down that info. Future you will be thankful. If not, write a section called “up next” that details where you’re running into challenges and need to make improvements.
I found a git repo with docker compose and the config files works well enough as long as you are willing to maintain a backup of the volumes and an .env file on KeePass (also backed up) for anything that might not be OK on a repo (even if private) like passwords and keys.
I started a blog specifically to make me document these things in a digestable manner. I doubt anyone will ever see it, but it’s for me. It’s a historical record of my projects and the steps and problems experienced when setting them up.
I’m using 11ty so I can just write markdown notes and publish static HTML using a very simple 11ty template. That takes all the hassle out of wrangling a website and all I have to do is markdown.
If someone stumbles across it in the slop ridden searchscape, I hope it helps them, but I know it will help me and that’s the goal.
Would love to see the blog
If a project doesn’t make it dead simple to manage via docker compose and environment variables, just don’t use it.
I run close to 100 services all using docker compose and it’s an incredibly simple, repeatable, self documenting process. Spinning up some new things is effortless and takes minutes to have it set up, accessible from the internet, and connected to my SSO.
Sometimes you see a program and it starts with “Clone this repo” and it has a docker compose file, six env files, some extra fig files, and consists of a front end container, back end container. Database container, message queueing container, etc… just close that web page and don’t bother with that project lol.
That being said, I think there’s a bigger issue at play here. If you “work in IT” and are burnt out from “15 containers and a lack of a gui” I’m afraid to say you’re in the wrong field of work and you’re trying to jam a square peg in a round hole
