Hi all!
So I want to get back into self hosting, but every time I have stopped is because I have lack of documentation to fix things that break. So I pose a question, how do you all go about keeping your setup documented? What programs do you use?
I have leaning towards open source software, so things like OneNote, or anything Microsoft are out of the question.
Edit: I didn’t want to add another post and annoy people, but had another inquiry:
What ReverseProxy do you use? I plan to run a bunch of services from docker, and would like to be able to reserve an IP:Port to something like service.mylocaldomain.lan
I already have Unbound setup on my PiHole, so I have the ability to set DNS records internally.
Bonus points if whatever ReverseProxy setup can accomplish SSL cert automation.
My documentation is a folder with the docker compose files I am using. And some notes in Nextcloud Notes if needed.
My reverse proxy is Traefik, since it’s docker aware. :)
Came to write basically this. I would try caddy but my compose file is 600 lines long now and half of that is traefik labels, I can’t be arsed with the migration.
Traefik or Caddy are the 2 I am bouncing back and forth between currently. I may spin up a nextcloud instance.
This might be a bit late, but from my perception Traefik has a touch more of a learning curve, but it integrates much better with solutions like Authlia/Authentik and Prometheus than Caddy does.
I might be wrong, I’ve never used Caddy, but that’s my perception.
Good to know. Thank you!
StandardNotes for me
So I want to get back into self hosting, but every time I have stopped is because I have lack of documentation to fix things that break. So I pose a question, how do you all go about keeping your setup documented? What programs do you use?
Joplin or Obsidian? Or… plain markdown files with your favorite text editor.
I use Joplin and it works great for this exact thing. Anytime I discover a new command that fixes something I’ll throw it into my Joplin notebook. “New Server Cheatsheet” goes to list in order common operations and commands for setting up SSH, UfW, making a non-root user, configuring wireguard, etc. I have hundreds of notes by now and they’re easily found via search bar.
I’m writing documentation in obsidian. I then expose it to the web so I can access from all my devices and share to others with quartz. Everything is markdown. It’s tunneled out of my network with cloudflare tunnels, which do handle SSL for me.
One day, I moved all services I really wanted from a couple of random VPS to a nice little proxmox machine at home (and then added some more services, of course). That was the day I swore to document stuff better, and I’m pretty satisfied with how well I was able to keep up with that.
In the proxmox web interface, you can leave notes per container. I note down which service the container is running including a link to the service’s web interface if applicable, plus the source, and a note about how it auto-updates (green check mark emoji) or if it requires manual updates (handiman emoji).
Further I made a concious effort to document everything into a gollum wiki running on that proxmox host (exposes a wiki like web interface, stores all entries as plaintext .md files into a local git repo - very “portable”). Most importantly, it also includes a page of easy to understand emergency measures in case I die or become unresponsive, which I regularly print out and put into a folder with other important documents. The page contains a QR code linking to itself on the wiki too in case the printed version might be outdated here or there.
The organization of the wiki itself (what goes into which folder) is a bit of a work in progress, but as it offers full text search, that’s not too much of a problem imo.
My documentation problem was largely fixed by using Nixos. The actual OS instances are self-documenting that way.
As far as the documentation for the network setup itself goes, a simple wiki does the rest of the trick for me.
I still want to get familiarized with NixOS and the concepts behind it. Just haven’t taken the time.
I have a couple Libre Office files where I document the non-technical stuff for my own quick reference, like network layout in Draw, or IP and port assignments in Calc. I use a git repo to store and organize podman scripts, systemd unit files, configs, etc. Probably not the most elegant solution, but it’s simple and FOSS.
Reverse proxy is Nginx Proxy Manager.
I use nginx for reverse proxy. You can get certbot working to automate ssl fairly easily. There is a learning curve, but most services I use have documentation for hosting their stuff with it.
i recently made the switch from doing k3s+flux to have everything in code with bundlewrap and anemos/makeimg
Wiki.js Nginx Proxy Manager.
I use markdown text files which are synced to my nextcloud instance.
This is somewhat tangential to your post, but I think using infrastructure as code and declarative technologies is great for reliability because you aren’t just running a bunch of commands until something works, you have the code which tells you exactly how things are set up, and you can version control it to roll back to a working state. The code itself can be a form of documentation in that case.
I think I need to utilize this strategy because I get lazy and don’t update external documentation.
Some examples of technologies which follow that paradigm are docker compose, ansible, nixOS and terraform. But it all depends on your workflow.
I think I am going down the docker compose route. When I started using docker, I didn’t use compose, however, now I plan to. Though, Ansible has been on my list of things to learn, as well as nixOS.
Another suggestion for you, I highly recommend specifying a version for the docker image you are using for a container, in the compose file. For example, nextcloud:29.0.1. If you just use :latest, it will pull a new version whenever you redeploy which you may not have tested against your setup, and the version upgrade may even be irreversible, as in the case of nextcloud. This will give you a lot more control over your setup. Just don’t forget to update images at reasonable intervals.
That is good advice, and honestly never really occurred to me to set specific versions for containers.
I write everything in markdown, and I mean just about everything. Tech notes, recipes, work procedures, shopping lists…everything. If you check my comment history from today, you can see a quick example of the kind of tech notes I keep (firewalld in this case).
I keep all of my plain text files synced across multiple devices using Syncthing. For desktop editors, I use mostly vim and VSCodium (though Kate is nice too), and I use Markor on Android. This workflow has been highly efficient for many years now, and I no longer waste time constantly reviewing the latest note-taking app.
I have an Ansible playbook that I use to setup everything and all troubleshooting steps I ever had to take to fix something get written down in an Obsidian.md vault.
I run a k3s cluster for selfhosted apps and keep all the configuration and docs in a git repo. That way I have history of changes and can rollback if needed. In that repo I have a docs folder with markdown documents about common operations and runbooks.
There are other ways to do this, but I like keeping docs next to the code and config so I can update them all at the same time. Deployed several wikis in the past but always forget to update them when I change things.
I really should spend time familiarizing with maintaining a git repo. I’ll likely find one I can self host.
If you want a git “server” quick and low maintenance then gitolite is most likely the best choice. https://gitolite.com/gitolite/index.html
It simply acts as a server that you can clone with any git client and the coolest part is that you use git commits to create repositories and manage users as well. Very very or no maintenance at all. I’ve been using it personally for years but also saw it being used at some large companies because it simply gets the job done and doesn’t bother anyone.
I will have to check out gitolite. Thank you!
https://forgejo.org selfhosted has been good for me, FOSS fork of Gitea.
Thank you for the suggestion. The fact that it’s FOSS wins my vote. I have been trying to go all open source where possible.
Right now, I’m using Obsidian. I think I’d like to transition to keeping docs in a wiki, but I worry that it’s part of the self-hosted infrastructure. In other words, if the wiki’s down, I no longer have the docs that I need to repair the wiki.
I have looked at Obsidian, it looks nice, but the closed source part is why I can’t personally use it. Though, from discussions I have seen Logseq be thrown out when talking about similar software.
The wiki idea is a good one. The way to handle that is to have the wiki backed up incrementally.