I believe this is a slightly controversial topic, at least from what I have gathered so far. Some say its best to leave the server on to spare the life time of the spinning rust. Other seem to prefer to save power and boot the server off each night. So wanted to chip in and hear what folks here do and why do what you do.
Bonus question; Do you guys have a UPS? Is it a must have for a homelab, or does it just depend on the usecase?
When my server was a laptop it was on 24/7. When my server was a desktop I had a cron job to turn it off at 2AM. Now that it’s a specialized hardware it’s on 24/7.
Being constantly on is very convenient, but if your services start quickly it’s not the end of the world to have to turn the machine on for them.
There’s a poweroff?!
Im pretty sure OP is just yanking our chain.
And ruin my uptime stats? Are you mad?!?!
Among the many things I run are my own email servers so, yeah gotta be up all the time. And yes I have a UPS behind every electronic device in my house except the TV because if that dies I get to buy a new one.
I’ve probably spent upwards of $2000 on UPSes and replacement batteries over the last 20 years, but if it saved even one of my servers from taking a hit it was worth it. Servers are expensive and my time is valuable to me.
Relevant xkcd: https://xkcd.com/705/
For a while I had a low-power server for my personal things that stayed on all the time, and a more powerful computer that hosted a minecraft server. As the player count dwindled, I decided to make the minecraft server automatically shut down at midnight, and wake up at 8 in the morning using
rtcwake. And eventually I disabled the rtcwake thing entirely, and made the smaller server run a webui that could wake up the minecraft server using wake-on-lan. So if anyone wanted to play, they would first have to remotely turn on the server through a web page. This was all password-protected ofcourse.Also, no, I don’t use a UPS. I’ve never seen anyone use a UPS in the country where I live, and I don’t think I’ve experienced a power outtage in like 4 years. Whether or not you need a UPS seems to be largely dependent on where you live.
Mine stays on 24/7/365 unless I am going to be out of town.
Entirely depends on the usecase. If it’s a NAS and you only watch a few movies in the evening: Turn it off.
I bult a fairly power-efficient server. Consumes less than 20W and spins down the harddisks if not in use.
I can’t turn it off because none of the lightbulbs in the house would turn on anymore, my website would go down, my Fediverse instance wouldn’t pull any posts from American people who are awake during parts of the night. My emails and chat messages wouldn’t get delivered.
I don’t have a UPS. Also depends on the circumstances. I use ext4 as a filesystem which is kind of robust enough to handle power outages. And they’re rare where I live. A UPS would draw additional power and cost money. It’s not worth it for me at home.
I can’t turn it off because none of the lightbulbs in the house would turn on anymore
If you have Hue bulbs, you can buy little radios that attach to your light switch (or replacement light switches) that will still operate your lights when the server is down or the network is unavailable. It’s a worthwhile upgrade.
I can’t turn it off because none of the lightbulbs in the house would turn on anymore
Personally I try to avoid making anything in my home actually dependant on my server. I have a single lamp that can only be controlled from my phone and that’s only because it’s so rarely used that I didn’t want to put in the effort. Everything else is local first and only gets extended functionality from my server running.
I’ve had a couple issues with my zigbee stuff over the years on the server side and I would be really pissed if I wouldn’t be able to turn my lights on because I haven’t gotten around to fixing my server yet.
Sure. All that stuff has consequences. I also used to run a DNS Adblocker on that machine. So after a power outage, all the lights in the kitchen and livingroom (those are the smart ones) would turn on at full brighness (their default state). The internet wouldn’t ever come back since it’s missing its DNS server. Obviously I can’t get any notification of the incident, since my server is down… And I fall back to being reachable via phone or SMS. If my wife tells me via chat or email… That’s down, too.
I’m still working on a better solution. It ain’t easy, though. It’s certainly easier to use some cloud services and have other people keep the infrastructure running in some datacenters which have more redundancy… We have light switches, though. All my smart home stuff is just retrofitted. So we can still turn it off or on with the wall switches. I won’t change that until I come up with a solution to this problem. Until then, I’ve dialed things a bit back and I refrain from making everything “smart” when I can’t do it 100% reliably.
And it’s just some lights and the washing machine. While I am a nerd and tinkerer, I don’t see any good reason to own a smart toaster, fridge or Alexa. YMMV, of course.
Doesn’t your phone switch back to mobile service if the internet isn’t reachable on your LAN?
yes, the phone switches to mobile. It’s just all the selfhosted services that are missing, like my nextcloud, matrix chat, etc. And several apps complaining they can’t sync anymore or send messages. I don’t use that many cloud services, so it’ll be a lot of things I rely on. Browsing the web works. But I’ve changed things and moved the adblocker. So now that one issue is gone. It still doesn’t solve the real issue… But at least the wifi comes back on its own.
Do you selfhost email too?
Currently yes. I have a bit of a non-standard setup though. Like a business contract with my internet service provider, which includes a few perks that are required to send mail and are missing on a normal residential internet connection (static ip, dns reverse pointer). And generally 95% of people recommend not to do email yourself. I might change in the future. Back in the days it was a few bucks more a month, but they increased prices substantially. Either I move my mailserver to a VPS or save me some time and effort and switch to some email service like everyone else.
UPS depends on usecase and on the stability of your electrical supply (which varies greatly from place to place). I just leave everything running and have it configured to restart automatically on power restore (if it fails, which it rarely does).
I have a small 6U rack in my hallway which is where all the server stuff sits. There are 1U UPS units, but I haven’t had the need for it yet. However after replacing motherboard on this current machine I forgot to turn on option for auto start after power failure. My servers are mostly for collecting data regarding temperature, humidity and other metrics around the house, glass house and other parts. Same machine also collects surveillance data from cameras around the property which detect human and animal shapes.
So since machine rarely does long term calculations or data processing it’s okay that it doesn’t have UPS, since no data would be coming anyway without power.
It runs 24/7. My Lemmy instance among other things, is running on the server.
I use a Power Station from Allpower with UPS capabilities.
Server is on 24/7 and it has a UPS for the momentary brown outs I have during heavy winds. It would be silly if it’s off for any reason besides maintenance, even more so since it holds multiple game worlds in addition to some web and chat stuff.
No one should be powering off their servers. Thats really not the way to go about anything. Now there’s nothing stopping you from doing that either if you want to and it makes you happy or your life easier.
But if you want a simple answer to a simple question, no, nobody sane is doing that lol
We power off servers in the enterprise all the time and on schedules 😂. Its called saving money.
Uh, in your “enterprise” maybe lol.
In pretty much any enterprise using the public cloud. Everything is auto scaling, so shutdowns when not needed. Dev environments shutdown over night… If you’re not shutting down and scaling in the public cloud, you’re doing it wrong.
Ok scaling is not what we’re talking about here lol.
Is it shutting down servers… Yes. it just does it based on parameters and thresholds.
Then you get things like VDI servers and jump boxes that only need to be on between certain hours, so get shutdown outside them hours.
Right you don’t shut them down, you scale them down. My server also uses less power off peak demand.
No we shut them down. They get deallocated the same way as shutting down a virtual server does. They’re not containers, the scaling part just turns them on and off based on workload or schedule
No Donny it’s not. You’re out of your element here.
But it is. They’re stopped and deallocated. They start up when demanded. And shutdown when below a threshold or a certain schedule.
They’re fuckin’ nihilists dude, they don’t believe in anything
Most of us don’t have clusters so shutting down the server means taking the server and all associated services completely offline.
Do you take your product completely offline for 8 hours every single day?
I see where your head is at here, but it sounds like you’re focusing on containerized items. A lot of people are going to look at you real weird if you think of scaling down a container as equivalent to shutting down a server. We can all see where your mind is going and there is logic there, but it’s more akin to closing chrome when you’re not using it than it is to shutting down the computer running chrome.
Even physical hardware, if your paying power you can have clusters of physical hardware power up and down based on usage. There is no point in having 10 physical hosts running when the workload for n+1 means 3 servers overnight. With bnc, ipmi, ilo, idrac it will power them up as needed.
Finally someone who gets it.
Yup. I run those kind of clusters. But unless your in home datacenter territory, that sort of config isn’t likely to happen in self hosted.
Everything runs 24/7, but now I am thinking about theoretical power saving modes. Besides any built in power saving whatever (a little clueless), you could always throttle the CPU more. Not sure if it would be worth it without testing with a power meter.
On/off:
I have 5 main chassis excluding desktops. Prod cluster is all flash, standalone host has one flash array, one spinning rust array, NAS is all spinning rust. I have a big enough server disk array that spinning it up is actually a power sink and the Dell firmware takes a looong time to get all the drives up on reboot.TLDR: Not off as a matter of day/night, off as a matter of summer/winter for heat.
Winter: all on
Summer:
- prod cluster on (3x vSAN - it gets really angry if it doesn’t have cluster consistency)
- NAS on
- standalone server off, except to test ESXi patches and when vCenter reboots cause it to be WoL’d (vpxd sends a wake to all stand by hosts on program init)
- main desktop on
- alt desktops off
VMs are a different story. Normally I just turn them on and off as needed regardless of season, though I will typically turn off more of my “optional” VMs to reduce summer workload in addition to powering off the one server. Rough goal is to reduce thermal load as to not kill my AC as quickly which is probably running above its duty cycle to keep up. Physical wise, these servers are virtualized so this on/off load doesn’t cycle the array.
Because all four of my main servers are the same hypervisor (for now, VMware ESXi), VMs can move among the prod cluster to balance load autonomously, and I can move VMs on or off the standalone host by drag-and-drop. When the standalone host is off, I usually move turn it’s VMs off and move them onto the prod cluster so I don’t get daily “backup failure” emails from the NAS.
UPS: Power in my area is pretty stable, but has a few phase hiccups in the summer. (I know it’s a phase hiccup because I mapped out which wall plus are on which phase, confirmed with a multimeter than I’m on two legs of a 3-phase grid hand-off, and watched which devices blip off during an event) For something like a light that will just flicker or a laptop/phone charger that has a high capacitance, such blips are a non issue. Smaller ones can even be eaten by the massive power supplies my Dell servers have. But, my Cisco switches are a bit sensitive to it and tend to sing me the song of their people when the power flickers - aka fan speed 100% boot up whining. Larger blips will also boop the Dell servers, but I don’t usually see breaks more than 3-5m.
Current UPS setup is:
- rack split into A/B power feeds, with servers plugged into both and every other one flipped A or B as it’s primary
- single plug devices (like NAS) plugged into just one
- “common purpose” devices on the same power feed (ex: my primary firewall, primary switches, and my NAS for backups are on feed A, but my backup disks and my secondary switches are on feed B)
- one 1500VA UPS per feed (two total) - aggregate usage is 600-800w
- one 1500VA desktop UPS handling my main tower, one monitor, and my PS5 (which gets unreasonably upset about losing power, so it gets the battery backup)
With all that setup, the gauges in the front of the 3 UPSes all show roughly 15-20m run time in summer, and 20-25m in winter. I know one may be lower than displayed because it’s battery is older, but even if it fails and dumps it’s redundant load onto the main newer UPS I’ll still have 7-10m of battery at worst case and that’s all I really need to weather most power related issues at my location.
24/7 of course, that’s the point of it. But I have solar, so I don’t mind consuming power, and its not thatuxh a yway, so, anyway…
What’s the point in turning it off at all???
Could be that it is hosting services that are only actively used instead of passively doing stuff, so no real need to have the server on when you’re sleeping. For me turning it off and on again would be more of a hassle than it’s worth it.
Agreed. First of all that would make running backups more complex, and would require either manual interaction, or very careful automation of some kind.
And any public facing service (like blog and some stuff) would still need to be accessible somehow, so…
Saving power of course.
Reducing noise and temps is another.
I have solar, so the power consumption is negligible, I am already mostly selling yo the network and not “consuming” most of the days. Also, the server stuff sits in a sound proofed and ventilated compartment in the most remote area of the house.
20 years of planning ahahahahah
Past the times when it was sitting in my bedroom.
I just said what the general benefits are since you apparently were unable to get that yourself.
I’m not saying you specifically should do or not do something, I don’t care.
My NAS and production server run 24/7, I’ve got a dev server that I turn off if I’m not expecting to use it for a week or so. Usually when I do that, I immediately need it for something and I’m away from home. I have chosen equipment to try and minimize energy use to allow for constant running.
My view on UPS is it’s a crucial part of getting your availability percentage up. As my home lab turned into crucial services I used to replace commercial cloud options, that became more important to me. Whether it is to you will depend on what you’re running and why.
I’ve heard that one of the most likely times for hard drives to fail is on power up, and it also makes sense to me that the heating/cooling cycles would be bad for the magnetic coating, so my NAS is configured to keep them spinning, and it hasn’t been turned off since I last did a drive change.
