FSE's on a fancy new box! There was a bit more downtime than expected. If the site's blank, clear your local storage; there's a bug in theme handling where it attempts to load themes from previous versions of Pleroma (cool) but then fails (okay, fine) and then crashes the UI and shows a blank page (booo).
Other than that, we're finally up to date again.
Some stuff is still migrating, so uploads that occurred before the move might load more slowly for a while. CofeSpace hasn't moved over yet but it will soon. I expected to hate libvirt but I do not. I vastly underestimated the time it would take to copy the DB from one server to another; transfer time ended up being very long, despite it going from one datacenter to another rather than down the pipe at me. Not much to say other than that.
The upside is that now Postgres is running on an actual machine instead of a VM, and it's using NVMe, all that. FSE and associated services still live in a VM (KVM) but they got a RAM bump. The box itself has about 12 times as much RAM as FSE's web/DB servers had combined, which is pretty nice. CofeSpace and some other services will be running on it, so it's not all FSE, but FSE can now have as much space as it wants without even disturbing the other boxes. FSE's moved from Devuan to Slackware (because I can upload whatever ISO I like); that's pleasant. I'll be selling hosting services soon, though there's no storefront to plug yet, and I'm mostly selling to people I know. (If it does well, I'll expand operations, but ping me on fedi if you want a slice of the new machine and you don't mind that I am relatively new to the hosting business so there will probably be the occasional hiccup.)
When the site's down and I can't immediately bring it back up, I'll usually post small updates on the 502 page. This time, the updates were boring but I did want to let people know it was still going on so I logged my consumption of psychoactives along with "It's still doing the same thing as last time." Here is the log:
(2021-01-20, 04:03 PST / 12:03 UTC) The server upgrade has officially started. Like I said (I'd link to the announcement, but FSE is down until the upgrade's done, so you can't see it), this'll take about 4-6 hours. This is an estimate based on some tests I did, we'll see how that actually goes. As usual, I will update here. Caffeine metric: I think it was about 450mg at this point but I wasn't counting yet.
(2021-01-20, 07:37 PST / 15:37 UTC) Still here, still copying stuff! Had a problem with Postgres, but that's fixed. (All the copying still happens while I'm fixing stuff, so luckily it didn't cost much time.) Caffeine metric: 900mg.
(2021-01-20, 08:47 PST / 16:47 UTC) COPY public.activities takes forever. This is the boring part. Also fc-cache runs way faster if you replace the binary with a symlink to /bin/true. Caffeine: 1060mg, English cheddar: 0.6lb (estimated; I have eaten about half a 1.2lb block), rye whiskey: 2 shots.
(2021-01-20, 10:30 PST / 18:30 UTC) COPY public.activities is still running. Looks like we're a couple of hours out still. Uploads are still copying, too. Caffeine: still 1060mg, rye whiskey: 4 shots.
(2021-01-20, 11:09 PST / 19:09 UTC) COPY public.activities is done, now we're doing COPY public.objects! Uploads still getting copied. Squinting at certbot. SlackBuilds is much nicer than apt. FFS, it's good to be running Slackware on the server again. No update on cheese, whiskey, or caffeine consumption. Also I have no idea where to buy cocaine but I feel like that would probably make the process more fun. Oh, also, also, SBo has an up-to-date version of Elixir.
(2021-01-20, 13:19 PST / 21:19 UTC) COPY public.objects still. After this is done, just the migrations left. Most of the services are already up and running (bloat, etc.), got some config stuff to do, but mostly just waiting now. Caffeine: 1360mg plus however much caffeine is in some stale espresso, rye whiskey: 5 shots, bourbon: 1 shot (in the coffee).
(2021-01-20, 13:45 PST / 21:45 UTC) All right, sorry about the delays, but also if I'm just waiting for data to go through the pipes, I might as well kick my feet up and close my eyes for a minute, tell the robot to start blasting something noisy when the transfer's done.
(2021-01-20, 18:32 PST / 2021-01-21 02:32 UTC) SO THE REASON WHY NOT TO DO THAT IS THAT IF IT HAPPENS YOU WILL OVERSLEEP. (Resetting the alcohol and caffeine counters.)
(2021-01-20, 19:56 PST / 2021-01-21 03:56 UTC) Brief argument with nginx, services will be moving in a few. Current estimate is about two hours.
(2021-01-20, 23:01 PST / 2021-01-21 07:01 UTC) Migrations! CERTBOT WAS WRITTEN BY JACKASSES. etc.
(2021-01-21 00:02 PST / 08:02 UTC) Pleroma's up and running. DB migrated. All fo that. Uploads may be partially janky for a moment. Fixing certs, final testing, then changing the DNS to point to our shiny new fuckin' server. Stuff like the blog and bloatfe and git repos will be a minute or two (they're still pointing at the old box), but we're almost there, and the estimate was only off by...10 hours.
(2021-01-21 01:05 PST / 09:05 UTC) r.color is undefined I think this is related to themes, probably; we have a lot of these and the format's changed. Someone that has an alt complain to one of the FE devs about this. Gonna clear the themes and see if that fixes it.