Author Topic: About the downtime  (Read 2544 times)

0 Members and 1 Guest are viewing this topic.

Offline Gryzor

  • Administrator
  • 6128 Plus
  • *****
  • Posts: 17.351
  • Country: gr
  • CPC-Wiki maintainer
    • CPCWiki
    • Awards
About the downtime
« on: 10:41, 28 September 21 »
Hello guys!


I apologise for the downtime. Getting the system up and running again took longer than expected, so let me give you the story.


The problem
On Saturday our site certificate expired. This should normally not be a problem because Letsencrypt/certbot was set up to auto-renew. Alarm bells went off when this didn't happen and some of you landed on the site to be greeted with stern browser messages about hackers and Elbonians stealing your data.


So, ok, let's renew manually, that's easy, right? Well yes, if you have Letsenrypt installed. In our case, it was gone. Turns out, the OS update we did a while back did not agree with Letsencrypt and it was removed.


So, ok, let's update the server, right? Right. And that's what was done, down to the kernel. Except the new OS version came with PHP7, and our forum version did not support PHP7 (in fact that was one of the reasons we had stayed back a few versions).


So, ok, let's update the forum software! Have done this dozens of times over the years, this should be a breeze (actually started doing it on my mobile during a grocery run). But no! It shouldn't be that easy, fate decided, and a key file was corrupted. Tracking down that issue alone (which was giving an Error 500 when visiting) took me a whole day. Once understood, it was an easy fix, so last night I finished the installation and put the forum in working mode this morning.


The status
We are now on the latest stable forum software version. Because that was a large update, most of the mods have been removed. Only those that rely on hooks survived, those that depended on changing code, or the manual code changes I had done, are gone. That was inevitable because the installation replaced all modified files with the official ones. More on this later in this post.


A major pain point at the moment: if you look at your browser's address bar you'll see a not-so-friendly URL. Pretty URLs was a mod we had running that provides human-readable URLs, and it'll have to be reinstalled. Not a huge issue by itself, but it means that all URLs that are out there (bookmarks, links from other sites, Google results) give a 404. So that'a  priority.


As a bonus, though, we can now update the wiki software, too. That side is also stale because newer mediawiki versions required more up-to-date platforms. It will take some time because I'll have to be very careful and set up basic functionality like email, but we'll get there.


The Future
Since the discussion we had some time ago about changing platforms did not bear any fruits, we're staying with SMF.


To minimize disruption from OS updates in the future we're looking into making a docker container out of the platform. This comes with its own set of problems but has very specific benefits in managing the forum.


Also: the 2.0 branch of SMF is, by all measures, pretty old-fashioned. There's a 2.1 in the horizon which is much more modern (and has a proper mobile version), which is now at RC4 and is nearing release. Now, we could jump onto it because it looks like it's pretty useable, *but* going from a RC to the stable release later on would cause the loss of any mods, again. So I'm trying to decide what to do...


Current issues/mods
As I said, most mods are missing. Because we have amassed quite a few modifications over the years I'm not sure I remember all of them. Please let me know of any functionality you're missing and I'll put it in the list below along with a status.


Thanks for listening,
T


Mods list (done ones with green)
  • Pretty URLs
  • Likes - won't do as it affects performance in a big way. Will wait for the next release of SMF which includes such functionality as core.
  • Mobile theme
  • Mentions
  • Flags
  • Media embedding
  • Unread posts (https://www.cpcwiki.eu/forum/unread/?all;start=0)
  • Donations
  • Inline attachments
« Last Edit: 16:32, 01 October 21 by Gryzor »
like
1
Members reacted like:
Shining,

Offline pelrun

  • Supporter
  • 6128 Plus
  • *
  • Posts: 825
  • Country: au
    • index.php?action=treasury
    • Awards
Re: About the downtime
« Reply #1 on: 11:03, 28 September 21 »
*goes to like the post* Oh wait...  :laugh: :picard2:
like
0
No reactions

Offline Gryzor

  • Administrator
  • 6128 Plus
  • *****
  • Posts: 17.351
  • Country: gr
  • CPC-Wiki maintainer
    • CPCWiki
    • Awards
Re: About the downtime
« Reply #2 on: 11:07, 28 September 21 »
*goes to like the post* Oh wait...  :laugh: :picard2:

Oh right.
like
0
No reactions

Offline luckpro

  • CPC464
  • **
  • Posts: 37
  • Country: es
    • Awards
Re: About the downtime
« Reply #3 on: 11:20, 28 September 21 »
Don't worry, these things happen. Thank you very much Gryzor.
like
0
No reactions

Offline thomas

  • CPC664
  • ***
  • Posts: 55
  • Country: de
    • Awards
Re: About the downtime
« Reply #4 on: 11:54, 28 September 21 »
Welcome to the Upgrade Hell ... thank you Gryzor!
like
0
No reactions

Offline Gryzor

  • Administrator
  • 6128 Plus
  • *****
  • Posts: 17.351
  • Country: gr
  • CPC-Wiki maintainer
    • CPCWiki
    • Awards
Re: About the downtime
« Reply #5 on: 12:16, 28 September 21 »
Likes reinstalled. Worth saying that the upcoming smf version has such functionality built into it so I'm not sure what will happen then.
like
0
No reactions

Offline ervin

  • Supporter
  • 6128 Plus
  • *
  • Posts: 1.464
  • Country: au
    • index.php?action=treasury
    • Awards
Re: About the downtime
« Reply #6 on: 13:40, 28 September 21 »
All good Gryzor, great to have you and the wiki back!
I have to admit I did panic a little when I found the website dead.

Where is the donations button?
I feel like throwing you some moneys after what you went through!
like
0
No reactions

Offline Gryzor

  • Administrator
  • 6128 Plus
  • *****
  • Posts: 17.351
  • Country: gr
  • CPC-Wiki maintainer
    • CPCWiki
    • Awards
Re: About the downtime
« Reply #7 on: 14:24, 28 September 21 »
Likes removed from now, causing issues with server performance; will revisit.

Donations mod must be reapplied, but no need really :)
like
0
No reactions

Offline reidrac

  • Supporter
  • 6128 Plus
  • *
  • Posts: 1.068
  • Country: gb
  • Trying to gamedev!
    • index.php?action=treasury
    • usebox.net
    • Awards
Re: About the downtime
« Reply #8 on: 14:28, 28 September 21 »
What an adventure!

The forum takes a lot to load now, at least for me.

e.g. this post took 8 secs to get the response. Then the modify was ~500ms. Then the "save" operation 39 seconds.

Just FYI.
« Last Edit: 14:31, 28 September 21 by reidrac »
like
0
No reactions
Released The Return of Traxtor, Golden Tail, Magica, The Dawn of Kernel, Kitsune`s Curse and Brick Rick for the CPC.

If you like my games and want to show some appreciation, you can always buy me a coffee.

Offline Gryzor

  • Administrator
  • 6128 Plus
  • *****
  • Posts: 17.351
  • Country: gr
  • CPC-Wiki maintainer
    • CPCWiki
    • Awards
Re: About the downtime
« Reply #9 on: 15:17, 28 September 21 »
Well, I think that now the forum is faster than ever. URLs have returned too, so if you have any bookmarks feel free to try them out!
like
0
No reactions

Offline Gryzor

  • Administrator
  • 6128 Plus
  • *****
  • Posts: 17.351
  • Country: gr
  • CPC-Wiki maintainer
    • CPCWiki
    • Awards
Re: About the downtime
« Reply #10 on: 15:23, 28 September 21 »
like
0
No reactions

Offline Gryzor

  • Administrator
  • 6128 Plus
  • *****
  • Posts: 17.351
  • Country: gr
  • CPC-Wiki maintainer
    • CPCWiki
    • Awards
Re: About the downtime
« Reply #11 on: 15:48, 28 September 21 »
Testing mentions. Paging dr @Bryce ...
like
0
No reactions

Offline Gryzor

  • Administrator
  • 6128 Plus
  • *****
  • Posts: 17.351
  • Country: gr
  • CPC-Wiki maintainer
    • CPCWiki
    • Awards
Re: About the downtime
« Reply #12 on: 15:59, 28 September 21 »
Ok, some mods installed. I've had enough for now, if you run into any issues let me know :)
like
0
No reactions

Offline eto

  • Supporter
  • 6128 Plus
  • *
  • Posts: 577
  • Country: de
    • Awards
Re: About the downtime
« Reply #13 on: 16:00, 28 September 21 »
Mobile theme

Not sure if that is really a good idea to get it back. Imho it made more problems than it helped. Without it, I can at least fully access the forum with my mobile phone. With the mobile theme I could hardly use it at all.
like
0
No reactions

Offline pelrun

  • Supporter
  • 6128 Plus
  • *
  • Posts: 825
  • Country: au
    • index.php?action=treasury
    • Awards
Re: About the downtime
« Reply #14 on: 16:36, 28 September 21 »
Donations are back! *kaching*  8)
like
0
No reactions

Offline SkulleateR

  • 6128 Plus
  • ******
  • Posts: 506
  • Country: de
    • Awards
Re: About the downtime
« Reply #15 on: 16:38, 28 September 21 »
I think mobile Theme would be nice, right now I got massive differences in text size, buttons below are so small I cannot even read whats on there ...
like
0
No reactions

Offline eto

  • Supporter
  • 6128 Plus
  • *
  • Posts: 577
  • Country: de
    • Awards
Re: About the downtime
« Reply #16 on: 16:54, 28 September 21 »
I think mobile Theme would be nice, right now I got massive differences in text size, buttons below are so small I cannot even read whats on there ...


you can zoom in and out. With the mobile theme, depending on your exact resolution it could happen that elements were completely hidden.

It's a bit annoying to zoom in, but at least you get what you want. But if the element is simply not there, you are doomed.
like
0
No reactions

Offline Gryzor

  • Administrator
  • 6128 Plus
  • *****
  • Posts: 17.351
  • Country: gr
  • CPC-Wiki maintainer
    • CPCWiki
    • Awards
Re: About the downtime
« Reply #17 on: 17:04, 28 September 21 »
Mobile view IMO, though lacking in features, offered a pretty nice mobile experience. Sure, every now and then I'd want to use a feature that was missing but basic functionality was there... so you were doing maybe 90% of the stuff you usually do at 100% instead of 100% of features at a 50% ease of use.

Yes, donations are back, thanks so much @pelrun !! Much, much appreciated :)

Something I forgot to mention:

At no point was the database at risk. All files had been specifically backed up before I started tinkering with the site (note to self: delete the backup, that's several 10s of GB on the disk), and both files and the db are on a daily rsync job and transferred to an external server as redundancy, with a rotation of several days' worth of backups. As a matter of fact when it became clear a file was corrupted I just pulled the latest backup and fixed it in seconds.


This, just in case anyone was worried.
like
0
No reactions

Offline Gryzor

  • Administrator
  • 6128 Plus
  • *****
  • Posts: 17.351
  • Country: gr
  • CPC-Wiki maintainer
    • CPCWiki
    • Awards
Re: About the downtime
« Reply #18 on: 17:08, 28 September 21 »
Now let me ask a question.

The reason it wasn't fixed a bit sooner and indeed the reason why nowadays it takes me a bit more time to do stuff on the server (well, apart from the toddler that climbs on me the moment I enter our home), is that the best times to do such stuff is at work: I have lots of hours in front of a PC so taking a little break here and there is the best opportunity to work on the server.

However, at my current job the wise IT stuff have decided to close all "non-standard" ports, which means that SSH at port 22 (or indeed any other random port) is a no-go. Any ideas how I could access the server with such a restriction in place? Cheers!
like
0
No reactions

Offline pelrun

  • Supporter
  • 6128 Plus
  • *
  • Posts: 825
  • Country: au
    • index.php?action=treasury
    • Awards
Re: About the downtime
« Reply #19 on: 17:31, 28 September 21 »
What the hell do they consider "standard", then? Lemme guess, only 80 and 443, because anything that isn't web browsing must be hacking  :picard2:
Nobody said you had to send only HTTP over port 80 though...  :laugh: https://daniel.haxx.se/docs/sshproxy.html
like
0
No reactions

Offline Gryzor

  • Administrator
  • 6128 Plus
  • *****
  • Posts: 17.351
  • Country: gr
  • CPC-Wiki maintainer
    • CPCWiki
    • Awards
Re: About the downtime
« Reply #20 on: 17:36, 28 September 21 »
Yeah that's their logic, naturally.

But if we do it on 80 then what happens with the web server's 80?
like
0
No reactions

Online Skunkfish

  • Supporter
  • 6128 Plus
  • *
  • Posts: 745
  • Country: gb
  • Editor of Colour Personal Computing
    • CPCSTORE.CO.UK - Dedicated to the Amstrad CPC
    • Awards
Re: About the downtime
« Reply #21 on: 18:13, 28 September 21 »
Great to see the site back up! And thanks for all the work that you put in Gryzor on keeping this thing afloat, we all appreciate it!  :)
like
0
No reactions
An expanding array of hardware available at www.cpcstore.co.uk (and issue 3 of CPC Fanzine!)

Offline Animalgril987

  • Supporter
  • CPC6128
  • *
  • Posts: 241
  • Country: gb
    • Awards
Re: About the downtime
« Reply #22 on: 18:57, 28 September 21 »
Great to see the site back up! And thanks for all the work that you put in Gryzor on keeping this thing afloat, we all appreciate it! 

Agreed :D
like
0
No reactions

Offline reidrac

  • Supporter
  • 6128 Plus
  • *
  • Posts: 1.068
  • Country: gb
  • Trying to gamedev!
    • index.php?action=treasury
    • usebox.net
    • Awards
Re: About the downtime
« Reply #23 on: 20:35, 28 September 21 »
Well, I think that now the forum is faster than ever. URLs have returned too, so if you have any bookmarks feel free to try them out!

It is, indeed! Whatever was happening, it is gone now!
like
0
No reactions
Released The Return of Traxtor, Golden Tail, Magica, The Dawn of Kernel, Kitsune`s Curse and Brick Rick for the CPC.

If you like my games and want to show some appreciation, you can always buy me a coffee.

Offline lmimmfn

  • CPC464
  • **
  • Posts: 49
  • Country: ie
    • Awards
Re: About the downtime
« Reply #24 on: 21:11, 28 September 21 »
Called it on TLS cert issue(deal with that crap regularly lol), great to see the site back, great job @Gryzor
like
0
No reactions
6128 for the win!!!