Libelektra: Create a Website

[markus2330]

Following essentials should be done before the webpage can be redirected to www.libelektra.org:

• [x] PRs merged
• [x] installed on libelektra.org server (+ tutorial how to install)
• [x] nice main page with main points present
• [x] Registration stable (no discarding of accounts)
• [x] URLs stable (+ ability to link to all documents)
• [x] entry points of docu are nice and proof-read
• [x] documentation mostly fixed, so it looks good on the website and has a uniform style
• [x] news/changelog page
• [x] equal stability also for the backend
• [x] full automatic pipeline to deploy frontend and backend
• [x] improve security #1153 (at least add security flags to homepage pipeline)
• [x] reliable start/stop (see #1181)
• [x] write release notes for the new snippet service/web page

Is something else missing?

[Namoshek]

• documentation fixed, so it looks good on the website and has a uniform style
• news/changelog page
• equal stability also for the backend
• todos in #1030 done

I think registration stability is no issue, this is something that is our responsibility to not delete the database.

[markus2330]

Thanks for your input!

I added your points except that it says "documentation _mostly_ fixed" and I excluded that #1030 needs to be done completely. Would of course be nice to have it, but it is not essential. Instead I added "entry points of docu are nice and proof-read".

[markus2330]

I added deployment pipeline and security

[Namoshek]

Deployment through build should actually work already.

[markus2330]

Thank you! Deployment works! But please get rid of the sleep(s) during deployment. This adds a lot of downtime.

[Namoshek]

I removed the sleep between the backend stop and start, but the other sleep should stay for a good reason:
The start script runs the backend in the background and because the cache needs to be loaded first, we cannot be sure the backend is reachable already (necessary for sitemap generation and this time will increase with the amount of snippets). But the 60 seconds sleep is no problem because the last frontend version is still online and reachable the whole time - we just override the frontend files without taking it down at all.

[markus2330]

Perfect, is resolved then!

[markus2330]

I also added "write release notes for the new snippet service/web page". It should be nothing new and I can help. Can we also mention the homebrew and other things (not directly related to the software)?

Btw. what is the easiest way to create preliminary news that should not yet be displayed on the website?

[Namoshek]

A PR or new branch should be the easiest way to avoid early publishing.

[markus2330]

What about not having the correct file extension? Or do you use every file from the news directory?

[Namoshek]

The news script fails on inappropriate filenames, but this could be changed easily so that it ignores them.

[markus2330]

Yes, would be nice to have some way of news preparation without the need of extra branches or similar. What about having explicit synax for it? (Maybe starting with _ or something?) Then you can still fail for malformed entries.

[Namoshek]

It fails intentionally on file names that do not match a certain regex, but I can change it to a warning, so the script runs through. Then you can use anything not matching the regex, e.g. an underscore as prefix. I will do this when preparing the news.

[markus2330]

I mean that we should only allow a very specific convention for news preparation. It is useful to fail on wrong names otherwise.

Btw. did you check: Does the installation (and the homepage build) fails when npm fails?

[Namoshek]

I mean that we should only allow a very specific convention for news preparation. It is useful to fail on wrong names otherwise.

No, actually a warning is enough, because there is not really an issue with wrong filenames. We could also have a CMakeLists.txt there for example. Unrelated files do not directly mean something is broken.

Btw. did you check: Does the installation (and the homepage build) fails when npm fails?

Building the homepage clearly fails (no grunt or no dependencies = no homepage), but if the cmake installation failes... no idea. You had problems with npm in the first hand, so you should know better?

[markus2330]

We could also have a CMakeLists.txt there for example.

Why would we do this? And if we add a file there (lets say README.md) I certainly do not want to have a warning every time ;)

Unrelated files do not directly mean something is broken.

Actually in this case it most likely does: it could be a typo. But a warning should be okay, too. (It is an obvious thing if a news item is missing.) But in general it is better if the building of the homepage fails in the case of error. (Then I get a mail that the homepage could not be built by jenkins)

Btw. Is jenkins able to collect the warnings that npm emits?

but if the cmake installation failes... no idea.

Could you please try? Simply create a PR which unconditionally leads to an error and start the hp build with the phrase.

You had problems with npm in the first hand, so you should know better?

The cmake installation went on, but I did not look what the overall exit status was.

[Namoshek]

Why would we do this? And if we add a file there (lets say README.md) I certainly do not want to have a warning every time ;)

But you suggest to get an error instead of a warning... that doesn't make sense. :smile:

Could you please try? Simply create a PR which unconditionally leads to an error and start the hp build with the phrase.

What is your desired output? That cmake fails on an npm-fail? I mean, there will be clearly an error message stating the problem, e.g.

npm executable was not found
npm package <xyz> not in the npm registry
...

so I doubt that it is an issue at all.

[markus2330]

But you suggest to get an error instead of a warning... that doesn't make sense. :smile:

Yes, an error should abort everything and then we add explicit support for such files (if ever needed).

What is your desired output? That cmake fails on an npm-fail? I mean, there will be clearly an error message stating the problem, e.g.

The build job should abort before it corrupts the homepage.

Maybe there is no issue anyway.

[Namoshek]

Hm well, letting grunt tasks fail is an issue; it can potentially corrupt the homepage. So we should probably avoid that.

[markus2330]

Maybe it would be a better design to run task during the compilation phase (then failing would be no issue), and during installation phase we would only copy files?

Even if you cannot do it because the changes are too big, we should at least get this right for @omnidan web GUI.

[Namoshek]

It is a small change in the package.json of the rest-frontend and no problem; but I cannot do it for our homepage build job only - it would be for all installations.

But I don't see the advantage to be honest, because if you start the frontend build script, it wipes the website content and re-creates it step-by-step through various scripts. And because the script is the same for cmake and the build script, it would fail both times.

[markus2330]

It is a small change in the package.json of the rest-frontend and no problem; but I cannot do it for our homepage build job only - it would be for all installations.

Of course.

But I don't see the advantage to be honest

Maybe we talk past each other, so the idea is:

make # also compiles/builds the frontend, might fail
make install # only copies files, only fails on access permissions/disc full

The advantage is that if we somehow corrupt translations/en.json, resources/structure.json.in, or other essential stuff we

1. have make fail (makes clear that something is wrong here early)
2. have the build job fail (because of 1.)
3. do not have partial installations (the installation would never start if make does not pass successfully)

because if you start the frontend build script, it wipes the website content and re-creates it step-by-step through various scripts. And because the script is the same for cmake and the build script, it would fail both times.

Maybe I have a wrong understanding, but for me make install was enough to get the new website, I do not re-execute the frontend build script. In which situations do I need to re-execute the frontend build script?

Maybe the frontend script should not wipe out the website content, but build in some temporary directory and then move the successfully build public directory to its destination?

[Namoshek]

Maybe we talk past each other, so the idea is:

Now I got it. If there is a way to build the homepage in the binary dir, this could work. I'm not 100% sure about the paths that are configured though, have to look at it.

In which situations do I need to re-execute the frontend build script?

The cmake installation executes grunt install, while the build script runs grunt full. The only difference is that grunt full also creates a sitemap, which requires the rest-backend to be up (so that the snippets can be listed).

Maybe the frontend script should not wipe out the website content, but build in some temporary directory and then move the successfully build public directory to its destination?

Also an idea, but I'm still not sure if it is worth the effort. We could just not let the grunt command fail at all; then we always have a working website (if not someone changes permissions or something else happens that is not the scripts fault).

[markus2330]

also creates a sitemap, which requires the rest-backend to be up

Ahh ok. Wouldn't it be nicer then if the script would exactly produces the sitemap and nothing else? Maybe we can even run this as cron job?

We could just not let the grunt command fail at all; then we always have a working website

Yes, always having a working website is by far the most important issue here!

[Namoshek]

Ahh ok. Wouldn't it be nicer then if the script would exactly produces the sitemap and nothing else? Maybe we can even run this as cron job?

You can run each task separately as well, grunt full (which is actually grunt default or only grunt) and grunt install are just a list of tasks being executed sequentially. So yes, a cronjob for the sitemap would make sense (but we have to be careful that the file is not written with the wrong permissions/user, otherwise other installations may fail).

But as the whole front-end build does only take few seconds, I don't think removing the build script form the build job has much of an impact.

[markus2330]

@e1528532 Can you take a look if the main entry points of the website are nice (and provide fixes if they do not)?

[markus2330]

Thanks, great site! Now we need more snippets ;)

[markus2330]

This is the first 1.0.0 issue solved! :fireworks: