Libelektra: Migration of build.libelektra.org

[markus2330]

This issue is about the migration of build.libelektra.org to the new server. #160 is only for the essential stuff (downtime), here we can discuss details.

From #2967:

Unfortunately the amount of free space on the Jenkins master is still very low (~ 3.9G).

I hope we can finally move to the new build server before we run into this problem again!

@Mistreated what is the status?

[dbulatovic]

@Mistreated what is the status?

Currently we have a Jenkins server online. Pipeline needs to be created, so does the nodes.
I want to avoid interference with the old server, so I am gonna try to construct a new pipeline tonight over night.

[mpranj]

I personally have Jenkins as a Docker image, with the home being a volume I don't delete (-v jenkins_home:/var/jenkins_home). I can then simply remove outdated Jenkins Docker images by fetching new ones, with the home always persisted, thus having quite simple upgrade paths. I used this for >2 years with regular updates to everything.

Maybe I have a too simplistic view but is it not possible to simply:

1. Shortly stop Jenkins and make a backup.
2. Migrate the jenkins home directory to the new server.
3. Possibly adjust some config parameters.
4. Start jenkins on the new node.

[markus2330]

I would have also done it like @mpranj describes (I did step 1). But we can check if @Mistreated is able to recreate everything. The advantage of recreating would be that we get a more up-to-date docu and everything is clean afterwards (at the moment we have many outdated jobs).

[dbulatovic]

Status update:

• all plugins installed
• pipeline recognized
• some PRs go through, the following one is not:

Checking pull request #809

Currently Im having this error:

Checking pull request #809 ERROR: [Wed Nov 06 14:19:08 UTC 2019] Could not fetch branches from source blueocean java.lang.NullPointerException at org.jenkinsci.plugins.github_branch_source.PullRequestSCMHead.<init>(PullRequestSCMHead.java:95) at org.jenkinsci.plugins.github_branch_source.GitHubSCMSource.retrievePullRequest(GitHubSCMSource.java:1168) at org.jenkinsci.plugins.github_branch_source.GitHubSCMSource.retrieve(GitHubSCMSource.java:1027) at jenkins.scm.api.SCMSource._retrieve(SCMSource.java:373) at jenkins.scm.api.SCMSource.fetch(SCMSource.java:283) at jenkins.branch.MultiBranchProject.computeChildren(MultiBranchProject.java:635) at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:277) at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:164) at jenkins.branch.MultiBranchProject$BranchIndexing.run(MultiBranchProject.java:1026) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) [Wed Nov 06 14:19:08 UTC 2019] Finished branch indexing. Indexing took 1 min 42 sec FATAL: Failed to recompute children of libelektra java.lang.NullPointerException at org.jenkinsci.plugins.github_branch_source.PullRequestSCMHead.<init>(PullRequestSCMHead.java:95) at org.jenkinsci.plugins.github_branch_source.GitHubSCMSource.retrievePullRequest(GitHubSCMSource.java:1168) at org.jenkinsci.plugins.github_branch_source.GitHubSCMSource.retrieve(GitHubSCMSource.java:1027) at jenkins.scm.api.SCMSource._retrieve(SCMSource.java:373) at jenkins.scm.api.SCMSource.fetch(SCMSource.java:283) at jenkins.branch.MultiBranchProject.computeChildren(MultiBranchProject.java:635) at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:277) at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:164) at jenkins.branch.MultiBranchProject$BranchIndexing.run(MultiBranchProject.java:1026) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) Finished: FAILURE

Edit:
From the system log:

Failed to add GitHub webhook for GitHubRepositoryName[host=github.com,username=ElektraInitiative,repository=libelektra]
java.lang.NullPointerException: There are no credentials with admin access to manage hooks on GitHubRepositoryName[host=github.com,username=ElektraInitiative,repository=libelektra]

[markus2330]

Thank you for the status report!

According the system log the credentials for hooks are missing. I gave elektrabot more rights.

[dbulatovic]

According the system log the credentials for hooks are missing. I gave elektrabot more rights.

Hooks should be fine now.

Is there a possibility to close the #809 PR?
Maybe temporary?

[markus2330]

Is there a possibility to close the #809 PR?

done

[dbulatovic]

Same thing happens with #769.

Could not fetch branches from source blueocean
java.lang.NullPointerException

Could it be that these PRs are outdated?
@markus2330 Closing last 2 PRs: #769 and #323 would be of use if they are not so important.

Edit: I took the liberty to close #769. Now the repo is scanned completely.
I dont have docker-hub-elektra-jenkins credentials and therefore cant check if everything is ok.

[markus2330]

Same thing happens with #769.

What happens?

Could it be that these PRs are outdated?

Yes, they are. Good that you closed them.

323 might still be relevant, though. But it is possible to reopen, so it should not be a big deal.

Now the repo is scanned completely.

Good hint, the rescan also helped on the old buildserver to function again. I hope this rescan of the old server did not make troubles on the new server?

I dont have docker-hub-elektra-jenkins credentials

Please write me always and immediately by email about all credentials you (might) need. But what are docker-hub-elektra-jenkins credentials? Do you mean the docker registry on a7?

[dbulatovic]

What happens?

This:

Could not fetch branches from source blueocean
java.lang.NullPointerException

323 might still be relevant, though. But it is possible to reopen, so it should not be a big deal.

323 isnt closed. There was no null pointer there.

Please write me always and immediately by email about all credentials you (might) need. But what are docker-hub-elektra-jenkins credentials? Do you mean the docker registry on a7?

I wrote an email.

[markus2330]

@ingwinlu can you help us out with the docker-hub-elektra-jenkins credentials?

@Mistreated is something else missing besides the docker-hub-elektra-jenkins credentials?

[dbulatovic]

Status update:

Build fails on todo

No such DSL method 'openTasks' found among steps

Probalby some plugin error.

Edit: fixed.

[markus2330]

Probalby some plugin error.

Yes.

Edit: fixed.

Which plugin was it?

So the status now is that copying the artifacts fails?

[dbulatovic]

Which plugin was it?

Pipeline Utility Steps

So the status now is that copying the artifacts fails?

Jenkins publishes on 2 servers, a7 and doc.libelektra.org. I configured it just on a7.
doc.libelektra.org needs to be migrated first

[markus2330]

Maybe we can publish everything on doc.libelektra.org? It would be good if we reduce what is running on a7 as it would make the setup easier. (maybe except of the docker registry, for performance reasons)

As I see it, the Debian packages are the only thing to be published on a7, which does not work satisfactory anyway as it does not support multiple repositories, but we need this now: Debian and Ubuntu.

[markus2330]

Migration mostly done.