Everything was logging fine but then mysteriously stopped. Tried:
- Removing gh-pages branch. I thought it would put it back but it doesn't so I then had to re-instate it.
- Restarting the jenkins-x-controllerbuild pod.
I want to try doing a jx upgrade boot but that gets stuck on issues with the cherry-picks.
ryandawsonuk
All 12 comments
Managed to 'fix' it by doing a jx upgrade platform... is that the intended fix?
ryandawsonuk
on 4 Dec 2019
Hi! Same problem here.
From time to time, JX stops putting logs to gh-pages. Last time, I performed a 'jx upgrade platform' that solved the issue...
marlovil
on 9 Dec 2019
I can confirm that this is still an issue in:
NAME VERSION
jx 2.0.1029
Kubernetes cluster v1.14.8-gke.12
kubectl v1.16.3
helm client Client: v2.13.0+g79d0794
git 2.17.1
Operating System Ubuntu 18.04.3 LTS
axsaucedo
on 16 Dec 2019
We had this issue again - we couldn't just run jx upgrade platform nor jx upgrade boot as it was giving us merge conflicts, so we had to uninstall the cluster and do a full installation. This fixed it now. Next time this happens I want to try the following steps:
1) Remove the gh-pages branch
2) Change all storage components from jx-requirements to false (disabled) and wait for gitops update
3) Change all storage components from jx-requirements back to true
The assumption is that this would re-create the gh-pages and the relevant triggers. If anyone else gets this issue again it would be great to hear if this fixes it for you.
axsaucedo
on 24 Jan 2020
Today, again, the same problem.
But I've found the responsible piece that is in charge of doing this.
This is the pod: jenkins-x-controllerbuild-yyyyyyyyyy-zzzzz and these are the logs:
{"level":"info","msg":"storing logs for activity mycompany-jxqsapp-master-81 into storage at jenkins-x/logs/mycompany/jxqsapp/master/81.log","time":"2020-02-05T12:58:39Z"}
{"level":"info","msg":"shallow cloning https://github.com/mycompany/jxqsapp.git branch gh-pages","time":"2020-02-05T12:58:39Z"}
{"level":"warning","msg":"failed to shallow clone https://github.com/mycompany/jxqsapp.git branch gh-pages due to: failed to init a new git repository in directory /tmp/jenkins-x-collect632787736: git output: : failed to run 'git init' command in directory '/tmp/jenkins-x-collect632787736', output: ''","time":"2020-02-05T12:58:39Z"}
{"level":"error","msg":"failed to store logs for activity mycompany-jxqsapp-master-81 into storage at jenkins-x/logs/mycompany/jxqsapp/master/81.log: failed to init a new git repository in directory /tmp/jenkins-x-collect369966487: git output: : failed to run 'git init' command in directory '/tmp/jenkins-x-collect369966487', output: ''","time":"2020-02-05T12:58:39Z"}
To solve the issue simply restart the pod. All pending logs are written immediately.
@ryandawsonuk @axsaucedo try this next time ;)
marlovil
on 5 Feb 2020
@marlovil this worked like a charm - you saved us yet another tedious re-install! Thanks!
axsaucedo
on 28 Feb 2020
Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://jenkins-x.io/community.
/lifecycle stale
jenkins-x-bot
on 28 May 2020
Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Provide feedback via https://jenkins-x.io/community.
/lifecycle rotten
jenkins-x-bot
on 27 Jun 2020
/remove-lifecycle rotten
axsaucedo
on 29 Jun 2020
I can confirm this is also present in lighthouse, it seems the issue is with controllerbuild pod. Using the latest version as of 07/07/2020, it seems bouncing the controllerbuild pod can resolve this and it "catches up" on previous notifications. At present a quickfix is basically to have a presistent bounce of the controllerbuild container.
axsaucedo
on 7 Jul 2020
Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://jenkins-x.io/community.
/lifecycle stale
jenkins-x-bot
on 5 Oct 2020
/remove-lifecycle stale
marlovil
on 5 Oct 2020
Related issues
ccojocar
路
3Comments
tdcox
路
3Comments
shahnewazrifat
路
3Comments
sourabhg
路
4Comments
vgallissot
路
5Comments
Most helpful comment
Today, again, the same problem.
But I've found the responsible piece that is in charge of doing this.
This is the pod: jenkins-x-controllerbuild-yyyyyyyyyy-zzzzz and these are the logs:
{"level":"info","msg":"storing logs for activity mycompany-jxqsapp-master-81 into storage at jenkins-x/logs/mycompany/jxqsapp/master/81.log","time":"2020-02-05T12:58:39Z"} {"level":"info","msg":"shallow cloning https://github.com/mycompany/jxqsapp.git branch gh-pages","time":"2020-02-05T12:58:39Z"} {"level":"warning","msg":"failed to shallow clone https://github.com/mycompany/jxqsapp.git branch gh-pages due to: failed to init a new git repository in directory /tmp/jenkins-x-collect632787736: git output: : failed to run 'git init' command in directory '/tmp/jenkins-x-collect632787736', output: ''","time":"2020-02-05T12:58:39Z"} {"level":"error","msg":"failed to store logs for activity mycompany-jxqsapp-master-81 into storage at jenkins-x/logs/mycompany/jxqsapp/master/81.log: failed to init a new git repository in directory /tmp/jenkins-x-collect369966487: git output: : failed to run 'git init' command in directory '/tmp/jenkins-x-collect369966487', output: ''","time":"2020-02-05T12:58:39Z"}To solve the issue simply restart the pod. All pending logs are written immediately.
@ryandawsonuk @axsaucedo try this next time ;)