Configuration-as-code-plugin: Config as code runs too early in the lifecycle

Created on 27 Aug 2019 · 15Comments · Source: jenkinsci/configuration-as-code-plugin

Your checklist for this issue

🚨 Please review the guidelines for contributing to this repository.

[x] Jenkins version (any / 2.176.2)
[x] Plugin version (any / 1.29)
[x] OS (any)

Description

Config as code applies the configuration after InitMilestone.EXTENSIONS_AUGMENTED. However the configuration of Jenkins is loaded after this and before JOBS_LOADED and as such as Config-As-Code will be manipulating state it can and will end up save()ing to disk whilst Jenkins is loading up. This will mean that potential changes that Config-As-Code has made may be lost and you will start up with a Jenkins that is not correctly configured.

see

bug

Source

jtnord

All 15 comments

amuniz on 27 Aug 2019

It is. We need to introduce more milestones in the Jenkins Core to make configuration management pluggable. Otherwise we just do hacks architecture-wise

oleg-nenashev on 27 Aug 2019

I believe that they are related but distinct.

jtnord on 27 Aug 2019

FTR there is a fix applied in the recent weekly.
https://jenkins.io/changelog/#v2.199

On Tue, Oct 15, 2019, 07:33 runzexia notifications@github.com wrote:

I think I may have encountered this problem.
I use the casc and groovy script to initialize jenkins .
In the case of the exact same configuration, some jenkins casc
configuration is not loaded correctly, I confirmed by config.xml
Is there any way to avoid this now? Avoid calling Jenkins.save in a groovy
script?

@jtnord https://github.com/jtnord @obreitenbach
https://github.com/obreitenbach

Jenkins version (any / 2.176.2)

Plugin version (any / 1.27)

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/jenkinsci/configuration-as-code-plugin/issues/1026?email_source=notifications&email_token=AAW4RIC7VWG4C5AJZ2XWJSDQOVI2VA5CNFSM4IQCOWCKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBHPAZQ#issuecomment-542044262,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAW4RIAJ22Y2NRU3IENC5U3QOVI2VANCNFSM4IQCOWCA
.

oleg-nenashev on 15 Oct 2019

This is a partial fix, we still need wider architecture changes to make
JCasC more stable

On Tue, Oct 15, 2019, 13:55 Oleg Nenashev o.v.nenashev@gmail.com wrote:

FTR there is a fix applied in the recent weekly.
https://jenkins.io/changelog/#v2.199

On Tue, Oct 15, 2019, 07:33 runzexia notifications@github.com wrote:

I think I may have encountered this problem.
I use the casc and groovy script to initialize jenkins .
In the case of the exact same configuration, some jenkins casc
configuration is not loaded correctly, I confirmed by config.xml
Is there any way to avoid this now? Avoid calling Jenkins.save in a
groovy script?

@jtnord https://github.com/jtnord @obreitenbach
https://github.com/obreitenbach

Jenkins version (any / 2.176.2)

Plugin version (any / 1.27)

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/jenkinsci/configuration-as-code-plugin/issues/1026?email_source=notifications&email_token=AAW4RIC7VWG4C5AJZ2XWJSDQOVI2VA5CNFSM4IQCOWCKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBHPAZQ#issuecomment-542044262,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAW4RIAJ22Y2NRU3IENC5U3QOVI2VANCNFSM4IQCOWCA
.

oleg-nenashev on 15 Oct 2019

@oleg-nenashev
I am not sure if my situation is related to this issue, I just want to confirm it.

I use the casc and groovy script to initialize jenkins .
In the case of the exact same configuration, sometimes casc configuration is not loaded correctly, I confirmed by config.xml
Is there any way to avoid this now? Avoid calling Jenkins.save in a groovy script?
Jenkins version (2.176.2)
Plugin version (1.27)

runzexia on 15 Oct 2019

I wouldn't recommend configuring Jenkins with both groovy init scripts and casc. why do you need to do that?

jtnord on 15 Oct 2019

@jtnord There are some reasons why I can't use casc all.
Mainly related to role-strategy, in my organization will use REST api to add and assign roles, and casc configuration role strategy will cause these to be overwritten after reboot

I also know now that it is best to use only one way to initialize, I think we should add some documentation to the user.
By the way, is it possible to judge whether the casc is completed in the first half of the groovy initialization script, and then execute the init script?

runzexia on 16 Oct 2019

@jtnord
I looked at the Jenkins code init groovy hook and casc are in different init stages, they should not interfere with each other? Maybe other configuration loads caused the problem?

https://github.com/jenkinsci/configuration-as-code-plugin/blob/94a2a73095a290ed2cb92b7ace5bf3afbd64b1ba/plugin/src/main/java/io/jenkins/plugins/casc/ConfigurationAsCode.java#L269
https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/init/impl/GroovyInitScript.java#L39

runzexia on 16 Oct 2019

looked at the Jenkins code init groovy hook and casc are in different init stages, they should not interfere with each other?

correct.

depending on what configuration you are loosing you may just be hitting this race condition that, or there could be other plugins trying to save some jenkins config during startup. given (mostly) jcasc would be trying to set the configuration that is already on disk on restart I would be surprised you are loosing things (I would expect more a situation where new updates do not get correctly applied - but given the race condition all manner of things could go haywire).

I would suggest you raise a specific issue for your scenario with all of the CasC files and details of the plugins/versions and jenkins version you are using. It may well be that this is the root cause, but it could also be something different (and having a set of files where it does reproduce is extremely helpful. Given timing may be involved if you are running in a public cloud provider providing details of the instance type and disk etc may also help in tracking down the issue (I spent many weeks tracking down a similar issue elsewhere that one person could reliably hit but no one else could reproduce)

jtnord on 16 Oct 2019

❤1

https://github.com/kubesphere/ks-installer/pull/299/files
As a workaround, I load casc again in the groovy hook

runzexia on 18 Oct 2019

Picked up JAVA_TOOL_OPTIONS: -Xms512m -Xmx512m -XX:MaxRAM=2g -Dhudson.slaves.NodeProvisioner.initialDelay=20 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85 -Dhudson.model.LoadStatistics.clock=5000 -Dhudson.model.LoadStatistics.decay=0.2 -Dhudson.slaves.NodeProvisioner.recurrencePeriod=5000 -verbose:gc -Xloggc:/var/jenkins_home/gc-%t.log -XX:NumberOfGCLogFiles=2 -XX:+UseGCLogFileRotation -XX:GCLogFileSize=100m -XX:+PrintGC -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintHeapAtGC -XX:+PrintGCCause -XX:+PrintTenuringDistribution -XX:+PrintReferenceGC -XX:+PrintAdaptiveSizePolicy -XX:+UseG1GC -XX:+UseStringDeduplication -XX:+ParallelRefProcEnabled -XX:+DisableExplicitGC -XX:+UnlockDiagnosticVMOptions -XX:+UnlockExperimentalVMOptions
Running from: /usr/share/jenkins/jenkins.war
webroot: EnvVars.masterEnvVars.get("JENKINS_HOME")
2019-11-19 03:02:27.156+0000 [id=1] INFO    org.eclipse.jetty.util.log.Log#initialized: Logging initialized @673ms to org.eclipse.jetty.util.log.JavaUtilLog
2019-11-19 03:02:27.365+0000 [id=1] INFO    winstone.Logger#logInternal: Beginning extraction from war file
2019-11-19 03:02:27.442+0000 [id=1] WARNING o.e.j.s.handler.ContextHandler#setContextPath: Empty contextPath
2019-11-19 03:02:27.554+0000 [id=1] INFO    org.eclipse.jetty.server.Server#doStart: jetty-9.4.z-SNAPSHOT; built: 2019-05-02T00:04:53.875Z; git: e1bc35120a6617ee3df052294e433f3a25ce7097; jvm 1.8.0_222-b10
2019-11-19 03:02:28.129+0000 [id=1] INFO    o.e.j.w.StandardDescriptorProcessor#visitServlet: NO JSP Support for /, did not find org.eclipse.jetty.jsp.JettyJspServlet
2019-11-19 03:02:28.235+0000 [id=1] INFO    o.e.j.s.s.DefaultSessionIdManager#doStart: DefaultSessionIdManager workerName=node0
2019-11-19 03:02:28.235+0000 [id=1] INFO    o.e.j.s.s.DefaultSessionIdManager#doStart: No SessionScavenger set, using defaults
2019-11-19 03:02:28.237+0000 [id=1] INFO    o.e.j.server.session.HouseKeeper#startScavenging: node0 Scavenging every 660000ms
Jenkins home directory: /var/jenkins_home found at: EnvVars.masterEnvVars.get("JENKINS_HOME")
2019-11-19 03:02:29.242+0000 [id=1] INFO    o.e.j.s.handler.ContextHandler#doStart: Started w.@5d908d47{Jenkins v2.199,/,file:///var/jenkins_home/war/,AVAILABLE}{/var/jenkins_home/war}
2019-11-19 03:02:29.255+0000 [id=1] INFO    o.e.j.server.AbstractConnector#doStart: Started ServerConnector@64616ca2{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
2019-11-19 03:02:29.351+0000 [id=1] INFO    org.eclipse.jetty.server.Server#doStart: Started @2868ms
2019-11-19 03:02:29.352+0000 [id=21]    INFO    winstone.Logger#logInternal: Winstone Servlet Engine v4.0 running: controlPort=disabled
2019-11-19 03:02:31.263+0000 [id=27]    INFO    jenkins.InitReactorRunner$1#onAttained: Started initialization
2019-11-19 03:02:32.260+0000 [id=26]    INFO    jenkins.InitReactorRunner$1#onAttained: Listed all plugins
2019-11-19 03:02:33.756+0000 [id=26]    INFO    ruby.RubyRuntimePlugin#start: Injecting JRuby into XStream
2019-11-19 03:02:59.769+0000 [id=26]    INFO    jenkins.InitReactorRunner$1#onAttained: Prepared all plugins
2019-11-19 03:03:00.260+0000 [id=27]    INFO    jenkins.InitReactorRunner$1#onAttained: Started all plugins
2019-11-19 03:03:04.695+0000 [id=27]    INFO    h.ExtensionFinder$GuiceFinder$FaultTolerantScope$1#error: Failed to instantiate optional component hudson.plugins.build_timeout.operations.AbortAndRestartOperation$DescriptorImpl; skipping
2019-11-19 03:03:05.033+0000 [id=27]    INFO    jenkins.InitReactorRunner$1#onAttained: Augmented all extensions
2019-11-19 03:03:09.131+0000 [id=27]    SEVERE  jenkins.model.Jenkins#save: An attempt to save Jenkins'' global configuration before it has been loaded has been made during milestone Augmented all extensions.  This is indicative of a bug in the caller and may lead to full or partial loss of configuration.
java.lang.IllegalStateException: call trace
    at jenkins.model.Jenkins.save(Jenkins.java:3299)
    at hudson.util.PersistedList.onModified(PersistedList.java:173)
    at jenkins.model.Jenkins$CloudList.onModified(Jenkins.java:525)
    at hudson.util.PersistedList.replaceBy(PersistedList.java:85)
    at io.jenkins.plugins.casc.impl.attributes.DescribableListAttribute.setValue(DescribableListAttribute.java:18)
    at io.jenkins.plugins.casc.impl.attributes.DescribableListAttribute.setValue(DescribableListAttribute.java:9)
    at io.jenkins.plugins.casc.BaseConfigurator.configure(BaseConfigurator.java:359)
    at io.jenkins.plugins.casc.BaseConfigurator.configure(BaseConfigurator.java:271)
    at io.jenkins.plugins.casc.ConfigurationAsCode.lambda$configureWith$4(ConfigurationAsCode.java:694)
    at io.jenkins.plugins.casc.ConfigurationAsCode.invokeWith(ConfigurationAsCode.java:664)
    at io.jenkins.plugins.casc.ConfigurationAsCode.configureWith(ConfigurationAsCode.java:694)
    at io.jenkins.plugins.casc.ConfigurationAsCode.configureWith(ConfigurationAsCode.java:585)
    at io.jenkins.plugins.casc.ConfigurationAsCode.configure(ConfigurationAsCode.java:280)
    at io.jenkins.plugins.casc.ConfigurationAsCode.init(ConfigurationAsCode.java:272)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:104)
    at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:175)
    at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
    at jenkins.model.Jenkins$5.runTask(Jenkins.java:1124)
    at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
    at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
2019-11-19 03:03:09.139+0000 [id=27]    SEVERE  jenkins.InitReactorRunner$1#onTaskFailed: Failed ConfigurationAsCode.init
java.lang.IllegalStateException: An attempt to save the global configuration was made before it was loaded
    at jenkins.model.Jenkins.save(Jenkins.java:3308)
    at hudson.util.PersistedList.onModified(PersistedList.java:173)
    at jenkins.model.Jenkins$CloudList.onModified(Jenkins.java:525)
    at hudson.util.PersistedList.replaceBy(PersistedList.java:85)
    at io.jenkins.plugins.casc.impl.attributes.DescribableListAttribute.setValue(DescribableListAttribute.java:18)
    at io.jenkins.plugins.casc.impl.attributes.DescribableListAttribute.setValue(DescribableListAttribute.java:9)
    at io.jenkins.plugins.casc.BaseConfigurator.configure(BaseConfigurator.java:359)
Caused: io.jenkins.plugins.casc.ConfiguratorException: cloud: Failed to set attribute clouds(class: class hudson.slaves.Cloud, multiple: true)
    at io.jenkins.plugins.casc.BaseConfigurator.configure(BaseConfigurator.java:361)
    at io.jenkins.plugins.casc.BaseConfigurator.configure(BaseConfigurator.java:271)
Caused: io.jenkins.plugins.casc.ConfiguratorException: Failed to save hudson.model.Hudson@484ad4e9
    at io.jenkins.plugins.casc.BaseConfigurator.configure(BaseConfigurator.java:274)
    at io.jenkins.plugins.casc.ConfigurationAsCode.lambda$configureWith$4(ConfigurationAsCode.java:694)
    at io.jenkins.plugins.casc.ConfigurationAsCode.invokeWith(ConfigurationAsCode.java:664)
Caused: io.jenkins.plugins.casc.ConfiguratorException: jenkins: error configuring 'jenkins' with class io.jenkins.plugins.casc.core.JenkinsConfigurator configurator
    at io.jenkins.plugins.casc.ConfigurationAsCode.invokeWith(ConfigurationAsCode.java:670)
    at io.jenkins.plugins.casc.ConfigurationAsCode.configureWith(ConfigurationAsCode.java:694)
    at io.jenkins.plugins.casc.ConfigurationAsCode.configureWith(ConfigurationAsCode.java:585)
    at io.jenkins.plugins.casc.ConfigurationAsCode.configure(ConfigurationAsCode.java:280)
    at io.jenkins.plugins.casc.ConfigurationAsCode.init(ConfigurationAsCode.java:272)
Caused: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:104)
Caused: java.lang.Error
    at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:110)
    at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:175)
    at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
    at jenkins.model.Jenkins$5.runTask(Jenkins.java:1124)
    at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
    at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
2019-11-19 03:03:09.142+0000 [id=20]    SEVERE  hudson.util.BootFailure#publish: Failed to initialize Jenkins
java.lang.IllegalStateException: An attempt to save the global configuration was made before it was loaded
    at jenkins.model.Jenkins.save(Jenkins.java:3308)
    at hudson.util.PersistedList.onModified(PersistedList.java:173)
    at jenkins.model.Jenkins$CloudList.onModified(Jenkins.java:525)
    at hudson.util.PersistedList.replaceBy(PersistedList.java:85)
    at io.jenkins.plugins.casc.impl.attributes.DescribableListAttribute.setValue(DescribableListAttribute.java:18)
    at io.jenkins.plugins.casc.impl.attributes.DescribableListAttribute.setValue(DescribableListAttribute.java:9)
    at io.jenkins.plugins.casc.BaseConfigurator.configure(BaseConfigurator.java:359)
Caused: io.jenkins.plugins.casc.ConfiguratorException: cloud: Failed to set attribute clouds(class: class hudson.slaves.Cloud, multiple: true)
    at io.jenkins.plugins.casc.BaseConfigurator.configure(BaseConfigurator.java:361)
    at io.jenkins.plugins.casc.BaseConfigurator.configure(BaseConfigurator.java:271)
Caused: io.jenkins.plugins.casc.ConfiguratorException: Failed to save hudson.model.Hudson@484ad4e9
    at io.jenkins.plugins.casc.BaseConfigurator.configure(BaseConfigurator.java:274)
    at io.jenkins.plugins.casc.ConfigurationAsCode.lambda$configureWith$4(ConfigurationAsCode.java:694)
    at io.jenkins.plugins.casc.ConfigurationAsCode.invokeWith(ConfigurationAsCode.java:664)
Caused: io.jenkins.plugins.casc.ConfiguratorException: jenkins: error configuring 'jenkins' with class io.jenkins.plugins.casc.core.JenkinsConfigurator configurator
    at io.jenkins.plugins.casc.ConfigurationAsCode.invokeWith(ConfigurationAsCode.java:670)
    at io.jenkins.plugins.casc.ConfigurationAsCode.configureWith(ConfigurationAsCode.java:694)
    at io.jenkins.plugins.casc.ConfigurationAsCode.configureWith(ConfigurationAsCode.java:585)
    at io.jenkins.plugins.casc.ConfigurationAsCode.configure(ConfigurationAsCode.java:280)
    at io.jenkins.plugins.casc.ConfigurationAsCode.init(ConfigurationAsCode.java:272)
Caused: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:104)
Caused: java.lang.Error
    at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:110)
    at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:175)
    at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
    at jenkins.model.Jenkins$5.runTask(Jenkins.java:1124)
    at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
    at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused: org.jvnet.hudson.reactor.ReactorException
    at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:282)
    at jenkins.InitReactorRunner.run(InitReactorRunner.java:48)
    at jenkins.model.Jenkins.executeReactor(Jenkins.java:1158)
    at jenkins.model.Jenkins.<init>(Jenkins.java:965)
    at hudson.model.Hudson.<init>(Hudson.java:85)
    at hudson.model.Hudson.<init>(Hudson.java:81)
    at hudson.WebAppMain$3.run(WebAppMain.java:233)
Caused: hudson.util.HudsonFailedToLoad
    at hudson.WebAppMain$3.run(WebAppMain.java:250)
2019-11-19 03:03:09.150+0000 [id=20]    INFO    jenkins.model.Jenkins#cleanUp: Stopping Jenkins
2019-11-19 03:03:09.327+0000 [id=26]    WARNING hudson.model.ComputerSet#<clinit>: Failed to instantiate NodeMonitors
java.nio.channels.ClosedByInterruptException
    at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
    at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:164)
    at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:65)
    at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:109)
    at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
    at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
    at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
    at java.io.FilterInputStream.read(FilterInputStream.java:83)
    at java.io.PushbackInputStream.read(PushbackInputStream.java:139)
    at com.thoughtworks.xstream.core.util.XmlHeaderAwareReader.getHeader(XmlHeaderAwareReader.java:79)
    at com.thoughtworks.xstream.core.util.XmlHeaderAwareReader.<init>(XmlHeaderAwareReader.java:61)
    at com.thoughtworks.xstream.io.xml.AbstractXppDriver.createReader(AbstractXppDriver.java:65)
Caused: com.thoughtworks.xstream.io.StreamException:  : null
    at com.thoughtworks.xstream.io.xml.AbstractXppDriver.createReader(AbstractXppDriver.java:69)
    at com.thoughtworks.xstream.XStream.fromXML(XStream.java:1053)
    at hudson.XmlFile.read(XmlFile.java:147)
Caused: java.io.IOException: Unable to read /var/jenkins_home/nodeMonitors.xml
    at hudson.XmlFile.read(XmlFile.java:149)
    at hudson.model.ComputerSet.<clinit>(ComputerSet.java:445)
    at jenkins.metrics.impl.JenkinsHealthCheckProviderImpl.getHealthChecks(JenkinsHealthCheckProviderImpl.java:89)
    at jenkins.metrics.api.HealthCheckProviderListener.onChange(HealthCheckProviderListener.java:93)
    at jenkins.metrics.api.HealthCheckProviderListener.attach(HealthCheckProviderListener.java:79)
    at jenkins.metrics.api.Metrics.afterExtensionsAugmented(Metrics.java:314)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:104)
    at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:175)
    at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
    at jenkins.model.Jenkins$5.runTask(Jenkins.java:1124)
    at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
    at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
2019-11-19 03:03:09.340+0000 [id=20]    INFO    jenkins.model.Jenkins$18#onAttained: Started termination
2019-11-19 03:03:09.365+0000 [id=20]    INFO    jenkins.model.Jenkins$18#onAttained: Completed termination
2019-11-19 03:03:09.365+0000 [id=20]    INFO    jenkins.model.Jenkins#_cleanUpDisconnectComputers: Starting node disconnection
2019-11-19 03:03:09.367+0000 [id=20]    INFO    jenkins.model.Jenkins#_cleanUpShutdownPluginManager: Stopping plugin manager
2019-11-19 03:03:09.441+0000 [id=20]    INFO    jenkins.model.Jenkins#_cleanUpPersistQueue: Persisting build queue
2019-11-19 03:03:09.457+0000 [id=20]    INFO    jenkins.model.Jenkins#cleanUp: Jenkins stopped

runzexia on 19 Nov 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.