Micrometer: JvmGcMetrics causes deployment failures on Wildfly

Created on 2 Nov 2020 · 3Comments · Source: micrometer-metrics/micrometer
We just started seeing one of Boot's Wildfly deployment tests fail. I don't yet understand why it didn't fail as soon as we upgraded to Micrometer 1.6, but I believe the failure is due to the changes made for #2306. com.sun.management.GcInfo now appears in a method signature which causes a failure when the class is being introspected. Something similar has been seen and fixed in the past: #401. I think another similar change is needed for GcInfo.
The failure that we're seeing is the following:
2020-11-02 16:38:19,678 INFO  [stdout] (ServerService Thread Pool -- 91) 
2020-11-02 16:38:19,678 INFO  [stdout] (ServerService Thread Pool -- 91)   .   ____          _            __ _ _
2020-11-02 16:38:19,678 INFO  [stdout] (ServerService Thread Pool -- 91)  /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
2020-11-02 16:38:19,679 INFO  [stdout] (ServerService Thread Pool -- 91) ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
2020-11-02 16:38:19,679 INFO  [stdout] (ServerService Thread Pool -- 91)  \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
2020-11-02 16:38:19,679 INFO  [stdout] (ServerService Thread Pool -- 91)   '  |____| .__|_| |_|_| |_\__, | / / / /
2020-11-02 16:38:19,680 INFO  [stdout] (ServerService Thread Pool -- 91)  =========|_|==============|___/=/_/_/_/
2020-11-02 16:38:19,681 INFO  [stdout] (ServerService Thread Pool -- 91)  :: Spring Boot ::       (v2.4.0-SNAPSHOT)
2020-11-02 16:38:19,681 INFO  [stdout] (ServerService Thread Pool -- 91) 
2020-11-02 16:38:19,689 INFO  [org.hibernate.validator.internal.util.Version] (background-preinit) HV000001: Hibernate Validator 6.0.20.Final
2020-11-02 16:38:19,765 INFO  [sample.app.DeploymentTestApplication] (ServerService Thread Pool -- 91) Starting DeploymentTestApplication using Java 11.0.8 on 52c0eb6f7ecd with PID 103 (started by jboss in /opt/jboss)
2020-11-02 16:38:19,767 INFO  [sample.app.DeploymentTestApplication] (ServerService Thread Pool -- 91) No active profile set, falling back to default profiles: default
2020-11-02 16:38:20,901 INFO  [io.undertow.servlet] (ServerService Thread Pool -- 91) Initializing Spring embedded WebApplicationContext
2020-11-02 16:38:20,902 INFO  [org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext] (ServerService Thread Pool -- 91) Root WebApplicationContext: initialization completed in 1096 ms
2020-11-02 16:38:21,136 WARN  [org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext] (ServerService Thread Pool -- 91) Exception encountered during context initialization - cancelling refresh attempt: org.s
pringframework.context.ApplicationContextException: Unable to start web server; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'webMvcMetricsFilter' defined in class path resource 
[org/springframework/boot/actuate/autoconfigure/metrics/web/servlet/WebMvcMetricsAutoConfiguration.class]: Unsatisfied dependency expressed through method 'webMvcMetricsFilter' parameter 0; nested exception is org.springframework.beans.factory.BeanC
reationException: Error creating bean with name 'simpleMeterRegistry' defined in class path resource [org/springframework/boot/actuate/autoconfigure/metrics/export/simple/SimpleMetricsExportAutoConfiguration.class]: Initialization of bean failed; ne
sted exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jvmGcMetrics' defined in class path resource [org/springframework/boot/actuate/autoconfigure/metrics/JvmMetricsAutoConfiguration.class]: Post-p
rocessing of merged bean definition failed; nested exception is java.lang.IllegalStateException: Failed to introspect Class [io.micrometer.core.instrument.binder.jvm.JvmGcMetrics] from ClassLoader [ModuleClassLoader for Module "deployment.spring-boo
t.war" from Service Module Loader]
2020-11-02 16:38:21,150 INFO  [org.springframework.boot.autoconfigure.logging.ConditionEvaluationReportLoggingListener] (ServerService Thread Pool -- 91) 

Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2020-11-02 16:38:21,169 ERROR [org.springframework.boot.SpringApplication] (ServerService Thread Pool -- 91) Application run failed: org.springframework.context.ApplicationContextException: Unable to start web server; nested exception is org.springf
ramework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'webMvcMetricsFilter' defined in class path resource [org/springframework/boot/actuate/autoconfigure/metrics/web/servlet/WebMvcMetricsAutoConfiguration.class]: Unsa
tisfied dependency expressed through method 'webMvcMetricsFilter' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'simpleMeterRegistry' defined in class path resource [org/sprin
gframework/boot/actuate/autoconfigure/metrics/export/simple/SimpleMetricsExportAutoConfiguration.class]: Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jvmGc
Metrics' defined in class path resource [org/springframework/boot/actuate/autoconfigure/metrics/JvmMetricsAutoConfiguration.class]: Post-processing of merged bean definition failed; nested exception is java.lang.IllegalStateException: Failed to intr
ospect Class [io.micrometer.core.instrument.binder.jvm.JvmGcMetrics] from ClassLoader [ModuleClassLoader for Module "deployment.spring-boot.war" from Service Module Loader]
    at deployment.spring-boot.war//org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.onRefresh(ServletWebServerApplicationContext.java:162)
    at deployment.spring-boot.war//org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:582)
    at deployment.spring-boot.war//org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:144)
    at deployment.spring-boot.war//org.springframework.boot.SpringApplication.refresh(SpringApplication.java:767)
    at deployment.spring-boot.war//org.springframework.boot.SpringApplication.refresh(SpringApplication.java:759)
    at deployment.spring-boot.war//org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:426)
    at deployment.spring-boot.war//org.springframework.boot.SpringApplication.run(SpringApplication.java:326)
    at deployment.spring-boot.war//org.springframework.boot.web.servlet.support.SpringBootServletInitializer.run(SpringBootServletInitializer.java:173)
    at deployment.spring-boot.war//org.springframework.boot.web.servlet.support.SpringBootServletInitializer.createRootApplicationContext(SpringBootServletInitializer.java:153)
    at deployment.spring-boot.war//org.springframework.boot.web.servlet.support.SpringBootServletInitializer.onStartup(SpringBootServletInitializer.java:95)
    at deployment.spring-boot.war//org.springframework.web.SpringServletContainerInitializer.onStartup(SpringServletContainerInitializer.java:174)
    at [email protected]//io.undertow.servlet.core.DeploymentManagerImpl$1.call(DeploymentManagerImpl.java:204)
    at [email protected]//io.undertow.servlet.core.DeploymentManagerImpl$1.call(DeploymentManagerImpl.java:186)
    at [email protected]//io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:42)
    at [email protected]//io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
    at [email protected]//org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105)
    at [email protected]//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1530)
    at [email protected]//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1530)
    at [email protected]//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1530)
    at [email protected]//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1530)
    at [email protected]//io.undertow.servlet.core.DeploymentManagerImpl.deploy(DeploymentManagerImpl.java:252)
    at [email protected]//org.wildfly.extension.undertow.deployment.UndertowDeploymentService.startContext(UndertowDeploymentService.java:96)
    at [email protected]//org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:78)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at [email protected]//org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
    at [email protected]//org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982)
    at [email protected]//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
    at [email protected]//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
    at java.base/java.lang.Thread.run(Thread.java:834)
    at [email protected]//org.jboss.threads.JBossThread.run(JBossThread.java:485)
Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'webMvcMetricsFilter' defined in class path resource [org/springframework/boot/actuate/autoconfigure/metrics/web/servlet/WebMvcMetricsAutoConf
iguration.class]: Unsatisfied dependency expressed through method 'webMvcMetricsFilter' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'simpleMeterRegistry' defined in class pa
th resource [org/springframework/boot/actuate/autoconfigure/metrics/export/simple/SimpleMetricsExportAutoConfiguration.class]: Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating
 bean with name 'jvmGcMetrics' defined in class path resource [org/springframework/boot/actuate/autoconfigure/metrics/JvmMetricsAutoConfiguration.class]: Post-processing of merged bean definition failed; nested exception is java.lang.IllegalStateExc
eption: Failed to introspect Class [io.micrometer.core.instrument.binder.jvm.JvmGcMetrics] from ClassLoader [ModuleClassLoader for Module "deployment.spring-boot.war" from Service Module Loader]
    at deployment.spring-boot.war//org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:798)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:539)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1336)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1179)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:571)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:531)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:213)
    at deployment.spring-boot.war//org.springframework.boot.web.servlet.ServletContextInitializerBeans.getOrderedBeansOfType(ServletContextInitializerBeans.java:212)
    at deployment.spring-boot.war//org.springframework.boot.web.servlet.ServletContextInitializerBeans.getOrderedBeansOfType(ServletContextInitializerBeans.java:203)
    at deployment.spring-boot.war//org.springframework.boot.web.servlet.ServletContextInitializerBeans.addServletContextInitializerBeans(ServletContextInitializerBeans.java:97)
    at deployment.spring-boot.war//org.springframework.boot.web.servlet.ServletContextInitializerBeans.<init>(ServletContextInitializerBeans.java:86)
    at deployment.spring-boot.war//org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.getServletContextInitializerBeans(ServletWebServerApplicationContext.java:259)
    at deployment.spring-boot.war//org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.selfInitialize(ServletWebServerApplicationContext.java:233)
    at deployment.spring-boot.war//org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.createWebServer(ServletWebServerApplicationContext.java:190)
    at deployment.spring-boot.war//org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.onRefresh(ServletWebServerApplicationContext.java:159)
    ... 30 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'simpleMeterRegistry' defined in class path resource [org/springframework/boot/actuate/autoconfigure/metrics/export/simple/SimpleMetricsExportAutoConfi
guration.class]: Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jvmGcMetrics' defined in class path resource [org/springframework/boot/actuate/autoconfigure/
metrics/JvmMetricsAutoConfiguration.class]: Post-processing of merged bean definition failed; nested exception is java.lang.IllegalStateException: Failed to introspect Class [io.micrometer.core.instrument.binder.jvm.JvmGcMetrics] from ClassLoader [M
oduleClassLoader for Module "deployment.spring-boot.war" from Service Module Loader]
    at deployment.spring-boot.war//org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:617)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:531)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
    at deployment.spring-boot.war//org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1367)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1287)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:885)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:789)
    ... 47 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jvmGcMetrics' defined in class path resource [org/springframework/boot/actuate/autoconfigure/metrics/JvmMetricsAutoConfiguration.class]: Post-processi
ng of merged bean definition failed; nested exception is java.lang.IllegalStateException: Failed to introspect Class [io.micrometer.core.instrument.binder.jvm.JvmGcMetrics] from ClassLoader [ModuleClassLoader for Module "deployment.spring-boot.war" 
from Service Module Loader]
    at deployment.spring-boot.war//org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:586)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:531)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
    at deployment.spring-boot.war//org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.DefaultListableBeanFactory.addCandidateEntry(DefaultListableBeanFactory.java:1592)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:1549)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveMultipleBeans(DefaultListableBeanFactory.java:1393)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1325)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.DefaultListableBeanFactory$DependencyObjectProvider.resolveStream(DefaultListableBeanFactory.java:2095)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.DefaultListableBeanFactory$DependencyObjectProvider.orderedStream(DefaultListableBeanFactory.java:2089)
    at deployment.spring-boot.war//org.springframework.boot.actuate.autoconfigure.metrics.MeterRegistryConfigurer.addBinders(MeterRegistryConfigurer.java:86)
    at deployment.spring-boot.war//org.springframework.boot.actuate.autoconfigure.metrics.MeterRegistryConfigurer.configure(MeterRegistryConfigurer.java:68)
    at deployment.spring-boot.war//org.springframework.boot.actuate.autoconfigure.metrics.MeterRegistryPostProcessor.postProcessAfterInitialization(MeterRegistryPostProcessor.java:64)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsAfterInitialization(AbstractAutowireCapableBeanFactory.java:444)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1792)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:609)
    ... 57 more
Caused by: java.lang.IllegalStateException: Failed to introspect Class [io.micrometer.core.instrument.binder.jvm.JvmGcMetrics] from ClassLoader [ModuleClassLoader for Module "deployment.spring-boot.war" from Service Module Loader]
    at deployment.spring-boot.war//org.springframework.util.ReflectionUtils.getDeclaredMethods(ReflectionUtils.java:481)
    at deployment.spring-boot.war//org.springframework.util.ReflectionUtils.doWithLocalMethods(ReflectionUtils.java:321)
    at deployment.spring-boot.war//org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.buildLifecycleMetadata(InitDestroyAnnotationBeanPostProcessor.java:232)
    at deployment.spring-boot.war//org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.findLifecycleMetadata(InitDestroyAnnotationBeanPostProcessor.java:210)
    at deployment.spring-boot.war//org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessMergedBeanDefinition(InitDestroyAnnotationBeanPostProcessor.java:149)
    at deployment.spring-boot.war//org.springframework.context.annotation.CommonAnnotationBeanPostProcessor.postProcessMergedBeanDefinition(CommonAnnotationBeanPostProcessor.java:294)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyMergedBeanDefinitionPostProcessors(AbstractAutowireCapableBeanFactory.java:1100)
    at deployment.spring-boot.war//org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:583)
    ... 75 more
Caused by: java.lang.NoClassDefFoundError: com/sun/management/GcInfo
    at java.base/java.lang.Class.getDeclaredMethods0(Native Method)
    at java.base/java.lang.Class.privateGetDeclaredMethods(Class.java:3166)
    at java.base/java.lang.Class.getDeclaredMethods(Class.java:2309)
    at deployment.spring-boot.war//org.springframework.util.ReflectionUtils.getDeclaredMethods(ReflectionUtils.java:463)
    ... 82 more
Caused by: java.lang.ClassNotFoundException: com.sun.management.GcInfo from [Module "deployment.spring-boot.war" from Service Module Loader]
    at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:255)
    at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:410)
    at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:398)
    at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:116)
    ... 86 more
bug
Source
wilkinsona
Most helpful comment

Thanks, both. Things look good with 1.6.1-SNAPSHOT and without the workaround to make the com.sun.management classes available.
wilkinsona on 5 Nov 2020
👍2
All 3 comments

@wilkinsona Thanks for the issue. I created #2332 to try to resolve this.
izeye on 2 Nov 2020
@wilkinsona thanks for the report. @izeye's fix should be available in 1.6.1-SNAPSHOT versions now if you can confirm there are no remaining issues.