Micrometer: java.lang.InternalError: Memory Pool not found

[wenfengsui]

jvmArgs:

-server -Xms2g -Xmx3g -XX:MaxMetaspaceSize=256m -Xmn2g -XX:SurvivorRatio=6 -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=80 -XX:+ScavengeBeforeFullGC -XX:+CMSScavengeBeforeRemark -XX:+PrintGCDateStamps -verbose:gc -XX:+PrintGCDetails -Xloggc:/logs/srch-goods-info-http-redis-gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/logs/date.hprof -Dspring.profiles.active=local

java -version

java version "1.8.0_191"
Java(TM) SE Runtime Environment (build 1.8.0_191-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)

error log:

Caused by: java.lang.InternalError: Memory Pool not found
at sun.management.MemoryPoolImpl.getUsage0(Native Method)
at sun.management.MemoryPoolImpl.getUsage(MemoryPoolImpl.java:96)
at io.micrometer.core.instrument.binder.jvm.JvmMemoryMetrics.lambda$bindTo$0(JvmMemoryMetrics.java:81)
at io.micrometer.core.instrument.internal.DefaultGauge.value(DefaultGauge.java:40)
at io.micrometer.prometheus.PrometheusMeterRegistry.lambda$newGauge$3(PrometheusMeterRegistry.java:235)
at io.micrometer.prometheus.MicrometerCollector.collect(MicrometerCollector.java:69)
at io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.findNextElement(CollectorRegistry.java:183)
at io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.nextElement(CollectorRegistry.java:216)
at io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.nextElement(CollectorRegistry.java:137)
at io.prometheus.client.exporter.common.TextFormat.write004(TextFormat.java:22)
at org.springframework.boot.actuate.metrics.export.prometheus.PrometheusScrapeEndpoint.scrape(PrometheusScrapeEndpoint.java:50)

[shakuzen]

I wasn't able to reproduce this using Micrometer 1.1.2 in a sample Spring Boot application, using the JVM args you provided with Oracle's java version "1.8.0_201". Can you provide more information? A sample we can run to reproduce the issue would be ideal.

[izeye]

Based on its Javadoc, it seems a JDK bug to me:

    /**
     * Returns an estimate of the memory usage of this memory pool.
     * This method returns <tt>null</tt>
     * if this memory pool is not valid (i.e. no longer exists).
     *
     * <p>
     * This method requests the Java virtual machine to make
     * a best-effort estimate of the current memory usage of this
     * memory pool. For some memory pools, this method may be an
     * expensive operation that requires some computation to determine
     * the estimate.  An implementation should document when
     * this is the case.
     *
     * <p>This method is designed for use in monitoring system
     * memory usage and detecting low memory condition.
     *
     * <p>
     * <b>MBeanServer access</b>:<br>
     * The mapped type of <tt>MemoryUsage</tt> is
     * <tt>CompositeData</tt> with attributes as specified in
     * {@link MemoryUsage#from MemoryUsage}.
     *
     * @return a {@link MemoryUsage} object; or <tt>null</tt> if
     * this pool not valid.
     */
    public MemoryUsage getUsage();

@wenfengsui Is it easily reproducible or just encountered only once?

[wenfengsui]

i can reproduce it in local java environment(oracle 1.8.0_191) or in docker ( openjdk:8-jre-alpine).

[shakuzen]

Could you share a sample application that we could run to reproduce the issue? Does it happen on any machine or only a certain kind?

[izeye]

@wenfengsui Thanks for the quick feedback!

I can reproduce it with a sample with the following steps:

1. Run the sample as follows:

xxx:sample-micrometer-spring-boot izeye$ java -Xms2g -Xmn2g -XX:SurvivorRatio=6 -XX:+UseConcMarkSweepGC -jar build/libs/sample-micrometer-spring-boot.jar 

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.1.2.RELEASE)

2019-02-12 10:58:07.046  INFO 2781 --- [           main] com.izeye.sample.Application             : Starting Application on xxx.local with PID 2781 (/Users/izeye/IdeaProjects/sample-micrometer-spring-boot/build/libs/sample-micrometer-spring-boot.jar started by izeye in /Users/izeye/IdeaProjects/sample-micrometer-spring-boot)
2019-02-12 10:58:07.049  INFO 2781 --- [           main] com.izeye.sample.Application             : No active profile set, falling back to default profiles: default
2019-02-12 10:58:08.305  INFO 2781 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8080 (http)
2019-02-12 10:58:08.340  INFO 2781 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2019-02-12 10:58:08.341  INFO 2781 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.14]
2019-02-12 10:58:08.360  INFO 2781 --- [           main] o.a.catalina.core.AprLifecycleListener   : The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: [/Users/izeye/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:.]
2019-02-12 10:58:08.602  INFO 2781 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2019-02-12 10:58:08.602  INFO 2781 --- [           main] o.s.web.context.ContextLoader            : Root WebApplicationContext: initialization completed in 1508 ms
2019-02-12 10:58:09.219  INFO 2781 --- [           main] o.s.s.concurrent.ThreadPoolTaskExecutor  : Initializing ExecutorService 'applicationTaskExecutor'
2019-02-12 10:58:09.521  INFO 2781 --- [           main] o.s.b.a.e.web.EndpointLinksResolver      : Exposing 15 endpoint(s) beneath base path '/actuator'
2019-02-12 10:58:09.617  INFO 2781 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
2019-02-12 10:58:09.621  INFO 2781 --- [           main] com.izeye.sample.Application             : Started Application in 3.057 seconds (JVM running for 3.476)
2019-02-12 10:58:15.257  INFO 2781 --- [nio-8080-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring DispatcherServlet 'dispatcherServlet'
2019-02-12 10:58:15.257  INFO 2781 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Initializing Servlet 'dispatcherServlet'
2019-02-12 10:58:15.264  INFO 2781 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Completed initialization in 7 ms

1. Hit the Spring Boot Actuator jvm.memory.used metrics link and then the following error occurs:

2019-02-12 10:58:15.368 ERROR 2781 --- [nio-8080-exec-1] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Handler dispatch failed; nested exception is java.lang.InternalError: Memory Pool not found] with root cause

java.lang.InternalError: Memory Pool not found
        at sun.management.MemoryPoolImpl.getUsage0(Native Method) ~[na:1.8.0_191]
        at sun.management.MemoryPoolImpl.getUsage(MemoryPoolImpl.java:96) ~[na:1.8.0_191]
        at io.micrometer.core.instrument.binder.jvm.JvmMemoryMetrics.lambda$bindTo$0(JvmMemoryMetrics.java:81) ~[micrometer-core-1.1.2.jar!/:1.1.2]
        at io.micrometer.core.instrument.internal.DefaultGauge.value(DefaultGauge.java:40) ~[micrometer-core-1.1.2.jar!/:1.1.2]
        at io.micrometer.core.instrument.Measurement.getValue(Measurement.java:39) ~[micrometer-core-1.1.2.jar!/:1.1.2]
        at org.springframework.boot.actuate.metrics.MetricsEndpoint.lambda$mergeMeasurements$5(MetricsEndpoint.java:139) ~[spring-boot-actuator-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
        at java.util.Collections$SingletonList.forEach(Collections.java:4822) ~[na:1.8.0_191]
        at org.springframework.boot.actuate.metrics.MetricsEndpoint.mergeMeasurements(MetricsEndpoint.java:138) ~[spring-boot-actuator-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
        at org.springframework.boot.actuate.metrics.MetricsEndpoint.lambda$getSamples$4(MetricsEndpoint.java:133) ~[spring-boot-actuator-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
        at java.util.ArrayList.forEach(ArrayList.java:1257) ~[na:1.8.0_191]
        at org.springframework.boot.actuate.metrics.MetricsEndpoint.getSamples(MetricsEndpoint.java:133) ~[spring-boot-actuator-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
        at org.springframework.boot.actuate.metrics.MetricsEndpoint.metric(MetricsEndpoint.java:89) ~[spring-boot-actuator-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_191]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_191]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_191]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_191]
        at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:246) ~[spring-core-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
        at org.springframework.boot.actuate.endpoint.invoke.reflect.ReflectiveOperationInvoker.invoke(ReflectiveOperationInvoker.java:76) ~[spring-boot-actuator-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
        at org.springframework.boot.actuate.endpoint.annotation.AbstractDiscoveredOperation.invoke(AbstractDiscoveredOperation.java:61) ~[spring-boot-actuator-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
        at org.springframework.boot.actuate.endpoint.web.servlet.AbstractWebMvcEndpointHandlerMapping$ServletWebOperationAdapter.handle(AbstractWebMvcEndpointHandlerMapping.java:294) ~[spring-boot-actuator-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
        at org.springframework.boot.actuate.endpoint.web.servlet.AbstractWebMvcEndpointHandlerMapping$OperationHandler.handle(AbstractWebMvcEndpointHandlerMapping.java:355) ~[spring-boot-actuator-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_191]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_191]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_191]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_191]
        at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:189) ~[spring-web-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
        at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138) ~[spring-web-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
        at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:102) ~[spring-webmvc-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895) ~[spring-webmvc-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:800) ~[spring-webmvc-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
        at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1038) ~[spring-webmvc-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:942) ~[spring-webmvc-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1005) ~[spring-webmvc-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
        at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:897) ~[spring-webmvc-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:634) ~[tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:882) ~[spring-webmvc-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) ~[tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) ~[tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) ~[tomcat-embed-websocket-9.0.14.jar!/:9.0.14]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.springframework.boot.actuate.web.trace.servlet.HttpTraceFilter.doFilterInternal(HttpTraceFilter.java:90) ~[spring-boot-actuator-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) ~[spring-web-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:92) ~[spring-web-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93) ~[spring-web-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:117) ~[spring-boot-actuator-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
        at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:106) ~[spring-boot-actuator-2.1.2.RELEASE.jar!/:2.1.2.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200) ~[spring-web-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-5.1.4.RELEASE.jar!/:5.1.4.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199) ~[tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:490) [tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) [tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) [tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) [tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) [tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408) [tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) [tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:834) [tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1417) [tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_191]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_191]
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-9.0.14.jar!/:9.0.14]
        at java.lang.Thread.run(Thread.java:748) [na:1.8.0_191]

The test environment is as follows:

xxx:sample-micrometer-spring-boot izeye$ uname -a
Darwin xxx.local 17.7.0 Darwin Kernel Version 17.7.0: Wed Oct 10 23:06:14 PDT 2018; root:xnu-4570.71.13~1/RELEASE_X86_64 x86_64
xxx:sample-micrometer-spring-boot izeye$ java -version
java version "1.8.0_191"
Java(TM) SE Runtime Environment (build 1.8.0_191-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)
xxx:sample-micrometer-spring-boot izeye$

Note I stripped down the original JVM options to minimum to reproduce it and noticed "-Xmn" seems too high.

[izeye]

FWIW with changing "-Xmn2g" to "-Xmn1g" the error won't occur. I'm not sure whether this setup is valid or not although it doesn't seem to be usual to me.

[izeye]

I don't have experience with "-Xmn" JVM option and I thought it's for young space and commented here based on the IBM document but the Oracle document says it's for minimum heap space. I'm confused which document is correct.

[izeye]

The previous Oracle document seems wrong and this Oracle document says the same thing as the IBM document, hence it's for young space.

[izeye]

If the setup is valid, we can be defensive (simply not providing memory usage metrics).

[shakuzen]

Just sharing some research: seemingly the same issue has been reported to the Prometheus JMX exporter previously (see https://github.com/prometheus/jmx_exporter/issues/251).

As @izeye pointed out, it does sound like a JDK bug in that it should be returning null, not throwing the error. Not quite the same but a similar JDK bug was fixed before: JDK-8025089. Another ticket JDK-8154458 for a test failure with this error is still open. Maybe we should report this or at least ask on the (I guess hotspot?) mailing list?

While being defensive about this seems good, I also don't want to add too much complexity for something that shouldn't be happening. I'm also not sure if that combination of args are valid. Elasticsearch decided to not handle it (see https://github.com/elastic/elasticsearch/pull/29625), though probably that is more because they haven't had any (reproducible) reports of the issue from users. Geode had the same reported error GEODE-1349 and guards against it now:

https://github.com/apache/geode/blob/33077b3dab41260c70cece5b4f7ff1c42501b01c/geode-core/src/main/java/org/apache/geode/internal/stats50/VMStats50.java#L482-L503

If this only happens with +UseConcMarkSweepGC, I would also note that GC has been deprecated from Java 9 for eventual removal, though we still support Java 8 and it is not deprecated there.

[izeye]

@shakuzen I noticed null MemoryUsage handling is missing, so I created #1225. I also included defensive code for this in the PR as I think it's a reasonable compromise for now but I'll revert it if inappropriate.

[wenfengsui]

changing "-Xmn2g" to "-Xmn1g", works for me.