Skywalking: java.lang.OutOfMemoryError: Java heap space

[SummerOfServenteen]

Please answer these questions before submitting your issue.

• Why do you submit this issue?
• [ ] Question or discussion
• [x] Bug
• [ ] Requirement
• [ ] Feature or performance improvement

---

Question

• What do you want to know?

---

Bug

• Which version of SkyWalking, OS and JRE?
  SkyWalking：6.1.0,jre:1.8
• Which company or project?
  jingling
• What happen?
  If possible, provide a way for reproducing the error. e.g. demo application, component version.

---

Requirement or improvement

• Please describe about your requirements or improvement suggestions.

[kezhenxu94]

@SummerOfServenteen Can you provide more information?

[wu-sheng]

@kezhenxu94 We only set version if we plan to accept PR. Before that, we discuss.

[SummerOfServenteen]

hello.When we use skyWalking ,it cause Java heap space。after analyze the dump file ,we find it's conflict with Sentinel.
Here is the detail.

Finally,i find it's method of org.apache.skywalking.apm.agent.core.context.util.ThrowableTransformer.convert2String that cause outOfMemory .

Code analyzing...

Because of Sentinel had overWrite the method of fillInStackTrace(),so the stackTrace of size is 0,
the return value of printStackElement is always false.Then the method of convert2String is in Infinite loop.

My idea is rewrite the method of printStackElement like this :

[SummerOfServenteen]

@wu-sheng,@kezhenxu94,can you help me to solve this problem~

[kezhenxu94]

@SummerOfServenteen Thanks for reporting and elaborate on this, seems you've found a solution, could you please send a pull request to fix this?

[SummerOfServenteen]

👌

[kezhenxu94]

I'll take a look at the issue tomorrow

[wu-sheng]

Why don't we fix append listenser bug and make the overMaxLength works?

[wu-sheng]

Quote from expert

1. Worst case: little memory and very deep stack.
2. Use StackWalker and unsafe API before Java 9.
3. Like Sentinel, do intentionally.

I will raise a PR to make a better fix.

[wu-sheng]

@SummerOfServenteen @kezhenxu94 Please review #2931

[arugal]

https://github.com/apache/skywalking/blob/fa7a1983f9be13280a3d5842a6b99d6b5c43563a/apm-sniffer/apm-agent-core/src/main/java/org/apache/skywalking/apm/agent/core/context/util/ThrowableTransformer.java#L50

This is a weird line @wu-sheng

[wu-sheng]

Why get the cause for deeper stack is weird? In this case, they create a self-ref, which cause OOM only.

[arugal]

Looks like the same throwable is returning

[wu-sheng]

Good catch. That is a bug. I will fix that.

[wu-sheng]

Merged. @SummerOfServenteen Could you try the master branch?

[SummerOfServenteen]

could you please tell me what's the time of your next publication?

[wu-sheng]

I hope I could froze codes next week.

[SummerOfServenteen]

I and my co-worker tried to package a few days ago, but it always failed and he had said that in qq group 2

[wu-sheng]

How fail? There are many messages every day. I can't follow all.
CI checked package for every merge. Should be fine.