Dubbo: Hessian2SerializerFactory 序列化问题

[JoeXie]

有个奇怪的问题:我们使用spring+dubbo2.5.6版本最近发现,在服务重启后,远端调用dubbo provider接口,发现总出现 classcastexception 异常: com.alibaba.com.caucho.hessian.io.SerializerFactory.getDeserializer Hessian/Burlap: 'com.ba.org.MarketPlace' is an unknown class in null: 发现
此处的loader=null,dubbo在发现为null时候, 反序列化对象会使用默认的HashMap序列化对象,导致对象反序列化失败, 问题: 为什么此处的classloader为null ?

[zonghaishang]

'com.ba.org.MarketPlace' is an unknown class in null， I think it should be a class not found， it doesn't matter to classloader

@JoeXie Can you provide a demo to github?

[JoeXie]

@zonghaishang
抱歉,可能描述不清除, 应用在web容器初始化完毕后, 调用远端provider提供的接口,发现:
Caused by: java.lang.ClassCastException: java.util.HashMap cannot be cast to com.ba.org.QueryInst$BackupStatus 跟踪类型异常的时候代码走到如下标红

正常情况下应该是if 判断通过, 拿到相应类型的序列化器

跟踪发现此时deserializer==null,原因是Hessian2SerializerFactory对象在初始化时,拿到的classloader==null

我并未在dubbo中发现设置Thread.currentThread.contextclassloader=null的设置.
debug如下, 我很好奇什么情况下Thread.currentThread().getContextClassLoader() 对象的Thread.currentThread().getContextClassLoader() =null

[zonghaishang]

@JoeXie
You should check if the provider has this class com.ba.org.QueryInst$BackupStatus. If you still can't solve it, please provide a demo for me to check and solve.

[JoeXie]

@zonghaishang
There is a class we need. It is not caused by Class not Found.
If I didn't call rest url immediately and wait 30s, the service will be OK, and the thread context class loader is not null. If I call the url immediately when spring mvc started, the servcie is bad, and the class loader is null.

It seens that the Hessian2SerializerFactory hasn't initiated itself when spring context initated finished.

[chenk008]

i met this bug too in SpringBoot2

[chenk008]

I use service consumer in RocketMq's consumer，“an unknown class in null” appears

[beiwei30]

this issue could be caused by missing class, would you mind to share a project to reproduce this issue on your GitHub space? @chenk008 @JoeCao

[NeymarLiu]

We've had this problem before，Finally, we found that this was a tomcat problem。I suggest you have a look org.apache.catalina.startup.SafeForkJoinWorkerThreadFactory in tomcat-embed-core-xxx.jar，I think you can find the answer in that. You can also find clues through this link
https://bz.apache.org/bugzilla/show_bug.cgi?id=60620

[NeymarLiu]

There are two solutions
No.1 Improve dubbo version at least dubbo 2.5.10
NO.2 Reduced tomcat version Version prior to January 2017 exp：tomcat8.0.1

[JoeXie]

We've had this problem before，Finally, we found that this was a tomcat problem。I suggest you have a look org.apache.catalina.startup.SafeForkJoinWorkerThreadFactory in tomcat-embed-core-xxx.jar，I think you can find the answer in that. You can also find clues through this link
https://bz.apache.org/bugzilla/show_bug.cgi?id=60620
@NeymarLiu
thank you. I will look into tomcat class later.

[JoeXie]

There are two solutions
No.1 Improve dubbo version at least dubbo 2.5.10
@NeymarLiu
If we want to upgrade dubbo to fix this, do you know which class or commit dubbo developers fix this issue?

[NeymarLiu]

sorry ,i don't know ,you can debug this class->SerializerFactory ,you will find thread executed under different versions of dubbo are different ,in dubbo2.5.6 ,dubbo will create a IO work thread to load class ,it use forkjninthread's contextclassload,but ,in dubbo2.5.10 dubbo will use a nettyClientThread,dubbo didn't solve the problem,it just use different thread. but it's not important
so The heart of the problem is this forkjoinpool , you should avoid use parallelStream ,It has a lot of problems.
The default thread pool does not change Thread.contextClassLoader to the system class loader. When ForkJoinPool.commonPool is used in a multi-class loader environment (e.g. a JavaEE container) this causes references to be retained to the class loader associated with the code that first uses the pool. This is highly likely to cause a memory leak.
The InnocuousForkJoinWorkerThread does not modify Thread.contextClassLoader so undesirable class loader references are retained even when using the InnocuousForkJoinWorkerThreadFactory

[beiwei30]

We've had this problem before，Finally, we found that this was a tomcat problem。I suggest you have a look org.apache.catalina.startup.SafeForkJoinWorkerThreadFactory in tomcat-embed-core-xxx.jar，I think you can find the answer in that. You can also find clues through this link
https://bz.apache.org/bugzilla/show_bug.cgi?id=60620

@NeymarLiu is it all because security manager is installed? I didn't look into the details, but I recalled a similar problem I met before when I see ForkJoinPool.

[devpers]

@JoeXie 请问最后是否有问题解决方案