Quarkus: ConcurrentModificationException when using EntityManager from async scheduler tasks

[Postremus]

Describe the bug
We currently have an production application running on wildfly 10.
I created a POC a while back for the same application running in quarkus. The poc was based on quarkus 1.3.0.CR2, and never updated.

We need to run some tasks asynchron, like handling of our recalculation queues.

Basicly, this is what I do in my code:
StuffImporterScheduler.java

import io.quarkus.scheduler.Scheduled;

import javax.enterprise.context.ApplicationScoped;
import javax.inject.Inject;
import javax.transaction.Transactional;
import javax.xml.bind.JAXBException;
import java.util.List;

@ApplicationScoped
@Transactional(Transactional.TxType.REQUIRES_NEW)
public class StuffImporterScheduler{

@Inject
StuffImporterSchedulerService service;

    @Scheduled(every = SCHEDULER_INTERVAL + "m")
    public void importScheduler() {
        service.processImports();
    }
}

StuffImporterSchedulerService.java

import io.quarkus.scheduler.Scheduled;

import javax.enterprise.context.ApplicationScoped;
import javax.inject.Inject;
import javax.transaction.Transactional;
import javax.xml.bind.JAXBException;
import java.util.List;

@ApplicationScoped
public class StuffImporterSchedulerService{
    @Inject
    ManagedExecutor executor;

    @Inject
    StuffDAO dao;

    @Transactional(Transactional.TxType.NOT_SUPPORTED)
    public void processImports() {

        executor.runAsync(() -> {
// Import some stuff Entities
// By using jpa methods of the entitymanager 
             List<Stuff> stuffList = dao.getStuff()
        });
    }
}

StuffDAO.java

import io.quarkus.scheduler.Scheduled;

import javax.enterprise.context.ApplicationScoped;
import javax.inject.Inject;
import javax.transaction.Transactional;
import javax.xml.bind.JAXBException;
import java.util.List;

@ApplicationScoped
public class StuffDAO{
    @Inject
    EntityManager em;

    public List<Stuff> getStuff() {
     // get Stuff entities, by using jpa criteria queries on the em.
      ....
   }
}

I use the following modules:

23.04.2020 04:17:02 INFO  [io.quarkus] Installed features: [agroal, cdi, config-yaml, hibernate-orm, hibernate-validator, jdbc-postgresql, narayana-jta, resteasy, resteasy-jackson, scheduler, smallrye-context-propagation, smallrye-health, smallrye-metrics, smallrye-openapi, swagger-ui]

Expected behavior
No sporadic exception when using the entity manager from an async task.

Actual behavior
Sometimes, I get the following error. I guess it has something to do with the asynchronous job?

23.04.2020 04:37:03 ERROR  [io.quarkus.arc.impl.InstanceHandleImpl] Error occured while destroying instance of bean [io.quarkus.hibernate.orm.runtime.RequestScopedEntityManagerHolder_Bean]: java.util.ConcurrentModificationException
23.04.2020 04:52:03 WARN  [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] SQL Error: 0, SQLState: 55000
23.04.2020 04:52:03 ERROR  [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] This statement has been closed.

To Reproduce
I am myself not able to reproduce this reliable.

Configuration

# Swagger and Openapi Settings
quarkus.swagger-ui.always-include=true
#Logging Settings
quarkus.log.console.format=%d{dd.MM.yyyy HH:mm:ss} %p  [%c] %s%e%n
quarkus.log.console.color=false
quarkus.log.console.async=true
quarkus.log.file.enable=true
quarkus.log.file.path=historyservice/server.log
quarkus.log.file.format=%d{dd.MM.yyyy HH:mm:ss} %p  [%c] %s%e%n
quarkus.log.file.async=true
# CORS Settings
quarkus.http.cors=true
# Metrics Settings
quarkus.datasource.metrics.enabled=true
quarkus.hibernate-orm.metrics.enabled=false
# Database settings
quarkus.hibernate-orm.log.sql=false
quarkus.hibernate-orm.database.generation=update
quarkus.datasource.driver=org.postgresql.Driver
# Dev Settings (overwrites normal settings when in dev mode)
%dev.quarkus.hibernate-orm.log.sql=false
%dev.quarkus.hibernate-orm.database.generation=none
%dev.quarkus.log.category."org.hibernate.type.descriptor.sql.BasicBinder".level=INFO

Environment (please complete the following information):

• Output of uname -a or ver:
  MSYS_NT-10.0 NANB7NLNVP2 2.10.0(0.325/5/3) 2018-06-13 23:34 x86_64 Msys
• Output of java -version:
  java version "1.8.0_241"
  Java(TM) SE Runtime Environment (build 1.8.0_241-b07)
  Java HotSpot(TM) 64-Bit Server VM (build 25.241-b07, mixed mode)
• Quarkus version or git rev:
  1.3.0.CR2
• Build tool (ie. output of mvnw --version or gradlew --version):
  Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
  Maven home: C:eclipsetoolsapache-mavenbin..
  Java version: 1.8.0_181, vendor: Oracle Corporation, runtime: C:Program FilesJavajdk1.8.0_181jre
  Default locale: de_DE, platform encoding: Cp1252
  OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"

[machi1990]

@Postremus can you try https://github.com/quarkusio/quarkus/pull/8870 ?

[gsmet]

An important information is missing: how do you get your EntityManager?

[Postremus]

@gsmet

@Inject
EntityManager em;

I updated the issue description with a more complet example.

[Postremus]

@machi1990
I tested with your PR.
Still geht the same error message.

[machi1990]

Yes thanks for checking. @gsmet @stuartwdouglas suggested that it is something deep that's need further checking. I'll do that later today. Sorry for the noise :-)

[machi1990]

@Postremus d you think it is possible to strip out some things from your application and prepare a simple reproducer? Thanks.

[Postremus]

@machi1990 I had no luck creating a reproducer the first time I saw this error - about a month ago.
I will try to create a reproducer, but I might not have any luck in this endeavor.

[Postremus]

Was not able to reproduce this anymore on current quarkus versions, closing this for now.
Thank you for your help anyway!