Dropwizard: Dropwizard process hanging after exception thrown in Application.run(config, env)

[randyp]

It appears Server/EnvironmentCommand does not shutdown managed executors when an exception is thrown in application.run, thus causing dropwizard applications to hang even though the application has shutdown.

Historically the behavior I expected was for managed objects to be shutdown, even if they had not been started. If this is no longer the expected behavior, my apologies, just close this issue/pr.

Technically this is affecting all managed objects, not just managed executors, but is presenting itself most seriously through executors.

Unit test to demonstrate issue in #1459

[spudone]

AbstractServerFactory.createThreadPool looks like it does not use daemon threads, which would be one way to fix this issue (not sure if that is the only place though).

[randyp]

That would 'fix' the zombie process problem, but would not address the core issue: managed objects in the LifecycleEnvironment are not being stopped if an exception is thrown in Application#run.

Unless the current behavior is intentional dropwizard does not shutdown/stop all managed objects such as object/connection pools, executor services, etc.

In addition, I suspect that threads in the executor services were intentionally not daemonized, but could be wrong.

[isaki]

To get around this behavior you can do several things.

1. If you are using a standard executor under the hood, you could provide your own thread factory that has an UnhandledExceptionHandler that invokes System.exit.
2. If you are using a scheduled executor under the hood, #1 won't work. You will have handle the exceptions in your runnable.
3. Write a service that watches other services and restarts/aborts if they have a problem. Again, UnhandledExceptionHandler is useful here. For scheduled tasks, you can track the ScheduledFutures and periodically call get() to capture the ExecutionException's thrown (they should contain the original exception).

You could also write an event driven system using Guava's AsynchronousEventBus that can restart services/abort your process. You will need to engineer your implementations of Managed to generate events instead of exceptions (or wrappers that catch the exceptions and generate events). This is a bit more work but is the most flexible. I strongly encourage you to use the AsynchronousEventBus as the normal EventBus executes in the thread that generated the event, which may be a problem.

[randyp]

Closing this issue, as there is a reasonable fix for this issue.

Creation of executor services should deferred until server start, and the dropwizard does not yet support this out of this box.