Spring-framework: PrematureCloseException when using connection pooling and server returns "Connection: Close"

[jnizet]

Using Spring Boot 2.1.3-RELEASE, I consistently get the following exception when doing two consecutive requests to the Nominatim OpenStreetMap API using WebClient.
Disabling the pooling of connections makes it work.
I have no idea if this is a WebClient bug, or a reactor-netty bug, or a bug in the remote host. But it took me some time to at least find the workaround, and if it's a downstream bug, I guess WebClient and/or reactor-netty could at least give a hint about how to workaround the issue in the exception message.

Here's a complete Spring Boot 2.1.3 application reproducing the issue:

package com.example.webclientissue;

import java.util.List;
import java.util.Map;

import com.fasterxml.jackson.annotation.JsonProperty;
import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.WebApplicationType;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.web.reactive.function.client.WebClient;
import reactor.core.publisher.Mono;

@SpringBootApplication
public class WebclientissueApplication implements CommandLineRunner {

    private final WebClient webClient;

    public WebclientissueApplication(WebClient.Builder webClientBuilder) {
        this.webClient = webClientBuilder
            .baseUrl("https://nominatim.openstreetmap.org")
            // .clientConnector(new ReactorClientHttpConnector(HttpClient.newConnection().compress(true)))
            .build();
    }

    @Override
    public void run(String... args) throws Exception {
        DetailsResult details = details(44, 1.5);
        System.out.println("details = " + details);
    }

    public static void main(String[] args) {
        SpringApplication app = new SpringApplication(WebclientissueApplication.class);
        app.setWebApplicationType(WebApplicationType.NONE);
        app.run(args);
    }

    private DetailsResult details(double latitude, double longitude) {
        return loadReverse(latitude, longitude).flatMap(it -> loadDetails(it)).block();
    }

    private Mono<ReverseResult> loadReverse(double latitude, double longitude) {
        return webClient.get().uri(builder ->
            builder.path("/reverse")
                   .queryParam("format", "json")
                   .queryParam("lat", Double.toString(latitude))
                   .queryParam("lon", Double.toString(longitude))
                   .build()
        ).retrieve().bodyToMono(ReverseResult.class);
    }

    private Mono<DetailsResult> loadDetails(ReverseResult reverseResult) {
        return webClient.get().uri(builder ->
            builder.path("/details")
                   .queryParam("format", "json")
                   .queryParam("osmtype", reverseResult.getOsmType().substring(0, 1).toUpperCase())
                   .queryParam("osmid", reverseResult.getOsmId().toString())
                   .queryParam("hierarchy", "0")
                   .queryParam("addressdetails", "1")
                   .build()
        ).retrieve().bodyToMono(DetailsResult.class);
    }

    public static class DetailsResult {
        private List<Map<String, Object>> address;

        public List<Map<String, Object>> getAddress() {
            return address;
        }

        public void setAddress(List<Map<String, Object>> address) {
            this.address = address;
        }

        @Override
        public String toString() {
            return "DetailsResult{" +
                "address=" + address +
                '}';
        }
    }

    public static class ReverseResult {
        @JsonProperty("osm_id")
        private Long osmId;

        @JsonProperty("osm_type")
        private String osmType;

        public Long getOsmId() {
            return osmId;
        }

        public void setOsmId(Long osmId) {
            this.osmId = osmId;
        }

        public String getOsmType() {
            return osmType;
        }

        public void setOsmType(String osmType) {
            this.osmType = osmType;
        }
    }
}

You can uncomment the line

            // .clientConnector(new ReactorClientHttpConnector(HttpClient.newConnection().compress(true)))

to make the code run without any exception.

Here's the full stack trace of the exception:

java.lang.IllegalStateException: Failed to execute CommandLineRunner
    at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:816) ~[spring-boot-2.1.3.RELEASE.jar:2.1.3.RELEASE]
    at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:797) ~[spring-boot-2.1.3.RELEASE.jar:2.1.3.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:324) ~[spring-boot-2.1.3.RELEASE.jar:2.1.3.RELEASE]
    at com.example.webclientissue.WebclientissueApplication.main(WebclientissueApplication.java:35) [classes/:na]
Caused by: reactor.core.Exceptions$ReactiveException: reactor.netty.http.client.PrematureCloseException: Connection prematurely closed BEFORE response
    at reactor.core.Exceptions.propagate(Exceptions.java:326) ~[reactor-core-3.2.6.RELEASE.jar:3.2.6.RELEASE]
    at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:91) ~[reactor-core-3.2.6.RELEASE.jar:3.2.6.RELEASE]
    at reactor.core.publisher.Mono.block(Mono.java:1494) ~[reactor-core-3.2.6.RELEASE.jar:3.2.6.RELEASE]
    at com.example.webclientissue.WebclientissueApplication.details(WebclientissueApplication.java:39) [classes/:na]
    at com.example.webclientissue.WebclientissueApplication.run(WebclientissueApplication.java:28) [classes/:na]
    at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:813) ~[spring-boot-2.1.3.RELEASE.jar:2.1.3.RELEASE]
    ... 3 common frames omitted
    Suppressed: java.lang.Exception: #block terminated with an error
        at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:93) ~[reactor-core-3.2.6.RELEASE.jar:3.2.6.RELEASE]
        ... 7 common frames omitted
Caused by: reactor.netty.http.client.PrematureCloseException: Connection prematurely closed BEFORE response

[scvzerng]

@jnizet Thank you very much. I also met this problem and solved it through your solution

[rstoyanchev]

I've created https://github.com/reactor/reactor-netty/issues/632. Thanks for the report.

[jnizet]

Thanks for looking at the issue @rstoyanchev.

[violetagg]

@jnizet You should be GOOD with the latest Reactor Netty 0.8.6.BUILD-SNAPSHOT

[jnizet]

@violetagg thanks a lot. Nothing urgent on my side since there's a workaround, so I'll wait until Spring Boot releases a version with the fix. But it's great to have it fixed so quickly. And thanks for letting me know, too.

[sylvain-rouquette]

I still have this issue with version 0.8.8.RELEASE (spring boot 2.1.5.RELEASE) and the quick fix described above allowed me to fix the problem in the meantime.

[violetagg]

@sylvain-rouquette This exception might have different causes, so please provide a reproducible scenario in a new ticket. The cause for this particular issue is https://github.com/reactor/reactor-netty/issues/632 and it is fixed (a header Connection with more than one value).