Wiremock: GZIP compression does not work for responses to POST requests.
Wiremock registers GzipFilter with default configuration:
com.github.tomakehurst.wiremock.jetty9.JettyHttpServer
mockServiceContext.addFilter(GzipFilter.class, "/*", EnumSet.of(DispatcherType.REQUEST, DispatcherType.FORWARD));
But by default the aforementioned filter handles only GET request.
org.eclipse.jetty.servlets.GzipFilter
// public void init
tmp=filterConfig.getInitParameter("methods");
if (tmp!=null)
{
StringTokenizer tok = new StringTokenizer(tmp,",",false);
while (tok.hasMoreTokens())
_methods.add(tok.nextToken().trim().toUpperCase(Locale.ENGLISH));
}
else
_methods.add(HttpMethod.GET.asString());
// public void doFilter()
if (!_methods.contains(request.getMethod()) || isExcludedPath(requestURI))
{
super.doFilter(request,response,chain);
return;
}
arokhmistrov
All 23 comments
Has this been updated yet?
carthoris
on 14 Jun 2018
WireMock 2.18.0 switched to the new Jetty GZip handler. I haven't specifically tested it for this case yet, but it's worth giving it a try.
tomakehurst
on 14 Jun 2018
Thanks,
I have never used gzip data in wiremock before but have been asked to start.
Does wiremock accept gzip data during the seeding process via the endpoints or does it take uncompressed data and gzips it before sending it if the Accept-Encoding is gzip?
From: Tom Akehurst [mailto:[email protected]]
Sent: Thursday, June 14, 2018 9:00 AM
To: tomakehurst/wiremock wiremock@noreply.github.com
Cc: Jay Goldberg jgoldberg@pros.com; Comment comment@noreply.github.com
Subject: Re: [tomakehurst/wiremock] GZIP compression does not work for responses to POST requests. (#787)
WireMock 2.18.0 switched to the new Jetty GZip handler. I haven't specifically tested it for this case yet, but it's worth giving it a try.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftomakehurst%2Fwiremock%2Fissues%2F787%23issuecomment-397306031&data=02%7C01%7Cjgoldberg%40pros.com%7C456e56c2cc724103abdc08d5d1ff21e8%7C094cfb7ad13146379047e339e7d04359%7C0%7C0%7C636645816078219778&sdata=DgT%2FwpHUDmtD3cmoLbV24r65vIGM%2Bayckab%2FE1oSD6E%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAD0hEOLWXUL6dVjEFnFXPg8LehXbaFHLks5t8mxlgaJpZM4QEJHW&data=02%7C01%7Cjgoldberg%40pros.com%7C456e56c2cc724103abdc08d5d1ff21e8%7C094cfb7ad13146379047e339e7d04359%7C0%7C0%7C636645816078219778&sdata=wdzkLlEfO1cbZlwLfb2lYQWBzTO9lAWLBS1LufCEqYs%3D&reserved=0.
carthoris
on 14 Jun 2018
Stubs are always loaded uncompressed, then WireMock (well, Jetty's handler specifically) gzips/gunzips requests and responses.
tomakehurst
on 14 Jun 2018
Hi Tom,
Regarding the change in 2.18.0 for the new Jetty Gzip Handler, does it mean that I can change the Jetty version to 9.4 when I build wiremock locally? I'm using the standalone jar and the Jetty version incuded seems 9.2.x , so I'm not sure if there's something else I need to allow "POST" methods to apply Gzip Handler.. Thanks!
joshuacura
on 4 Jul 2018
I just bumped into this scenario with v2.18.0. Do we have updates on this feature request? Thanks.
jecklgamis
on 2 Aug 2018
@jecklgamis , i did checked-out the wiremock code, did a few modifications then build it locally, to enable gzip-compression on HTTP POST requests.. @tomakehurst , could we have it configurable when running wiremock standalone on the next release? Thanks!
joshuacura
on 2 Aug 2018
What was the change you had to make?
Are we talking about compressing the request or response body?
tomakehurst
on 9 Aug 2018
Did not make a change. I don’t think I got it to work. It has been awhile. I was trying to compress the response from wiremock to my client so that the client can test its un-compress method.
From: Tom Akehurst notifications@github.com
Sent: Thursday, August 9, 2018 9:56 AM
To: tomakehurst/wiremock wiremock@noreply.github.com
Cc: Jay Goldberg jgoldberg@pros.com; Comment comment@noreply.github.com
Subject: Re: [tomakehurst/wiremock] GZIP compression does not work for responses to POST requests. (#787)
What was the change you had to make?
Are we talking about compressing the request or response body?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftomakehurst%2Fwiremock%2Fissues%2F787%23issuecomment-411787132&data=02%7C01%7Cjgoldberg%40pros.com%7Cf553d3b8c4bd467d2d5808d5fe084ac0%7C094cfb7ad13146379047e339e7d04359%7C0%7C0%7C636694233925221921&sdata=AgRD1VETFSyI0ThjLDmqr1K%2BowVbECr4vc3XAvGac%2BM%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAD0hEBeaabwNIk5Mw9hKi0HV6SIa6XLpks5uPE2dgaJpZM4QEJHW&data=02%7C01%7Cjgoldberg%40pros.com%7Cf553d3b8c4bd467d2d5808d5fe084ac0%7C094cfb7ad13146379047e339e7d04359%7C0%7C0%7C636694233925231926&sdata=DugOHSKuTm6lLVKWYRp4o0pkspHDhleoYhwrQ95zZaA%3D&reserved=0.
carthoris
on 9 Aug 2018
OK, I've found the problem. I'll include some kind of fix in the next release.
tomakehurst
on 9 Aug 2018
Nice!
From: Tom Akehurst notifications@github.com
Sent: Thursday, August 9, 2018 10:32 AM
To: tomakehurst/wiremock wiremock@noreply.github.com
Cc: Jay Goldberg jgoldberg@pros.com; Comment comment@noreply.github.com
Subject: Re: [tomakehurst/wiremock] GZIP compression does not work for responses to POST requests. (#787)
OK, I've found the problem. I'll include some kind of fix in the next release.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftomakehurst%2Fwiremock%2Fissues%2F787%23issuecomment-411799553&data=02%7C01%7Cjgoldberg%40pros.com%7C31f68d16a2194af6545108d5fe0d5048%7C094cfb7ad13146379047e339e7d04359%7C0%7C0%7C636694255492170379&sdata=4G%2FzDRX%2FbJUgcQtq7sds1SfOaQMYAXH%2Fm16X8o8DZ6k%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAD0hEMwKEwMsk8ArL9wsQwo9tKtvBWLuks5uPFYKgaJpZM4QEJHW&data=02%7C01%7Cjgoldberg%40pros.com%7C31f68d16a2194af6545108d5fe0d5048%7C094cfb7ad13146379047e339e7d04359%7C0%7C0%7C636694255492180388&sdata=lkNCnClMP0BQ%2FBaeZ4bw5%2FXgMNVtOwVcH2kn32hzP8E%3D&reserved=0.
carthoris
on 9 Aug 2018
I updated to 2.18.0 today, and noticed calling WireMock.reset() sent an empty POST to __admin/reset , which in turn caused an EOFException when trying to unzip the request body.
jamespope101
on 9 Aug 2018
Can you provide some more details so I can replicate?
tomakehurst
on 9 Aug 2018
Using libraries:
jetty-server:9.4.11.v20180605
wiremock:2.18.0
WireMockServer wireMockServer = new WireMockServer(STUB_TEST_PORT);
WireMock.configureFor("localhost", STUB_TEST_PORT);
wireMockServer.start();
WireMock.reset();
Stack trace:
java.lang.RuntimeException: java.io.EOFException
at com.github.tomakehurst.wiremock.servlet.WireMockHttpServletRequestAdapter.getBody(WireMockHttpServletRequestAdapter.java:137)
at com.github.tomakehurst.wiremock.servlet.WireMockHttpServletRequestAdapter.getBodyAsString(WireMockHttpServletRequestAdapter.java:159)
at com.github.tomakehurst.wiremock.http.AdminRequestHandler.handleRequest(AdminRequestHandler.java:67)
at com.github.tomakehurst.wiremock.http.AbstractRequestHandler.handle(AbstractRequestHandler.java:44)
at com.github.tomakehurst.wiremock.servlet.WireMockHandlerDispatchingServlet.service(WireMockHandlerDispatchingServlet.java:108)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:535)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:203)
at org.eclipse.jetty.server.Dispatcher.error(Dispatcher.java:81)
at com.github.tomakehurst.wiremock.jetty9.NotFoundHandler.handle(NotFoundHandler.java:41)
at org.eclipse.jetty.server.Response.sendError(Response.java:655)
at org.eclipse.jetty.server.Response.sendError(Response.java:590)
at com.github.tomakehurst.wiremock.servlet.WireMockHandlerDispatchingServlet$ServletHttpResponder.respondTo(WireMockHandlerDispatchingServlet.java:175)
at com.github.tomakehurst.wiremock.servlet.WireMockHandlerDispatchingServlet$ServletHttpResponder.respondSync(WireMockHandlerDispatchingServlet.java:138)
at com.github.tomakehurst.wiremock.servlet.WireMockHandlerDispatchingServlet$ServletHttpResponder.respond(WireMockHandlerDispatchingServlet.java:132)
at com.github.tomakehurst.wiremock.http.AbstractRequestHandler.handle(AbstractRequestHandler.java:63)
at com.github.tomakehurst.wiremock.servlet.WireMockHandlerDispatchingServlet.service(WireMockHandlerDispatchingServlet.java:108)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:535)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:674)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:531)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
at java.lang.Thread.run(Thread.java:748)
Caused by:
java.io.EOFException
at java.util.zip.GZIPInputStream.readUByte(GZIPInputStream.java:268)
at java.util.zip.GZIPInputStream.readUShort(GZIPInputStream.java:258)
at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:164)
at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:79)
at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:91)
at com.github.tomakehurst.wiremock.common.Gzip.unGzip(Gzip.java:36)
at com.github.tomakehurst.wiremock.servlet.WireMockHttpServletRequestAdapter.getBody(WireMockHttpServletRequestAdapter.java:135)
at com.github.tomakehurst.wiremock.servlet.WireMockHttpServletRequestAdapter.getBodyAsString(WireMockHttpServletRequestAdapter.java:159)
at com.github.tomakehurst.wiremock.http.AdminRequestHandler.handleRequest(AdminRequestHandler.java:67)
at com.github.tomakehurst.wiremock.http.AbstractRequestHandler.handle(AbstractRequestHandler.java:44)
at com.github.tomakehurst.wiremock.servlet.WireMockHandlerDispatchingServlet.service(WireMockHandlerDispatchingServlet.java:108)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:535)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:203)
at org.eclipse.jetty.server.Dispatcher.error(Dispatcher.java:81)
at com.github.tomakehurst.wiremock.jetty9.NotFoundHandler.handle(NotFoundHandler.java:41)
at org.eclipse.jetty.server.Response.sendError(Response.java:655)
at org.eclipse.jetty.server.Response.sendError(Response.java:590)
at com.github.tomakehurst.wiremock.servlet.WireMockHandlerDispatchingServlet$ServletHttpResponder.respondTo(WireMockHandlerDispatchingServlet.java:175)
at com.github.tomakehurst.wiremock.servlet.WireMockHandlerDispatchingServlet$ServletHttpResponder.respondSync(WireMockHandlerDispatchingServlet.java:138)
at com.github.tomakehurst.wiremock.servlet.WireMockHandlerDispatchingServlet$ServletHttpResponder.respond(WireMockHandlerDispatchingServlet.java:132)
at com.github.tomakehurst.wiremock.http.AbstractRequestHandler.handle(AbstractRequestHandler.java:63)
at com.github.tomakehurst.wiremock.servlet.WireMockHandlerDispatchingServlet.service(WireMockHandlerDispatchingServlet.java:108)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:535)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:674)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:531)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
at java.lang.Thread.run(Thread.java:748)
jamespope101
on 9 Aug 2018
@tomakehurst this is the code I've modified to enable POST request to use compression when the request has "Accept-Encoding:gzip" .. I applied this to use it on running wiremock as standalone jar.
on class JettyHttpServer:addGzipHandler(...)
gzipWrapper.addIncludedMethods(HttpMethod.POST.asString());
joshuacura
on 10 Aug 2018
Thanks. I've actually got a patch for this on in my workspace now so it'll be on master shortly.
tomakehurst
on 10 Aug 2018
Thanks @tomakehurst, @joshuacura . I have a similar change in my local. Looks like Dropwizard app has a similar behavior and it has to be explicitly configured (https://www.dropwizard.io/1.3.5/docs/manual/configuration.html: server.gzip.includedMethods).
jecklgamis
on 13 Aug 2018
Hey @tomakehurst! Did you manage to push the fix to master for this?
I am making a POST request with "Content-Encoding: gzip" header and I get a java.util.zip.ZipException: Not in GZIP format. If I remove this header, I get the response as expected. Is there a way to fix this? I am using WireMock as standalone via JSON.
@joshuacura Where did you apply this fix?
stelarelas
on 24 Jun 2020
@stelarelas yes, I should have closed this issue ages ago, as it's been fixed.
It sounds to me like you're trying to send a non-gzipped request with a gzip header, so I'd expect it to throw an exception in that instance.
tomakehurst
on 24 Jun 2020
Fixed in df78bfca
tomakehurst
on 24 Jun 2020
@tomakehurst Thanks for the quick response! The situation is exactly that. My production endpoint returns a gzip response that's why there is a "Content-Encoding: gzip" header in it. How am I suppose to mock this response in my json file that WireMock uses? I currently have it as a json and I get the Not in GZIP format error back.
stelarelas
on 24 Jun 2020
It should happen automatically provided you're sending Accept-Encoding:gzip in the request.
tomakehurst
on 24 Jun 2020
@tomakehurst - I am also facing the same problem and trying to find a solution on this.
- Using Wiremock standalone version.
- My production URL returns the response with headers- "Content-Encoding":[ "deflate,gzip"]
- I mocked this request with RR pairs but ending up getting -HTTP 500 error
java.lang.RuntimeException: java.io.EOFException
- how should I build the request so that it will accept the request and respond.
- I tried following writing in my request body but didn't solve the problem-
{
"id" : "3ee520c4-905b-4be2-97d8-1ec69ea992d9",
"name" : "rm_shopping_v2_flights",
"request" : {
"url" : "/rm/shopping/v2/flights",
"method" : "POST",
"headers" : {
"Accept-Encoding" : {
"contains" : "deflate,gzip"
}
}
} - I tried removing headers from the request being sent to mock, it works but with header "Content-Encoding":[ "deflate,gzip"], it returning in 500 error.
7.Kindly help. This is an urgent issue and we have many apis like that to work with. Appreciate your help.
AR4348
on 17 Dec 2020
Related issues
HasseGbg
·
16Comments
rciovati
·
16Comments
ntam
·
13Comments
mdomasevicius
·
10Comments
asarkar
·
19Comments