Searchkick: Elasticsearch::Transport::Transport::Errors::BadRequest - too_long_frame_exception

[justi]

I'm using ES Scroll API to retrieve a large amount of data.

results = []
MyModel.search(
      where: {
        ...
      },
      load: false,
      select: %w[field_1 field_2],
      scroll: '1m'
    ).scroll { |batch| results << batch }

Searchkick version: 4.3.0
Elasticsearch version: 6.8.9

on scroll method call I get

Elasticsearch::Transport::Transport::Errors::BadRequest ([400] {"error":{"root_cause":[{"type":"too_long_frame_exception","reason":"An HTTP line is larger than 4096 bytes."}],"type":"too_long_frame_exception","reason":"An HTTP line is larger than 4096 bytes."},"status":400})

Maybe change the request type from GET to POST will fix this?

[ankane]

Hey @justi, haven't seen that error before. Can you enable logging for the Elasticsearch client:

Searchkick.client.transport.logger = ActiveSupport::Logger.new(STDOUT)

And paste the output as well as the full stack trace?

Edit: For log output, interested in what the GET http://localhost:9200/_search/scroll/... line looks like before it fails.

[ankane]

Should have dug into this more before responding. Looks like changing the request type should fix it. Thanks for reporting and the fix!

[ankane]

Did not mean to reopen, but this just went out with a release