Searchkick: [Idea] Support "refresh: wait_for" option when reindexing

[mattvague]

Hey there, in the app I'm working on we have the requirement that if we query elastic search immediately after creating a new record, that this record shows up. Because of the 1s delay between index refreshes this is unreliable as-is.

I've explored several different options and it seems like the best one is to use ES's refresh: wait_for option during index updates. This will make it so that our API will not respond after a request until elastic search has finished refreshing.

Is there any chance of adding a new callbacks: :wait_for option or something? Failing that, would you accept a pull request to allow a bit more configurability in Searchkick to allow us to implement this ourselves?

Thanks!

[ankane]

Hey @mattvague, a new callback mode makes sense to me. Happy to review a PR.

[mattvague]

I was exploring this, and I don't think it will be too difficult besides figuring out how to pass this new refresh option through all the way through to Searchkick::Indexer so we can do Searchkick.client.bulk(body: items, refresh: refresh).

What would you advise here? Store the option on Searchkick::Index and then add the optional refresh argument to Searchkick::Indexer#queue and Searchkick::Indexer#perform?

[ankane]

I don't fully follow the description/question. I think this one will be easier to discuss when there's code.

[mattvague]

I don't fully follow the description/question. I think this one will be easier to discuss when there's code.

Hah, wasn't my finest piece of writing. For sure, will throw up a PR for you to look at

[ankane]

Cleaning up issues, but still happy to review a PR.

[TikiTDO]

@ankane I'm looking at throwing together a PR for this issue, but I don't see a way to add this as a callback mode without adding a whole lot of confusion around testing.

Would you be open to adding a new top-level config option, something similar to Searchkick.client_options, but more like Searchkick.refresh_mode?