Elasticsearch-rails: Elasticsearch::Transport::Transport::Errors::NotFound on `object.destroy()`

[githorse]

I've got a very simple, vanilla ES setup using the feature extraction pattern, with no custom callbacks or anything like that -- see my other issue for details. I'm getting the following error consistently every time I try to destroy an ES-enabled model:

Elasticsearch::Transport::Transport::Errors::NotFound: [404] {"found":false,"_index":"surveys","_type":"survey","_id":"60","_version":1,"_shards":{"total":2,"successful":1,"failed":0}}

Obviously, this one occurs when I run Survey.find(60).destroy(). Survey has_many Questions which are marked dependent: destroy; the Questions appear to be deleted without issue, and then this error gets raised on the call DELETE FROM "surveys" WHERE "surveys"."id" = ? [["id", 60]]. The call does actually succeed in deleting the survey before the error is raised.

Any idea what might cause this issue?

[karmi]

This is weird, how have you set up the callback for deleting the document from the index?

[githorse]

Hi @karmi -- I have no custom callbacks set up. My understanding was updating the index after deleting a record would be handled by Elasticsearch::Model::Callbacks. I've included that according to the feature extraction pattern from the README:

module Searchable
  extend ActiveSupport::Concern

  included do
    include Elasticsearch::Model
    include Elasticsearch::Model::Callbacks

    class << self
      def search( query )
        # etc
      end
    end
  end
end

Definitely seems like a callback issue though ... I'll try experimenting with an after_destroy callback.

[CamJN]

I've got the same issue.

[thomasnal]

The same issue here, appearing randomly, together with too many connection resets (due to closed stream - IOError) after 0 requests on 70235882588320, last used 1471107194.7592144 seconds ago and sometimes succeeding.

There is no guide or help anywhere on the web to resolve this issue. Makes elasticsearch unusable, there is no workaround.

Elasticsearch running on localhost:9200.

[adamdilek]

+1

[anthonator]

I'm seeing this as well. I only see it in my production environment (Heroku/Bonsai). My dev environment (Docker) doesn't have this problem.

[arvindravi]

I've got the same issue. Happens only on production (Heroku/Bonsai) when the callback is fired on object creation, and it appears on object.destroy() randomly.

[ahsankamal]

I was also getting the same error on my rails console whenever i was calling destroy method on ES-enabled model. But it vanished when i replaced automatic callbacks by custom callbacks.
I removed "include Elasticsearch::Model::Callbacks" and included custom callbacks:

after_commit on: [:create] do
  __elasticsearch__.index_document
end

after_commit on: [:update] do
  __elasticsearch__.index_document
end

after_commit on: [:destroy] do
  __elasticsearch__.delete_document
end

It seems that issue lies in automatic callback.

[zauzaj]

Hi,
I'm using Sidekiq for async updating/deleting items on ES but getting this error in logs/sidekiq.log

2017-02-03T12:30:00Z 7487 TID-ougrrn8sc ERROR: Sidekiq::Processor crashed! Elasticsearch::Transport::Transport::Errors::NotFound: [404] {"found":false,"_index":"development_app_scoped_index","_type":"staff","_id":"14","_version":1,"_shards":{"total":2,"successful":1,"failed":0}}

This is how I handle errors but it doesn't work :

private
def worker(action, id, klass)
  begin
    ElasticSearch::ReindexWorker.perform_async(action, id, klass)
  rescue Elasticsearch::Transport::Transport::Error => e
    puts e.inspect
  end
end

Any help here please ?

[karmi]

I'm wondering if it's possible that the document has been removed because the database record has ben deleted in the meantime?

It's weird that rescuing Elasticsearch::Transport::Transport::Error wouldn't rescue the Elasticsearch::Transport::Transport::Errors::NotFound exception, I'll check it.

What does the ReindexWorker do? Just re-indexes a document based on ID?

[zauzaj]

Hey @karmi,

thanks on fast reply.
I moved handling errors to perform action of sidekiq worker and it works perfect now. Also, I had to restart my sidekiq.

Everything is ok on my side !

Thanks again.

[karmi]

Thanks for the reply as well! I've quickly tried this:

begin
  client.perform_request 'GET', 'foo/bar/1'
rescue Elasticsearch::Transport::Transport::Error => e  
  puts e
end

and the exception has been catched -- I was worried there's some incorrect class hierarchy at play.

[sechix]

Hello, What is the solution exactly? I have same issue

[zauzaj]

@sechix Sidekiq should be restarted when you do some changes in worker class.

[karmi]

Hi @sechix, this thread is quite long, what "same issue" do you have?

[soujanya24]

I got this error. What does this mean and any solution quickly would be appreciated. error_class="Elasticsearch::Transport::Transport::Errors::NotFound" error="[404] \r\n\r\n\r\n

404 Not Found

\r\n

---

nginx/1.9.5\r\n\r\n\r\n" plugin_id="object:3febccf36194"
Thanks

[karmi]

@soujanya24, I'm sorry, but you would have to provide full details about your environment, the versions of the Rubygem and Elasticsearch used, preferably in a new issue, so we're able to help you. Posting a question like this isn't helpful at all.

[sechix]

Thanks for your answers. I finally solved my issue creating index. My problem was with heroku and elasticsearch. The solution is install gems elasticsearch rails / model, and for bonsai heroku gem elasticsearch-bonsai. After run in rails index create for corresponding class.

[karmi]

@sechix, glad it's working!

[AntoninMarchardDev]

Hi !
Same issue :
Elasticsearch::Transport::Transport::Errors::NotFound ([404] {"_index":"travels","_type":"travel","_id":"11","_version":4,"result":"not_found","_shards":{"total":2,"successful":2,"failed":0},"_seq_no":40,"_primary_term":1}):

Here is my config :

gem 'elasticsearch-model', github: 'elastic/elasticsearch-rails', branch: '5.x'
gem 'elasticsearch-rails', github: 'elastic/elasticsearch-rails', branch: '5.x'
gem 'bonsai-elasticsearch-rails'

In the Travel model :

  include Elasticsearch::Model
  include Elasticsearch::Model::Callbacks

And when I call @travel.destroy, I have the error.
Any help will be appreciate !