Elasticsearch-rails: Problems with update callback on models with #as_indexed_json defined

[mschulkind]

I noticed a few problems with the update callback on models with #as_indexed_json defined.

Fairly minor, but if you look at this line you'll notice that #as_indexed_json is called once for each changed field, which is pretty suboptimal if #as_indexed_json is at all complex. This is clearly solvable fairly easily.

A little more concerning is that with #as_indexed_json defined, the optimization of checking for changes only saves bandwidth when sending to elasticsearch which is probably a rather minor concern, especially because if you defined #as_indexed_json, it's probably so that you can index associated objects, and the DB requests to fetch those objects will take far longer than sending an extra few bytes to elasticsearch.

Without #as_indexed_json defined, you're not saving any DB requests, since I believe the entire object must necessarily be loaded already and there's no possibility of associated objects being defined. I won't question this case here.

The major issue, and why I'm writing this at all, is that the dirty checking completely breaks updates to any fields that are not named the same as or directly map to a database attribute in the indexed json. Say you have a #name method in your model that forms the full name from #first_name and #last_name, and you add this as the name field in #as_indexed_json. As far as I can tell, there's no way #name will ever get updated by the callbacks, and there's not even an override option that I can see.

I propose we just disable the dirty-checking optimization when #as_indexed_json is defined. What do you think?

[karmi]

as_indexed_json is called once for each changed field

That's a very good catch!, embarrasing, will fix it.

only saves bandwidth when sending to elasticsearch which is probably a rather minor concern

That depends on the use case, but apart from that, you're minimizing the risk of conflicts.

Say you have a #name method in your model that forms the full name from #first_name and #67

Yes, and we been discussing this in other issues. You simply have to define your own indexing hooks/routines -- the bundled callbacks are just starting point, just a default.

I propose we just disable the dirty-checking optimization

I do think it's a good default option, since it gets you started quickly and with minimal effort. For almost any more complicated use case (complex associations, custom serialisation, counters, ...) you have to write your own indexing routines. Look into the examples and example Rails application templates.

The README and documentation tries to emphasize this point, but I guess it can be emphasized even more -- I'll look into proper wording in proper context there.

[mschulkind]

Sounds good to me.

[drueck]

I'm running into this issue as well where my calculated methods specified in as_indexed_json aren't getting saved when update_document is called. How do you go about disabling the dirty checking or working around this?

[drueck]

Update: I figured out a workaround and thought I'd record it here in case someone else comes across this. Not sure if this is the ideal and/or correct solution, but it seems to work for me. In my custom callback I made the following change:

after_commit on: [:update] do
  # update_document uses dirty checking which ignores calculated methods
  # __elasticsearch__.update_document
  __elasticsearch__.index_document
end

[abscondment]

+1 for the workaround explanation from @drueck. Thanks!

[estolfo]

Closing as this is out of date.