Elasticsearch-rails: How do you manage mappings changes?

[tastycode]

We have a field that describes a list of tags. The default tokenizer won't work for us so we've defined one like this (this is on an existing elasticsearch model)

settings do
   mapping dynamic: false do
      indexes :tag_list, analyzer: 'whitespace'
   end
end

but when we start our elasticsearch test cluster and inspect the mappings on the model, we don't see the changes.

Also, we've looked at a few tutorials like http://tutorials.pluralsight.com/ruby-ruby-on-rails/elasticsearch-with-ruby-on-rails and don't see where they are explicitly calling create_index! force:true.

In a real world production application, how does updating mappings work? Do you have to call create_index! on every single test that depends on elasticsearch functionality based on mappings? Is there something in elasticsearch-rails that checks to see if the mappings that elasticsearch has match the configured mappings and automatically migrates them if there is a difference?

[karmi]

Hi, there are multiple aspects to your question.

First, create_index! is useful when you want or need to re-create the index with different settings/mappings and load all the data. One of the prime examples for that is integration tests. Or, when you're still developing the application, and want to frequently rebuild the indices and import all data.

If you have an existing index, let's say in production environment, and you want to update mappings, there are several things to consider:

In Elasticsearch itself, you cannot _change_ mapping for an existing field, eg. to use a different analyzer. In this case, you have to create another index with desired mappings, and re-index the data.

You can, however, _add_ a field into the mapping. The integration with your model won't help by itself, because it only applies when you create the index. Therefore, you have to call the indices.put_mapping method externally, passing it the mappings.

In practical terms, you can do something like this:

Article.__elasticsearch__.client.indices.put_mapping \
  index: Article.index_name, 
  type: Article.document_type, 
  body: Article.mappings.to_hash

Does it answer your question?

[tastycode]

This does! I'm surprised that there isn't something similar to db:migrate
for elasticsearch indexes!

On Sat, Nov 19, 2016 at 12:08 PM Karel Minarik [email protected]
wrote:

Hi, there are multiple aspects to your question.

First, create_index! is useful when you want or need to re-create the
index with different settings/mappings and load all the data. One of the
prime examples for that is integration tests. Or, when you're still
developing the application, and want to frequently rebuild the indices and
import all data.

If you have an existing index, let's say in production environment, and
you want to update mappings, there are several things to consider:

In Elasticsearch itself, you cannot _change_ mapping for an existing
field, eg. to use a different analyzer. In this case, you have to create
another index with desired mappings, and re-index the data.

You can, however, _add_ a field into the mapping. The integration with
your model won't help by itself, because it only applies when you create
the index. Therefore, you have to call the indices.put_mapping method
externally, passing it the mappings.

In practical terms, you can do something like this:

Article.elasticsearch.client.indices.put_mapping index: Article.index_name, type: Article.document_type, body: Article.mappings.to_hash

Does it answer your question?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/elastic/elasticsearch-rails/issues/640#issuecomment-261729696,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAxb4evglfyd_e1tD5ttWD_ks1rWnOpiks5q_zstgaJpZM4KzKSV
.

[karmi]

Cool!

(Something like db:migrate would be a bit complicated to achieve in Elasticsearch model, because people might have incorrect assumptions, like the case with changing the mapping for an existing attribute. In practice, it's important to understand the Elasticsearch model behind the Ruby facade, and in practical terms, be ready for frequent re-indexing, at least during the initial phase of development :)

[stuaxo]

You can say this about migrations in general, but I don't think it's an argument against having them.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.