Elasticsearch-rails: Setting Elasticsearch for production

[tarebyte]

Hi there I'm trying to deploy a toy app with this gem to heroku, and I didn't see any documentation how how to set the production environment url.

I'm sorry if I overlooked something, but any help would be much appreciated.

[mkrisher]

You can set by host name, or include things like port and scheme.

Docs here:
https://github.com/elasticsearch/elasticsearch-ruby/tree/master/elasticsearch-transport#setting-hosts

[tarebyte]

I figured it would be something I looked over, thank you.

Any suggestions as to how I would override the client that Rails uses in production. I'm currently working on this, but it doesn't look right

require 'elasticsearch/transport'

if Rails.env.production?
  client = Elasticsearch::Client.new host: ENV['BONSAI_URL']
end

This is just setting the host for a client instance.

[tarebyte]

Cool I got that bit to work, so in your initializer file you can set the ElasticModel to a new host

require 'elasticsearch/model'
require 'elasticsearch/transport'

if Rails.env.production?
  Elasticsearch::Model.client = Elasticsearch::Client.new url: ENV['BONSAI_URL']
end 

I tried to migrate the data over using

heroku run bundle exec rake environment elasticsearch:import:model CLASS='User'

And I got

Elasticsearch::Transport::Transport::Errors::NotFound: [404] {"error":"IndexMissingException[[users] missing]","status":404}

Elasticsearch::Transport::Transport::Errors::NotFound: [404] {"error":"Resource not found. Please contact [email protected] for assistance.","status":404}

So I'm going to contact bonsai for this probably.
Thanks @mkrisher for your help!

If you have any other pieces of advice on how to deal with this please let me know.

[karmi]

Yeah, you should configure the client for the app (all models) with Elasticsearch::Model.client, as you do. This is fairly prominent in the README, so I'm quite surprised you haven't found it initially?

Regarding the IndexMissingException error, I think on Bonsai, you'll get a special index name (which includes a token, wo consider it private), which you have to then use in index_name (doc).

Last note, the require 'elasticsearch/transport' shouldn't be needed -- require 'elasticsearch/model' should do that for you.

[tarebyte]

I'm attributing it to a lack of sleep. Thank you for the help it was very visible in the README it just slipped right past me.

[karmi]

Cool :) I'm always interested in how people parse the documentation, as I tend to put a lot of time and care into it :)

[tarebyte]

Hey @karmi I got it working :smile:

For anyone who sees this issue later this was how I did it.

# config/initializers/bonsai.rb

require 'elasticsearch/model'

if ENV['BONSAI_URL']
  Elasticsearch::Model.client = Elasticsearch::Client.new({url: ENV['BONSAI_URL'], logs: true})
  BONSAI_INDEX_NAME = ENV['BONSAI_URL'][/[^\/]+$/]
else
  app_name = Rails.application.class.parent_name.underscore.dasherize
  BONSAI_INDEX_NAME = "#{app_name}-#{Rails.env}"
end

Because Bonsai doesn't support lazy index creation you have to run FORCE=true on creation.

For example:

heroku run bundle exec rake environment elasticsearch:import:model CLASS='User' FORCE=true

And then if you run it again without FORCE it should import everything correctly.

[karmi]

@tarebyte Cool, thanks for the writeup!

Regarding the index creation, you can, alternatively, create some class method / rake task, so you can call it during the application bootstrap process a _and_ test setups.

[nz]

Hey all, Nick with bonsai.io here — anything we can do or contribute to make this case a bit easier?

For what it's worth, all the special-case index_name and BONSAI_INDEX_NAME stuff is no longer relevant with Bonsai. We should pretty much just work with conventional Elasticsearch usage the way you'd expect, including the Tire-era index-per-model convention.

The one exception, noted by @tarebyte, is lazy index creation. Most index missing exceptions on our systems are from trying to use or insert into an index that doesn't yet exist. We're pretty strict on that one — indices must be explicitly created before using them.

Seems like it might be time for us to refresh our setup docs for elasticsearch-rails!

[fertobar]

Thanks for share @tarebyte! With FORCE=true option works like a charm in HK.

[atcoco]

Hello!

I have some issues with AWS. I have installed and configured perfectly about apache2 and passenger. But this times out to the deployment app on AWS. So I tried to deploy sample project. As a result it worked. Only My real project doesn't work. I need your help! :)

In apache2 error.log file
App 2289 stdout:
App 2289 stdout: BONSAI_URL not present, proceeding with Elasticsearch defaults.
App 2430 stdout:
I think, BONSAI_URL doesn't matter.
Thanks