Cartodb: Disable loading of other rake tasks for resques [DEPLOY ON 12/19]

[javitonino]

Today we ran into an issue with the importer after deploying a PR (#11031) which just included changes to another separate rake task. The issue was that the new rake added some objects to the global namespace (later fixed in #11044) and that caused the importer to fail to report some metrics.

Currently, when running rake tasks, we load all the tasks in the lib/tasks directory (which is the Rails default), so if any rake tasks is behaving badly (like modifying core objects), it will affect the importer. A
solution is to have a second Rakefile that is used only for resque:work which avoids loading all the rest of rakes, in order to isolate it and minimize problems like the one we had today.

The proposed solution is just to create a new Rakefile.resque, which is identical to our current one except for adding CartoDB::Application.paths['lib/tasks'] = [] so it does not loads all the tasks in lib/tasks. The benefit is a more robust importer that is not affected by potential errors while coding other rakes, and also a marginally faster startup time.

We would have to create the new Rakefile and modify scripts/resque to use it (passing a new parameter), as well as all resque service files.

Opinions? Do you think it's worth it?
cc @CartoDB/builder-backend @CartoDB/dataservices @CartoDB/systems

[jgoizueta]

Then I guess we should be careful with cases where the resque:work task is used directly, rather than through the scripts/resque, like here: https://github.com/CartoDB/cartodb-platform/blob/087145b031c91e4277580068f7a5cd2c8fa9d8d6/cookbooks/resque/templates/default/resque-service.initd.rhel.erb#L18

If I understand correctly we'd need to always use the script to avoid loading the regular Rakefile, is that so?

[javitonino]

Yeah, we would have to modify a lot of platform scripts, to add the -f Rakefile.resque parameter. It could be a pain, indeed, but I am not sure about the magnitude of the task, that's why I summoned you guys.

[javitonino]

I talked with @ethervoid and another idea is to add the following to the Rakefile:

if Rake.application.top_level_tasks.include?('resque:work')
  CartoDB::Application.paths['lib/tasks'] = []
end

This automatically disables loading of other rake tasks if the task to be ran is resque:work, so no need to do further changes to scripts.

[rafatower]

jefazos! :clap: :clap:
looks good to me

[juanignaciosl]

First of all, the solution that you came up with looks great :-)

I'd like to be _really_ picky in order to avoid further issues: if for some reason top_level_tasks included resque:work _and_ others, that might not work. Wouldn't Rake.application.top_level_tasks == ['resque:work'] be more accurate?

[javitonino]

By default, there are other tasks included, the array normally contains ['environment', 'resque:work'] for a normal invocation. Maybe we can compare with that. You are right that the correct comparison should be that there aren't any tasks aside from resque:work, not the opposite.

[juanignaciosl]

I have never looked at what it contains; I was mostly thinking aloud in case it might help :_)