Luigi: Add LSF Support

[boombard]

Make a production-ready version of https://github.com/dattalab/luigi/blob/lsf/luigi/lsf.py based on the existing SGE implementation. I will make a pull request

[dlstadther]

@boombard Status update on PR?

[Tarrasch]

I'm closing this issue.

• [x] It has been inactive for +4 months.
• [ ] It's not about luigi core, so not as many users are affected about this.
• [ ] The change seems quite big, it's unlikely to be sporadically picked up.
• [x] The owner haven't responded or disappeared.
• [ ] I don't understand what this issue is about.
• [ ] There exists a reasonable workaround for this.
• [ ] We need to check if this hasn't been fixed by now (for old issues).
• [ ] This is kind of by design and not a bug.
• [ ] Resolving this would probably add a lot of complexity.
• [ ] Cannot reproduce

Every open issue adds some clutter, and we try to make the issues fewer and make it easier for new collaborators to find. Currently we try to close any issue that meets the first checkbox + one other.

Feel free to reopen this issue at any point if you have the intent to continue to work this. :)

[ypriverol]

Any progress on this. Would be great to have support for LSF clusters in Luigi.

[markmcdowall]

Hi @Tarrasch,

I have been in contact with @alexbw about the Luigi LSF update. We now have a working update on the LSF branch of a fork.

Would you like the changes merged to the master branch of the fork and then for a pull request to be made to Luigi or would you like it to remain on the branch?

Cheers,

Mark

[dlstadther]

@markmcdowall Feature branch PR will be fine!