Super-linter: Is bash the right language?

[Hanse00]

I'd like to bring this up for consideration are a general question. I understand there are pros and cons of any language, this isn't so much an academic debate, as a practical consideration.

Pros for bash:

• The code is already written in bash, duh.
• Relatively simple and lightweight

Cons for bash:

• Gets harder to manage as the complexity grows
• Relatively hard to test

At the end of the day, it's a decision beyond my control. But I'm starting to think that as the complexity grows, we may wish we had some unit tests, and cleaner code.

Just some food for thought.

[Almenon]

A 2k+ line bash file is definitely going to scare off some newcomers.

[zkoppert]

I’d be interested to hear some suggestions on a language that could provide some benefit to the maintainability aspect.

[Hanse00]

I have a personal bias towards Python, mostly based on years of experience. It does have pretty solid unit testing capabilities, and some of the code duplication we’ve done so far, could easily be replaced with looping constructs. Calling out to shell commands is easy enough with the standard libraries too.

That’s just one suggestion though, I’m happy to hear others as well.

[Almenon]

Typescript is easy to work with ala python with the additional benefit of having a great type system. However the people probably interested in this project (devops) would be more likely to have experience with python.

How much is speed a concern for this project? How long does it take to run super-linter against a big monolith project, disregarding the time it takes for the linters themselves? How does that compare to the full timing? I'm presuming speed is negligible compared to the time to takes to run the linters, and that maintainability is a more important consideration.

[nemchik]

I can't make the argument that bash is the best option, or that any other suggestion is any less valid than what I'm about to say, but I wanted to chime in on the maintainability consideration with bash.

Just by looking over the repo and making my first PR my opinion is that maintainability could be improved by splitting out the functions in some of the larger script into separate files. Here is a repo I maintain https://github.com/GhostWriters/DockSTARTer that is a script written in bash. The largest script in the repo is https://github.com/GhostWriters/DockSTARTer/blob/master/main.sh which has a specific section https://github.com/GhostWriters/DockSTARTer/blob/c9384746aa8766cd656bdd26ecb7fdc7aebace9e/main.sh#L335-L345 that runs functions from the files in https://github.com/GhostWriters/DockSTARTer/tree/master/.scripts so that the code can be modular in a way, and easier for contributors to read and understand. It doesn't make sense to split EVERY function into a separate file (it really depends on each individual project).

From a unit testing perspective I have an example of how that can be done as well https://github.com/GhostWriters/DockSTARTer/blob/c9384746aa8766cd656bdd26ecb7fdc7aebace9e/main.sh#L348-L364 . This method may not be the right answer for every project (or this project) but just an example of what can be done. In each script file that the above runner function is able to load there is a function and a test_function and this code runs whatever is necessary to test the function. I have not put a lot of effort into the testing in my project to do things like assert or confirm exact values or actions, rather I just ensure that the code runs without a bad exit code, but that was just a personal choice for my project. This project could write tests to do whatever is needed to truly confirm code works exactly as expected in all scenarios (as much as possible).

I do think there are some readability/maintainability improvements that can be made in the bash scripts in this repo, and I'll likely be digging in to contribute as much as I can. Just to highlight another example of a bash script I wrote that uses a very similar structure to the one linked above here is https://github.com/nemchik/ShellSuite/blob/master/shellsuite.sh which is actually just a single file script, but quite readable. You can for the most part skip down to the main function in this script and read exactly what it's doing very easily.

tl;dr
Making maintainable bash scripts is doable. Unit testing bash scripts is doable.

[admiralAwkbar]

At some point, making this a few bash scripts is the right answer... When it was started, it was very small and made sense... Now its a bit bigger and needs to look at framework and maintainability.

[Hanse00]

@admiralAwkbar I could whip up a Python prototype with some testing, if you'd like to see that as an option.

[admiralAwkbar]

I think we're good on leaving this as a bash project for the time being :) its simple, flexible, and with the newer version to pull it into more functions, should be easier to maintain :)

[Hanse00]

Alright :)