Fpm: implement basic run subcommand in Fortran fpm

[urbanjost]

Looking at the subcommand "build" and just following that as a model I think I can make a functioning "run" subcommand for the Fortran fpm if the CLI is in place. I would like to give that a try, as I do not see that listed anywhere. If that works I believe the "test" subcommand would b virtually identical except it would use the test names instead of the executable names.

[everythingfunctional]

That would be great. One quick note about a subtle difference between run and test, run should call build to make sure everything is up to date, which does not necessarily need to fetch or build the dev-dependencies or the tests. However, test does need to make sure that build does do that first. I think so far we are missing the --test switch for the build command that would signify this behavior for that command.

[urbanjost]

I new build was being called from run and test, but in the version so far I have left build as-is for the most part so for now the run and test commands just have a placeholder. The build in the version I was expanding upon seems to unconditionally when called. The build of course is the core, but I was not trying to tackle that till the CLI/new/run/test commands were in place. To match what is in the Haskell fpm it needs to do quite a bit more and is doing a few things that might be intentional or might not be. The *.o files are built with the original file prefix intact -- if name.f90 is built name.f90.o is output; but I have seen this done intentionally to avoid collisions between Fortran and C files of the same basename; the Haskell version builds seperate programs in seperate directories but this one puts everything in the app directory. Not sure if that is an intentional refactoring or needs expanded and so on. So I had too many questions about that and skipped calling it automatically. Considering what you say here I should at least call it if the executable does not exist; so it would be created if not updated. I will make that change and as soon as "build" is complete at least for files local to the project directory I think it will be a one-line change to call it unconditionally.

[LKedward]

Your proposed solution to call build when the app/test doesn't exist seems reasonable until conditional compilation is implemented in build. Matching Haskell fpm behaviour with respect to object files doesn't seem to be a necessary goal.

[urbanjost]

Changed the PR to call build if executable does not exist with a command that that is temporary. I had made an earlier model that emulated the Haskell version partly because that then gave me existing packages to test against and a pass/fail criteria that it generated the same files. But the build routine has been expanded since then, and when I call it in some of the existing Haskell fpm packages it fails, but I am not sure if that is intentional or not. Everything goes into app/ and test/ instead of specific directories that mirror the input file path name for example. But I see several other discussions about redoing how files are automatically discovered and how the build/ directory might be restructured and it is not clear to me as to what should be changed so
1) should the Fortran fpm be compatible with the Haskell fpm in that it can build any current H-fpm package
2) should the generated files all go into the directories they are currently placed in?
So the problem is that as-is a bunch of current fpm packages I have to not build. The gfortran command being built could probably be changed to correct for where the files ARE going, or the files can be re-arranged to go where they do in the Haskell version, which would also change the gfortran command. It is essentially unclear which way the community plans on this going so I think I am stuck at this point until/if the CLI interface is merged and then the same for this one, and I am not sure when/if that is happening. So is there a functional spec anywhere that says what F-fpm will do? I wanted to look at whether a topological sort of the build files is reasonable to do in fortran to get the right compile order or if that is already in the current build and I missed it ( I haven't gotten through understanding all of it yet). So I think I am basically frozen at this point (waiting for Gadot). This will create the same files H-fpm does for the "new" command and "build", "run", and "test" it robustly which I was hoping would be merged so the bigger issue of complex builds could be resolved. That would create a functional F-fpm for local projects and only leave the lib-curl part remaining and non-Fortran. So you would have to have gfortran and git and probably ar or its equivalent but you could build projects on a relatively normal system (and git isn't really needed to do the development) without having to install Haskell. I was hoping we would be at that point and refining fpm to be a useful tool for building local projects; and then when the project sharing was enabled that would be the show-stopper. The toml-f package was a huge step forward, and now things seem stalled on simpler issues I thought I could get out of the way. Seems like I am missing something but Fortran needs an easy module sharing mechanism and other package managers have not seemed to catch on with the Fortran community in my experience. Seems like a project plan is missing.

[LKedward]

Thanks for your comments @urbanjost, you raise some good points.

Regarding the object file paths, I have no objection to matching the behaviour of Haskell fpm; but since I see little benefit to this, beyond allowing side-by-side use of h-fpm and fpm, it isn't something I will be personally addressing soon. Your 'test' for matching object file output between fpm and h-fpm seems quite arbitrary since no promises have been made about the structure of intermediate build files. Am I missing something here? Is there an important reason why you want the intermediate objects to match the Haskell version beyond your test between the two versions?

Fortran fpm can build any H-fpm package that doesn't have any external dependencies or build system - if you have a self-contained package that builds with h-fpm but not with fortran fpm please let me know! If you haven't already, please also see this index of example packages that I put together that summarises the current feature gap between fpm and h-fpm.

Regarding the topo sort, this is already implemented within the build system via a depth-first search; there are improvements that can be made to this, in particular to enable parallel builds, but this isn't a high priority IMO.

[everythingfunctional]

In my opinion, no promises about intermediate build files or locations should be made. Even the locations of desired artifacts (e.g. library files or executables) should probably be left unspecified other than where an install command would put them. Ideally, user's should not need to know or rely on such details. Of course Hyrum's Law will almost certainly come into effect pretty quickly, so we should probably at least put some thought into it.

[urbanjost]

There were several comments that supported the goal of making the f-fpm version at first duplicate h-fpm and nothing I could find that clearly stated otherwise so for lack of a better goal I was trying to get it to duplicate the h-fpm version so that I could interchangeably use either h-fpm and f-fpm in the same directory, but that was primarily useful for testing the build functionality that at the time did not exist. Now that I had some time to look I see build is essentially done except for not rebuilding and what you mentioned. Nice job. So I have a bunch that fail but a quick look shows it is probably the same reason. See the trivial build

   M_msg          = { git = "https://github.com/urbanjost/M_msg.git" }

which shows a setup that works with h-fpm and not f-fpm. After I removed the packages with external dependencies and the ones with custom make files and ones with system dependencies like X11, ncurses that uses an extension in a private version it worked very well on 17 non-trival projects which was quite exciting. In those I could (using PR #189) do build, build -release, run --list, test --list, run NAME, test NAME, and your build method worked with the files made with a new command as well, so for local builds it looks great. Several packages only have a custom makefile because they include C code which your version seems to support so I am going to try that. But if f-fpm should be able to build anything h-fpm does if you could take a look at the M_msg module which is the smallest example hitting the problem that would be great.

Is supporting two languages why the .f90 suffix is left on the relocatable files? Seems to work fine but unexpected; and there was no libNAME.a files which I was not sure if was intentional but as long as using the *.o files works about the only side-effect is a slightly bigger executable. I have seen leaving the suffix on as a way to avoid conflicts in projects that have NAME.c and NAME.f90 sources going into a single lib.a file but it is rare (actually it was me and I have seen it no where else :>).

[LKedward]

Apologies, yes I forgot to say that f-fpm also does not yet create static libraries libName.a as you point out.

... it worked very well on 17 non-trival projects ...

I'm glad to hear this!

Is supporting two languages why the .f90 suffix is left on the relocatable files?

Yes, although we really only need to leave the suffix on non-fortran files, so it is still possible to have full compatibility with h-fpm in the intermediate files if desired.

... if you could take a look at the M_msg module which is the smallest example hitting the problem that would be great.

I've looked into this and opened an issue (https://github.com/urbanjost/M_msg/issues/1). In summary, the difference in behaviour with h-fpm is because f-fpm does not yet support linking with static libraries.

[LKedward]

Functionality now implemented.