Fpm: Package layout

[certik]

We've been working with @everythingfunctional on the standardization of the layout. First iteration:

$ tree .
.
├── fpm.toml
└── src
    ├── a.f90
    ├── b
    │   └── utils.f90
    └── utils.f90

2 directories, 4 files
$ cat fpm.toml 
Name "a"
$ cat src/a.f90 
module a
use a_utils, only: util1
use a_b_utils, only: util2

implicit none
private
public util1, util2
end module
$ cat src/utils.f90 
module a_utils
implicit none
private
public util1

integer :: util1

end module
$ cat src/b/utils.f90 
module a_b_utils
implicit none
private
public util2

integer :: util2

end module

[certik]

Here is a second iteration of the above layout:

$ tree .
.
├── app
│   └── main.f90
├── fpm.toml
└── src
    ├── a
    │   ├── b
    │   │   └── utils.f90
    │   └── utils.f90
    ├── a.f90
    ├── x
    └── x.f90

5 directories, 6 files

[milancurcic]

I'm really confused now.

• This thread doesn't say that the source file should have the same name as the module;
• It actually implies that it can be different, e.g. utils.f90 defines module a_utils.
• fpm can't build this example because of #57.

So based on feedback so far, I assume that source file should be named the same as the module, and perhaps the spec in this thread is out of date. Then you have (ignoring app and x.f90 for simplicity):

$ tree .
.
├── fpm.toml
└── src
    ├── a.f90
    ├── a_utils.f90
    └── b
        └── a_b_utils.f90

2 directories, 4 files

Now the source files have the same name as the modules they define. This doesn't work either because fpm builds b_a_b_utils.o out of src/b/a_b_utils.f90:

# gfortran (for build/debug/library/a_utils.o build/debug/library/a_utils.mod)
# gfortran (for build/debug/library/b_a_b_utils.o build/debug/library/b_a_b_utils.mod)
fpm-exe: Error when running Shake build system:
  at want, called at src/Build.hs:188:11 in fpm-0.1.0.0-9zFE4ut013U9YSOSmXT3I3:Build
* Depends on: build/debug/library/package-name.a
  at need, called at src/Build.hs:186:13 in fpm-0.1.0.0-9zFE4ut013U9YSOSmXT3I3:Build
* Depends on: build/debug/library/b_a_b_utils.o
  at &%>, called at src/Build.hs:166:11 in fpm-0.1.0.0-9zFE4ut013U9YSOSmXT3I3:Build
* Depends on: build/debug/library/b_a_b_utils.o build/debug/library/b_a_b_utils.mod
  at error, called at src/Development/Shake/Internal/Rules/Files.hs:245:13 in shake-0.18.5-44KSA7uQF2VObxzEvLYZx:Development.Shake.Internal.Rules.Files
* Raised the exception:
Error, &%> rule failed to produce 1 file (out of 2)
  build/debug/library/b_a_b_utils.o
  build/debug/library/b_a_b_utils.mod - MISSING

Okay, so fpm does some renaming of files under the hood--b/a_b_utils.f90 compiles to b_a_b_utils.f90. But the compiler emits a_b_utils.mod and fpm expects b_a_b_utils.mod.
A-ha! Maybe I need to call my module b_a_b_utils:

$ head -1 src/b/a_b_utils.f90 
module b_a_b_utils

Great, fpm now builds correctly.

Perhaps this is what Brad meant in #57 when he said

we want the file name to match the module name, and the module name to include the path components of the file name.

I didn't understand this because the 1st half of the sentence conflicts the 2nd half of the sentence.

But I think the 2nd half is key: If you have src/a/b/c/utils.f90, then the module should be called a_b_c_utils.

Given this, perhaps I can reverse engineer the correct spec. Let's try a third iteration:

$ tree .
.
├── fpm.toml
└── src
    ├── a.f90
    ├── b
    │   └── b_utils.f90
    └── utils.f90

Then, the contents are:

$ cat src/a.f90 
module a
use utils, only: util1
use b_b_utils, only: util2

implicit none
private
public util1, util2
end module
$ cat src/utils.f90 
module utils
implicit none
private
public util1

integer :: util1

end module
$ cat src/b/b_utils.f90 
module b_b_utils
implicit none
private
public util2

integer :: util2

end module

fpm builds this correctly.

Now, if I went through this much trouble to figure this out with help from fpm developers, imagine other people trying to build their thing with fpm. :)

We need a clear, clean, explicit spec.

[certik]

@everythingfunctional and I discussed this, and the solution that we both liked in the end is precisely as I posted above (which is different to your comment), I just didn't have time to write it more explicitly. @milancurcic, please help us, we are doing what we can in our little time. If you have time, please help us write the spec.

The most natural way seems to have a file src/a/b/utils.f90, which means a package "a.b", and module "a.b.utils" (using Python syntax). However, in Fortran we do not have nested packages, until https://github.com/j3-fortran/fortran_proposals/issues/86 is implemented. But what we can do now is to put the package name in the name of the module, so the file src/a/b/utils.f90 would start with module a_b_utils and fpm would check / enforce it. To move files around, you can simply move them, and fpm will have a mode to rename the first module line correctly.

What you are proposing is to name the file as src/a/b/a_b_utils.f90 which is redundant and we feel it would be quite a pain to keep correctly named. It's much easier to simply call it src/a/b/utils.f90. However, the first module line must have the full name, to disambiguate src/a/utils.f90 from src/a/b/utils.f90.

Now we are just working on fpm to work like this. If there are some bugs, then we need to fix them.

In particular, we need to write more tests, which would clarify what is meant to work.

[everythingfunctional]

Ok, yeah. Here's what I think the specification about that would be.

fpm replaces the path separators with underscores when determining the name of the .o file to be created when compiling a given source file. For modules, it also expects compiling that source file to create a .mod file with the same name as the .o file. This means that a source file a/b/utils.f90 should contain a module named a_b_utils.

I'm sure this could use a bit more wordsmithing or clarification. It also needs to be fit into a larger specification about the expected (default) organization of an fpm compatible project, with instructions about how to override the defaults.

[milancurcic]

@milancurcic, please help us, we are doing what we can in our little time. If you have time, please help us write the spec.

What do you think I'm doing? :)

What you are proposing is to name the file as src/a/b/a_b_utils.f90 which is redundant and we feel it would be quite a pain to keep correctly named. It's much easier to simply call it src/a/b/utils.f90. However, the first module line must have the full name, to disambiguate src/a/utils.f90 from src/a/b/utils.f90.

I don't propose that, it's how I originally understood Brad, but that's not what he meant and I understood it later. What led me astray is that the 2nd iteration of your tree wasn't consistent with the module names in the original post.

Based on the feedback, here's the package structure (same as Ondrej's 2nd iteration):

$ tree .
.
├── fpm.toml
└── src
    ├── a.f90
    ├── a_utils.f90
    └── b
        └── utils.f90

2 directories, 4 files

And here are the contents:

$ cat src/a.f90 
module a
use a_utils, only: util1
use a_b_utils, only: util2

implicit none
private
public util1, util2
end module
$ cat src/a_utils.f90 
module a_utils
implicit none
private
public util1

integer :: util1

end module
$ cat src/b/utils.f90 
module a_b_utils
implicit none
private
public util2

integer :: util2

end module

Does this look correct?

[certik]

Almost. You are still putting the prefix into the filename in a_utils.f90. It should look like in https://github.com/fortran-lang/fpm/issues/39#issuecomment-590597668. So put a_utils.f90 into a/utils.f90.

I also had the b module nested under a, but that just depends what you are trying to show. Both can be done.

[milancurcic]

Oops, you're right, I did that. If we ignore src/x* and app, we get:

$ tree .
.
├── fpm.toml
└── src
    ├── a
    │   ├── b
    │   │   └── utils.f90 # module a_b_utils
    │   └── utils.f90 # module a_utils
    └── a.f90 # module a

3 directories, 4 files

I put module names as # comments next to each file. I think this is correct now and I agree with it.

If you agree, I'll submit a PR to document this.

[milancurcic]

Btw, current master of fpm builds this correctly.

[certik]

@milancurcic yes, if you could please document this and the reasoning behind this decision, that would be awesome. I am really happy you agree with this also. It's different to what I've been used to doing, but only in the fact that each module has the full name in the module line (and when you think about it, it makes sense to do it that way until https://github.com/j3-fortran/fortran_proposals/issues/86 is implemented). Otherwise I've seen lots of projects structured in directories like this.

[everythingfunctional]

It looks like we're on the same page now. Thanks for struggling through this @milancurcic . I know we didn't do a very good job documenting it, so your efforts are hugely appreciated.

[everythingfunctional]

I think this is pretty well settled now. Should we close this?

[certik]

We should document these choices and why we chose it. Then we can close this. Because this will come up again.

On Wed, Apr 29, 2020, at 9:30 PM, Brad Richardson wrote:

I think this is pretty well settled now. Should we close this?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/fortran-lang/fpm/issues/39#issuecomment-621591611,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAFAWES7J5OW4SJVYLBDYLRPDWENANCNFSM4K2TQKOQ.

[certik]

This issue does not have much info anyway and it is documented in the tutorial a bit, so let's close this one.