Spack: How to find the name of module to build before it gets installed

[gweodoo]

Hi all,

This may be a dumb question/suggestion, but I did not find anywhere a way to get the actual package name that will be effectively created. In my case, I use environment-modules and I would like to have the module name BEFORE it gets actually installed.

Rationale

I understand it needs the package to be concretized beforehand and that's not a problem. I can also understand that concretization may fail if the actual recipe is updated with new versions and/or defaults. But, given a recipe, with known version, how hard would it be to get the module name early enough?

Description

I'm currently building software to run tasks remotely on a supercomputer, especially for the parallel testing purpose. For example, I want to test a runtime with the OSU benchmark. I have to create my batch before actually running it. Two tasks will then be generated :

• a command like spack install osu-micro-benchmarks ^mpi@...
• a command like spack load osu-micro-benchmarks... depending on the above task.
  I agree this package is not the best illustration of my situation as it does not expose any binary in PATH but consider a Corals app. It that case, I'm currently relying on spack location --install-dir instead.

Using spack load(as for some others commands) is not optimal because it requires Shell integration and each task being run into its own shell. It means setup-env.sh would have to be loaded before each task starts, leading to an important overhead. At this point, I would like to use the actual module name, but at the task generating step, the concretization has not been done yet and I did not find how to get it without effectively installing the package.

If such a thing was possible, building Spack commands could be separated from their actual execution, without paying the price of loading the whole Spack environment just to run an MPI application.

Notes

Clearly, this is specific, maybe a public API to help program-to-program communication (through JSON, for instance) could clearly achieve that and would standardize a lot of things (the fact that Spack is not always returning a non-zero value when an error occurs can also be frustrating).

I'm sure such an interface already came to your minds and I might have missed something from the documentation.

Let me know if something is unclear, I would be happy to provide additional information.
Thanks for your help and sorry for any English typo.

[alalazo]

This may be a dumb question/suggestion, but I did not find anywhere a way to get the actual package name that will be effectively created. In my case, I use environment-modules and I would like to have the module name BEFORE it gets actually installed.

Would this help:

$ . share/spack/setup-env.sh 
$ spack-python
Spack version 0.12.1
Python 2.7.15rc1, Linux x86_64
>>> import spack.spec
>>> zlib =  spack.spec.Spec('zlib')
>>> zlib.concretize()
>>> import spack.modules.tcl as tcl
>>> writer = tcl.TclModulefileWriter(zlib)
>>> print(writer.layout.filename)
/home/mculpo/PycharmProjects/spack/share/spack/modules/linux-ubuntu18.04-x86_64/zlib-1.2.11-gcc-8.2.0-ivqu252

?

[gweodoo]

It definitively could!
I'm gonna test it within the end of the day to let you know if this ticket could be closed.
Thanks

[gweodoo]

it seems writer.layout.use_name could do the trick. I probably have enough with that.
Thanks for your help.