Spack: creating a spack package for closed-source software

[cwsmith]

Our code optionally links against the Simmetrix SimModSuite closed-source libraries that support parallel unstructured mesh operations. Simmetrix provides a set of tarballs (one per logical component) with headers and libraries, and an MPI wrapper layer that gets compiled on the destination system (to support the user's MPI). Is there a clean way to create a spack package that downloads/locates the tarballs, unpacks them, and compiles the MPI wrapper layer? Any example packages for developing such a package would be appreciated.

At this point, my specific concerns are:

1. Is this a good use case for spack?
2. Accessing the tarballs - The tarballs are available in a password protected portion of a website. I use wget with the --user= and --password= arguments to download them. Does spack support this? Alternatively, can spack be told where to find local copies of the tarballs?
3. Unpacking multiple tarballs in a single package - There is one tarball per logical component. Does spack support this? If not, could each tarball need to be broken out into a separate package and a meta package defined for the set?

Thank-you,
Cameron

[adamjstewart]

1. Yes. Spack has multiple closed-source libraries or libraries that install pre-compiled binaries. Obviously it is ideal to build from source for optimization, but you can still use Spack to install closed-source software. Spack even supports licensed software.

2. I don't know of any way to handle the user/password logic, but you can tell Spack to install from an already downloaded tarball. See matlab, star-ccm-plus, and vizglow for examples. With these packages, Spack will first check to see if your tarball is installed in a mirror, and if not checks the current directory for a tarball.

3. Usually, a package has a single primary tarball, but may have additional resources it needs to download and extract before installation. There are several examples of this in Spack if you grep for them.

[tgamblin]

@cwsmith: you could consider using a local package repository. See http://spack.readthedocs.io/en/latest/repositories.html. Your users would have to type spack repo add <path/to/repo> to register it initially, or you can stick the path in /etc/spack/repos.yaml on the system where you want to use this.

If you do this, you can actually add your username and password to your special packages via the curl_options attribute. Look at the jdk package for an example. Obviously, you wouldn't upstream these packages, but it would allow you to embed the username/password in your package file.

[cwsmith]

Thank you. I'll work on creating the new package. If I hit any snags I'll post in slack and/or create a new issue.