Conda-build: Conda build needs to be better about cleaning up after itself

[groutr]

1. Remove build directory as last step of building, not first step of testing.
2. Remove _test environment. Somehow clear local packages from the cache.

The state of conda should, ideally, be the same after running conda-build as before running conda-build.

[ivoflipse]

If the _test environment got deleted (regardless of success or failure), it would be pretty hard to debug any problems that might have come up during testing.

Building a new package at least removes the _test environment, so it shouldn't interfere with that.

[pelson]

Actually, when writing new recipes, the remnants of the _build env are supremely useful. I'd definitely not like to lose that ability, though I'd be happy enough to turn the functionality on as a config itme.

[ivoflipse]

Would it help to have a conda clean command get rid of such artifacts?

[groutr]

The main priority is that build artifacts don't end up polluting the package cache as it can sometimes cause problems (like shadowing a package in a repo).

I see three possible solutions (open to more)

1. Possibly enhance conda clean to clean a specific package from cache (something like conda clean mypackage) (@ivoflipse, this is a possibility). Then we can clean all packages that are listed in the conda-bld/platform repodata.
2. Remove _build and _test envs after a successful build, followed by a conda clean.
3. See if there is a way to create a temporary package cache for conda-build. This is only an idea and I have no idea how it would be implemented right now.

Questions:
@pelson, how is _build env useful to you in writing new recipes?
@ivoflipse, I'm not sure I understand you earlier comment about debugging problems that come up during testing. Are you referring to the tests that are run by conda-build?

[ivoflipse]

Yes, the _test env that get's created when building a package. Likewise, I tend to look inside the _build env if some of my commands fail during building. To see if everything is in the state I expected.

[sscherfke]

Newer versions of conda-build leave a lot of <recipe>_<timestamp> directories (for broken builds) in the conda-bld directory. It would be nice if those would be picked up by conda clean [--all].

[msarahan]

@sscherfke, at the end of each build, you should see the message created by:

   print("Source and build intermediates have been left in " + config.croot + ".")
    build_folders = get_build_folders(config.croot)
    print("There are currently {num_builds} accumulated.".format(num_builds=len(build_folders)))
    print("To remove them, you can run the ```conda build purge``` command")

if there are any of those folders. Is that not clear? I agree that having conda clean handle these would be conceptually nicer, but conda clean is part of conda, not conda-build, and thus the implementation across projects becomes a pretty ugly maintenance issue.

Thanks for highlighting this issue, though. I think that conda-build 2's individual folders handle this problem, and that this issue can be closed.

[sscherfke]

There's so much output, that I must have simply overlooked that message. :)

Maybe you could use some colors or bold printing to highlight important messages? I’ve been using click for all my CLI apps lately. It is awesome for command line arg handling, composing complex apps with sub commands and ships with utilities like colorful printing and progressbars.

[msarahan]

Click is very nice, and I'd like to use it. We're stuck with argparse for now, because changing means also changing conda, since conda-build's commands are actually subcommands of the conda command. That's a very hairy implementation, and finding a different way would be very nice.

Your suggestion of coloring output is a very good one, though, and I'll see if there's a way to do that.