Task: Allow skiping task if generated files exists

[smyrman]

Given the following example

task1: # Download some files from the internet that are always the same!
  cmds:
    - ./download_from_internet.bash
  generates:
    - files/v1.0.0/*
task2: # Task 2 need task1 to run only if files/v1.0.0/* does not exist.
  deps:
    - task1
  sources:
    - files/v1.0.0/file1.xml
  cmds:
    - ./use_files.bash

Given we try ro run task task2, could we somehow tell task1 not to run if "files/v1.0.0/*" have been generated?

[andreynering]

Today, Task requires both sources and generates to be able to compare files timestamps. Currently you can do something like this:

dl:
  cmds:
    - touch dl.lock
    - curl https://google.com/ > index.html
  sources:
    - dl.lock
  generates:
    - index.html

dl.lock means the file was downloaded.

I think it makes sense to somehow allow this without this trick, but I'm not sure we could do it. I'll think about it, ideas welcome.

[smyrman]

As a FYI, I wanted dl.lock to only be generated if the other cmds succeeded, so I ended up with something like this:

dl:
  cmds:
    - download-a-bunch-of-files.bash
    # Prevent task from running again by creating a lock file in the past.
    - touch -t 197001010101 .dl.lock
  sources:
    - .dl.lock
  generates:
    - files/*

I think it makes sense to somehow allow this without this trick, but I'm not sure we could do it. I'll think about it, ideas welcome.

Yeah, it's hard to come up with something that's generic, easy to understand and don't compromises the simplicity of your format;-) As long as the trick works there is no rush.

[smyrman]

I have one idea actually...

What if we added a "status" keyword so that I could do something like this?

dl:
  cmds:
    - download-a-bunch-of-files.bash
  status:
    - test -d ./files
    - test -f ./files/myfile.txt

The idea being that "status" contains a list of commands, and if all commands succeed (exit status 0), assume task is up-to-date. Should be pretty generic, and pretty powerful.

[andreynering]

That's a good idea.

[andreynering]

Implemented on master branch.