Prefect: Fix Codecov bot to only update after all reports are in

[cicdw]

Currently, our helpful Codecov bot will begin posting coverage stats when one of the lighter CI jobs is complete and has submitted its report. For example, on #1101 the first post by codecov was:

This can be confusing and misleading to new contributors.

Ideally, the bot should wait for all "expected" reports to be submitted before posting a coverage report. I tried looking into this at one point but couldn't figure out if it was possible.

[zangell44]

I can look into it this week, I'm tired of getting roasted by the Codecov bot 😄

[cicdw]

Yea, that'd be great! In each CI job that uploads coverage, there are only two things that are codecov related:

• in each pytest command that is intended to generated coverage, we use the --cov=prefect flag
• after each pytest is run, we upload coverage to codecov with bash <(curl -s https://codecov.io/bash) -cF python

Here is an example: https://github.com/PrefectHQ/prefect/blob/master/.circleci/config.yml#L89-L95

[zangell44]

would it be okay for me to open a dummy PR just to test changes to the circleci config file?

Struggling to find a way to test workflows locally

[cicdw]

Yea, go for it @zangell44

[zangell44]

Sorry @cicdw, I couldn't figure this one out this week, closed my PR so it doesn't linger.

For future reference, I suspect the best way to do it is using codecov flags. This would involve flagging different reports by

1. Changing -cF python in the circleci config to -cF <moreSpecificTestFlag> (e.g. python35)
2. Enumerating flags properly in codecov.yml

[cicdw]

No worries @zangell44 ! Thanks for giving it a solid effort, it is definitely appreciated.