Skaffold: Infer deployables from git tagger in skaffold deploy flow

[tejal29]

Context: This is left over idea up issue from #922
In #922, we added a flag to --build-artifact to skaffold deploy which is json representation of the build output. This is an explicit way of connecting build to deploy.

Current behavior

User has to explicitly generate the json file and feed that to deploy, even when using git tagger.

Expected behavior

If users are using the GitTagger for tagging the build outputs, deploy should be able to pick up the built images without passing in --build-artifact

[pepjo]

Hi,

I am not sure if this is the right place to mention this. If I should open a new issue please tell me and I would gladly do so.

We are running skaffold build --quiet > build.out and skaffold deploy --build-artifacts build.out as different stages in our CI. Build happens when a pull request is created and integration tests run against that. Deploy only happens when the branch is merged.

We have a problem with skaffold build --quiet > build.out since we can no longer see errors that might be generated during the build phase. Is there any way to avoid that? Could the file not be created with an option like skaffold build --out build.out without interfering with the output?

Thanks for your time and effort with skaffold! It is an amazing project.

[derekjanni]

My interpretation from #922 was that we want the actual behavior to look like this:

1. skaffold build # to build and/or push images to registry
2. skaffold deploy  # this infers latest build from registry by using GitTagger 

Presumably, 1 & 2 could happen at different times, or on different machines and deploy would still work as expected. If there is only local context, 2 should act in accordance with build's default behavior.

I would also go as far as to say that deploy should not build unless instructed to because that is what build is for. There is a ton of implicit, duplicated functionality between build, deploy, dev and run - it's worth the effort to start confining each of these commands to their particular domains.

[pepjo]

Hi @derekjanni ,

That would indeed be even better. My comment was just meanwhile this does not happen.

Also, it would be nice if there is a way to parse the output of skaffold build like the current JSON. This is useful to perform end to end testing with the docker images generated by skaffold.

[derekjanni]

@pepjo I believe your issue is separate from this one and warrants another ticket. It sounds like you're either asking for a new parameter (-o / --out) or for error traces to be automatically logged to files on completion, or both. This issue specifically deals with a suggestion for using the tagger to infer build output from a previous step instead of generating new images when using deploy

[pepjo]

Hi @derekjanni , yes it is asking for a new parameter. I will open a different issue.

Is there any timeline for implementing the behavior described in this issue?

best,
Pep

[derekjanni]

@pepjo I'm not sure when this will be finished, I'm hoping to personally get around it soon but I need to learn go first.

[balopat]

@pepjo not sure if you opened an issue, but I did independently: https://github.com/GoogleContainerTools/skaffold/issues/2249
@derekjanni let us know if you're stuck, feel free to open a PR while in progress! :)

[tstromberg]

Thank you for the idea!

I'm closing this issue as it's been open a while, and it is not clear if it's still an open issue. No one has recently stated an interest in addressing this, but if you feel strongly about it, please feel free to add a comment or send us a PR!

[pepjo]

Hi @tstromberg ,

This is still an issue for us, maybe we de something wrong?

Our current CI/CD flow is:

Run linter & unit tests -> skaffold build —quite & scaffold build -> integration and functional tests using the generated image -> skaffold deploy

As you can see, we use the skaffold docker image to run our tests. I think this is nice as it is the same image that will be deployed. However, if we run ‘skaffold build’ with the quite flag and redirecting the output we cannot see the build errors which is VERY annoying. To solve this, we run skaffold build without the quite flag in parallel, even though we are not using its output and is a complete waste.

With the feature described in the issue, we would be able to run skaffold build AND we would avoid passing the output of build as an artefact to a different CI/CD stage which adds complication.

UPDATE: I see that now there is --build-artifacts that partially solves the problem. I still think that not having to pass output files would be amazing and would simplify the pipeline. Specially since skaffold has all the needed information, that output file is just a gimmick in my use case.

[nkubala]

i'm gonna reopen this one since i do think it's still a valid use case. doubtful that our team will have any time to prioritize it anytime soon so i'll mark it as P3, but if anyone wants to submit a PR contributions certainly welcome here.