Packit-service: Use-case: continuous builds into a Koji tag

[jlebon]

Hi,

Project looks very interesting!

One use-case here I don't see explicitly listed (though I think can reasonably fall under Packit's scope) is doing continuous builds from an upstream repo. In Fedora CoreOS, we'd like our build tools to be rebuilt continuously. We've recently acquired a Koji tag to do this, which works well, but we really want to automate this process.

IOW, would be nice if Packit-a-a-S supported the following:

• watch for every git push in an upstream repo
• on push events, create an SRPM and submit Koji build targeting specific tag

At least Packit knows how to build SRPMs from upstream content, so the first part seems covered. What would be nice is if that feature was kept separate from the propose-update workflow. I.e. we want to be able to have both continuous builds, as well as regular "release" builds which follow the usual dist-git -> Koji -> Bodhi flow.

(To confirm, there's no Packit-a-a-S deployed anywhere yet, right?)

---

Written by @TomasTomecek
The work which needs to be done to make this work:

• [ ] Packit service can send a build to koji as a reaction to a specific event
• [ ] Job definition is implemented (with a handler and the specific event type)
• [ ] People can configure if the builds are meant to be scratch or pick a koji target

[TomasTomecek]

(To confirm, there's no Packit-a-a-S deployed anywhere yet, right?)

not yet, should be done by the end of the sprint, which is this week :D My assumption is that it will be deployed and usable by the end of the month.

Your proposal is actually fairly close to what we are planning on doing: continuously build in copr (for sake of CI).

For production builds, you may be aware of DLRN, which does exactly what you need. Right now we are in process of learning more about DLRN and figuring out if packit could integrate with it.

[jlebon]

not yet, should be done by the end of the sprint, which is this week :D My assumption is that it will be deployed and usable by the end of the month.

Nice! Looking forward to it. :)

Your proposal is actually fairly close to what we are planning on doing: continuously build in copr (for sake of CI).

Yeah, we actually moved away from COPR to Koji because the former doesn't support aarch64, which we need for FCOS. Would be great to have the backend infrastructure be configurable so we could target Koji instead of COPR.

For production builds, you may be aware of DLRN, which does exactly what you need.

Hmm, interesting, hadn't heard of it before. Though it looks like it's just using mock to build packages locally? We want to be able to delegate builds to Koji for multi-arch.

[TomasTomecek]

AFAIK DLRN can also build in koji.

Okay, let's leave this one opened as an RFE to support continuous builds in koji.

[TomasTomecek]

@jlebon @dustymabe guys, what's the status here? are you still interested in this?

[dustymabe]

yes!

[TomasTomecek]

I finally did some work to start addressing this.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[jlebon]

This issue is still relevant to us.

[dhodovsk]

related to https://github.com/packit-service/packit-service/issues/74

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Thank you for your contributions.
We are doing this to be sure that the issue is still relevant. Anyone can comment to remove the stale state. (The issues marked with pinned, security, bug or EPIC label
are not considered stale.)

[jlebon]

This issue is still relevant to us.

[lachmanfrantisek]

This issue is still relevant to us.

@jlebon Thanks for the update. Sorry about the stale bot..

[TomasTomecek]

Me, Franta, Dusty and Christian talked about this the other day and we finally understand the request. In coming weeks, we'll have a ton of stuff on our plate, so I can't guarantee when we'll be able to start working on this. Also moving this to packit-service repo.

[dustymabe]

hey @TomasTomecek - if we could enumerate some of the work to be done it might make it easy for a new team member or an external contributor to help. Do you think you could describe the work that needs to be done?

[TomasTomecek]

@dustymabe I added a brief task list to Jonathan's first post.

The good thing is that we've already done the plumbing code in packit itself, now we just need to define the job in packit-service and implement a handler for it. I hope that next sprint (in one week) we should start adding such code in packit-service since we need it for the centos stream work anyway.

[TomasTomecek]

Franta is working on koji builds this sprint, unassigning myself

CC @lachmanfrantisek

[jlebon]

Any updates on this? Or something you'd like us to take for a spin to test?

[lachmanfrantisek]

@jlebon Sorry, still too many other things around. But the sending part is in review: #547

When it's merged, it will be available on our staging app. The missing pieces after that are showing the response and document everything.

[lachmanfrantisek]

You will be able to have builds (or scratch ones) for release, branch or PR commits.

[TomasTomecek]

@lachmanfrantisek this is relevant, correct? https://github.com/packit-service/packit-service/pull/657

[lachmanfrantisek]

@lachmanfrantisek this is relevant, correct? #657

Yes, this should bring the last part -- reporting back the results. (Only commit status for now. No retriggering possible yet.)

[TomasTomecek]

@lachmanfrantisek this is relevant, correct? #657

Yes, this should bring the last part -- reporting back the results. (Only commit status for now. No retriggering possible yet.)

nice, shall we close this then?

we can also provide guidance who's interested in setting this up for their projects

[LorbusChris]

:+1: Documentation or example configs for this feature would be highly appreciated! :)

[lachmanfrantisek]

+1 Documentation or example configs for this feature would be highly appreciated! :)

It's already in our docs page:
https://packit.dev/docs/configuration/#supported-jobs -> production_build

[lachmanfrantisek]

nice, shall we close this then?

@TomasTomecek I am not sure if the original issue is covered but all of your bullet points are done.

on push events, create an SRPM and submit Koji build targeting specific tag

We don't support setting tags yet.

[TomasTomecek]

let's close this then

We don't support setting tags yet.

seems like we do :)

targets – a (list of) targets we want to build for, list of supported targets can be listed using with koji list-targets

@LorbusChris please let us know if the documentation is sufficient, and if not, we can work with you to set this up

[LorbusChris]

Thanks, it looks pretty straight forward, I'll set something up soon :)

I do have one question though:
What user does the builds in Koji?
Is it my own user, or a packit user? If it's the latter, do I need to add it to the list of maintainers for the given RPM?

[lachmanfrantisek]

What user does the builds in Koji?

It's same for everything we do:

• If you use packit CLI, it will use your own user/authentication.
• If you use packit-service (the GitHub app), it will use our user -- so yes, you will probably need to give us(=our user) the permissions.

[lachmanfrantisek]

seems like we do :)

Yes, my bad. I was confused about tags/targets difference... (Funny that it was me who implement that..;)

[stale[bot]]

This issue has been marked as stale because it hasn't seen any
activity for the last 60 days.

Stale issues are closed after 14 days, unless the label is removed
by a maintainer or someone comments on it.

This is done in order to ensure that open issues are still relevant.

Thank you for your contribution! :unicorn: :rocket: :robot:

(Note: issues labeled with pinned, security, bug or EPIC are
never marked as stale.)

[lachmanfrantisek]

let's close this then

OK, closing the issue since the koji-builds are supported.

Please, create a new, specific, issue if you want a new trigger or miss some feature. Thanks!