Packit-service: p-s ignored a PR for F34 on a non-main branch

[TomasTomecek]

use case: Anaconda team is setting up a maintenance branch for F34 where they configure packit to build and test only for F34. Packit did not set any status for this PR: https://github.com/rhinstaller/anaconda/pull/3165/files

Logs for a /packit build comment:

[2021-02-15 12:13:27,181: DEBUG/ForkPoolWorker-1] Found a config file '.packit.yml' on ref '436141408b363482ad6f79b35e420c41aecc9a38' of the 'rhinstaller/anaconda' repository.
[2021-02-15 12:13:27,379: DEBUG/ForkPoolWorker-1] No spec file found in 'rhinstaller/anaconda'
[2021-02-15 12:13:27,379: DEBUG/ForkPoolWorker-1] Package config:
{
    "specfile_path": "anaconda.spec",
    "upstream_package_name": "anaconda",
    "upstream_tag_template": "anaconda-{version}-1",
    "actions": {
        "post-upstream-clone": [
            "./autogen.sh",
            "./configure"
        ],
        "create-archive": [
            "make release",
            "bash -c \"ls -1 anaconda-*.tar.bz2\""
        ]
    },
    "jobs": [
        {
            "job": "tests",
            "trigger": "pull_request",
            "metadata": {
                "targets": [
                    "fedora-34"
                ]
            }
        },
        {
            "job": "copr_build",
            "trigger": "commit",
            "metadata": {
                "targets": [
                    "fedora-34"
                ],
                "branch": "f34-devel",
                "owner": "@rhinstaller",
                "project": "Anaconda-devel",
                "preserve_project": true,
                "additional_repos": [
                    "copr://@storage/blivet-daily"
                ]
            }
        },
        {
            "job": "copr_build",
            "trigger": "commit",
            "metadata": {
                "targets": [
                    "fedora-rawhide",
                    "fedora-eln"
                ],
                "branch": "master",
                "owner": "@rhinstaller",
                "project": "Anaconda",
                "preserve_project": true,
                "additional_repos": [
                    "copr://@storage/blivet-daily"
                ]
            }
        }
    ]
}
[2021-02-15 12:13:27,381: DEBUG/ForkPoolWorker-1] PackageConfig(config_file_path='.packit.yml', specfile_path='anaconda.spec', synced_files='SyncFilesConfig([])', jobs='[JobConfig(job=JobType.tests, trigger=JobConfigTriggerType.pull_request, meta=JobMetadataConfig(targets={'fedora-34'}, timeout=7200, owner=None, project=None, dist_git_branches=set(), branch=None, scratch=False, list_on_homepage=False, preserve_project=False, additional_packages=[], additional_repos=[]), config_file_path='.packit.yml', specfile_path='anaconda.spec', synced_files='SyncFilesConfig([])', dist_git_namespace='rpms', upstream_project_url='None', upstream_package_name='anaconda', downstream_project_url='https://src.fedoraproject.org/rpms/anaconda.git', downstream_package_name='anaconda', dist_git_base_url='https://src.fedoraproject.org/', create_tarball_command='None', current_version_command='None', actions='{<ActionName.post_upstream_clone: 'post-upstream-clone'>: ['./autogen.sh', './configure'], <ActionName.create_archive: 'create-archive'>: ['make release', 'bash -c "ls -1 anaconda-*.tar.bz2"']}', upstream_ref='None', allowed_gpg_keys='None', create_pr='True', sync_changelog='False', spec_source_id='Source0', upstream_tag_template='anaconda-{version}-1', patch_generation_ignore_paths='[]',copy_upstream_release_description='False'), JobConfig(job=JobType.copr_build, trigger=JobConfigTriggerType.commit, meta=JobMetadataConfig(targets={'fedora-34'}, timeout=7200, owner=@rhinstaller, project=Anaconda-devel, dist_git_branches=set(), branch=f34-devel, scratch=False, list_on_homepage=False, preserve_project=True, additional_packages=[], additional_repos=['copr://@storage/blivet-daily']), config_file_path='.packit.yml', specfile_path='anaconda.spec', synced_files='SyncFilesConfig([])', dist_git_namespace='rpms', upstream_project_url='None', upstream_package_name='anaconda', downstream_project_url='https://src.fedoraproject.org/rpms/anaconda.git', downstream_package_name='anaconda', dist_git_base_url='https://src.fedoraproject.org/', create_tarball_command='None', current_version_command='None', actions='{<ActionName.post_upstream_clone: 'post-upstream-clone'>: ['./autogen.sh', './configure'], <ActionName.create_archive: 'create-archive'>: ['make release', 'bash -c "ls -1 anaconda-*.tar.bz2"']}', upstream_ref='None', allowed_gpg_keys='None', create_pr='True', sync_changelog='False', spec_source_id='Source0', upstream_tag_template='anaconda-{version}-1', patch_generation_ignore_paths='[]',copy_upstream_release_description='False'), JobConfig(job=JobType.copr_build, trigger=JobConfigTriggerType.commit, meta=JobMetadataConfig(targets={'fedora-eln', 'fedora-rawhide'}, timeout=7200, owner=@rhinstaller, project=Anaconda, dist_git_branches=set(), branch=master, scratch=False, list_on_homepage=False, preserve_project=True, additional_packages=[], additional_repos=['copr://@storage/blivet-daily']), config_file_path='.packit.yml', specfile_path='anaconda.spec', synced_files='SyncFilesConfig([])', dist_git_namespace='rpms', upstream_project_url='None', upstream_package_name='anaconda', downstream_project_url='https://src.fedoraproject.org/rpms/anaconda.git', downstream_package_name='anaconda', dist_git_base_url='https://src.fedoraproject.org/', create_tarball_command='None', current_version_command='None', actions='{<ActionName.post_upstream_clone: 'post-upstream-clone'>: ['./autogen.sh', './configure'], <ActionName.create_archive: 'create-archive'>: ['make release', 'bash -c "ls -1 anaconda-*.tar.bz2"']}', upstream_ref='None', allowed_gpg_keys='None', create_pr='True', sync_changelog='False', spec_source_id='Source0', upstream_tag_template='anaconda-{version}-1', patch_generation_ignore_paths='[]',copy_upstream_release_description='False')]', dist_git_namespace='rpms', upstream_project_url='None', upstream_package_name='anaconda', downstream_project_url='https://src.fedoraproject.org/rpms/anaconda.git', downstream_package_name='anaconda', dist_git_base_url='https://src.fedoraproject.org/', create_tarball_command='None', current_version_command='None', actions='{<ActionName.post_upstream_clone: 'post-upstream-clone'>: ['./autogen.sh', './configure'], <ActionName.create_archive: 'create-archive'>: ['make release', 'bash -c "ls -1 anaconda-*.tar.bz2"']}', upstream_ref='None', allowed_gpg_keys='None', create_pr='True', sync_changelog='False', spec_source_id='Source0', upstream_tag_template='anaconda-{version}-1', archive_root_dir_template={upstream_pkg_name}-{version}', patch_generation_ignore_paths='[]', copy_upstream_release_description='False')
12:13:27.421 jobs.py           DEBUG  No config found, let's see the jobs that requires this handler.
[2021-02-15 12:13:27,421: DEBUG/ForkPoolWorker-1] No config found, let's see the jobs that requires this handler.
TomasTomecek is admin, you shall pass.
[2021-02-15 12:13:27,421: INFO/ForkPoolWorker-1] TomasTomecek is admin, you shall pass.
12:13:27.421 abstract.py       DEBUG  Getting signature of a Celery task task.run_copr_build_handler.
[2021-02-15 12:13:27,421: DEBUG/ForkPoolWorker-1] Getting signature of a Celery task task.run_copr_build_handler.
[2021-02-15 12:13:27,469: INFO/ForkPoolWorker-1] Task task.steve_jobs.process_message[ebf89894-d918-4428-b7f1-0144277f81e1] succeeded in 1.9188994984142482s: [{'success': True, 'details': {'msg': 'Job created.', 'event': {...}, 'package_config': {...}, 'job': 'tests', 'job_config': {...}}}]

The matching copr project does not exist: https://copr.fedorainfracloud.org/coprs/packit/rhinstaller-anaconda-3165/

Why did packit ignored this event?

[lachmanfrantisek]

Is this related to #973?

[TomasTomecek]

there was no info in logs about creating any copr projects - actually the logs are way too sparse

[jkonecny12]

This is happening also for commit trigger for COPR builds. If you look on last commit https://github.com/rhinstaller/anaconda/commit/cd8b5edcaf0ce54654ddea56160d9b2f75bbf18b Packit service did not report status but when I'm looking on COPR build you will see that the build happened and was successful.

[jkonecny12]

Another weird behavior is that my configuration for f34-devel branch is not applied correctly. When you look on the configuration it should be started for Rawhide, 34 and ELN but it's started for Rawhide and ELN only. It seems to me that it's getting configuration from master branch instead of f34-devel branch.

You can see that on this PR: https://github.com/rhinstaller/anaconda/pull/3173

[csomh]

Another weird behavior is that my configuration for f34-devel branch is not applied correctly. When you look on the configuration it should be started for Rawhide, 34 and ELN but it's started for Rawhide and ELN only. It seems to me that it's getting configuration from master branch instead of f34-devel branch.

You can see that on this PR: rhinstaller/anaconda#3173

Hm, I was working on this not so long ago, but then it seems the fix was not a complete one. I'll check once more.

[martinpitt]

I now tried to enable packit for all releases in cockpit-podman. F32/33/rawhide now actually work (except for issue #987), but F34 is forever stuck in "packit/testing-farm-fedora-34-x86_64 Pending — RPM build is in progress... ". The link points to https://prod.packit.dev/copr-build/83879 , which finished some 25 minutes ago already. I also don't see a single F34 test on https://dashboard.packit.dev/jobs → Tests, which seems to suggest that F34 tests are generally ignored/broken right now? Some missing config in packit infra perhaps?

Not sure if that is the same root cause as this bug. If not, I'm happy to open a new issue for this.

[jpopelka]

seems to suggest that F34 tests are generally ignored/broken right now?

There was F34 compose missing in Testing farm, but that (#979) should be fixed now.
Your F34 test was not triggered because of #984 (a list of chroots from Copr client is sometimes missing fedora-34-*) which I'm struggling to reproduce.

EDIT: And the not updated checks for packit/testing-farm-fedora-[32/33]-x86_64 are #967

[mfocko]

So just to make sure what's required output of this issue:

• [x] anaconda
  
  
  
  • [x] investigate no statuses for https://github.com/rhinstaller/anaconda/pull/3165/files
    
    
    
    
    
    • [x] check relation to #973 (imo irrelevant in this scenario)
      
      
      I'd expect it to be more related to Laura's #996
      
      
      (merged 11 days ago, anaconda's PR opened 25 days ago, merged 22 days ago)
    
    
    
  
  
  • [x] commit triggers; no longer relevant?
    
    currently appears to be building on master and f34-devel, statuses are set
    
    statuses - maybe celery time out? timewise it overlaps with that 3k long queue in COPR
  
  
  • [x] check git ref of .packit.yml and potentially come up with better solution if needed
  
  
• [x] cockpit-podman seems to be part of #967

[mfocko]

Another weird behavior is that my configuration for f34-devel branch is not applied correctly. When you look on the configuration it should be started for Rawhide, 34 and ELN but it's started for Rawhide and ELN only. It seems to me that it's getting configuration from master branch instead of f34-devel branch.

You can see that on this PR: rhinstaller/anaconda#3173

@jkonecny12 I've checked .packit.yaml on the last commit on the mentioned PR, can't see mentioned targets, could it be missed rebase? Config conforms to set statuses (pull_request trigger). Please confirm :)

[jkonecny12]

Everything seems to be working correctly a newer PR https://github.com/rhinstaller/anaconda/pull/3221. I may have been a problem of rebase in that case sorry for the noise.

[TomasTomecek]

@jkonecny12 no worries, rebasing is hard :) @mfocko thank you for such a thorough investigation!

is there anything else which needs to be done here or shall we close this one?

[mfocko]

@jkonecny12 better to check than have a bug :D

@TomasTomecek I'll check the statuses for rhinstaller/anaconda#3165, cause I'd expect at least SRPM build, otherwise we're done here