katello-nightly-rpm-pipeline 944 failed

Katello nightly pipeline failed:

https://ci.theforeman.org/job/katello-nightly-rpm-pipeline/944/

foreman-pipeline-katello-nightly-centos8-install (passed) (remote job)
foreman-pipeline-katello-nightly-centos8-upgrade (passed) (remote job)
foreman-pipeline-katello-nightly-centos7-install (passed) (remote job)
foreman-pipeline-katello-nightly-centos7-upgrade (passed) (remote job)

Wooo tests passed. Pushing RPMs failed…

@evgeni / @ekohl any ideas on the failure here?

[2021-04-20T18:36:22.516Z] cp: cannot create hard link ���/var/www/vhosts/yum/htdocs/katello/nightly/katello/.el7-20210420183622/x86_64/katello-repos-latest.rpm��� to ���/var/www/vhosts/yum/htdocs/katello/nightly/katello/el7/x86_64/katello-repos-latest.rpm���: Operation not permitted

Looks like file permissions are incorrect. Files are owned by root instead of yumrepo so it can’t overwrite the symlink. We haven’t seen this before because it hasn’t passed since we started to push the repositories to yum.theforeman.org. Should be fixed now:

# chown yumrepo: -h /var/www/vhosts/yum/htdocs/katello/nightly/katello/el7/x86_64/katello-repos-latest.rpm /var/www/vhosts/yum/htdocs/katello/nightly/katello/el8/x86_6/katello-repos-latest.rpm
# ls -l /var/www/vhosts/yum/htdocs/katello/nightly/katello/el7/x86_64/katello-repos-latest.rpm /var/www/vhosts/yum/htdocs/katello/nightly/katello/el8/x86_64/katello-repos-latest.rpm
lrwxrwxrwx. 1 yumrepo yumrepo 44 Mar 24 14:18 /var/www/vhosts/yum/htdocs/katello/nightly/katello/el7/x86_64/katello-repos-latest.rpm -> katello-repos-4.1-0.1.nightly.el7.noarch.rpm
lrwxrwxrwx. 1 yumrepo yumrepo 44 Mar 24 14:18 /var/www/vhosts/yum/htdocs/katello/nightly/katello/el8/x86_64/katello-repos-latest.rpm -> katello-repos-4.1-0.1.nightly.el8.noarch.rpm

Thanks for fixing. We have pushed to yum.theforeman.org via the job previously [1] as that is how the initial set of repositories got there. Did something change in the infra or our config that could have caused this? I am hoping to understand if we think this will happen again or was a one time change due to something we changed.

[1] Jenkins

I might or might not have “fixed” the links as root and forgot the perms.

Whistles innocently

We did indeed discover we forgot to update them. This was merged to automatically fix it:
https://github.com/theforeman/foreman-infra/commit/30e477b5c357a59dbd9c9b5843fbc5cdb00f6be4

Since nightly was failing, it was then updated by hand. As all things done by hand, there’s a chance for mistakes.

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.