Problem:
on the week end all syncs have frozen, they will not complete past 74%
Error message: the server returns an error HTTP status code: 500 Response headers: {“date”=>“Wed, 21 Apr 2021 02:42:27 GMT”, “server”=>“gunicorn/20.0.4”, “content-type”=>“text/html; charset=UTF-8”, “x-frame-options”=>“SAMEORIGIN”, “content-length”=>“27”, “vary”=>“Cookie”, “via”=>"1.1 ", “connection”=>“close”} Response body:
Server Error (500)
I also get this error now even trying to add a repo
Foreman and Proxy versions:
2.3.3
Foreman and Proxy plugin versions:
foreman-tasks 3.0.5
foreman_discovery 16.3.4
foreman_hooks 0.3.17
foreman_remote_execution 4.2.2
foreman_setup 7.0.0
foreman_templates 9.0.1
katello 3.18.2.1
puppetdb_foreman 5.0.0
Distribution and version:
RHEL 7.8
additionally this was working with no issue and performed a patching cycle flawlessly before this weekend and there was no change in the environment.
I have since found that this actually seems to have been broken since the update cycle. after investigation (restore from checkpoint then attempt patching) the update of these packages breaks it
Updating:
createrepo_c x86_64 0.17.1-1.el7 pulpcore 59 k
createrepo_c-libs x86_64 0.17.1-1.el7 pulpcore 85 k
katello noarch 3.18.2-1.el7 katello 9.6 k
katello-common noarch 3.18.2-1.el7 katello 18 k
katello-debug noarch 3.18.2-1.el7 katello 11 k
katello-repos noarch 3.18.2-1.el7 katello 12 k
libsolv x86_64 0.7.17-1.el7 pulpcore 371 k
libuv x86_64 1:1.41.0-1.el7 epel 153 k
pulp-docker-plugins noarch 3.2.9-1.el7 pulp 107 k
python-pulp-docker-common noarch 3.2.9-1.el7 pulp 36 k
python2-kombu noarch 10:4.0.2-14.el7 pulp 343 k
python2-solv x86_64 0.7.17-1.el7 pulpcore 106 k
python3-createrepo_c x86_64 0.17.1-1.el7 pulpcore 43 k
python3-django noarch 2.2.20-1.el7 pulpcore 5.1 M
python3-pulp-2to3-migration noarch 0.11.0-1.el7 pulpcore 216 k
python3-pulp-rpm noarch 3.10.0-1.el7 pulpcore 280 k
python3-pulpcore noarch 3.7.5-1.el7 pulpcore 486 k
python3-rq noarch 1.5.2-2.el7 pulpcore 120 k
python3-solv x86_64 0.7.17-1.el7 pulpcore 107 k
qpid-dispatch-router x86_64 1.15.0-1.el7 epel 1.1 M
tfm-rubygem-foreman-tasks noarch 3.0.5-1.fm2_3.el7 foreman-plugins 2.0 M
tfm-rubygem-hammer_cli_katello noarch 0.24.3-1.el7 katello 127 k
tfm-rubygem-katello noarch 3.18.2.1-1.el7 katello 9.3 M
tfm-rubygem-pulp_2to3_migration_client noarch 1:0.7.0-1.el7 katello 22 k
tfm-rubygem-pulp_rpm_client noarch 3.9.0-1.el7 katello 51 k
Installing for dependencies:
libnghttp2 x86_64 1.33.0-1.1.el7 epel 68 k
Transaction Summary
Install ( 1 Dependent package)
Upgrade 25 Packages
Hi @bcoleman,
Can you please show us any backtraces you see on the failed Foreman Dynflow task? Also, do you see backtraces in /var/log/messages from around the same time the sync task errored out?
1 Like
