Feedback for Foreman 3.11 & Katello 4.13

Support
61
  1. Update all packages for new release
# dnf update
Updating Subscription Management repositories.
Last metadata expiration check: 0:02:21 ago on Tue 30 Jul 2024 07:20:29 PM CEST.
Dependencies resolved.
==============================================================================================================================================================================================================================================
 Package                                                                      Architecture                             Version                                                        Repository                                         Size
==============================================================================================================================================================================================================================================
Installing:
 foreman-obsolete-packages                                                    noarch                                   1.9-1.el8                                                      foreman                                           8.5 k
     replacing  rubygem-request_store.noarch 1.6.0-1.el8
Upgrading:
 candlepin                                                                    noarch                                   4.4.12-1.el8                                                   candlepin                                          72 M
 candlepin-selinux                                                            noarch                                   4.4.12-1.el8                                                   candlepin                                         579 k
 createrepo_c-libs                                                            x86_64                                   1.1.3-1.el8                                                    pulpcore                                          106 k
 foreman                                                                      noarch                                   3.11.1-1.el8                                                   foreman                                            36 M
 foreman-cli                                                                  noarch                                   3.11.1-1.el8                                                   foreman                                            34 k
 foreman-dynflow-sidekiq                                                      noarch                                   3.11.1-1.el8                                                   foreman                                            37 k
 foreman-installer                                                            noarch                                   1:3.11.1-1.el8                                                 foreman                                           1.5 M
 foreman-installer-katello                                                    noarch                                   1:3.11.1-1.el8                                                 foreman                                            93 k
 foreman-postgresql                                                           noarch                                   3.11.1-1.el8                                                   foreman                                            35 k
 foreman-proxy                                                                noarch                                   3.11.1-1.el8                                                   foreman                                           155 k
 foreman-redis                                                                noarch                                   3.11.1-1.el8                                                   foreman                                            35 k
 foreman-selinux                                                              noarch                                   3.11.1-1.el8                                                   foreman                                            57 k
 foreman-service                                                              noarch                                   3.11.1-1.el8                                                   foreman                                            37 k
 katello                                                                      noarch                                   4.13.1-1.el8                                                   katello                                            17 k
 katello-certs-tools                                                          noarch                                   2.10.0-1.el8                                                   foreman                                            69 k
 katello-common                                                               noarch                                   4.13.1-1.el8                                                   katello                                            25 k
 libcomps                                                                     x86_64                                   0.1.21-1.el8                                                   pulpcore                                           83 k
 python3-libcomps                                                             x86_64                                   0.1.21-1.el8                                                   pulpcore                                           53 k
 python3.11-aiodns                                                            noarch                                   3.0.0-7.el8                                                    pulpcore                                           18 k
 python3.11-aiofiles                                                          noarch                                   22.1.0-5.el8                                                   pulpcore                                           34 k
 python3.11-aiohttp                                                           x86_64                                   3.9.4-1.el8                                                    pulpcore                                          645 k
 python3.11-aiohttp-socks                                                     noarch                                   0.7.1-7.el8                                                    pulpcore                                           23 k
 python3.11-aiohttp-xmlrpc                                                    noarch                                   1.5.0-6.el8                                                    pulpcore                                           33 k
 python3.11-aioredis                                                          noarch                                   2.0.1-6.el8                                                    pulpcore                                          163 k
 python3.11-aiosignal                                                         noarch                                   1.3.1-5.el8                                                    pulpcore                                           18 k
 python3.11-asgiref                                                           noarch                                   3.6.0-5.el8                                                    pulpcore                                           54 k
 python3.11-async-lru                                                         noarch                                   1.0.3-5.el8                                                    pulpcore                                           17 k
 python3.11-async-timeout                                                     noarch                                   4.0.2-6.el8                                                    pulpcore                                           18 k
 python3.11-asyncio-throttle                                                  noarch                                   1.0.2-7.el8                                                    pulpcore                                           15 k
 python3.11-attrs                                                             noarch                                   21.4.0-6.el8                                                   pulpcore                                          116 k
 python3.11-backoff                                                           noarch                                   2.2.1-5.el8                                                    pulpcore                                           34 k
 python3.11-bandersnatch                                                      noarch                                   6.1.0-4.el8                                                    pulpcore                                          236 k
 python3.11-beautifulsoup4                                                    noarch                                   4.11.2-5.el8                                                   pulpcore                                          258 k
 python3.11-bindep                                                            noarch                                   2.11.0-5.el8                                                   pulpcore                                           59 k
 python3.11-bleach                                                            noarch                                   3.3.1-6.el8                                                    pulpcore                                          315 k
 python3.11-bleach-allowlist                                                  noarch                                   1.0.3-7.el8                                                    pulpcore                                           23 k
 python3.11-bracex                                                            noarch                                   2.2.1-6.el8                                                    pulpcore                                           30 k
 python3.11-brotli                                                            x86_64                                   1.0.9-6.el8                                                    pulpcore                                          314 k
 python3.11-certifi                                                           noarch                                   2022.12.7-5.el8                                                pulpcore                                           15 k
 python3.11-cffi                                                              x86_64                                   1.15.1-5.el8                                                   pulpcore                                          293 k
 python3.11-charset-normalizer                                                noarch                                   2.1.1-5.el8                                                    pulpcore                                           92 k
 python3.11-click                                                             noarch                                   8.1.3-5.el8                                                    pulpcore                                          219 k
 python3.11-click-shell                                                       noarch                                   2.1-7.el8                                                      pulpcore                                           27 k
 python3.11-colorama                                                          noarch                                   0.4.4-7.el8                                                    pulpcore                                           37 k
 python3.11-commonmark                                                        noarch                                   0.9.1-9.el8                                                    pulpcore                                          132 k
 python3.11-contextlib2                                                       noarch                                   21.6.0-7.el8                                                   pulpcore                                           30 k
 python3.11-createrepo_c                                                      x86_64                                   1.1.3-1.el8                                                    pulpcore                                           71 k
 python3.11-cryptography                                                      x86_64                                   42.0.5-1.el8                                                   pulpcore                                          1.1 M
 python3.11-dataclasses                                                       noarch                                   0.8-7.el8                                                      pulpcore                                           39 k
 python3.11-dateutil                                                          noarch                                   2.8.2-7.el8                                                    pulpcore                                          331 k
 python3.11-debian                                                            noarch                                   0.1.44-7.el8                                                   pulpcore                                          243 k
 python3.11-defusedxml                                                        noarch                                   0.7.1-7.el8                                                    pulpcore                                           55 k
 python3.11-deprecated                                                        noarch                                   1.2.13-5.el8                                                   pulpcore                                           24 k
 python3.11-diff-match-patch                                                  noarch                                   20200713-7.el8                                                 pulpcore                                           82 k
 python3.11-distro                                                            noarch                                   1.7.0-4.el8                                                    pulpcore                                           41 k
 python3.11-django-filter                                                     noarch                                   23.2-4.el8                                                     pulpcore                                          113 k
 python3.11-django-guid                                                       noarch                                   3.3.0-5.el8                                                    pulpcore                                           42 k
 python3.11-django-import-export                                              noarch                                   3.1.0-4.el8                                                    pulpcore                                          129 k
 python3.11-django-lifecycle                                                  noarch                                   1.0.0-4.el8                                                    pulpcore                                           34 k
 python3.11-django-readonly-field                                             noarch                                   1.1.2-4.el8                                                    pulpcore                                           18 k
 python3.11-djangorestframework                                               noarch                                   3.14.0-4.el8                                                   pulpcore                                          1.1 M
 python3.11-djangorestframework-queryfields                                   noarch                                   1.0.0-8.el8                                                    pulpcore                                           14 k
 python3.11-drf-access-policy                                                 noarch                                   1.3.0-4.el8                                                    pulpcore                                           32 k
 python3.11-drf-nested-routers                                                noarch                                   0.93.4-6.el8                                                   pulpcore                                           40 k
 python3.11-drf-spectacular                                                   noarch                                   0.26.5-5.el8                                                   pulpcore                                          206 k
 python3.11-dynaconf                                                          noarch                                   3.1.12-4.el8                                                   pulpcore                                          631 k
 python3.11-ecdsa                                                             noarch                                   0.18.0-5.el8                                                   pulpcore                                          296 k
 python3.11-enrich                                                            noarch                                   1.2.6-9.el8                                                    pulpcore                                           24 k
 python3.11-et-xmlfile                                                        noarch                                   1.1.0-6.el8                                                    pulpcore                                           15 k
 python3.11-filelock                                                          noarch                                   3.8.0-5.el8                                                    pulpcore                                           27 k
 python3.11-flake8                                                            noarch                                   5.0.0-3.el8                                                    pulpcore                                          139 k
 python3.11-frozenlist                                                        x86_64                                   1.3.3-5.el8                                                    pulpcore                                           46 k
 python3.11-future                                                            noarch                                   0.18.3-5.el8                                                   pulpcore                                          940 k
 python3.11-gitdb                                                             noarch                                   4.0.10-5.el8                                                   pulpcore                                          129 k
 python3.11-gitpython                                                         noarch                                   3.1.40-3.el8                                                   pulpcore                                          375 k
 python3.11-gnupg                                                             noarch                                   0.5.0-5.el8                                                    pulpcore                                           58 k
 python3.11-googleapis-common-protos                                          noarch                                   1.59.1-5.el8                                                   pulpcore                                          182 k
 python3.11-grpcio                                                            x86_64                                   1.56.0-5.el8                                                   pulpcore                                          4.2 M
 python3.11-gunicorn                                                          noarch                                   20.1.0-8.el8                                                   pulpcore                                          187 k
 python3.11-humanfriendly                                                     noarch                                   10.0-7.el8                                                     pulpcore                                          162 k
 python3.11-importlib-metadata                                                noarch                                   6.0.1-5.el8                                                    pulpcore                                           52 k
 python3.11-inflection                                                        noarch                                   0.5.1-7.el8                                                    pulpcore                                           20 k
 python3.11-iniparse                                                          noarch                                   0.4-40.el8                                                     pulpcore                                           58 k
 python3.11-jinja2                                                            noarch                                   3.1.4-1.el8                                                    pulpcore                                          321 k
 python3.11-jq                                                                x86_64                                   1.6.0-4.el8                                                    pulpcore                                          336 k
 python3.11-json_stream_rs_tokenizer                                          x86_64                                   0.4.25-4.el8                                                   pulpcore                                           27 k
 python3.11-jsonschema                                                        noarch                                   4.10.3-4.el8                                                   pulpcore                                          185 k
 python3.11-libcomps                                                          x86_64                                   0.1.21-1.el8                                                   pulpcore                                           53 k
 python3.11-lxml                                                              x86_64                                   4.9.2-5.el8                                                    pulpcore                                          1.4 M
 python3.11-markdown                                                          noarch                                   3.4.1-4.el8                                                    pulpcore                                          183 k
 python3.11-markuppy                                                          noarch                                   1.14-7.el8                                                     pulpcore                                           24 k
 python3.11-markupsafe                                                        x86_64                                   2.1.2-5.el8                                                    pulpcore                                           31 k
 python3.11-mccabe                                                            noarch                                   0.7.0-4.el8                                                    pulpcore                                           24 k
 python3.11-multidict                                                         x86_64                                   6.0.4-5.el8                                                    pulpcore                                           54 k
 python3.11-odfpy                                                             noarch                                   1.4.1-10.el8                                                   pulpcore                                          276 k
 python3.11-openpyxl                                                          noarch                                   3.1.0-5.el8                                                    pulpcore                                          545 k
 python3.11-packaging                                                         noarch                                   21.3-6.el8                                                     pulpcore                                           88 k
 python3.11-parsley                                                           noarch                                   1.3-6.el8                                                      pulpcore                                          225 k
 python3.11-pbr                                                               noarch                                   5.8.0-7.el8                                                    pulpcore                                          181 k
 python3.11-pillow                                                            x86_64                                   9.5.0-5.el8                                                    pulpcore                                          817 k
 python3.11-pkginfo                                                           noarch                                   1.9.6-4.el8                                                    pulpcore                                           72 k
 python3.11-productmd                                                         noarch                                   1.33-7.el8                                                     pulpcore                                           96 k
 python3.11-protobuf                                                          noarch                                   4.21.6-5.el8                                                   pulpcore                                          293 k
 python3.11-psycopg                                                           noarch                                   3.1.9-5.el8                                                    pulpcore                                          384 k
 python3.11-pulp-ansible                                                      noarch                                   1:0.21.6-1.el8                                                 pulpcore                                          463 k
 python3.11-pulp-cli                                                          noarch                                   0.25.6-1.el8                                                   pulpcore                                          203 k
 python3.11-pulp-container                                                    noarch                                   2.20.1-1.el8                                                   pulpcore                                          395 k
 python3.11-pulp-deb                                                          noarch                                   3.2.0-1.el8                                                    pulpcore                                          437 k
 python3.11-pulp-glue                                                         noarch                                   0.25.6-1.el8                                                   pulpcore                                          105 k
 python3.11-pulp-python                                                       noarch                                   3.11.1-1.el8                                                   pulpcore                                          175 k
 python3.11-pulp-rpm                                                          noarch                                   3.26.1-1.el8                                                   pulpcore                                          559 k
 python3.11-pulpcore                                                          noarch                                   3.49.10-1.el8                                                  pulpcore                                          1.3 M
     replacing  python3.11-pulp-certguard.noarch 1.7.1-2.el8
     replacing  python3.11-pulp-file.noarch 1.15.1-2.el8
 python3.11-pyOpenSSL                                                         noarch                                   24.1.0-1.el8                                                   pulpcore                                          120 k
 python3.11-pycares                                                           x86_64                                   4.1.2-6.el8                                                    pulpcore                                          103 k
 python3.11-pycodestyle                                                       noarch                                   2.9.1-3.el8                                                    pulpcore                                           85 k
 python3.11-pycparser                                                         noarch                                   2.21-6.el8                                                     pulpcore                                          250 k
 python3.11-pyflakes                                                          noarch                                   2.5.0-3.el8                                                    pulpcore                                          164 k
 python3.11-pygtrie                                                           noarch                                   2.5.0-5.el8                                                    pulpcore                                           45 k
 python3.11-pyjwkest                                                          noarch                                   1.4.2-9.el8                                                    pulpcore                                          111 k
 python3.11-pyjwt                                                             noarch                                   2.5.0-5.el8                                                    pulpcore                                           56 k
 python3.11-pyparsing                                                         noarch                                   3.1.1-4.el8                                                    pulpcore                                          225 k
 python3.11-pypi-simple                                                       noarch                                   0.9.0-6.el8                                                    pulpcore                                           41 k
 python3.11-pyrsistent                                                        x86_64                                   0.18.1-6.el8                                                   pulpcore                                          126 k
 python3.11-pytz                                                              noarch                                   2022.2.1-6.el8                                                 pulpcore                                          245 k
 python3.11-redis                                                             noarch                                   4.3.4-5.el8                                                    pulpcore                                          480 k
 python3.11-requests                                                          noarch                                   2.31.0-5.el8                                                   pulpcore                                          130 k
 python3.11-requirements-parser                                               noarch                                   0.2.0-7.el8                                                    pulpcore                                           22 k
 python3.11-rhsm                                                              x86_64                                   1.19.2-7.el8                                                   pulpcore                                          123 k
 python3.11-rich                                                              noarch                                   13.3.1-8.el8                                                   pulpcore                                          558 k
 python3.11-ruamel-yaml                                                       noarch                                   0.17.21-6.el8                                                  pulpcore                                          264 k
 python3.11-ruamel-yaml-clib                                                  x86_64                                   0.2.7-5.el8                                                    pulpcore                                          130 k
 python3.11-schema                                                            noarch                                   0.7.5-6.el8                                                    pulpcore                                           41 k
 python3.11-semantic-version                                                  noarch                                   2.10.0-5.el8                                                   pulpcore                                           45 k
 python3.11-six                                                               noarch                                   1.16.0-6.el8                                                   pulpcore                                           34 k
 python3.11-smmap                                                             noarch                                   5.0.0-6.el8                                                    pulpcore                                           54 k
 python3.11-socks                                                             noarch                                   2.0.3-6.el8                                                    pulpcore                                          101 k
 python3.11-soupsieve                                                         noarch                                   2.3.1-7.el8                                                    pulpcore                                           98 k
 python3.11-sqlparse                                                          noarch                                   0.5.0-1.el8                                                    pulpcore                                          104 k
 python3.11-tablib                                                            noarch                                   3.3.0-5.el8                                                    pulpcore                                          108 k
 python3.11-tenacity                                                          noarch                                   7.0.0-7.el8                                                    pulpcore                                           51 k
 python3.11-toml                                                              noarch                                   0.10.2-7.el8                                                   pulpcore                                           51 k
 python3.11-types-cryptography                                                noarch                                   3.3.23.2-5.el8                                                 pulpcore                                           28 k
 python3.11-typing-extensions                                                 noarch                                   4.7.1-5.el8                                                    pulpcore                                           73 k
 python3.11-uritemplate                                                       noarch                                   4.1.1-6.el8                                                    pulpcore                                           36 k
 python3.11-url-normalize                                                     noarch                                   1.4.3-8.el8                                                    pulpcore                                           20 k
 python3.11-urllib3                                                           noarch                                   1.26.18-3.el8                                                  pulpcore                                          264 k
 python3.11-urlman                                                            noarch                                   2.0.1-5.el8                                                    pulpcore                                           21 k
 python3.11-uuid6                                                             noarch                                   2023.5.2-5.el8                                                 pulpcore                                           18 k
 python3.11-wcmatch                                                           noarch                                   8.3-6.el8                                                      pulpcore                                          102 k
 python3.11-webencodings                                                      noarch                                   0.5.1-7.el8                                                    pulpcore                                           29 k
 python3.11-whitenoise                                                        noarch                                   6.0.0-5.el8                                                    pulpcore                                           55 k
 python3.11-wrapt                                                             x86_64                                   1.14.1-5.el8                                                   pulpcore                                           64 k
 python3.11-xlrd                                                              noarch                                   2.0.1-9.el8                                                    pulpcore                                          198 k
 python3.11-xlwt                                                              noarch                                   1.3.0-7.el8                                                    pulpcore                                          219 k
 python3.11-yarl                                                              x86_64                                   1.8.2-5.el8                                                    pulpcore                                           87 k
 python3.11-zipp                                                              noarch                                   3.4.0-8.el8                                                    pulpcore                                           19 k
 rubygem-activerecord-import                                                  noarch                                   1.6.0-1.el8                                                    katello                                            37 k
 rubygem-audited                                                              noarch                                   5.6.0-1.el8                                                    foreman                                            26 k
 rubygem-css_parser                                                           noarch                                   1.17.1-1.el8                                                   foreman                                            27 k
 rubygem-dynflow                                                              noarch                                   1.8.4-1.el8                                                    foreman                                           381 k
 rubygem-excon                                                                noarch                                   0.110.0-1.el8                                                  foreman                                            45 k
 rubygem-facter                                                               noarch                                   4.7.0-1.el8                                                    foreman                                           291 k
 rubygem-fast_gettext                                                         noarch                                   2.3.0-1.el8                                                    foreman                                            35 k
 rubygem-foreman_maintain                                                     noarch                                   1:1.6.9-1.el8                                                  foreman                                           163 k
 rubygem-foreman_remote_execution                                             noarch                                   13.1.0-1.fm3_11.el8                                            foreman-plugins                                    17 M
 rubygem-graphql-batch                                                        noarch                                   0.6.0-1.el8                                                    foreman                                            15 k
 rubygem-hammer_cli                                                           noarch                                   3.11.0-1.el8                                                   foreman                                           134 k
 rubygem-hammer_cli_foreman                                                   noarch                                   3.11.0-1.el8                                                   foreman                                           234 k
 rubygem-hammer_cli_foreman_tasks                                             noarch                                   0.0.21-1.fm3_11.el8                                            foreman-plugins                                    48 k
 rubygem-hammer_cli_katello                                                   noarch                                   1.13.0-0.2.pre.master.el8                                      katello                                           251 k
 rubygem-i18n                                                                 noarch                                   1.14.5-1.el8                                                   foreman                                            54 k
 rubygem-katello                                                              noarch                                   4.13.1-1.el8                                                   katello                                            28 M
 rubygem-ldap_fluff                                                           noarch                                   0.7.0-1.el8                                                    foreman                                            23 k
 rubygem-marcel                                                               noarch                                   1.0.4-1.el8                                                    foreman                                            47 k
 rubygem-method_source                                                        noarch                                   1.1.0-1.el8                                                    foreman                                            15 k
 rubygem-mime-types-data                                                      noarch                                   3.2024.0206-1.el8                                              foreman                                           149 k
 rubygem-net-ssh                                                              noarch                                   7.2.3-1.el8                                                    foreman                                           129 k
 rubygem-nio4r                                                                x86_64                                   2.7.3-1.el8                                                    foreman                                            58 k
 rubygem-nokogiri                                                             x86_64                                   1.15.6-1.el8                                                   foreman                                           270 k
 rubygem-pg                                                                   x86_64                                   1.5.6-1.el8                                                    foreman                                           118 k
 rubygem-public_suffix                                                        noarch                                   5.0.5-1.el8                                                    foreman                                            95 k
 rubygem-pulp_ansible_client                                                  noarch                                   0.21.3-1.el8                                                   katello                                            98 k
 rubygem-pulp_certguard_client                                                noarch                                   3.49.6-1.el8                                                   katello                                            25 k
 rubygem-pulp_container_client                                                noarch                                   2.20.0-1.el8                                                   katello                                            63 k
 rubygem-pulp_deb_client                                                      noarch                                   3.2.0-1.el8                                                    katello                                            69 k
 rubygem-pulp_file_client                                                     noarch                                   3.49.6-1.el8                                                   katello                                            48 k
 rubygem-pulp_ostree_client                                                   noarch                                   2.3.0-1.el8                                                    katello                                            47 k
 rubygem-pulp_python_client                                                   noarch                                   3.11.1-1.el8                                                   katello                                            50 k
 rubygem-pulp_rpm_client                                                      noarch                                   3.26.1-1.el8                                                   katello                                            76 k
 rubygem-pulpcore_client                                                      noarch                                   1:3.49.6-1.el8                                                 katello                                           102 k
 rubygem-puma                                                                 x86_64                                   6.4.2-2.el8                                                    foreman                                           112 k
 rubygem-rack                                                                 noarch                                   2.2.9-1.el8                                                    foreman                                           132 k
 rubygem-rails-i18n                                                           noarch                                   7.0.9-1.el8                                                    foreman                                           140 k
 rubygem-roadie-rails                                                         noarch                                   3.2.0-1.el8                                                    foreman                                            16 k
 rubygem-safemode                                                             noarch                                   1.5.0-1.el8                                                    foreman                                            24 k
 rubygem-sequel                                                               noarch                                   5.80.0-1.el8                                                   foreman                                           577 k
 rubygem-zeitwerk                                                             noarch                                   2.6.14-1.el8                                                   foreman                                            29 k
Installing dependencies:
 rubygem-spidr                                                                noarch                                   0.7.1-1.el8                                                    katello                                            29 k

Transaction Summary
==============================================================================================================================================================================================================================================
Install    2 Packages
Upgrade  197 Packages

Total download size: 189 M
Is this ok [y/N]: y
Downloading Packages:
(1/199): foreman-obsolete-packages-1.9-1.el8.noarch.rpm                                                                                                                                                        98 kB/s | 8.5 kB     00:00
(2/199): rubygem-spidr-0.7.1-1.el8.noarch.rpm                                                                                                                                                                 327 kB/s |  29 kB     00:00
(3/199): foreman-cli-3.11.1-1.el8.noarch.rpm                                                                                                                                                                  2.2 MB/s |  34 kB     00:00
(4/199): foreman-dynflow-sidekiq-3.11.1-1.el8.noarch.rpm                                                                                                                                                      2.4 MB/s |  37 kB     00:00
(5/199): foreman-installer-katello-3.11.1-1.el8.noarch.rpm                                                                                                                                                    5.0 MB/s |  93 kB     00:00
(6/199): foreman-postgresql-3.11.1-1.el8.noarch.rpm                                                                                                                                                           1.8 MB/s |  35 kB     00:00
(7/199): foreman-installer-3.11.1-1.el8.noarch.rpm                                                                                                                                                             15 MB/s | 1.5 MB     00:00
...
(197/199): python3.11-zipp-3.4.0-8.el8.noarch.rpm                                                                                                                                                             1.6 MB/s |  19 kB     00:00
(198/199): python3.11-wrapt-1.14.1-5.el8.x86_64.rpm                                                                                                                                                           564 kB/s |  64 kB     00:00
[MIRROR] rubygem-katello-4.13.1-1.el8.noarch.rpm: Curl error (28): Timeout was reached for https://yum.theforeman.org/katello/4.13/katello/el8/x86_64/rubygem-katello-4.13.1-1.el8.noarch.rpm [Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds]
(199/199): rubygem-katello-4.13.1-1.el8.noarch.rpm                                                                                                                                                            865 kB/s |  28 MB     00:33
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                         5.5 MB/s | 189 MB     00:34
Foreman 3.11                                                                                                                                                                                                  1.6 MB/s | 1.7 kB     00:00
Importing GPG key 0x1790A6E6:
 Userid     : "Foreman Automatic Signing Key (3.11) <packages@theforeman.org>"
 Fingerprint: CD41 E7A8 7E35 3C3A 26F8 91D2 4E70 8246 1790 A6E6
 From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-foreman
Is this ok [y/N]: y
Key imported successfully
Candlepin: an open source entitlement management system.                                                                                                                                                      1.6 MB/s | 1.7 kB     00:00
Importing GPG key 0x008C5E2A:
 Userid     : "Candlepin Automatic Signing Key (4.4) <packages@theforeman.org>"
 Fingerprint: B97A 252D 01CF 67E5 617C 05DC 0C0C B0CD 008C 5E2A
 From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-candlepin
Is this ok [y/N]: y
Key imported successfully
pulpcore: Fetch, Upload, Organize, and Distribute Software Packages.                                                                                                                                          3.9 kB/s | 1.7 kB     00:00
Importing GPG key 0xC375BEFA:
 Userid     : "Pulpcore Automatic Signing Key (3.49) <packages@theforeman.org>"
 Fingerprint: F330 4779 EBC3 1BA9 4037 09F9 7AA8 E392 C375 BEFA
 From       : https://yum.theforeman.org/pulpcore/3.49/GPG-RPM-KEY-pulpcore
Is this ok [y/N]: y
Key imported successfully
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                                                                                                                                      1/1
  Running scriptlet: python3.11-six-1.16.0-6.el8.noarch                                                                                                                                                                                   1/1
  Upgrading        : python3.11-six-1.16.0-6.el8.noarch                                                                                                                                                                                 1/399
  Upgrading        : python3.11-typing-extensions-4.7.1-5.el8.noarch                                                                                                                                                                    2/399
...
  Running scriptlet: foreman-3.11.1-1.el8.noarch                                                                                                                                                                                      158/399
  Upgrading        : foreman-3.11.1-1.el8.noarch                                                                                                                                                                                      158/399
  Running scriptlet: foreman-3.11.1-1.el8.noarch                                                                                                                                                                                      158/399
  Upgrading        : foreman-postgresql-3.11.1-1.el8.noarch                                                                                                                                                                           159/399
  Upgrading        : rubygem-foreman_remote_execution-13.1.0-1.fm3_11.el8.noarch                                                                                                                                                      160/399
  Upgrading        : rubygem-katello-4.13.1-1.el8.noarch                                                                                                                                                                              161/399
  Upgrading        : foreman-installer-1:3.11.1-1.el8.noarch                                                                                                                                                                          162/399
warning: /etc/foreman-installer/scenarios.d/foreman-answers.yaml created as /etc/foreman-installer/scenarios.d/foreman-answers.yaml.rpmnew

  Running scriptlet: foreman-installer-1:3.11.1-1.el8.noarch                                                                                                                                                                          162/399
  Upgrading        : foreman-installer-katello-1:3.11.1-1.el8.noarch                                                                                                                                                                  163/399
warning: /etc/foreman-installer/scenarios.d/katello-answers.yaml created as /etc/foreman-installer/scenarios.d/katello-answers.yaml.rpmnew

  Running scriptlet: foreman-installer-katello-1:3.11.1-1.el8.noarch                                                                                                                                                                  163/399
  Upgrading        : katello-4.13.1-1.el8.noarch                                                                                                                                                                                      164/399
  Upgrading        : foreman-dynflow-sidekiq-3.11.1-1.el8.noarch                                                                                                                                                                      165/399
  Running scriptlet: foreman-dynflow-sidekiq-3.11.1-1.el8.noarch                                                                                                                                                                      165/399
...
  Verifying        : python3.11-yarl-1.8.2-4.el8.x86_64                                                                                                                                                                               397/399
  Verifying        : python3.11-zipp-3.4.0-8.el8.noarch                                                                                                                                                                               398/399
  Verifying        : python3.11-zipp-3.4.0-7.el8.noarch                                                                                                                                                                               399/399
Installed products updated.

Upgraded:
  candlepin-4.4.12-1.el8.noarch                                  candlepin-selinux-4.4.12-1.el8.noarch                            createrepo_c-libs-1.1.3-1.el8.x86_64                 foreman-3.11.1-1.el8.noarch
...
  rubygem-zeitwerk-2.6.14-1.el8.noarch
Installed:
  foreman-obsolete-packages-1.9-1.el8.noarch                                                                                 rubygem-spidr-0.7.1-1.el8.noarch

Complete!

To be continued…

62

And finally!

  1. Run installer, it will get through some time but if you look in foreman-tail after 1250 appears you will start to see the candlepin connection errors
# foreman-installer 
2024-07-30 19:28:50 [NOTICE] [root] Loading installer configuration. This will take some time.
2024-07-30 19:28:53 [NOTICE] [root] Running installer with log based terminal output at level NOTICE.
2024-07-30 19:28:53 [NOTICE] [root] Use -l to set the terminal output log level to ERROR, WARN, NOTICE, INFO, or DEBUG. See --full-help for definitions.
2024-07-30 19:28:56 [NOTICE] [pre] Performing upgrade of PostgreSQL to 13
2024-07-30 19:28:56 [NOTICE] [pre] Upgrading PostgreSQL packages
2024-07-30 19:29:22 [NOTICE] [pre] Migrating PostgreSQL data
2024-07-30 19:29:31 [NOTICE] [pre] Analyzing the new PostgreSQL cluster
2024-07-30 19:29:33 [NOTICE] [pre] Upgrade to PostgreSQL 13 completed
2024-07-30 19:29:35 [NOTICE] [configure] Starting system configuration.
2024-07-30 19:29:50 [NOTICE] [configure] 250 configuration steps out of 1351 steps complete.
2024-07-30 19:29:50 [NOTICE] [configure] 500 configuration steps out of 1353 steps complete.
2024-07-30 19:29:53 [NOTICE] [configure] 750 configuration steps out of 1358 steps complete.
2024-07-30 19:29:54 [NOTICE] [configure] 1000 configuration steps out of 1361 steps complete.
2024-07-30 19:31:39 [NOTICE] [configure] 1250 configuration steps out of 1363 steps complete.

2024-07-30T19:33:40 [E|app|eed0b027] Error occurred while starting Katello::CandlepinEventListener
2024-07-30T19:33:40 [E|app|eed0b027] Connection refused - connect(2) for "localhost" port 61613
2024-07-30T19:33:40 [E|app|eed0b027] /usr/share/gems/gems/stomp-1.4.10/lib/connection/netio.rb:461:in `initialize'
 eed0b027 | /usr/share/gems/gems/stomp-1.4.10/lib/connection/netio.rb:461:in `open'
 eed0b027 | /usr/share/gems/gems/stomp-1.4.10/lib/connection/netio.rb:461:in `block in open_ssl_socket'
 eed0b027 | /usr/share/ruby/timeout.rb:78:in `timeout'
 eed0b027 | /usr/share/gems/gems/stomp-1.4.10/lib/connection/netio.rb:460:in `open_ssl_socket'
 eed0b027 | /usr/share/gems/gems/stomp-1.4.10/lib/connection/netio.rb:520:in `open_socket'
 eed0b027 | /usr/share/gems/gems/stomp-1.4.10/lib/connection/utils.rb:116:in `block in socket'
 eed0b027 | /usr/share/gems/gems/stomp-1.4.10/lib/connection/utils.rb:109:in `synchronize'
 eed0b027 | /usr/share/gems/gems/stomp-1.4.10/lib/connection/utils.rb:109:in `socket'
 eed0b027 | /usr/share/gems/gems/stomp-1.4.10/lib/stomp/connection.rb:173:in `initialize'
 eed0b027 | /usr/share/gems/gems/stomp-1.4.10/lib/stomp/client.rb:134:in `new'
 eed0b027 | /usr/share/gems/gems/stomp-1.4.10/lib/stomp/client.rb:134:in `create_connection'
 eed0b027 | /usr/share/gems/gems/stomp-1.4.10/lib/stomp/client.rb:101:in `block in initialize'
 eed0b027 | /usr/share/ruby/timeout.rb:95:in `block in timeout'
 eed0b027 | /usr/share/ruby/timeout.rb:33:in `block in catch'
 eed0b027 | /usr/share/ruby/timeout.rb:33:in `catch'
 eed0b027 | /usr/share/ruby/timeout.rb:33:in `catch'
 eed0b027 | /usr/share/ruby/timeout.rb:110:in `timeout'
 eed0b027 | /usr/share/gems/gems/stomp-1.4.10/lib/stomp/client.rb:99:in `initialize'
 eed0b027 | /usr/share/gems/gems/katello-4.13.1/app/lib/katello/messaging/stomp_connection.rb:69:in `new'
 eed0b027 | /usr/share/gems/gems/katello-4.13.1/app/lib/katello/messaging/stomp_connection.rb:69:in `client'
 eed0b027 | /usr/share/gems/gems/katello-4.13.1/app/lib/katello/messaging/stomp_connection.rb:43:in `subscribe'
 eed0b027 | /usr/share/gems/gems/katello-4.13.1/app/services/katello/candlepin_event_listener.rb:34:in `run'
 eed0b027 | /usr/share/gems/gems/katello-4.13.1/app/lib/katello/event_daemon/monitor.rb:40:in `block in check_services'
 eed0b027 | /usr/share/gems/gems/katello-4.13.1/app/lib/katello/event_daemon/monitor.rb:30:in `each'
 eed0b027 | /usr/share/gems/gems/katello-4.13.1/app/lib/katello/event_daemon/monitor.rb:30:in `check_services'
 eed0b027 | /usr/share/gems/gems/katello-4.13.1/app/lib/katello/event_daemon/monitor.rb:16:in `block (2 levels) in start'
 eed0b027 | /usr/share/gems/gems/activesupport-6.1.7.8/lib/active_support/execution_wrapper.rb:87:in `wrap'
 eed0b027 | /usr/share/gems/gems/katello-4.13.1/app/lib/katello/event_daemon/monitor.rb:15:in `block in start'
 eed0b027 | /usr/share/gems/gems/katello-4.13.1/app/lib/katello/event_daemon/monitor.rb:14:in `loop'
 eed0b027 | /usr/share/gems/gems/katello-4.13.1/app/lib/katello/event_daemon/monitor.rb:14:in `start'
 eed0b027 | /usr/share/gems/gems/katello-4.13.1/app/lib/katello/event_daemon/runner.rb:84:in `block (2 levels) in start_monitor_thread'
 eed0b027 | /usr/share/gems/gems/activesupport-6.1.7.8/lib/active_support/execution_wrapper.rb:91:in `wrap'
 eed0b027 | /usr/share/gems/gems/katello-4.13.1/app/lib/katello/event_daemon/runner.rb:83:in `block in start_monitor_thread'
 eed0b027 | /usr/share/gems/gems/logging-2.3.1/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'

Also… sorry for the flood of text :upside_down_face:

1 Like
63

I believe @Ceiu now has an internal reproducer and is working on discovering the cause of the issue. But thanks even so for going through this! :slight_smile:

64

Oh! That’s good to hear :slight_smile:
And sure, used the day off pretty work well ^^

65

Those of you on systems where the halting behavior is observed, try running the following command a few times while Candlepin is hung and let me know what the output is:

time cat /dev/random | head -c 32

66

Interesting thing, has been a long time since I had rng issues.

1st
eal	0m14.792s
user	0m0.001s
sys	0m0.004s

2nd
real	3m40.438s
user	0m0.003s
sys	0m0.001s

3rd
real	4m2.567s
user	0m0.000s
sys	0m0.004s

It keeps being slow now.
This is a VM on ESXi 8.0.2, rngd is not running on that system, can install/start that if this might help.

67

Okay yes, after installing rng-tools and starting rngd it’s working with Managed setting now :+1:

3 Likes
68

Nope, that’s not necessary. I’m pretty confident I’ve discovered the root cause here and it’s a bit of an adventure.

The immediate workaround for those of you hitting this will be to use a utility daemon like haveged (available from the EPEL repos) which populate the system with entropy for /dev/random.

We’ll be putting together a build that tries to work around this soon™ so that’s not necessary.

For those interested in the underlying details:
As established above, Candlepin uses Liquibase for managing its db schema. As part of the various operations it does while comparing the existing schema to the changesets it has, it occasionally generates random strings. It does so by calling RandomStringUtil.random(...) [1] from the Apache Commons’ lang3 library.

While this has generally been fine, recently-ish (v3.15+) [2], the folks managing the lang3 library updated the RandomStringUtil to use the blocking SecureRandom implementation for generating these strings. This, unfortunately is hard-coded to pull from /dev/random under the hood.

To Liquibase’s credit, their library depends on v3.14 – before this change to the RandomStringUtil was made. However, Candlepin uses v3.15, and due to the dependency resolution magic and packaging, means Liquibase is also forced to use it, leading to this issue. It’s likely their devs aren’t aware of this yet, and/or will be addressing it in an upcoming build.

Our short-term fix is going to be to back up the dependency on Apache Commons lang3 to v3.14 which rolls back the reliance on SecureRandom in this path. Unfortunately because the issue is being hit multiple libraries away from our code in a way that isn’t configurable at all, this is our only immediate recourse; and I’d like to not incur a system-level dependency of some kind of rng tooling to workaround it.

So that’s where we’re at. My apologies to those who burned their time dealing with this, and I appreciate all the logs and details provided. It really helped narrow down the root cause.

[1] liquibase/liquibase-standard/src/main/java/liquibase/util/StringUtil.java at master · liquibase/liquibase · GitHub

[2] commons-lang/src/main/java/org/apache/commons/lang3/RandomStringUtils.java at rel/commons-lang-3.15.0 · apache/commons-lang · GitHub

3 Likes
69

Well that’s one kind of an issue unfolding…
Thank you for the full explanation as well!
I was not expecting it to be something so hidden. Awesome that you found it!

I only remember fixing entropy issues in the past with starting rngd, read that in one of the RHEL knowledgebase articles a long time ago.

Also out of curiosity I noticed that on the system itself there is apache-commons-lang3.noarch 3.7-3.module+el8.6.0+843+5a13dac3 installed, while EL 9 has 3.12 then.

70

Candlepin uses its bundled dependencies wherever possible in an attempt to avoid issues like the one we’re hitting now. Any version of the commons-lang library already present on the system shouldn’t interfere so long as it doesn’t find its way into the classpath Tomcat is using to load stuff.

1 Like
71

Hi, candlepin-4.4.13-1 is now available, which contains the fix.

1 Like
72

Can confirm, works!
Thank you very much for the whole troubleshooting and fast fixing :raised_hands:t2:

1 Like
73

Did you kept the rngd service up while upgrading ?

I tried the update on an instance that was a little bit older and still on candelpin 4.4.10 and got stuck the same way until I installed rng-tools and run the service as explained here: Status code: 403 for Error: Failed to download metadata for repo after upgrade to 3.11

Anyway, I can confirm that the workaround with rng-tools is fully working on 2 different instances with the same issue but not the same databases contents (they are not linked, one is my personal instance at home while the other is a production instance at work).

Best Regards,
Nicolas.

1 Like
74

That‘s a good point was too fast there…
After disabling rngd the problem is back :grimacing:

75

Seems Candlepin had some issues with the build and included the problematic lang3 unintentionally. Stay tuned…

2 Likes
76

Thank you all for your work on this. I too was impacted today post-upgrade by extremely slow response and the same errors in the log when communications to Candlepin were attempted. I was able to rollback the candlepin package to candlepin-4.4.10-1, rerun the foreman-installer, and performance is restored with all other upgrades still in place. I’ll stay tuned for the outcome of @jeremylenz updates.

1 Like
77

Okay folks… Candlepin 4.4.14-1 is on the way, which should (hopefully) have lang3 v3.14 and not the problematic 3.15. Let us know if it works…

2 Likes
78

Test was successful. Running “hammer status” returns quickly with no system issues reported. Update involved the upgrade of the following two packages:

candlepin-4.4.14-1.el8.noarch
candlepin-selinux-4.4.14-1.el8.noarch

Current running system version:
Foreman: 3.11.1

Current Plugins:
Katello: 4.13.1
Foreman Remote Execution: 13.1.0
Foreman Puppet: 6.3.0
Foreman Ansible: 14.0.0
Foreman Tasks: 9.1.1

Well done. The system is performing well and seems stable. I’ll continue to monitor all functions as they are performed, but initial testing shows success. Thank you, @jeremylenz !

3 Likes
79

Just upgraded from 3.10 to 3.11.1 and everything is looking good. I have also built a RHEL server and no issues so far. :slight_smile:

3 Likes
80

As the others already noted, yes it worked now!
Thank you once again :raised_hands:

1 Like