Bug: Host group settings not saved (Foreman 2.2/Katello 3.17)

Support
, ,
1

Problem:
After upgrading to Foreman 2.2 I’ve noticed that not all Host group settings are saved. On the ‘Host Group’ tab of any group the following settings are not saved:

  • Lifecycle environment
  • Content View
  • Content Source

As a side-effect, the Ansible modules also run into issues:

failed: [foreman.example.com] (item={'name': 'CentOS7-Base', 'arch': ['x86_64'], 'network': 'Ontwikkel', 'root_pass': 'awesome-password', 'dns_zone': 'example.com', 'smart_proxy': 'foreman.example.com', 'ptable': ['Kickstart default first disk only'], 'lifecycle': 'Ontwikkel', 'pxe_loader': 'PXELinux BIOS', 'os': 'CentOS 7', 'kickstart': True, 'kickstart_repo': 'CentOS7-Base', 'parameters': [{'name': 'kt_activation_keys', 'value': 'CentOS7-Base-Ontwikkel'}, {'name': 'autopart_options', 'value': '--nohome'}, {'name': 'remote_execution_create_user', 'value': 'true', 'parameter_type': 'boolean'}, {'name': 'remote_execution_ssh_user', 'value': 'ansible'}, {'name': 'remote_execution_effective_user_method', 'value': 'sudo'}, {'name': 'remote_execution_ssh_keys', 'value': 'xxxx'}, {'name': 'enable-epel', 'value': 'false', 'parameter_type': 'boolean'}]}) => changed=false 
  ansible_loop_var: hostgroup
  error:
    errors:
      content_facet.base:
      - |-
        The selected kickstart repository is not part of the assigned content view, lifecycle environment,
                          content source, operating system, and architecture
    full_messages:
    - |-
      Base The selected kickstart repository is not part of the assigned content view, lifecycle environment,
                        content source, operating system, and architecture
    id: 1
  hostgroup:
    arch:
    - x86_64
    dns_zone: example.com
    kickstart: true
    kickstart_repo: CentOS7-Base
    lifecycle: Ontwikkel
    name: CentOS7-Base
    network: Ontwikkel
    os: CentOS 7
    parameters:
    - name: kt_activation_keys
      value: CentOS7-Base-Ontwikkel
    - name: autopart_options
      value: --nohome
    - name: remote_execution_create_user
      parameter_type: boolean
      value: 'true'
    - name: remote_execution_ssh_user
      value: ansible
    - name: remote_execution_effective_user_method
      value: sudo
    - name: remote_execution_ssh_keys
      value: xxxxxxxx
    - name: enable-epel
      parameter_type: boolean
      value: 'false'
    ptable:
    - Kickstart default first disk only
    pxe_loader: PXELinux BIOS
    root_pass: awesome-password
    smart_proxy: foreman.example.com
  msg: 'Error while performing update on hostgroups: 422 Client Error: Unprocessable Entity'

Expected outcome: Stuff still works as it did in 2.1 :slight_smile:

Foreman and Proxy versions:
Foreman 2.2
Katello 3.17

Foreman and Proxy plugin versions:


Installed Packages

    candlepin-3.1.22-1.el7.noarch
    candlepin-selinux-3.1.22-1.el7.noarch
    foreman-2.2.1-1.el7.noarch
    foreman-cli-2.2.1-1.el7.noarch
    foreman-debug-2.2.1-1.el7.noarch
    foreman-dynflow-sidekiq-2.2.1-1.el7.noarch
    foreman-installer-2.2.1-1.el7.noarch
    foreman-installer-katello-2.2.1-1.el7.noarch
    foreman-postgresql-2.2.1-1.el7.noarch
    foreman-proxy-2.2.1-1.el7.noarch
    foreman-release-2.2.1-1.el7.noarch
    foreman-selinux-2.2.1-1.el7.noarch
    foreman-service-2.2.1-1.el7.noarch
    foreman.lbhr.htm.lan-apache-1.0-1.noarch
    foreman.lbhr.htm.lan-foreman-client-1.0-1.noarch
    foreman.lbhr.htm.lan-foreman-proxy-1.0-1.noarch
    foreman.lbhr.htm.lan-foreman-proxy-client-1.0-1.noarch
    foreman.lbhr.htm.lan-puppet-client-1.0-1.noarch
    foreman.lbhr.htm.lan-qpid-broker-1.0-1.noarch
    foreman.lbhr.htm.lan-qpid-client-cert-1.0-1.noarch
    foreman.lbhr.htm.lan-qpid-router-client-1.0-1.noarch
    foreman.lbhr.htm.lan-qpid-router-server-1.0-1.noarch
    foreman.lbhr.htm.lan-tomcat-1.0-1.noarch
    katello-3.17.0-1.el7.noarch
    katello-certs-tools-2.7.1-2.el7.noarch
    katello-client-bootstrap-1.7.5-1.el7.noarch
    katello-common-3.17.0-1.el7.noarch
    katello-debug-3.17.0-1.el7.noarch
    katello-default-ca-1.0-1.noarch
    katello-repos-3.17.0-1.el7.noarch
    katello-selinux-3.4.0-1.el7.noarch
    katello-server-ca-1.0-1.noarch
    pulp-client-1.0-1.noarch
    pulp-deb-plugins-1.10.2-1.el7.noarch
    pulp-docker-plugins-3.2.8-1.el7.noarch
    pulp-katello-1.0.3-1.el7.noarch
    pulp-puppet-plugins-2.21.4-1.el7.noarch
    pulp-puppet-tools-2.21.4-1.el7.noarch
    pulp-rpm-plugins-2.21.4-2.el7.noarch
    pulp-selinux-2.21.4-2.el7.noarch
    pulp-server-2.21.4-2.el7.noarch
    pulpcore-selinux-1.0.0-2.el7.x86_64
    python-gofer-qpid-2.12.5-3.el7.noarch
    python-pulp-common-2.21.4-2.el7.noarch
    python-pulp-deb-common-1.10.2-1.el7.noarch
    python-pulp-docker-common-3.2.8-1.el7.noarch
    python-pulp-oid_validation-2.21.4-2.el7.noarch
    python-pulp-puppet-common-2.21.4-1.el7.noarch
    python-pulp-repoauth-2.21.4-2.el7.noarch
    python-pulp-rpm-common-2.21.4-2.el7.noarch
    python-pulp-streamer-2.21.4-2.el7.noarch
    python2-qpid-1.37.0-4.el7.noarch
    python2-qpid-proton-0.32.0-2.el7.x86_64
    python2-qpid-qmf-1.39.0-1.el7.x86_64
    python3-pulp-2to3-migration-0.5.0-1.el7.noarch
    python3-pulp-certguard-1.0.2-1.el7.noarch
    python3-pulp-container-2.0.1-1.el7.noarch
    python3-pulp-file-1.2.0-1.el7.noarch
    python3-pulp-rpm-3.6.2-1.el7.noarch
    python3-pulpcore-3.6.3-2.el7.noarch
    qpid-cpp-client-1.39.0-1.el7.x86_64
    qpid-cpp-client-devel-1.39.0-1.el7.x86_64
    qpid-cpp-server-1.39.0-1.el7.x86_64
    qpid-cpp-server-linearstore-1.39.0-1.el7.x86_64
    qpid-dispatch-router-1.14.0-1.el7.x86_64
    qpid-proton-c-0.32.0-2.el7.x86_64
    qpid-qmf-1.39.0-1.el7.x86_64
    qpid-tools-1.39.0-1.el7.noarch
    rubygem-foreman_maintain-0.6.13-1.el7.noarch
    tfm-rubygem-actioncable-6.0.3.1-1.el7.noarch
    tfm-rubygem-actionmailbox-6.0.3.1-1.el7.noarch
    tfm-rubygem-actionmailer-6.0.3.1-1.el7.noarch
    tfm-rubygem-actionpack-6.0.3.1-1.el7.noarch
    tfm-rubygem-actiontext-6.0.3.1-1.el7.noarch
    tfm-rubygem-actionview-6.0.3.1-1.el7.noarch
    tfm-rubygem-activejob-6.0.3.1-1.el7.noarch
    tfm-rubygem-activemodel-6.0.3.1-1.el7.noarch
    tfm-rubygem-activerecord-6.0.3.1-1.el7.noarch
    tfm-rubygem-activerecord-import-1.0.0-2.el7.noarch
    tfm-rubygem-activerecord-session_store-1.1.1-4.el7.noarch
    tfm-rubygem-activestorage-6.0.3.1-1.el7.noarch
    tfm-rubygem-activesupport-6.0.3.1-1.el7.noarch
    tfm-rubygem-addressable-2.6.0-2.el7.noarch
    tfm-rubygem-algebrick-0.7.3-7.el7.noarch
    tfm-rubygem-amazing_print-1.1.0-1.el7.noarch
    tfm-rubygem-ancestry-3.0.7-1.el7.noarch
    tfm-rubygem-anemone-0.7.2-17.el7.noarch
    tfm-rubygem-angular-rails-templates-1.1.0-1.el7.noarch
    tfm-rubygem-ansi-1.5.0-2.el7.noarch
    tfm-rubygem-apipie-bindings-0.4.0-1.el7.noarch
    tfm-rubygem-apipie-dsl-2.2.9-1.el7.noarch
    tfm-rubygem-apipie-params-0.0.5-4.el7.noarch
    tfm-rubygem-apipie-rails-0.5.17-3.el7.noarch
    tfm-rubygem-arel-9.0.0-4.el7.noarch
    tfm-rubygem-audited-4.9.0-3.el7.noarch
    tfm-rubygem-awesome_print-1.8.0-5.el7.noarch
    tfm-rubygem-bcrypt-3.1.12-3.el7.x86_64
    tfm-rubygem-builder-3.2.4-1.el7.noarch
    tfm-rubygem-bundler_ext-0.4.1-5.el7.noarch
    tfm-rubygem-clamp-1.1.2-6.el7.noarch
    tfm-rubygem-concurrent-ruby-1.1.6-2.el7.noarch
    tfm-rubygem-concurrent-ruby-edge-0.6.0-2.fm2_1.el7.noarch
    tfm-rubygem-connection_pool-2.2.2-2.el7.noarch
    tfm-rubygem-crass-1.0.6-1.el7.noarch
    tfm-rubygem-css_parser-1.4.7-4.el7.noarch
    tfm-rubygem-daemons-1.2.3-6.el7.noarch
    tfm-rubygem-deacon-1.0.0-4.el7.noarch
    tfm-rubygem-deep_cloneable-3.0.0-3.el7.noarch
    tfm-rubygem-deface-1.5.3-2.el7.noarch
    tfm-rubygem-domain_name-0.5.20160310-4.el7.noarch
    tfm-rubygem-dynflow-1.4.7-1.fm2_2.el7.noarch
    tfm-rubygem-erubi-1.9.0-1.el7.noarch
    tfm-rubygem-ethon-0.12.0-1.el7.noarch
    tfm-rubygem-excon-0.76.0-1.el7.noarch
    tfm-rubygem-facter-2.4.0-7.el7.x86_64
    tfm-rubygem-faraday-0.15.4-2.el7.noarch
    tfm-rubygem-fast_gettext-1.4.1-4.el7.noarch
    tfm-rubygem-ffi-1.12.2-1.el7.x86_64
    tfm-rubygem-fog-core-2.1.0-3.el7.noarch
    tfm-rubygem-foreman-tasks-3.0.1-1.fm2_2.el7.noarch
    tfm-rubygem-foreman-tasks-core-0.3.4-1.fm2_1.el7.noarch
    tfm-rubygem-foreman_discovery-16.2.0-1.fm2_2.el7.noarch
    tfm-rubygem-foreman_remote_execution-4.1.0-1.fm2_2.el7.noarch
    tfm-rubygem-foreman_remote_execution_core-1.3.1-1.el7.noarch
    tfm-rubygem-formatador-0.2.1-12.el7.noarch
    tfm-rubygem-friendly_id-5.3.0-1.el7.noarch
    tfm-rubygem-fx-0.5.0-1.el7.noarch
    tfm-rubygem-get_process_mem-0.2.1-4.el7.noarch
    tfm-rubygem-gettext_i18n_rails-1.8.0-2.el7.noarch
    tfm-rubygem-gitlab-sidekiq-fetcher-0.6.0-1.el7.noarch
    tfm-rubygem-globalid-0.4.2-1.el7.noarch
    tfm-rubygem-graphql-1.8.14-2.el7.noarch
    tfm-rubygem-graphql-batch-0.3.10-2.el7.noarch
    tfm-rubygem-gssapi-1.2.0-7.el7.noarch
    tfm-rubygem-hammer_cli-2.2.1-1.el7.noarch
    tfm-rubygem-hammer_cli_foreman-2.2.0-1.el7.noarch
    tfm-rubygem-hammer_cli_foreman_bootdisk-0.3.0-1.el7.noarch
    tfm-rubygem-hammer_cli_foreman_docker-0.0.7-1.el7.noarch
    tfm-rubygem-hammer_cli_foreman_tasks-0.0.15-1.fm2_2.el7.noarch
    tfm-rubygem-hammer_cli_katello-0.23.2-1.el7.noarch
    tfm-rubygem-hashie-3.6.0-2.el7.noarch
    tfm-rubygem-highline-1.7.8-5.el7.noarch
    tfm-rubygem-http-cookie-1.0.2-4.el7.noarch
    tfm-rubygem-i18n-1.8.2-1.el7.noarch
    tfm-rubygem-jwt-2.2.1-2.el7.noarch
    tfm-rubygem-kafo-5.0.1-1.el7.noarch
    tfm-rubygem-kafo_parsers-1.1.0-3.el7.noarch
    tfm-rubygem-kafo_wizards-0.0.1-4.el7.noarch
    tfm-rubygem-katello-3.17.0-1.el7.noarch
    tfm-rubygem-ldap_fluff-0.4.7-5.el7.noarch
    tfm-rubygem-little-plugger-1.1.4-2.el7.noarch
    tfm-rubygem-locale-2.0.9-14.el7.noarch
    tfm-rubygem-logging-2.2.2-5.el7.noarch
    tfm-rubygem-loofah-2.4.0-1.el7.noarch
    tfm-rubygem-mail-2.7.1-1.el7.noarch
    tfm-rubygem-marcel-0.3.3-1.el7.noarch
    tfm-rubygem-method_source-0.9.2-1.el7.noarch
    tfm-rubygem-mime-types-3.2.2-4.el7.noarch
    tfm-rubygem-mime-types-data-3.2018.0812-4.el7.noarch
    tfm-rubygem-mimemagic-0.3.5-1.el7.noarch
    tfm-rubygem-mini_mime-1.0.2-1.el7.noarch
    tfm-rubygem-mini_portile2-2.4.0-1.el7.noarch
    tfm-rubygem-multi_json-1.14.1-2.el7.noarch
    tfm-rubygem-multipart-post-2.0.0-2.el7.noarch
    tfm-rubygem-mustermann-1.0.2-4.el7.noarch
    tfm-rubygem-net-ldap-0.16.1-2.el7.noarch
    tfm-rubygem-net-ping-2.0.1-4.el7.noarch
    tfm-rubygem-net-scp-1.2.1-4.el7.noarch
    tfm-rubygem-net-ssh-4.2.0-2.el7.noarch
    tfm-rubygem-netrc-0.11.0-5.el7.noarch
    tfm-rubygem-nio4r-2.5.2-1.el7.x86_64
    tfm-rubygem-nokogiri-1.10.9-1.el7.x86_64
    tfm-rubygem-oauth-0.5.4-4.el7.noarch
    tfm-rubygem-parse-cron-0.1.4-4.fm2_1.el7.noarch
    tfm-rubygem-passenger-4.0.18-10.13.el7.x86_64
    tfm-rubygem-passenger-native-4.0.18-10.13.el7.x86_64
    tfm-rubygem-passenger-native-libs-4.0.18-10.13.el7.x86_64
    tfm-rubygem-pg-1.1.4-3.el7.x86_64
    tfm-rubygem-polyglot-0.3.5-2.el7.noarch
    tfm-rubygem-powerbar-2.0.1-2.el7.noarch
    tfm-rubygem-promise.rb-0.7.4-2.el7.noarch
    tfm-rubygem-public_suffix-3.0.3-2.el7.noarch
    tfm-rubygem-pulp_2to3_migration_client-0.5.0-1.el7.noarch
    tfm-rubygem-pulp_ansible_client-0.3.0-1.el7.noarch
    tfm-rubygem-pulp_certguard_client-1.0.2-1.el7.noarch
    tfm-rubygem-pulp_container_client-2.0.0-1.el7.noarch
    tfm-rubygem-pulp_file_client-1.2.0-1.el7.noarch
    tfm-rubygem-pulp_rpm_client-3.6.2-1.el7.noarch
    tfm-rubygem-pulpcore_client-3.6.0-1.el7.noarch
    tfm-rubygem-puma-4.3.3-4.el7.x86_64
    tfm-rubygem-qpid_messaging-1.38.0-1.el7.x86_64
    tfm-rubygem-rabl-0.14.3-1.el7.noarch
    tfm-rubygem-rack-2.2.3-1.el7.noarch
    tfm-rubygem-rack-cors-1.0.2-2.el7.noarch
    tfm-rubygem-rack-jsonp-1.3.1-9.el7.noarch
    tfm-rubygem-rack-protection-2.0.3-4.el7.noarch
    tfm-rubygem-rack-test-1.1.0-4.el7.noarch
    tfm-rubygem-rails-6.0.3.1-1.el7.noarch
    tfm-rubygem-rails-dom-testing-2.0.3-6.el7.noarch
    tfm-rubygem-rails-html-sanitizer-1.3.0-1.el7.noarch
    tfm-rubygem-rails-i18n-6.0.0-2.el7.noarch
    tfm-rubygem-railties-6.0.3.1-1.el7.noarch
    tfm-rubygem-rainbow-2.2.1-3.el7.noarch
    tfm-rubygem-rake-compiler-1.0.7-3.el7.noarch
    tfm-rubygem-rb-inotify-0.9.7-5.el7.noarch
    tfm-rubygem-record_tag_helper-1.0.1-3.el7.noarch
    tfm-rubygem-redis-4.1.2-2.el7.noarch
    tfm-rubygem-responders-3.0.0-3.el7.noarch
    tfm-rubygem-rest-client-2.0.2-3.el7.noarch
    tfm-rubygem-rkerberos-0.1.5-18.el7.x86_64
    tfm-rubygem-roadie-3.4.0-3.el7.noarch
    tfm-rubygem-roadie-rails-2.1.1-2.el7.noarch
    tfm-rubygem-robotex-1.0.0-21.el7.noarch
    tfm-rubygem-rsec-0.4.3-4.el7.noarch
    tfm-rubygem-ruby-libvirt-0.7.1-1.el7.x86_64
    tfm-rubygem-ruby2ruby-2.4.2-3.el7.noarch
    tfm-rubygem-ruby_parser-3.10.1-3.el7.noarch
    tfm-rubygem-rubyipmi-0.10.0-6.el7.noarch
    tfm-rubygem-runcible-2.13.1-1.el7.noarch
    tfm-rubygem-safemode-1.3.5-3.el7.noarch
    tfm-rubygem-scoped_search-4.1.9-1.el7.noarch
    tfm-rubygem-secure_headers-6.3.0-2.el7.noarch
    tfm-rubygem-sequel-5.7.1-3.el7.noarch
    tfm-rubygem-sexp_processor-4.10.0-6.el7.noarch
    tfm-rubygem-sidekiq-5.2.7-3.el7.noarch
    tfm-rubygem-sinatra-2.0.3-4.el7.noarch
    tfm-rubygem-smart_proxy_pulp-2.1.0-3.fm2_2.el7.noarch
    tfm-rubygem-sprockets-4.0.2-1.el7.noarch
    tfm-rubygem-sprockets-rails-3.2.1-6.el7.noarch
    tfm-rubygem-sshkey-1.9.0-4.el7.noarch
    tfm-rubygem-statsd-instrument-2.1.4-3.el7.noarch
    tfm-rubygem-stomp-1.4.9-1.el7.noarch
    tfm-rubygem-thor-1.0.1-2.el7.noarch
    tfm-rubygem-thread_safe-0.3.6-5.el7.noarch
    tfm-rubygem-tilt-2.0.8-4.el7.noarch
    tfm-rubygem-typhoeus-1.3.1-1.el7.noarch
    tfm-rubygem-tzinfo-1.2.6-1.el7.noarch
    tfm-rubygem-unf-0.1.3-8.el7.noarch
    tfm-rubygem-unf_ext-0.0.7.2-3.el7.x86_64
    tfm-rubygem-unicode-0.4.4.4-3.el7.x86_64
    tfm-rubygem-unicode-display_width-1.0.5-4.el7.noarch
    tfm-rubygem-validates_lengths_from_database-0.5.0-7.el7.noarch
    tfm-rubygem-webpack-rails-0.9.8-5.el7.noarch
    tfm-rubygem-websocket-driver-0.7.1-1.el7.x86_64
    tfm-rubygem-websocket-extensions-0.1.5-1.el7.noarch
    tfm-rubygem-will_paginate-3.1.7-3.el7.noarch
    tfm-rubygem-x-editable-rails-1.5.5-5.el7.noarch
    tfm-rubygem-xmlrpc-0.3.0-2.el7.noarch
    tfm-rubygem-zeitwerk-2.2.2-1.el7.noarch
    tfm-runtime-6.1-3.el7.x86_64

Distribution and version:
CentOS 7
Other relevant data:

2

This also seems to persist in Foreman 2.3/Katello 3.18RC1. Editing a group via the WebUI doesn’t save and the Ansible module reports the same error.

I also ran this in Foreman 2.1/Katello 3.16 and it works like a charm there :slight_smile:

3

Hey,

thanks for noticing this! I hope someone from @katello can have a look and say whether this is intentional or a bug.

In the meantime, does that also happen when you create a host with the same parameters directly?

Evgeni

4

As I’m working in a lab deploying a lot of times, I re-did this on a different VM:

  1. Installed with Foreman 2.1 using Forklift (so not upgraded from 2.0) and configured with Ansible
  2. Upgraded to 2.2
    a. Installed package updates (including Foreman)
    b. Reboot (wasn’t intended, but did anyway :slight_smile: )
    c. Ran installer
    d. Reboot
  3. All Host groups are now missing the settings as mentioned earlier and trying to set new ones don’t ‘stick’
  4. Created a single Host manually, setting do ‘stick’.

So I feel it’s something that broke in Katello 3.17 (and wasn’t fixed in 3.18 so far)

I have cloned the VM I was testing with earlier, so I can do some more digging (if someone can tell me which logs they need, I’d be happy to supply them)

As a final test, I’ll make another copy and install 2.2 out of the box, see if that changes anything

5

So here’s something strange:

I installed a fresh VM from the CentOS7 ISO and then installed Foreman 2.2 on it like I usually do.

  1. I can create a new hostgroup via the WebUI and the LCE / CV and content source are saved.
  2. Ansible still isn’t able to create hostgroups, citing the same error as earlier
  3. When creating the hostgroup Ansible wanted to create by hand, only setting:
  • LCE / Content View / Source
  • Architecture and OS (and let the form autofill the rest)

Ansible is capable of filling in the blanks, but only for that group, still can’t create new ones.

6

ping @katello

Can one of you please shed a light on this issue?

Thanks!

7

I don’t think this behavior is intentional. Are there any related errors in Foreman’s production.log?

8

So I created a new hostgroup called test, it did save the settings.

And I set only the LCE, CV and CS, which resulted in the following logs:

2020-11-24T00:11:50 [I|app|1299c7a1] Started GET "/hostgroups/new" for 192.168.255.11 at 2020-11-24 00:11:50 +0100
2020-11-24T00:11:50 [I|app|1299c7a1] Processing by HostgroupsController#new as HTML
2020-11-24T00:11:50 [I|app|1299c7a1]   Rendering hostgroups/new.html.erb within layouts/application
2020-11-24T00:11:50 [I|app|1299c7a1]   Rendered common/os_selection/_architecture.html.erb (Duration: 4.9ms | Allocations: 6057)
2020-11-24T00:11:50 [I|app|1299c7a1]   Rendered common/os_selection/_operatingsystem.html.erb (Duration: 23.6ms | Allocations: 13168)
2020-11-24T00:11:50 [I|app|1299c7a1]   Rendered common/os_selection/_initial.html.erb (Duration: 36.7ms | Allocations: 25652)
2020-11-24T00:11:50 [I|app|1299c7a1]   Rendered collection of puppetclasses/_class_parameters.html.erb [0 times] (Duration: 0.0ms | Allocations: 3)
2020-11-24T00:11:50 [I|app|1299c7a1]   Rendered puppetclasses/_classes_parameters.html.erb (Duration: 19.3ms | Allocations: 8566)
2020-11-24T00:11:50 [I|app|1299c7a1]   Rendered common_parameters/_parameter.html.erb (Duration: 8.1ms | Allocations: 6812)
2020-11-24T00:11:50 [I|app|1299c7a1]   Rendered common_parameters/_parameters.html.erb (Duration: 16.8ms | Allocations: 13501)
2020-11-24T00:11:50 [I|app|1299c7a1]   Rendered taxonomies/_loc_org_tabs.html.erb (Duration: 11.2ms | Allocations: 7506)
2020-11-24T00:11:50 [I|app|1299c7a1]   Rendered hostgroups/_form.html.erb (Duration: 171.3ms | Allocations: 86685)
2020-11-24T00:11:50 [I|app|1299c7a1]   Rendered hostgroups/new.html.erb within layouts/application (Duration: 172.4ms | Allocations: 86979)
2020-11-24T00:11:50 [I|app|1299c7a1]   Rendered layouts/_application_content.html.erb (Duration: 4.9ms | Allocations: 6045)
2020-11-24T00:11:50 [I|app|1299c7a1]   Rendering layouts/base.html.erb
2020-11-24T00:11:50 [I|app|1299c7a1]   Rendered layouts/base.html.erb (Duration: 23.9ms | Allocations: 25505)
2020-11-24T00:11:50 [I|app|1299c7a1] Completed 200 OK in 232ms (Views: 187.5ms | ActiveRecord: 24.0ms | Allocations: 131832)
2020-11-24T00:11:52 [I|app|c89af43a] Started GET "/notification_recipients" for 192.168.255.11 at 2020-11-24 00:11:52 +0100
2020-11-24T00:11:52 [I|app|c89af43a] Processing by NotificationRecipientsController#index as JSON
2020-11-24T00:11:52 [I|app|c89af43a] Completed 200 OK in 17ms (Views: 0.2ms | ActiveRecord: 6.8ms | Allocations: 2327)
2020-11-24T00:11:59 [I|app|99cc1df0] Started GET "/katello/api/v2/content_views/?environment_id=6&full_result=true" for 192.168.255.11 at 2020-11-24 00:11:59 +0100
2020-11-24T00:11:59 [I|app|99cc1df0] Processing by Katello::Api::V2::ContentViewsController#index as */*
2020-11-24T00:11:59 [I|app|99cc1df0]   Parameters: {"environment_id"=>"6", "full_result"=>"true", "api_version"=>"v2"}
2020-11-24T00:11:59 [I|app|99cc1df0]   Rendering /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.17.0/app/views/katello/api/v2/content_views/index.json.rabl within katello/api/v2/layouts/collection
2020-11-24T00:12:00 [I|app|99cc1df0]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.17.0/app/views/katello/api/v2/content_views/index.json.rabl within katello/api/v2/layouts/collection (Duration: 1231.7ms | Allocations: 233177)
2020-11-24T00:12:00 [I|app|99cc1df0] Completed 200 OK in 1268ms (Views: 994.4ms | ActiveRecord: 252.6ms | Allocations: 239051)
2020-11-24T00:12:02 [I|app|7e5d28a5] Started GET "/katello/api/v2/environments/6/activation_keys?content_view_id=15" for 192.168.255.11 at 2020-11-24 00:12:02 +0100
2020-11-24T00:12:02 [I|app|cacc5b44] Started GET "/hosts/puppet_environment_for_content_view?content_view_id=15&lifecycle_environment_id=6" for 192.168.255.11 at 2020-11-24 00:12:02 +0100
2020-11-24T00:12:02 [I|app|cacc5b44] Processing by HostsController#puppet_environment_for_content_view as */*
2020-11-24T00:12:02 [I|app|cacc5b44]   Parameters: {"content_view_id"=>"15", "lifecycle_environment_id"=>"6"}
2020-11-24T00:12:02 [I|app|7e5d28a5] Processing by Katello::Api::V2::ActivationKeysController#index as */*
2020-11-24T00:12:02 [I|app|7e5d28a5]   Parameters: {"content_view_id"=>"15", "api_version"=>"v2", "environment_id"=>"6"}
2020-11-24T00:12:02 [I|app|7e5d28a5]   Rendering /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.17.0/app/views/katello/api/v2/activation_keys/index.json.rabl within katello/api/v2/layouts/collection
2020-11-24T00:12:02 [I|app|7e5d28a5]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.17.0/app/views/katello/api/v2/activation_keys/index.json.rabl within katello/api/v2/layouts/collection (Duration: 178.1ms | Allocations: 8932)
2020-11-24T00:12:02 [I|app|7e5d28a5] Completed 200 OK in 256ms (Views: 56.2ms | ActiveRecord: 181.3ms | Allocations: 14201)
2020-11-24T00:12:03 [I|app|b8106a6e] Started GET "/notification_recipients" for 192.168.255.11 at 2020-11-24 00:12:03 +0100
2020-11-24T00:12:03 [I|app|b8106a6e] Processing by NotificationRecipientsController#index as JSON
2020-11-24T00:12:03 [I|app|cacc5b44] Completed 200 OK in 610ms (Views: 0.4ms | ActiveRecord: 408.2ms | Allocations: 6001)
2020-11-24T00:12:03 [I|app|b8106a6e] Completed 200 OK in 94ms (Views: 0.2ms | ActiveRecord: 66.6ms | Allocations: 2161)
2020-11-24T00:12:08 [I|app|4e5c4558] Started POST "/hostgroups" for 192.168.255.11 at 2020-11-24 00:12:08 +0100
2020-11-24T00:12:08 [I|app|4e5c4558] Processing by HostgroupsController#create as HTML
2020-11-24T00:12:08 [I|app|4e5c4558]   Parameters: {"utf8"=>"✓", "authenticity_token"=>"IOJsc8tUePJDpnvUi1stDHGzJ3shofjtl3cT2SfWfCOSBU1B+o4pnj014bsF26whX3RWPJgwZdgY+csSLh47fQ==", "hostgroup"=>{"parent_id"=>"", "name"=>"test", "description"=>"", "lifecycle_environment_id"=>"6", "content_view_id"=>"15", "content_source_id"=>"1", "environment_id"=>"", "compute_resource_id"=>"", "puppet_proxy_id"=>"", "puppet_ca_proxy_id"=>"", "domain_id"=>"", "realm_id"=>"", "architecture_id"=>"", "operatingsystem_id"=>"", "medium_id"=>"", "ptable_id"=>"", "root_pass"=>"[FILTERED]", "location_ids"=>["", "2"], "organization_ids"=>["", "3"], "id"=>""}, "commit"=>"Submit"}
2020-11-24T00:12:09 [I|aud|4e5c4558] Hostgroup (36) create event on name test
2020-11-24T00:12:09 [I|aud|4e5c4558] Hostgroup (36) create event on environment_id 
2020-11-24T00:12:09 [I|aud|4e5c4558] Hostgroup (36) create event on operatingsystem_id 
2020-11-24T00:12:09 [I|aud|4e5c4558] Hostgroup (36) create event on architecture_id 
2020-11-24T00:12:09 [I|aud|4e5c4558] Hostgroup (36) create event on medium_id 
2020-11-24T00:12:09 [I|aud|4e5c4558] Hostgroup (36) create event on ptable_id 
2020-11-24T00:12:09 [I|aud|4e5c4558] Hostgroup (36) create event on root_pass 
2020-11-24T00:12:09 [I|aud|4e5c4558] Hostgroup (36) create event on puppet_ca_proxy_id 
2020-11-24T00:12:09 [I|aud|4e5c4558] Hostgroup (36) create event on use_image 
2020-11-24T00:12:09 [I|aud|4e5c4558] Hostgroup (36) create event on image_file 
2020-11-24T00:12:09 [I|aud|4e5c4558] Hostgroup (36) create event on ancestry 
2020-11-24T00:12:09 [I|aud|4e5c4558] Hostgroup (36) create event on vm_defaults 
2020-11-24T00:12:09 [I|aud|4e5c4558] Hostgroup (36) create event on subnet_id 
2020-11-24T00:12:09 [I|aud|4e5c4558] Hostgroup (36) create event on domain_id 
2020-11-24T00:12:09 [I|aud|4e5c4558] Hostgroup (36) create event on puppet_proxy_id 
2020-11-24T00:12:09 [I|aud|4e5c4558] Hostgroup (36) create event on title test
2020-11-24T00:12:09 [I|aud|4e5c4558] Hostgroup (36) create event on realm_id 
2020-11-24T00:12:09 [I|aud|4e5c4558] Hostgroup (36) create event on compute_profile_id 
2020-11-24T00:12:09 [I|aud|4e5c4558] Hostgroup (36) create event on grub_pass 
2020-11-24T00:12:09 [I|aud|4e5c4558] Hostgroup (36) create event on lookup_value_matcher hostgroup=test
2020-11-24T00:12:09 [I|aud|4e5c4558] Hostgroup (36) create event on subnet6_id 
2020-11-24T00:12:09 [I|aud|4e5c4558] Hostgroup (36) create event on pxe_loader 
2020-11-24T00:12:09 [I|aud|4e5c4558] Hostgroup (36) create event on description 
2020-11-24T00:12:09 [I|aud|4e5c4558] Hostgroup (36) create event on compute_resource_id 
2020-11-24T00:12:09 [I|aud|4e5c4558] Hostgroup (36) create event on content_facet_attributes {"id"=>nil, "hostgroup_id"=>nil, "kickstart_repository_id"=>nil, "content_source_id"=>1, "content_view_id"=>15, "lifecycle_environment_id"=>6}
2020-11-24T00:12:09 [I|aud|4e5c4558] Hostgroup (36) create event on organization_ids [3]
2020-11-24T00:12:09 [I|aud|4e5c4558] Hostgroup (36) create event on location_ids [2]
2020-11-24T00:12:09 [I|aud|4e5c4558] Katello::Hostgroup::ContentFacet (1) create event on hostgroup_id 36
2020-11-24T00:12:09 [I|aud|4e5c4558] Katello::Hostgroup::ContentFacet (1) create event on kickstart_repository_id 
2020-11-24T00:12:09 [I|aud|4e5c4558] Katello::Hostgroup::ContentFacet (1) create event on content_source_id 1
2020-11-24T00:12:09 [I|aud|4e5c4558] Katello::Hostgroup::ContentFacet (1) create event on content_view_id 15
2020-11-24T00:12:09 [I|aud|4e5c4558] Katello::Hostgroup::ContentFacet (1) create event on lifecycle_environment_id 6
2020-11-24T00:12:09 [I|app|4e5c4558] Redirected to https://foreman.lbhr.htm.lan/hostgroups
2020-11-24T00:12:09 [I|app|4e5c4558] Completed 302 Found in 396ms (ActiveRecord: 236.7ms | Allocations: 31343)
2020-11-24T00:12:09 [I|app|67077b0a] Started GET "/hostgroups" for 192.168.255.11 at 2020-11-24 00:12:09 +0100
2020-11-24T00:12:09 [I|app|67077b0a] Processing by HostgroupsController#index as HTML
2020-11-24T00:12:09 [I|app|67077b0a]   Rendering hostgroups/index.html.erb within layouts/application
2020-11-24T00:12:10 [I|app|67077b0a]   Rendered common/_pagination.html.erb (Duration: 18.0ms | Allocations: 5780)
2020-11-24T00:12:10 [I|app|67077b0a]   Rendered hostgroups/index.html.erb within layouts/application (Duration: 871.5ms | Allocations: 81677)
2020-11-24T00:12:10 [I|app|67077b0a]   Rendered common/_searchbar.html.erb (Duration: 5.2ms | Allocations: 6034)
2020-11-24T00:12:10 [I|app|67077b0a]   Rendered layouts/_application_content.html.erb (Duration: 20.6ms | Allocations: 12084)
2020-11-24T00:12:10 [I|app|67077b0a]   Rendering layouts/base.html.erb
2020-11-24T00:12:10 [I|app|67077b0a]   Rendered layouts/base.html.erb (Duration: 66.9ms | Allocations: 25518)
2020-11-24T00:12:10 [I|app|67077b0a] Completed 200 OK in 1028ms (Views: 596.0ms | ActiveRecord: 402.2ms | Allocations: 130442)
2020-11-24T00:12:12 [I|app|a95db2a2] Started GET "/notification_recipients" for 192.168.255.11 at 2020-11-24 00:12:12 +0100
2020-11-24T00:12:12 [I|app|a95db2a2] Processing by NotificationRecipientsController#index as JSON
2020-11-24T00:12:12 [I|app|a95db2a2] Completed 200 OK in 61ms (Views: 0.2ms | ActiveRecord: 40.4ms | Allocations: 2327)
2020-11-24T00:12:12 [I|app|] CONTENT_INDEX type=Katello::Erratum duration=67024.22
2020-11-24T00:12:13 [I|app|] CONTENT_INDEX type=Katello::PackageGroup duration=789.61
2020-11-24T00:12:13 [I|app|] CONTENT_INDEX type=Katello::Srpm duration=294.11
2020-11-24T00:12:13 [I|bac|] Task {label: Actions::Katello::Repository::CapsuleSync, id: 345b45ba-70d5-4f72-bc43-3539e114e266, execution_plan_id: 7f9efddc-1979-4aa9-9309-574c266fc51e} state changed: planning 
2020-11-24T00:12:14 [I|bac|] Task {label: Actions::Katello::Repository::CapsuleSync, id: 345b45ba-70d5-4f72-bc43-3539e114e266, execution_plan_id: 7f9efddc-1979-4aa9-9309-574c266fc51e} state changed: planned 
2020-11-24T00:12:14 [I|bac|] Task {label: Actions::Katello::Repository::CapsuleSync, id: 345b45ba-70d5-4f72-bc43-3539e114e266, execution_plan_id: 7f9efddc-1979-4aa9-9309-574c266fc51e} state changed: running 
2020-11-24T00:12:14 [I|bac|] Task {label: Actions::Katello::Repository::CapsuleSync, id: 345b45ba-70d5-4f72-bc43-3539e114e266, execution_plan_id: 7f9efddc-1979-4aa9-9309-574c266fc51e} state changed: stopped  result: success
2020-11-24T00:12:14 [I|bac|] Task {label: Actions::Katello::Repository::CapsuleSync, id: 345b45ba-70d5-4f72-bc43-3539e114e266, execution_plan_id: 7f9efddc-1979-4aa9-9309-574c266fc51e} state changed: stopped  result: success
2020-11-24T00:12:14 [I|bac|] Task {label: Actions::Katello::Repository::Sync, id: 6c97fae4-b6c6-42cd-a88a-6f14d7c80931, execution_plan_id: 39b3f9ac-4dbc-4d71-abd3-774b0f31756e} state changed: stopped  result: success
2020-11-24T00:12:14 [I|app|239b5c69] Started GET "/rhsm/consumers/7bef9910-a37e-4ffd-8537-c732ed3dedb7/compliance" for 192.168.255.156 at 2020-11-24 00:12:14 +0100
2020-11-24T00:12:14 [I|app|239b5c69] Processing by Katello::Api::Rhsm::CandlepinProxiesController#get as JSON
2020-11-24T00:12:14 [I|app|239b5c69]   Parameters: {"id"=>"7bef9910-a37e-4ffd-8537-c732ed3dedb7"}
2020-11-24T00:12:14 [I|bac|] Task {label: Actions::Katello::Repository::Sync, id: 6c97fae4-b6c6-42cd-a88a-6f14d7c80931, execution_plan_id: 39b3f9ac-4dbc-4d71-abd3-774b0f31756e} state changed: stopped  result: success
2020-11-24T00:12:14 [E|app|239b5c69] RestClient::NotFound: 404 Not Found
 239b5c69 | Body: {"displayMessage":"Consumer with id 7bef9910-a37e-4ffd-8537-c732ed3dedb7 could not be found.","requestUuid":"e7469cb2-90f8-4e1a-b9be-378cc85382f9"}
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/abstract_response.rb:223:in `exception_with_response'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/abstract_response.rb:103:in `return!'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/request.rb:809:in `process_result'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/request.rb:725:in `block in transmit'
 239b5c69 | /opt/rh/rh-ruby25/root/usr/share/ruby/net/http.rb:910:in `start'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/request.rb:715:in `transmit'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/request.rb:145:in `execute'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/request.rb:52:in `execute'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/resource.rb:51:in `get'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.17.0/app/lib/katello/resources/candlepin/proxy.rb:26:in `get'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.17.0/app/controllers/katello/api/rhsm/candlepin_proxies_controller.rb:75:in `get'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_controller/metal/basic_implicit_render.rb:6:in `send_action'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/abstract_controller/base.rb:195:in `process_action'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_controller/metal/rendering.rb:30:in `process_action'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/abstract_controller/callbacks.rb:42:in `block in process_action'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.17.0/app/controllers/katello/api/rhsm/candlepin_proxies_controller.rb:35:in `repackage_message'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
 239b5c69 | /usr/share/foreman/app/controllers/api/v2/base_controller.rb:181:in `disable_json_root'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
 239b5c69 | /usr/share/foreman/app/controllers/concerns/foreman/controller/timezone.rb:10:in `set_timezone'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
 239b5c69 | /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:32:in `clear_thread'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
 239b5c69 | /usr/share/foreman/app/controllers/concerns/foreman/controller/topbar_sweeper.rb:12:in `set_topbar_sweeper_controller'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/audited-4.9.0/lib/audited/sweeper.rb:14:in `around'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/audited-4.9.0/lib/audited/sweeper.rb:14:in `around'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/callbacks.rb:139:in `run_callbacks'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/abstract_controller/callbacks.rb:41:in `process_action'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_controller/metal/rescue.rb:22:in `process_action'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_controller/metal/instrumentation.rb:33:in `block in process_action'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/notifications.rb:180:in `block in instrument'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/notifications.rb:180:in `instrument'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_controller/metal/instrumentation.rb:32:in `process_action'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_controller/metal/params_wrapper.rb:245:in `process_action'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.1/lib/active_record/railties/controller_runtime.rb:27:in `process_action'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.17.0/app/controllers/katello/concerns/api/api_controller.rb:40:in `process_action'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/abstract_controller/base.rb:136:in `process'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionview-6.0.3.1/lib/action_view/rendering.rb:39:in `process'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_controller/metal.rb:190:in `dispatch'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_controller/metal.rb:254:in `dispatch'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/route_set.rb:50:in `dispatch'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/route_set.rb:33:in `serve'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/journey/router.rb:49:in `block in serve'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/journey/router.rb:32:in `each'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/journey/router.rb:32:in `serve'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/route_set.rb:834:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.1/lib/rails/engine.rb:527:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.1/lib/rails/railtie.rb:190:in `public_send'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.1/lib/rails/railtie.rb:190:in `method_missing'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/mapper.rb:19:in `block in <class:Constraints>'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/mapper.rb:48:in `serve'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/journey/router.rb:49:in `block in serve'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/journey/router.rb:32:in `each'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/journey/router.rb:32:in `serve'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/route_set.rb:834:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.17.0/lib/katello/middleware/event_daemon.rb:10:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/static.rb:126:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/static.rb:126:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-dsl-2.2.9/lib/apipie_dsl/static_dispatcher.rb:67:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.5.17/lib/apipie/static_dispatcher.rb:66:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.5.17/lib/apipie/extractor/recorder.rb:137:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/static.rb:126:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/static.rb:126:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/static.rb:126:in `call'
 239b5c69 | /usr/share/foreman/lib/foreman/middleware/telemetry.rb:10:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.5.17/lib/apipie/middleware/checksum_in_headers.rb:27:in `call'
 239b5c69 | /usr/share/foreman/lib/foreman/middleware/catch_json_parse_errors.rb:9:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/tempfile_reaper.rb:15:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/etag.rb:27:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/conditional_get.rb:27:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/head.rb:12:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/http/content_security_policy.rb:18:in `call'
 239b5c69 | /usr/share/foreman/lib/foreman/middleware/logging_context_session.rb:22:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/session/abstract/id.rb:266:in `context'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/session/abstract/id.rb:260:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/cookies.rb:648:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/callbacks.rb:27:in `block in call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/callbacks.rb:101:in `run_callbacks'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/callbacks.rb:26:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/actionable_exceptions.rb:17:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/debug_exceptions.rb:32:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.1/lib/rails/rack/logger.rb:37:in `call_app'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.1/lib/rails/rack/logger.rb:28:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/sprockets-rails-3.2.1/lib/sprockets/rails/quiet_assets.rb:13:in `call'
 239b5c69 | /usr/share/foreman/lib/foreman/middleware/logging_context_request.rb:11:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/request_id.rb:27:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.17.0/lib/katello/prevent_json_parsing.rb:12:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/method_override.rb:24:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/runtime.rb:22:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/cache/strategy/local_cache_middleware.rb:29:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/executor.rb:14:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/static.rb:126:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/sendfile.rb:110:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/host_authorization.rb:76:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/secure_headers-6.3.0/lib/secure_headers/middleware.rb:11:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.1/lib/rails/engine.rb:527:in `call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.1/lib/rails/railtie.rb:190:in `public_send'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.1/lib/rails/railtie.rb:190:in `method_missing'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/urlmap.rb:74:in `block in call'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/urlmap.rb:58:in `each'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/urlmap.rb:58:in `call'
 239b5c69 | /usr/share/passenger/phusion_passenger/rack/thread_handler_extension.rb:74:in `process_request'
 239b5c69 | /usr/share/passenger/phusion_passenger/request_handler/thread_handler.rb:141:in `accept_and_process_next_request'
 239b5c69 | /usr/share/passenger/phusion_passenger/request_handler/thread_handler.rb:109:in `main_loop'
 239b5c69 | /usr/share/passenger/phusion_passenger/request_handler.rb:455:in `block (3 levels) in start_threads'
 239b5c69 | /opt/theforeman/tfm/root/usr/share/gems/gems/logging-2.2.2/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'
2020-11-24T00:12:14 [I|app|239b5c69]   Rendering text template
2020-11-24T00:12:14 [I|app|239b5c69]   Rendered text template (Duration: 0.0ms | Allocations: 2)
2020-11-24T00:12:14 [I|app|239b5c69] Completed 404 Not Found in 174ms (Views: 0.5ms | ActiveRecord: 19.6ms | Allocations: 4163)
2020-11-24T00:12:16 [I|app|735b30c8] Started GET "/rhsm/consumers/7bef9910-a37e-4ffd-8537-c732ed3dedb7" for 192.168.255.156 at 2020-11-24 00:12:16 +0100
2020-11-24T00:12:16 [I|app|735b30c8] Processing by Katello::Api::Rhsm::CandlepinProxiesController#consumer_show as JSON
2020-11-24T00:12:16 [I|app|735b30c8]   Parameters: {"id"=>"7bef9910-a37e-4ffd-8537-c732ed3dedb7"}
2020-11-24T00:12:16 [E|app|735b30c8] RestClient::NotFound: Katello::Resources::Candlepin::Consumer: 404 Not Found {"displayMessage":"Consumer with id 7bef9910-a37e-4ffd-8537-c732ed3dedb7 could not be found.","requestUuid":"820ca6b9-11b2-4744-9aba-9d088cfb7968"} (GET /candlepin/consumers/7bef9910-a37e-4ffd-8537-c732ed3dedb7)
 735b30c8 | Body: {"displayMessage":"Consumer with id 7bef9910-a37e-4ffd-8537-c732ed3dedb7 could not be found.","requestUuid":"820ca6b9-11b2-4744-9aba-9d088cfb7968"}
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/abstract_response.rb:223:in `exception_with_response'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/abstract_response.rb:103:in `return!'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/request.rb:809:in `process_result'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/request.rb:725:in `block in transmit'
 735b30c8 | /opt/rh/rh-ruby25/root/usr/share/ruby/net/http.rb:910:in `start'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/request.rb:715:in `transmit'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/request.rb:145:in `execute'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/request.rb:52:in `execute'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/resource.rb:51:in `get'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.17.0/app/lib/katello/http_resource.rb:87:in `get'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.17.0/app/lib/katello/resources/candlepin/consumer.rb:20:in `get'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.17.0/app/controllers/katello/api/rhsm/candlepin_proxies_controller.rb:304:in `block in find_host'
 735b30c8 | /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:108:in `as'
 735b30c8 | /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:114:in `as_anonymous_admin'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.17.0/app/controllers/katello/api/rhsm/candlepin_proxies_controller.rb:304:in `find_host'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/callbacks.rb:428:in `block in make_lambda'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/callbacks.rb:181:in `block (2 levels) in halting_and_conditional'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/abstract_controller/callbacks.rb:34:in `block (2 levels) in <module:Callbacks>'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/callbacks.rb:182:in `block in halting_and_conditional'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/callbacks.rb:513:in `block in invoke_before'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/callbacks.rb:513:in `each'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/callbacks.rb:513:in `invoke_before'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/callbacks.rb:110:in `block in run_callbacks'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.17.0/app/controllers/katello/api/rhsm/candlepin_proxies_controller.rb:35:in `repackage_message'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
 735b30c8 | /usr/share/foreman/app/controllers/api/v2/base_controller.rb:181:in `disable_json_root'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
 735b30c8 | /usr/share/foreman/app/controllers/concerns/foreman/controller/timezone.rb:10:in `set_timezone'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
 735b30c8 | /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:32:in `clear_thread'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
 735b30c8 | /usr/share/foreman/app/controllers/concerns/foreman/controller/topbar_sweeper.rb:12:in `set_topbar_sweeper_controller'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/audited-4.9.0/lib/audited/sweeper.rb:14:in `around'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/audited-4.9.0/lib/audited/sweeper.rb:14:in `around'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/callbacks.rb:139:in `run_callbacks'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/abstract_controller/callbacks.rb:41:in `process_action'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_controller/metal/rescue.rb:22:in `process_action'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_controller/metal/instrumentation.rb:33:in `block in process_action'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/notifications.rb:180:in `block in instrument'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/notifications.rb:180:in `instrument'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_controller/metal/instrumentation.rb:32:in `process_action'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_controller/metal/params_wrapper.rb:245:in `process_action'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.1/lib/active_record/railties/controller_runtime.rb:27:in `process_action'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.17.0/app/controllers/katello/concerns/api/api_controller.rb:40:in `process_action'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/abstract_controller/base.rb:136:in `process'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionview-6.0.3.1/lib/action_view/rendering.rb:39:in `process'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_controller/metal.rb:190:in `dispatch'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_controller/metal.rb:254:in `dispatch'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/route_set.rb:50:in `dispatch'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/route_set.rb:33:in `serve'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/journey/router.rb:49:in `block in serve'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/journey/router.rb:32:in `each'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/journey/router.rb:32:in `serve'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/route_set.rb:834:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.1/lib/rails/engine.rb:527:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.1/lib/rails/railtie.rb:190:in `public_send'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.1/lib/rails/railtie.rb:190:in `method_missing'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/mapper.rb:19:in `block in <class:Constraints>'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/mapper.rb:48:in `serve'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/journey/router.rb:49:in `block in serve'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/journey/router.rb:32:in `each'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/journey/router.rb:32:in `serve'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/route_set.rb:834:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.17.0/lib/katello/middleware/event_daemon.rb:10:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/static.rb:126:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/static.rb:126:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-dsl-2.2.9/lib/apipie_dsl/static_dispatcher.rb:67:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.5.17/lib/apipie/static_dispatcher.rb:66:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.5.17/lib/apipie/extractor/recorder.rb:137:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/static.rb:126:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/static.rb:126:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/static.rb:126:in `call'
 735b30c8 | /usr/share/foreman/lib/foreman/middleware/telemetry.rb:10:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.5.17/lib/apipie/middleware/checksum_in_headers.rb:27:in `call'
 735b30c8 | /usr/share/foreman/lib/foreman/middleware/catch_json_parse_errors.rb:9:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/tempfile_reaper.rb:15:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/etag.rb:27:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/conditional_get.rb:27:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/head.rb:12:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/http/content_security_policy.rb:18:in `call'
 735b30c8 | /usr/share/foreman/lib/foreman/middleware/logging_context_session.rb:22:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/session/abstract/id.rb:266:in `context'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/session/abstract/id.rb:260:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/cookies.rb:648:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/callbacks.rb:27:in `block in call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/callbacks.rb:101:in `run_callbacks'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/callbacks.rb:26:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/actionable_exceptions.rb:17:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/debug_exceptions.rb:32:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.1/lib/rails/rack/logger.rb:37:in `call_app'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.1/lib/rails/rack/logger.rb:28:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/sprockets-rails-3.2.1/lib/sprockets/rails/quiet_assets.rb:13:in `call'
 735b30c8 | /usr/share/foreman/lib/foreman/middleware/logging_context_request.rb:11:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/request_id.rb:27:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.17.0/lib/katello/prevent_json_parsing.rb:12:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/method_override.rb:24:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/runtime.rb:22:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/cache/strategy/local_cache_middleware.rb:29:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/executor.rb:14:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/static.rb:126:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/sendfile.rb:110:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.1/lib/action_dispatch/middleware/host_authorization.rb:76:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/secure_headers-6.3.0/lib/secure_headers/middleware.rb:11:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.1/lib/rails/engine.rb:527:in `call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.1/lib/rails/railtie.rb:190:in `public_send'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.1/lib/rails/railtie.rb:190:in `method_missing'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/urlmap.rb:74:in `block in call'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/urlmap.rb:58:in `each'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/urlmap.rb:58:in `call'
 735b30c8 | /usr/share/passenger/phusion_passenger/rack/thread_handler_extension.rb:74:in `process_request'
 735b30c8 | /usr/share/passenger/phusion_passenger/request_handler/thread_handler.rb:141:in `accept_and_process_next_request'
 735b30c8 | /usr/share/passenger/phusion_passenger/request_handler/thread_handler.rb:109:in `main_loop'
 735b30c8 | /usr/share/passenger/phusion_passenger/request_handler.rb:455:in `block (3 levels) in start_threads'
 735b30c8 | /opt/theforeman/tfm/root/usr/share/gems/gems/logging-2.2.2/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'
2020-11-24T00:12:16 [I|app|735b30c8]   Rendering text template
2020-11-24T00:12:16 [I|app|735b30c8]   Rendered text template (Duration: 0.0ms | Allocations: 2)
2020-11-24T00:12:16 [I|app|735b30c8] Completed 404 Not Found in 71ms (Views: 0.4ms | ActiveRecord: 9.2ms | Allocations: 4164)
2020-11-24T00:12:16 [I|bac|] Task {label: Actions::Katello::Applicability::Hosts::BulkGenerate, id: 2f3998ea-00e7-4841-aabc-99a21e029411, execution_plan_id: 8df9b41f-fbf0-402f-8aaf-5c64ad7a21cf} state changed: planning 
2020-11-24T00:12:16 [I|bac|] Task {label: Actions::Katello::Applicability::Hosts::BulkGenerate, id: 2f3998ea-00e7-4841-aabc-99a21e029411, execution_plan_id: 8df9b41f-fbf0-402f-8aaf-5c64ad7a21cf} state changed: planned 
2020-11-24T00:12:16 [I|bac|] Task {label: Actions::Katello::Applicability::Hosts::BulkGenerate, id: 2f3998ea-00e7-4841-aabc-99a21e029411, execution_plan_id: 8df9b41f-fbf0-402f-8aaf-5c64ad7a21cf} state changed: running 
2020-11-24T00:12:17 [I|bac|] Task {label: Actions::Katello::Applicability::Hosts::BulkGenerate, id: 2f3998ea-00e7-4841-aabc-99a21e029411, execution_plan_id: 8df9b41f-fbf0-402f-8aaf-5c64ad7a21cf} state changed: stopped  result: success
2020-11-24T00:12:17 [I|bac|] Task {label: Actions::Katello::Applicability::Hosts::BulkGenerate, id: 2f3998ea-00e7-4841-aabc-99a21e029411, execution_plan_id: 8df9b41f-fbf0-402f-8aaf-5c64ad7a21cf} state changed: stopped  result: success
2020-11-24T00:12:22 [I|app|556a4a4d] Started GET "/notification_recipients" for 192.168.255.11 at 2020-11-24 00:12:22 +0100
2020-11-24T00:12:22 [I|app|556a4a4d] Processing by NotificationRecipientsController#index as JSON
2020-11-24T00:12:22 [I|app|556a4a4d] Completed 200 OK in 11ms (Views: 0.2ms | ActiveRecord: 1.3ms | Allocations: 2155)
2020-11-24T00:12:32 [I|app|cf125bec] Started GET "/notification_recipients" for 192.168.255.11 at 2020-11-24 00:12:32 +0100
2020-11-24T00:12:32 [I|app|cf125bec] Processing by NotificationRecipientsController#index as JSON
2020-11-24T00:12:32 [I|app|cf125bec] Completed 200 OK in 9ms (Views: 0.1ms | ActiveRecord: 1.4ms | Allocations: 2161)

2020-11-24T00:11:50 [I|app|1299c7a1] Started GET "/hostgroups/new" for 192.168.255.11 at 2020-11-24 00:11:50 +0100
2020-11-24T00:11:50 [I|app|1299c7a1] Processing by HostgroupsController#new as HTML
2020-11-24T00:11:50 [I|app|1299c7a1]   Rendering hostgroups/new.html.erb within layouts/application
2020-11-24T00:11:50 [I|app|1299c7a1]   Rendered common/os_selection/_architecture.html.erb (Duration: 4.9ms | Allocations: 6057)
2020-11-24T00:11:50 [I|app|1299c7a1]   Rendered common/os_selection/_operatingsystem.html.erb (Duration: 23.6ms | Allocations: 13168)
2020-11-24T00:11:50 [I|app|1299c7a1]   Rendered common/os_selection/_initial.html.erb (Duration: 36.7ms | Allocations: 25652)
2020-11-24T00:11:50 [I|app|1299c7a1]   Rendered collection of puppetclasses/_class_parameters.html.erb [0 times] (Duration: 0.0ms | Allocations: 3)
2020-11-24T00:11:50 [I|app|1299c7a1]   Rendered puppetclasses/_classes_parameters.html.erb (Duration: 19.3ms | Allocations: 8566)
2020-11-24T00:11:50 [I|app|1299c7a1]   Rendered common_parameters/_parameter.html.erb (Duration: 8.1ms | Allocations: 6812)
2020-11-24T00:11:50 [I|app|1299c7a1]   Rendered common_parameters/_parameters.html.erb (Duration: 16.8ms | Allocations: 13501)
2020-11-24T00:11:50 [I|app|1299c7a1]   Rendered taxonomies/_loc_org_tabs.html.erb (Duration: 11.2ms | Allocations: 7506)
2020-11-24T00:11:50 [I|app|1299c7a1]   Rendered hostgroups/_form.html.erb (Duration: 171.3ms | Allocations: 86685)
2020-11-24T00:11:50 [I|app|1299c7a1]   Rendered hostgroups/new.html.erb within layouts/application (Duration: 172.4ms | Allocations: 86979)
2020-11-24T00:11:50 [I|app|1299c7a1]   Rendered layouts/_application_content.html.erb (Duration: 4.9ms | Allocations: 6045)
2020-11-24T00:11:50 [I|app|1299c7a1]   Rendering layouts/base.html.erb
2020-11-24T00:11:50 [I|app|1299c7a1]   Rendered layouts/base.html.erb (Duration: 23.9ms | Allocations: 25505)
2020-11-24T00:11:50 [I|app|1299c7a1] Completed 200 OK in 232ms (Views: 187.5ms | ActiveRecord: 24.0ms | Allocations: 131832)
2020-11-24T00:11:52 [I|app|c89af43a] Started GET "/notification_recipients" for 192.168.255.11 at 2020-11-24 00:11:52 +0100
2020-11-24T00:11:52 [I|app|c89af43a] Processing by NotificationRecipientsController#index as JSON
2020-11-24T00:11:52 [I|app|c89af43a] Completed 200 OK in 17ms (Views: 0.2ms | ActiveRecord: 6.8ms | Allocations: 2327)
2020-11-24T00:11:59 [I|app|99cc1df0] Started GET "/katello/api/v2/content_views/?environment_id=6&full_result=true" for 192.168.255.11 at 2020-11-24 00:11:59 +0100
2020-11-24T00:11:59 [I|app|99cc1df0] Processing by Katello::Api::V2::ContentViewsController#index as */*
2020-11-24T00:11:59 [I|app|99cc1df0]   Parameters: {"environment_id"=>"6", "full_result"=>"true", "api_version"=>"v2"}
2020-11-24T00:11:59 [I|app|99cc1df0]   Rendering /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.17.0/app/views/katello/api/v2/content_views/index.json.rabl within katello/api/v2/layouts/collection
2020-11-24T00:12:00 [I|app|99cc1df0]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.17.0/app/views/katello/api/v2/content_views/index.json.rabl within katello/api/v2/layouts/collection (Duration: 1231.7ms | Allocations: 233177)
2020-11-24T00:12:00 [I|app|99cc1df0] Completed 200 OK in 1268ms (Views: 994.4ms | ActiveRecord: 252.6ms | Allocations: 239051)
2020-11-24T00:12:02 [I|app|7e5d28a5] Started GET "/katello/api/v2/environments/6/activation_keys?content_view_id=15" for 192.168.255.11 at 2020-11-24 00:12:02 +0100
2020-11-24T00:12:02 [I|app|cacc5b44] Started GET "/hosts/puppet_environment_for_content_view?content_view_id=15&lifecycle_environment_id=6" for 192.168.255.11 at 2020-11-24 00:12:02 +0100
2020-11-24T00:12:02 [I|app|cacc5b44] Processing by HostsController#puppet_environment_for_content_view as */*
2020-11-24T00:12:02 [I|app|cacc5b44]   Parameters: {"content_view_id"=>"15", "lifecycle_environment_id"=>"6"}
2020-11-24T00:12:02 [I|app|7e5d28a5] Processing by Katello::Api::V2::ActivationKeysController#index as */*

I also updated a Hostgroup which existed before the upgrade by filling in the blanks again, still didn’t stick:

2020-11-24T00:16:23 [I|app|26ae8f20] Started GET "/hostgroups/11-CentOS7-Base-Acceptatie/edit" for 192.168.255.11 at 2020-11-24 00:16:23 +0100
2020-11-24T00:16:23 [I|app|26ae8f20] Processing by HostgroupsController#edit as HTML
2020-11-24T00:16:23 [I|app|26ae8f20]   Parameters: {"id"=>"11-CentOS7-Base-Acceptatie"}
2020-11-24T00:16:23 [I|app|26ae8f20]   Rendering hostgroups/edit.html.erb within layouts/application
2020-11-24T00:16:23 [I|app|26ae8f20]   Rendered common/os_selection/_architecture.html.erb (Duration: 13.4ms | Allocations: 7179)
2020-11-24T00:16:23 [I|app|26ae8f20]   Rendered common/os_selection/_pxe_loader.html.erb (Duration: 5.5ms | Allocations: 6389)
2020-11-24T00:16:23 [I|app|817e482b] Started GET "/notification_recipients" for 192.168.255.11 at 2020-11-24 00:16:23 +0100
2020-11-24T00:16:23 [I|app|26ae8f20]   Rendered common/os_selection/_operatingsystem.html.erb (Duration: 61.7ms | Allocations: 33670)
2020-11-24T00:16:23 [I|app|26ae8f20]   Rendered common/os_selection/_initial.html.erb (Duration: 82.3ms | Allocations: 47281)
2020-11-24T00:16:23 [I|app|817e482b] Processing by NotificationRecipientsController#index as JSON
2020-11-24T00:16:23 [I|app|26ae8f20]   Rendered collection of puppetclasses/_class_parameters.html.erb [0 times] (Duration: 0.0ms | Allocations: 3)
2020-11-24T00:16:23 [I|app|26ae8f20]   Rendered puppetclasses/_classes_parameters.html.erb (Duration: 19.8ms | Allocations: 9383)
2020-11-24T00:16:23 [I|app|817e482b] Completed 200 OK in 12ms (Views: 0.1ms | ActiveRecord: 1.3ms | Allocations: 2157)
2020-11-24T00:16:23 [I|app|26ae8f20]   Rendered common_parameters/_parameter.html.erb (Duration: 5.6ms | Allocations: 6849)
2020-11-24T00:16:23 [I|app|26ae8f20]   Rendered common_parameters/_parameter.html.erb (Duration: 5.5ms | Allocations: 6854)
2020-11-24T00:16:23 [I|app|26ae8f20]   Rendered common_parameters/_parameter.html.erb (Duration: 6.0ms | Allocations: 6853)
2020-11-24T00:16:23 [I|app|26ae8f20]   Rendered common_parameters/_parameter.html.erb (Duration: 5.5ms | Allocations: 6855)
2020-11-24T00:16:23 [I|app|26ae8f20]   Rendered common_parameters/_parameter.html.erb (Duration: 5.6ms | Allocations: 6853)
2020-11-24T00:16:23 [I|app|26ae8f20]   Rendered common_parameters/_parameter.html.erb (Duration: 5.5ms | Allocations: 6851)
2020-11-24T00:16:23 [I|app|26ae8f20]   Rendered common_parameters/_parameter.html.erb (Duration: 6.9ms | Allocations: 6853)
2020-11-24T00:16:23 [I|app|26ae8f20]   Rendered common_parameters/_parameter.html.erb (Duration: 6.2ms | Allocations: 6798)
2020-11-24T00:16:23 [I|app|26ae8f20]   Rendered common_parameters/_parameters.html.erb (Duration: 59.4ms | Allocations: 63299)
2020-11-24T00:16:23 [I|app|26ae8f20]   Rendered taxonomies/_loc_org_tabs.html.erb (Duration: 28.9ms | Allocations: 16291)
2020-11-24T00:16:23 [I|app|26ae8f20]   Rendered hostgroups/_form.html.erb (Duration: 279.4ms | Allocations: 172267)
2020-11-24T00:16:23 [I|app|26ae8f20]   Rendered hostgroups/edit.html.erb within layouts/application (Duration: 283.7ms | Allocations: 178063)
2020-11-24T00:16:23 [I|app|26ae8f20]   Rendered layouts/_application_content.html.erb (Duration: 4.7ms | Allocations: 6156)
2020-11-24T00:16:23 [I|app|26ae8f20]   Rendering layouts/base.html.erb
2020-11-24T00:16:23 [I|app|26ae8f20]   Rendered layouts/base.html.erb (Duration: 24.1ms | Allocations: 25505)
2020-11-24T00:16:23 [I|app|26ae8f20] Completed 200 OK in 346ms (Views: 282.4ms | ActiveRecord: 42.5ms | Allocations: 223747)
2020-11-24T00:16:25 [I|app|b2cc5ed0] Started GET "/notification_recipients" for 192.168.255.11 at 2020-11-24 00:16:25 +0100
2020-11-24T00:16:25 [I|app|b2cc5ed0] Processing by NotificationRecipientsController#index as JSON
2020-11-24T00:16:25 [I|app|b2cc5ed0] Completed 200 OK in 9ms (Views: 0.1ms | ActiveRecord: 1.2ms | Allocations: 2327)
2020-11-24T00:16:27 [I|app|c9c1f7d2] Started GET "/katello/api/v2/content_views/?environment_id=6&full_result=true" for 192.168.255.11 at 2020-11-24 00:16:27 +0100
2020-11-24T00:16:27 [I|app|c9c1f7d2] Processing by Katello::Api::V2::ContentViewsController#index as */*
2020-11-24T00:16:27 [I|app|c9c1f7d2]   Parameters: {"environment_id"=>"6", "full_result"=>"true", "api_version"=>"v2"}
2020-11-24T00:16:27 [I|app|c9c1f7d2]   Rendering /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.17.0/app/views/katello/api/v2/content_views/index.json.rabl within katello/api/v2/layouts/collection
2020-11-24T00:16:28 [I|app|c9c1f7d2]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.17.0/app/views/katello/api/v2/content_views/index.json.rabl within katello/api/v2/layouts/collection (Duration: 603.9ms | Allocations: 232945)
2020-11-24T00:16:28 [I|app|c9c1f7d2] Completed 200 OK in 619ms (Views: 534.4ms | ActiveRecord: 74.9ms | Allocations: 238820)
2020-11-24T00:16:30 [I|app|7fba4777] Started GET "/katello/api/v2/environments/6/activation_keys?content_view_id=15" for 192.168.255.11 at 2020-11-24 00:16:30 +0100
2020-11-24T00:16:30 [I|app|7fba4777] Processing by Katello::Api::V2::ActivationKeysController#index as */*
2020-11-24T00:16:30 [I|app|7fba4777]   Parameters: {"content_view_id"=>"15", "api_version"=>"v2", "environment_id"=>"6"}
2020-11-24T00:16:30 [I|app|7fba4777]   Rendering /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.17.0/app/views/katello/api/v2/activation_keys/index.json.rabl within katello/api/v2/layouts/collection
2020-11-24T00:16:30 [I|app|7fba4777]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.17.0/app/views/katello/api/v2/activation_keys/index.json.rabl within katello/api/v2/layouts/collection (Duration: 28.4ms | Allocations: 8932)
2020-11-24T00:16:30 [I|app|7fba4777] Completed 200 OK in 44ms (Views: 19.9ms | ActiveRecord: 14.6ms | Allocations: 14200)
2020-11-24T00:16:32 [I|app|fc4442a7] Started POST "/hostgroups/os_selected" for 192.168.255.11 at 2020-11-24 00:16:32 +0100
2020-11-24T00:16:32 [I|app|fc4442a7] Processing by HostgroupsController#os_selected as */*
2020-11-24T00:16:32 [I|app|fc4442a7]   Parameters: {"hostgroup"=>{"operatingsystem_id"=>"2", "lifecycle_environment_id"=>"6", "content_view_id"=>"15", "environment_id"=>"6", "content_source_id"=>"1", "architecture_id"=>"1"}}
2020-11-24T00:16:32 [I|app|fc4442a7]   Rendered common/os_selection/_pxe_loader.html.erb (Duration: 4.8ms | Allocations: 6378)
2020-11-24T00:16:32 [I|app|fc4442a7]   Rendered common/os_selection/_operatingsystem.html.erb (Duration: 103.5ms | Allocations: 45031)
2020-11-24T00:16:32 [I|app|fc4442a7] Completed 200 OK in 140ms (Views: 93.0ms | ActiveRecord: 17.8ms | Allocations: 56858)
2020-11-24T00:16:32 [I|app|2a3493f2] Started POST "/hostgroups/puppetclass_parameters" for 192.168.255.11 at 2020-11-24 00:16:32 +0100
2020-11-24T00:16:32 [I|app|2a3493f2] Processing by HostgroupsController#puppetclass_parameters as */*
2020-11-24T00:16:32 [I|app|2a3493f2]   Parameters: {"utf8"=>"✓", "authenticity_token"=>"IFRKX/u4g343LEebp9HF6ob6WW0j9CrWwPEsQtLLeN+Ss2ttymLSEkm/3fQpUUTHqD0oKpplt+NPf/SJ2wM/gQ==", "hostgroup"=>{"parent_id"=>"", "name"=>"CentOS7-Base-Acceptatie", "description"=>"Managed by Ansible, your changes will be lost", "lifecycle_environment_id"=>"6", "content_view_id"=>"15", "content_source_id"=>"1", "environment_id"=>"", "compute_resource_id"=>"", "puppet_proxy_id"=>"", "puppet_ca_proxy_id"=>"", "domain_id"=>"", "realm_id"=>"", "architecture_id"=>"1", "operatingsystem_id"=>"2", "medium_id"=>"", "kickstart_repository_id"=>"147", "ptable_id"=>"", "pxe_loader"=>"PXELinux BIOS", "group_parameters_attributes"=>{"0"=>{"name"=>"autopart_options", "parameter_type"=>"string", "value"=>"[FILTERED]", "hidden_value"=>"[FILTERED]", "_destroy"=>"false", "id"=>"84"}, "1"=>{"name"=>"enable-epel", "parameter_type"=>"boolean", "value"=>"[FILTERED]", "hidden_value"=>"[FILTERED]", "_destroy"=>"false", "id"=>"89"}, "2"=>{"name"=>"kt_activation_keys", "parameter_type"=>"string", "value"=>"[FILTERED]", "hidden_value"=>"[FILTERED]", "_destroy"=>"0", "id"=>"83"}, "3"=>{"name"=>"remote_execution_create_user", "parameter_type"=>"boolean", "value"=>"[FILTERED]", "hidden_value"=>"[FILTERED]", "_destroy"=>"false", "id"=>"85"}, "4"=>{"name"=>"remote_execution_effective_user_method", "parameter_type"=>"string", "value"=>"[FILTERED]", "hidden_value"=>"[FILTERED]", "_destroy"=>"false", "id"=>"87"}, "5"=>{"name"=>"remote_execution_ssh_keys", "parameter_type"=>"string", "value"=>"[FILTERED]", "hidden_value"=>"[FILTERED]", "_destroy"=>"false", "id"=>"88"}, "6"=>{"name"=>"remote_execution_ssh_user", "parameter_type"=>"string", "value"=>"[FILTERED]", "hidden_value"=>"[FILTERED]", "_destroy"=>"false", "id"=>"86"}}, "location_ids"=>["", "4", "5"], "organization_ids"=>["", "3"], "id"=>"11"}, "media_selector"=>"synced_content", "fakepassword"=>"[FILTERED]", "hostgroup_id"=>"11"}
2020-11-24T00:16:32 [I|app|2a3493f2]   Rendered collection of puppetclasses/_class_parameters.html.erb [0 times] (Duration: 0.0ms | Allocations: 3)
2020-11-24T00:16:32 [I|app|2a3493f2]   Rendered puppetclasses/_classes_parameters.html.erb (Duration: 14.0ms | Allocations: 8724)
2020-11-24T00:16:32 [I|app|2a3493f2] Completed 200 OK in 42ms (Views: 11.5ms | ActiveRecord: 8.5ms | Allocations: 19751)
2020-11-24T00:16:34 [I|app|ae4664af] Started PATCH "/hostgroups/11-CentOS7-Base-Acceptatie" for 192.168.255.11 at 2020-11-24 00:16:34 +0100
2020-11-24T00:16:34 [I|app|ae4664af] Processing by HostgroupsController#update as HTML
2020-11-24T00:16:34 [I|app|ae4664af]   Parameters: {"utf8"=>"✓", "authenticity_token"=>"IFRKX/u4g343LEebp9HF6ob6WW0j9CrWwPEsQtLLeN+Ss2ttymLSEkm/3fQpUUTHqD0oKpplt+NPf/SJ2wM/gQ==", "hostgroup"=>{"parent_id"=>"", "name"=>"CentOS7-Base-Acceptatie", "description"=>"Managed by Ansible, your changes will be lost", "lifecycle_environment_id"=>"6", "content_view_id"=>"15", "content_source_id"=>"1", "environment_id"=>"", "compute_resource_id"=>"", "puppet_proxy_id"=>"", "puppet_ca_proxy_id"=>"", "domain_id"=>"", "realm_id"=>"", "architecture_id"=>"1", "operatingsystem_id"=>"2", "medium_id"=>"", "kickstart_repository_id"=>"147", "ptable_id"=>"", "pxe_loader"=>"PXELinux BIOS", "group_parameters_attributes"=>{"0"=>{"name"=>"autopart_options", "parameter_type"=>"string", "value"=>"[FILTERED]", "hidden_value"=>"[FILTERED]", "_destroy"=>"false", "id"=>"84"}, "1"=>{"name"=>"enable-epel", "parameter_type"=>"boolean", "value"=>"[FILTERED]", "hidden_value"=>"[FILTERED]", "_destroy"=>"false", "id"=>"89"}, "2"=>{"name"=>"kt_activation_keys", "parameter_type"=>"string", "value"=>"[FILTERED]", "hidden_value"=>"[FILTERED]", "_destroy"=>"0", "id"=>"83"}, "3"=>{"name"=>"remote_execution_create_user", "parameter_type"=>"boolean", "value"=>"[FILTERED]", "hidden_value"=>"[FILTERED]", "_destroy"=>"false", "id"=>"85"}, "4"=>{"name"=>"remote_execution_effective_user_method", "parameter_type"=>"string", "value"=>"[FILTERED]", "hidden_value"=>"[FILTERED]", "_destroy"=>"false", "id"=>"87"}, "5"=>{"name"=>"remote_execution_ssh_keys", "parameter_type"=>"string", "value"=>"[FILTERED]", "hidden_value"=>"[FILTERED]", "_destroy"=>"false", "id"=>"88"}, "6"=>{"name"=>"remote_execution_ssh_user", "parameter_type"=>"string", "value"=>"[FILTERED]", "hidden_value"=>"[FILTERED]", "_destroy"=>"false", "id"=>"86"}}, "location_ids"=>["", "4", "5"], "organization_ids"=>["", "3"], "id"=>"11"}, "media_selector"=>"synced_content", "commit"=>"Submit", "id"=>"11-CentOS7-Base-Acceptatie"}
2020-11-24T00:16:34 [E|app|ae4664af] Failed to save: Base The selected kickstart repository is not part of the assigned content view, lifecycle environment,
 ae4664af |                   content source, operating system, and architecture
2020-11-24T00:16:34 [I|app|ae4664af]   Rendering hostgroups/edit.html.erb within layouts/application
2020-11-24T00:16:34 [I|app|ae4664af]   Rendered common/os_selection/_architecture.html.erb (Duration: 8.8ms | Allocations: 7179)
2020-11-24T00:16:34 [I|app|ae4664af]   Rendered common/os_selection/_pxe_loader.html.erb (Duration: 5.8ms | Allocations: 6376)
2020-11-24T00:16:34 [I|app|ae4664af]   Rendered common/os_selection/_operatingsystem.html.erb (Duration: 94.2ms | Allocations: 43351)
2020-11-24T00:16:34 [I|app|ae4664af]   Rendered common/os_selection/_initial.html.erb (Duration: 110.1ms | Allocations: 56962)
2020-11-24T00:16:34 [I|app|ae4664af]   Rendered collection of puppetclasses/_class_parameters.html.erb [0 times] (Duration: 0.0ms | Allocations: 3)
... (had to cut some lines)
2020-11-24T00:16:34 [I|app|ae4664af]   Rendered common_parameters/_parameters.html.erb (Duration: 55.6ms | Allocations: 62934)
2020-11-24T00:16:34 [I|app|ae4664af]   Rendered taxonomies/_loc_org_tabs.html.erb (Duration: 26.3ms | Allocations: 14376)
2020-11-24T00:16:34 [I|app|ae4664af]   Rendered hostgroups/_form.html.erb (Duration: 318.3ms | Allocations: 177997)
2020-11-24T00:16:34 [I|app|ae4664af]   Rendered hostgroups/edit.html.erb within layouts/application (Duration: 323.5ms | Allocations: 183791)
2020-11-24T00:16:34 [I|app|ae4664af]   Rendered layouts/_application_content.html.erb (Duration: 4.4ms | Allocations: 6025)
2020-11-24T00:16:34 [I|app|ae4664af]   Rendering layouts/base.html.erb
2020-11-24T00:16:34 [I|app|ae4664af]   Rendered layouts/base.html.erb (Duration: 22.9ms | Allocations: 25501)
2020-11-24T00:16:34 [I|app|ae4664af] Completed 200 OK in 433ms (Views: 324.0ms | ActiveRecord: 47.9ms | Allocations: 248882)
2020-11-24T00:16:36 [I|app|d6617385] Started GET "/notification_recipients" for 192.168.255.11 at 2020-11-24 00:16:36 +0100
2020-11-24T00:16:36 [I|app|d6617385] Processing by NotificationRecipientsController#index as JSON
2020-11-24T00:16:36 [I|app|d6617385] Completed 200 OK in 9ms (Views: 0.2ms | ActiveRecord: 1.2ms | Allocations: 2327)
9

From what I can find in the logs, no other errors except

2020-11-24T00:16:34 [E|app|ae4664af] Failed to save: Base The selected kickstart repository is not part of the assigned content view, lifecycle environment,
 ae4664af |                   content source, operating system, and architecture

Which is also what Ansible cited earlier. But it doesn’t make sense, this setup works fine on 2.1. And the Kickstart repo does exist, but for some reason any existing group seems broken.

As well as that Ansible is not able to create new groups using the theforeman.foreman.hostgroup module.

10

What is the CV and LCE you’re trying to assign to the hostgroup and is the kickstart repo available in those?

11

Below are all the settings as I want them to be:

[root@foreman ~]# hammer hostgroup show --id 11
Id:                    11
Name:                  CentOS7-Base-Acceptatie
Title:                 CentOS7-Base-Acceptatie
Description:           
  Managed by Ansible, your changes will be lost
Network:               
    Subnet ipv4: Acceptatie
    Domain:      lacc.htm.lan
Operating system:      
    Architecture:     x86_64
    Operating System: CentOS 7
    Partition Table:  Kickstart default first disk only
    PXE Loader:       PXELinux BIOS
Puppetclasses:         

Parameters:            
    autopart_options => --nohome
    enable-epel => false
    kt_activation_keys => CentOS7-Base-Acceptatie
    remote_execution_create_user => true
    remote_execution_effective_user_method => sudo
    remote_execution_ssh_keys => xxx
Locations:             
    DC-A
    DC-B
Organizations:         
    HTM
Content View:          
    ID:   15
    Name: COV CentOS7-Base
Lifecycle Environment: 
    ID:   6
    Name: Acceptatie
Content Source:        
    ID:   1
    Name: foreman.lbhr.htm.lan
Kickstart Repository:  
    ID:   147
    Name: 147

The above came from the original vm, which is still running 2.1. After upgrading it looks like this (the output below came from the upgraded clone):

[root@foreman ~]# hammer hostgroup show --id 11
Id:                    11
Name:                  CentOS7-Base-Acceptatie
Title:                 CentOS7-Base-Acceptatie
Description:           
  Managed by Ansible, your changes will be lost
Network:               
    Subnet ipv4: Acceptatie
    Domain:      lacc.htm.lan
Operating system:      
    Architecture:     x86_64
    Operating System: CentOS 7
    Partition Table:  Kickstart default first disk only
    PXE Loader:       PXELinux BIOS
Puppetclasses:         

Parameters:            
    autopart_options => --nohome
    enable-epel => false
    kt_activation_keys => CentOS7-Base-Acceptatie
    remote_execution_create_user => true
    remote_execution_effective_user_method => sudo
    remote_execution_ssh_keys => xxx
Locations:             
    DC-A
    DC-B
Organizations:         
    HTM
Content View:          
    ID:   
    Name:
Lifecycle Environment: 
    ID:   
    Name:
Content Source:        
    ID:   
    Name:
Kickstart Repository:  
    ID:
12

So the upgrade includes https://github.com/Katello/katello/commit/66b1f85c527dd6cee32d190fda2eb5a0795caed6 which means on upgrade these fields on the hostgroup got moved to the hostgroup content facet table.
Are you able to do everything as expected through the UI. I have a feeling some API changes might be needed for this to work with ansible modules.

14

Well, that’s the thing, I can partially do so in the UI.

  1. Creating a new group works
  2. Pre-existing groups from 2.1 are breaking with regards to LCE and CV/CS.

Did some schema/db conversion go haywire? I know my way around Foreman/Katello, but this goes a bit deeper than I know :slight_smile:

I dug up some info from the DB (this is all from a lab VM):

From the hostgroups table in Katello 3.16:

id,name,created_at,updated_at,environment_id,operatingsystem_id,architecture_id,medium_id,ptable_id,root_pass,puppet_ca_proxy_id,use_image,image_file,ancestry,vm_defaults,subnet_id,domain_id,puppet_proxy_id,title,realm_id,compute_profile_id,content_source_id,grub_pass,content_view_id,lifecycle_environment_id,lookup_value_matcher,kickstart_repository_id,subnet6_id,pxe_loader,description,compute_resource_id
11,CentOS7-Base-Acceptatie,2020-11-20 19:39:32.886917,2020-11-21 10:12:18.656595,,2,1,,136,$6$xxxxx.,,,,,,4,4,,CentOS7-Base-Acceptatie,,,1,$6$xxxxx,15,6,hostgroup=CentOS7-Base-Acceptatie,147,,PXELinux BIOS,"Managed by Ansible, your changes will be lost",

Same group and the newly created “test” group, Katello 3.17:

id,name,created_at,updated_at,environment_id,operatingsystem_id,architecture_id,medium_id,ptable_id,root_pass,puppet_ca_proxy_id,use_image,image_file,ancestry,vm_defaults,subnet_id,domain_id,puppet_proxy_id,title,realm_id,compute_profile_id,grub_pass,lookup_value_matcher,subnet6_id,pxe_loader,description,compute_resource_id
11,CentOS7-Base-Acceptatie,2020-11-20 19:39:32.886917,2020-11-21 10:12:18.656595,,2,1,,136,$6$xxx.,,,,,,4,4,,CentOS7-Base-Acceptatie,,,$6$xxx,hostgroup=CentOS7-Base-Acceptatie,,PXELinux BIOS,"Managed by Ansible, your changes will be lost",
36,test,2020-11-23 23:12:08.968592,2020-11-23 23:25:33.09145,,2,1,,136,"",,,,,,,,,test,,,"",hostgroup=test,,PXELinux BIOS,"",

katello_hostgroup_content_facets table (complete table, just has 1 line, which might have to do with the fact I created a complete new group that did work)

id,hostgroup_id,kickstart_repository_id,content_source_id,content_view_id,lifecycle_environment_id
1,36,147,1,15,6
15

This looks like a migration issue. You should ideally have a katello_hostgroup_content_facet record for the old hostgroup upon upgrade. I have raised Bug #31382: Old hostgroup CV,LCE,CE aren't migrated to the new hostgroup content facet table. - Katello - Foreman to get this fixed.

Could you confirm in the installer logs that the upgrade went through without any issues while migrating?

1 Like
16

Ok, here’s what I did:

  1. Reverted 2.1 snapshot on my VM
  2. Removed any foreman/katello related logging I could find
  3. Ran upgrade
  4. I’ve uploaded katello.log and my terminal output here: https://gist.github.com/Thulium-Drake/4724ab62178cc68672d76ce4329ee8c2
17

I did some quick grepping on the log for Host groups and ContentFacets, I see the following lines:

[DEBUG 2020-11-25T18:24:55 main]  /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/unless:   20180904122343 CreateHostgroupContentFacet
[DEBUG 2020-11-25T18:24:55 main]  /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/unless:   20200514092553 MoveKatelloFieldsFromHostgroups

But nothing that actually moves stuff over.

And the only ERROR output is stuff about foreman-maintain packages is-locked --assumeyes failed!

18

@sajha any update? Do you have all the information you need? Or is there anything else I can test/dig up/look up? :slight_smile:

19

@katello ping, do you guys have all information you need? The bug doesn’t seem to have an update (except a bump to 3.17.2)

20

It’s on our current sprint so someone will be diving into it in the coming weeks!

1 Like
21

Awesome, thanks for the update! If you need anything, let me know! :wink: