Monitor/Jobs/Overview/Host detail Takes 30-40 minutes to render screen with job output, after job has been successfully completed

Problem:
After a large job (Run Ansible Roles) with over 14,000 lines of output completes, when we try to view the Host details, it takes 30 - 40 minutes to render the screen. In Red Hat Satellite 6.9.x this only took about a minute, but following an upgrade we began to experience the issue.

Issue is the same for RHEL7 and RHEL8 hosts.

Expected outcome:
Even for jobs with large output, it should only take 2 - 3 minutes to render the screen.

Foreman and Proxy versions:
Red Hat Satellite 6.11.4 (Foreman 3.1.1.25)
Foreman and Proxy plugin versions:
ansiblerole-foreman_scap_client-0.2.0-2.el7sat.noarch
foreman-3.1.1.25-1.el7sat.noarch
foreman-bootloaders-redhat-202005201200-1.el7sat.noarch
foreman-bootloaders-redhat-tftpboot-202005201200-1.el7sat.noarch
foreman-cli-3.1.1.25-1.el7sat.noarch
foreman-debug-3.1.1.25-1.el7sat.noarch
foreman-dynflow-sidekiq-3.1.1.25-1.el7sat.noarch
foreman-ec2-3.1.1.25-1.el7sat.noarch
foreman-gce-3.1.1.25-1.el7sat.noarch
foreman-installer-3.1.2.8-1.el7sat.noarch
foreman-installer-katello-3.1.2.8-1.el7sat.noarch
foreman-libvirt-3.1.1.25-1.el7sat.noarch
foreman-openstack-3.1.1.25-1.el7sat.noarch
foreman-ovirt-3.1.1.25-1.el7sat.noarch
foreman-postgresql-3.1.1.25-1.el7sat.noarch
foreman-proxy-3.1.1.3-1.el7sat.noarch
foreman-selinux-3.1.2.1-2.el7sat.noarch
foreman-service-3.1.1.25-1.el7sat.noarch
foreman-vmware-3.1.1.25-1.el7sat.noarch
oitetnmulv112.vha.med.va.gov-foreman-client-1.0-1.noarch
oitetnmulv112.vha.med.va.gov-foreman-proxy-1.0-3.noarch
oitetnmulv112.vha.med.va.gov-foreman-proxy-client-1.0-1.noarch
puppet-foreman_scap_client-0.4.0-1.el7sat.noarch
rubygem-foreman_maintain-1.0.18-1.el7sat.noarch
tfm-rubygem-foreman_ansible-7.0.4.1-1.el7sat.noarch
tfm-rubygem-foreman_ansible_core-4.2.0-1.el7sat.noarch
tfm-rubygem-foreman_azure_rm-2.2.6-1.el7sat.noarch
tfm-rubygem-foreman_bootdisk-19.0.4.1-1.el7sat.noarch
tfm-rubygem-foreman_discovery-19.0.5-1.el7sat.noarch
tfm-rubygem-foreman_hooks-0.3.17-2.el7sat.noarch
tfm-rubygem-foreman_openscap-5.1.1-1.el7sat.noarch
tfm-rubygem-foreman_puppet-2.0.6-1.el7sat.noarch
tfm-rubygem-foreman_remote_execution-5.0.8-1.el7sat.noarch
tfm-rubygem-foreman_remote_execution_core-1.4.8-1.el7sat.noarch
tfm-rubygem-foreman_rh_cloud-5.0.41-1.el7sat.noarch
tfm-rubygem-foreman-tasks-5.2.3-1.el7sat.noarch
tfm-rubygem-foreman-tasks-core-0.3.6-1.el7sat.noarch
tfm-rubygem-foreman_templates-9.1.0-1.el7sat.noarch
tfm-rubygem-foreman_theme_satellite-9.0.0.11-1.el7sat.noarch
tfm-rubygem-foreman_virt_who_configure-0.5.9-1.el7sat.noarch
tfm-rubygem-hammer_cli_foreman-3.1.0.1-1.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_admin-1.1.0-1.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_ansible-0.3.4-1.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_azure_rm-0.2.2-1.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_bootdisk-0.3.0-2.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_discovery-1.1.0-1.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_openscap-0.1.13-1.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_puppet-0.0.5-1.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_remote_execution-0.2.2-1.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_tasks-0.0.17-1.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_templates-0.2.0-2.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_virt_who_configure-0.0.9-1.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_webhooks-0.0.2-1.el7sat.noarch
Distribution and version:
RH Satellite is running on RHEL7.9

Other relevant data:

This is a known issue Bug #33674: Viewing Remote Execution job invocation output scales very badly in the WebUI as output grows - Foreman Remote Execution - Foreman . There doesn’t seem to be an easy way out. The whole page will get rewritten eventually which should hopefully resolve this issue, but we’re not there yet.

Until then, the api (and hammer) doesn’t suffer from this. When looking into this some time ago, I also had some luck with having noscript enabled in my browser, but that obviously breaks almost everything else.

1 Like

This is just the information I needed. I can use Hammer to view the results of these large Remote Execution jobs.

I encounter the issue running the Ansible Roles RedHatOfficial.rhel7_stig and RedHatOfficial.rhel8_stig, which can produce 12,000 - 14,000 lines of output.

Thank you,