Importing manifest taking for ever

Running Status Services

Get status of applicable services:

Displaying the following service(s):
rh-redis5-redis, postgresql, pulpcore-api, pulpcore-content, qdrouterd, qpidd, rh-redis5-redis, pulpcore-worker@1.service, pulpcore-worker@2.service, pulpcore-worker@3.service, pulpcore-worker@4.service, pulpcore-worker@5.service, pulpcore-worker@6.service, tomcat, dynflow-sidekiq@orchestrator, foreman, httpd, puppetserver, dynflow-sidekiq@worker-1, dynflow-sidekiq@worker-hosts-queue-1, foreman-proxy

  • displaying rh-redis5-redis
    ● rh-redis5-redis.service - Redis persistent key-value database
    Loaded: loaded (/usr/lib/systemd/system/rh-redis5-redis.service; enabled; vendor preset: disabled)
    Drop-In: /etc/systemd/system/rh-redis5-redis.service.d
    └─90-limits.conf
    Active: active (running) since Tue 2021-09-21 15:50:27 CDT; 22min ago
    Main PID: 1424 (redis-server)
    Tasks: 4
    CGroup: /system.slice/rh-redis5-redis.service
    └─1424 /opt/rh/rh-redis5/root/usr/bin/redis-server 127.0.0.1:6379

Sep 21 15:50:27 hostname systemd[1]: Starting Redis persistent key-value database…
Sep 21 15:50:27 hostname systemd[1]: Started Redis persistent key-value database.
\ displaying postgresql
● rh-postgresql12-postgresql.service - PostgreSQL database server
Loaded: loaded (/usr/lib/systemd/system/rh-postgresql12-postgresql.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2021-09-21 15:50:27 CDT; 22min ago
Process: 1376 ExecStartPre=/opt/rh/rh-postgresql12/root/usr/libexec/postgresql-check-db-dir %N (code=exited, status=0/SUCCESS)
Main PID: 1453 (postmaster)
Tasks: 86
CGroup: /system.slice/rh-postgresql12-postgresql.service
β”œβ”€1453 postmaster -D /var/opt/rh/rh-postgresql12/lib/pgsql/data
β”œβ”€1665 postgres: logger
β”œβ”€1703 postgres: checkpointer
β”œβ”€1704 postgres: background writer
β”œβ”€1705 postgres: walwriter
β”œβ”€1706 postgres: autovacuum launcher
β”œβ”€1707 postgres: stats collector
β”œβ”€1708 postgres: logical replication launcher
β”œβ”€2548 postgres: pulp pulpcore 127.0.0.1(59552) idle
β”œβ”€2549 postgres: pulp pulpcore 127.0.0.1(59554) idle
β”œβ”€2550 postgres: pulp pulpcore 127.0.0.1(59556) idle
β”œβ”€2551 postgres: pulp pulpcore 127.0.0.1(59558) idle
β”œβ”€2555 postgres: pulp pulpcore 127.0.0.1(59560) idle
β”œβ”€2559 postgres: pulp pulpcore 127.0.0.1(59562) idle
β”œβ”€2797 postgres: pulp pulpcore 127.0.0.1(59564) idle
β”œβ”€2798 postgres: pulp pulpcore 127.0.0.1(59566) idle
β”œβ”€2799 postgres: pulp pulpcore 127.0.0.1(59568) idle
β”œβ”€2800 postgres: pulp pulpcore 127.0.0.1(59570) idle
β”œβ”€2801 postgres: pulp pulpcore 127.0.0.1(59572) idle
β”œβ”€2805 postgres: pulp pulpcore 127.0.0.1(59574) idle
β”œβ”€2815 postgres: pulp pulpcore 127.0.0.1(59576) idle
β”œβ”€2820 postgres: pulp pulpcore 127.0.0.1(59578) idle
β”œβ”€2821 postgres: pulp pulpcore 127.0.0.1(59580) idle
β”œβ”€2824 postgres: pulp pulpcore 127.0.0.1(59582) idle
β”œβ”€2825 postgres: pulp pulpcore 127.0.0.1(59584) idle
β”œβ”€2829 postgres: pulp pulpcore 127.0.0.1(59586) idle
β”œβ”€2832 postgres: pulp pulpcore 127.0.0.1(59588) idle
β”œβ”€2965 postgres: pulp pulpcore 127.0.0.1(59598) idle
β”œβ”€2966 postgres: pulp pulpcore 127.0.0.1(59600) idle
β”œβ”€2967 postgres: pulp pulpcore 127.0.0.1(59602) idle
β”œβ”€2969 postgres: pulp pulpcore 127.0.0.1(59604) idle
β”œβ”€2972 postgres: pulp pulpcore 127.0.0.1(59606) idle
β”œβ”€2974 postgres: pulp pulpcore 127.0.0.1(59608) idle
β”œβ”€2976 postgres: pulp pulpcore 127.0.0.1(59610) idle
β”œβ”€2977 postgres: pulp pulpcore 127.0.0.1(59612) idle
β”œβ”€2978 postgres: pulp pulpcore 127.0.0.1(59614) idle
β”œβ”€2979 postgres: pulp pulpcore 127.0.0.1(59616) idle
β”œβ”€3019 postgres: pulp pulpcore 127.0.0.1(59626) idle
β”œβ”€3022 postgres: pulp pulpcore 127.0.0.1(59630) idle
β”œβ”€3024 postgres: pulp pulpcore 127.0.0.1(59632) idle
β”œβ”€3111 postgres: foreman foreman [local] idle
β”œβ”€3208 postgres: foreman foreman [local] idle
β”œβ”€3209 postgres: foreman foreman [local] idle
β”œβ”€3210 postgres: foreman foreman [local] idle
β”œβ”€3211 postgres: foreman foreman [local] idle
β”œβ”€3212 postgres: foreman foreman [local] idle
β”œβ”€3215 postgres: foreman foreman [local] idle
β”œβ”€3218 postgres: foreman foreman [local] idle
β”œβ”€3220 postgres: foreman foreman [local] idle
β”œβ”€3221 postgres: foreman foreman [local] idle
β”œβ”€3222 postgres: foreman foreman [local] idle
β”œβ”€3224 postgres: foreman foreman [local] idle
β”œβ”€3226 postgres: foreman foreman [local] idle
β”œβ”€3231 postgres: foreman foreman [local] idle
β”œβ”€3234 postgres: foreman foreman [local] idle
β”œβ”€3235 postgres: foreman foreman [local] idle
β”œβ”€3236 postgres: foreman foreman [local] idle
β”œβ”€3241 postgres: foreman foreman [local] idle
β”œβ”€3243 postgres: foreman foreman [local] idle
β”œβ”€3245 postgres: foreman foreman [local] idle
β”œβ”€3249 postgres: foreman foreman [local] idle
β”œβ”€3251 postgres: foreman foreman [local] idle
β”œβ”€3258 postgres: foreman foreman [local] idle
β”œβ”€3268 postgres: foreman foreman [local] idle
β”œβ”€3318 postgres: foreman foreman [local] idle
β”œβ”€3326 postgres: foreman foreman [local] idle
β”œβ”€3327 postgres: foreman foreman [local] idle
β”œβ”€3328 postgres: foreman foreman [local] idle
β”œβ”€3357 postgres: foreman foreman [local] idle
β”œβ”€3361 postgres: foreman foreman [local] idle
β”œβ”€3362 postgres: foreman foreman [local] idle
β”œβ”€3377 postgres: foreman foreman [local] idle
β”œβ”€3381 postgres: foreman foreman [local] idle
β”œβ”€3382 postgres: foreman foreman [local] idle
β”œβ”€3393 postgres: foreman foreman [local] idle
β”œβ”€3398 postgres: foreman foreman [local] idle
β”œβ”€3399 postgres: foreman foreman [local] idle
β”œβ”€3542 postgres: foreman foreman [local] idle
β”œβ”€4744 postgres: candlepin candlepin 127.0.0.1(59983) idle
β”œβ”€4745 postgres: candlepin candlepin 127.0.0.1(59982) idle
β”œβ”€4746 postgres: candlepin candlepin 127.0.0.1(59986) idle
β”œβ”€4929 postgres: candlepin candlepin 127.0.0.1(60134) idle
β”œβ”€4930 postgres: candlepin candlepin 127.0.0.1(60136) idle
β”œβ”€4931 postgres: candlepin candlepin 127.0.0.1(60138) idle
β”œβ”€4932 postgres: candlepin candlepin 127.0.0.1(60140) idle
└─4933 postgres: candlepin candlepin 127.0.0.1(60142) idle

Sep 21 15:50:27 hostname systemd[1]: Starting PostgreSQL database server…
Sep 21 15:50:27 hostname sh[1453]: 2021-09-21 15:50:27 CDT LOG: starting PostgreSQL 12.7 on x86_64-redhat-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44), 64-bit
Sep 21 15:50:27 hostname sh[1453]: 2021-09-21 15:50:27 CDT LOG: listening on IPv4 address β€œ127.0.0.1”, port 5432
Sep 21 15:50:27 hostname sh[1453]: 2021-09-21 15:50:27 CDT LOG: listening on Unix socket β€œ/var/run/postgresql/.s.PGSQL.5432”
Sep 21 15:50:27 hostname sh[1453]: 2021-09-21 15:50:27 CDT LOG: listening on Unix socket β€œ/tmp/.s.PGSQL.5432”
Sep 21 15:50:27 hostname sh[1453]: 2021-09-21 15:50:27 CDT LOG: redirecting log output to logging collector process
Sep 21 15:50:27 hostname sh[1453]: 2021-09-21 15:50:27 CDT HINT: Future log output will appear in directory β€œlog”.
Sep 21 15:50:27 hostname systemd[1]: Started PostgreSQL database server.
Warning: rh-postgresql12-postgresql.service changed on disk. Run β€˜systemctl daemon-reload’ to reload units.
\ displaying pulpcore-api
● pulpcore-api.service - Pulp API Server
Loaded: loaded (/etc/systemd/system/pulpcore-api.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2021-09-21 15:50:29 CDT; 22min ago
Main PID: 1396 (gunicorn)
Status: β€œGunicorn arbiter booted”
Tasks: 2
CGroup: /system.slice/pulpcore-api.service
β”œβ”€1396 /usr/bin/python3 /usr/bin/gunicorn pulpcore.app.wsgi:application --timeout 90 -w 1 --access-logfile - --access-logformat pulp [%({correlation-id}o)s]: %(h)s %(l)s %(u)s %(t)s β€œ%(r)s” %(s)s %(b)s β€œ%(f)s” β€œ%(a)s”
└─1903 /usr/bin/python3 /usr/bin/gunicorn pulpcore.app.wsgi:application --timeout 90 -w 1 --access-logfile - --access-logformat pulp [%({correlation-id}o)s]: %(h)s %(l)s %(u)s %(t)s β€œ%(r)s” %(s)s %(b)s β€œ%(f)s” β€œ%(a)s”

Sep 21 15:50:27 hostname systemd[1]: Starting Pulp API Server…
Sep 21 15:50:29 hostname pulpcore-api[1396]: [2021-09-21 15:50:29 -0500] [1396] [INFO] Starting gunicorn 20.1.0
Sep 21 15:50:29 hostname pulpcore-api[1396]: [2021-09-21 15:50:29 -0500] [1396] [INFO] Listening at: unix:/run/pulpcore-api.sock (1396)
Sep 21 15:50:29 hostname pulpcore-api[1396]: [2021-09-21 15:50:29 -0500] [1396] [INFO] Using worker: sync
Sep 21 15:50:29 hostname systemd[1]: Started Pulp API Server.
Sep 21 15:50:29 hostname pulpcore-api[1396]: [2021-09-21 15:50:29 -0500] [1903] [INFO] Booting worker with pid: 1903
\ displaying pulpcore-content
● pulpcore-content.service - Pulp Content App
Loaded: loaded (/etc/systemd/system/pulpcore-content.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2021-09-21 15:50:30 CDT; 21min ago
Main PID: 1443 (gunicorn)
Status: β€œGunicorn arbiter booted”
Tasks: 40
CGroup: /system.slice/pulpcore-content.service
β”œβ”€1443 /usr/bin/python3 /usr/bin/gunicorn pulpcore.content:server --timeout 90 --worker-class aiohttp.GunicornWebWorker -w 13 --access-logfile -
β”œβ”€2061 /usr/bin/python3 /usr/bin/gunicorn pulpcore.content:server --timeout 90 --worker-class aiohttp.GunicornWebWorker -w 13 --access-logfile -
β”œβ”€2062 /usr/bin/python3 /usr/bin/gunicorn pulpcore.content:server --timeout 90 --worker-class aiohttp.GunicornWebWorker -w 13 --access-logfile -
β”œβ”€2066 /usr/bin/python3 /usr/bin/gunicorn pulpcore.content:server --timeout 90 --worker-class aiohttp.GunicornWebWorker -w 13 --access-logfile -
β”œβ”€2072 /usr/bin/python3 /usr/bin/gunicorn pulpcore.content:server --timeout 90 --worker-class aiohttp.GunicornWebWorker -w 13 --access-logfile -
β”œβ”€2078 /usr/bin/python3 /usr/bin/gunicorn pulpcore.content:server --timeout 90 --worker-class aiohttp.GunicornWebWorker -w 13 --access-logfile -
β”œβ”€2081 /usr/bin/python3 /usr/bin/gunicorn pulpcore.content:server --timeout 90 --worker-class aiohttp.GunicornWebWorker -w 13 --access-logfile -
β”œβ”€2086 /usr/bin/python3 /usr/bin/gunicorn pulpcore.content:server --timeout 90 --worker-class aiohttp.GunicornWebWorker -w 13 --access-logfile -
β”œβ”€2096 /usr/bin/python3 /usr/bin/gunicorn pulpcore.content:server --timeout 90 --worker-class aiohttp.GunicornWebWorker -w 13 --access-logfile -
β”œβ”€2105 /usr/bin/python3 /usr/bin/gunicorn pulpcore.content:server --timeout 90 --worker-class aiohttp.GunicornWebWorker -w 13 --access-logfile -
β”œβ”€2108 /usr/bin/python3 /usr/bin/gunicorn pulpcore.content:server --timeout 90 --worker-class aiohttp.GunicornWebWorker -w 13 --access-logfile -
β”œβ”€2109 /usr/bin/python3 /usr/bin/gunicorn pulpcore.content:server --timeout 90 --worker-class aiohttp.GunicornWebWorker -w 13 --access-logfile -
β”œβ”€2110 /usr/bin/python3 /usr/bin/gunicorn pulpcore.content:server --timeout 90 --worker-class aiohttp.GunicornWebWorker -w 13 --access-logfile -
└─2111 /usr/bin/python3 /usr/bin/gunicorn pulpcore.content:server --timeout 90 --worker-class aiohttp.GunicornWebWorker -w 13 --access-logfile -

Sep 21 15:50:30 hostname pulpcore-content[1443]: [2021-09-21 15:50:30 -0500] [2072] [INFO] Booting worker with pid: 2072
Sep 21 15:50:30 hostname pulpcore-content[1443]: [2021-09-21 15:50:30 -0500] [2078] [INFO] Booting worker with pid: 2078
Sep 21 15:50:30 hostname pulpcore-content[1443]: [2021-09-21 15:50:30 -0500] [2081] [INFO] Booting worker with pid: 2081
Sep 21 15:50:31 hostname pulpcore-content[1443]: [2021-09-21 15:50:31 -0500] [2086] [INFO] Booting worker with pid: 2086
Sep 21 15:50:31 hostname pulpcore-content[1443]: [2021-09-21 15:50:31 -0500] [2096] [INFO] Booting worker with pid: 2096
Sep 21 15:50:31 hostname pulpcore-content[1443]: [2021-09-21 15:50:31 -0500] [2105] [INFO] Booting worker with pid: 2105
Sep 21 15:50:31 hostname pulpcore-content[1443]: [2021-09-21 15:50:31 -0500] [2108] [INFO] Booting worker with pid: 2108
Sep 21 15:50:31 hostname pulpcore-content[1443]: [2021-09-21 15:50:31 -0500] [2109] [INFO] Booting worker with pid: 2109
Sep 21 15:50:31 hostname pulpcore-content[1443]: [2021-09-21 15:50:31 -0500] [2110] [INFO] Booting worker with pid: 2110
Sep 21 15:50:31 hostname pulpcore-content[1443]: [2021-09-21 15:50:31 -0500] [2111] [INFO] Booting worker with pid: 2111
\ displaying qdrouterd
● qdrouterd.service - Qpid Dispatch router daemon
Loaded: loaded (/usr/lib/systemd/system/qdrouterd.service; enabled; vendor preset: disabled)
Drop-In: /etc/systemd/system/qdrouterd.service.d
└─90-limits.conf
Active: active (running) since Tue 2021-09-21 15:50:27 CDT; 22min ago
Main PID: 1383 (qdrouterd)
Tasks: 7
CGroup: /system.slice/qdrouterd.service
└─1383 /usr/sbin/qdrouterd -c /etc/qpid-dispatch/qdrouterd.conf

Sep 21 15:50:28 hostname qdrouterd[1383]: 2021-09-21 15:50:28.210018 -0500 SERVER (notice) Process VmSize 279.91 MiB (31.26 GiB available memory)
Sep 21 15:50:28 hostname qdrouterd[1383]: 2021-09-21 15:50:28.218340 -0500 SERVER (notice) Listening on :5647
Sep 21 15:50:28 hostname qdrouterd[1383]: 2021-09-21 15:50:28.218499 -0500 SERVER (notice) Listening on :5646
Sep 21 15:50:28 hostname qdrouterd[1383]: 2021-09-21 15:50:28.242089 -0500 SERVER (info) [C1] Connection to localhost:5671 failed: proton:io Connection refused - disconnected localhost:5671
Sep 21 15:50:33 hostname qdrouterd[1383]: 2021-09-21 15:50:33.368790 -0500 ROUTER_CORE (info) [C2] Connection Opened: dir=out host=localhost:5671 vhost= encrypted=TLSv1/SSLv3 auth=EXTERNAL user=(null) container_id=989d5fb8-96d3-414a-804e-89403879a1e2 props={:product=β€œqpid-cpp”, :version=β€œ1.39.0”, :platform=β€œLinux”, :host=β€œhostname”}
Sep 21 15:50:33 hostname qdrouterd[1383]: 2021-09-21 15:50:33.369481 -0500 ROUTER_CORE (info) Link Route Activated β€˜linkRoute/0’ on connection broker
Sep 21 15:50:33 hostname qdrouterd[1383]: 2021-09-21 15:50:33.369512 -0500 ROUTER_CORE (info) Link Route Activated β€˜linkRoute/1’ on connection broker
Sep 21 15:50:33 hostname qdrouterd[1383]: 2021-09-21 15:50:33.369526 -0500 ROUTER_CORE (info) Link Route Activated β€˜linkRoute/2’ on connection broker
Sep 21 15:50:33 hostname qdrouterd[1383]: 2021-09-21 15:50:33.369537 -0500 ROUTER_CORE (info) Link Route Activated β€˜linkRoute/3’ on connection broker
Sep 21 15:50:33 hostname qdrouterd[1383]: 2021-09-21 15:50:33.369548 -0500 ROUTER_CORE (info) Link Route Activated β€˜linkRoute/4’ on connection broker
\ displaying qpidd
● qpidd.service - An AMQP message broker daemon.
Loaded: loaded (/usr/lib/systemd/system/qpidd.service; enabled; vendor preset: disabled)
Drop-In: /etc/systemd/system/qpidd.service.d
└─90-limits.conf, wait-for-port.conf
Active: active (running) since Tue 2021-09-21 15:50:30 CDT; 21min ago
Docs: man:qpidd(1)
http://qpid.apache.org/
Process: 1380 ExecStartPost=/bin/bash -c while ! ss --no-header --tcp --listening --numeric sport = :5671 | grep -q β€œ^LISTEN.*:5671”; do sleep 1; done (code=exited, status=0/SUCCESS)
Main PID: 1379 (qpidd)
Tasks: 8
CGroup: /system.slice/qpidd.service
└─1379 /usr/sbin/qpidd --config /etc/qpid/qpidd.conf

Sep 21 15:50:27 hostname systemd[1]: Starting An AMQP message broker daemon…
Sep 21 15:50:30 hostname systemd[1]: Started An AMQP message broker daemon…
\ displaying rh-redis5-redis
● rh-redis5-redis.service - Redis persistent key-value database
Loaded: loaded (/usr/lib/systemd/system/rh-redis5-redis.service; enabled; vendor preset: disabled)
Drop-In: /etc/systemd/system/rh-redis5-redis.service.d
└─90-limits.conf
Active: active (running) since Tue 2021-09-21 15:50:27 CDT; 22min ago
Main PID: 1424 (redis-server)
Tasks: 4
CGroup: /system.slice/rh-redis5-redis.service
└─1424 /opt/rh/rh-redis5/root/usr/bin/redis-server 127.0.0.1:6379

Sep 21 15:50:27 hostname systemd[1]: Starting Redis persistent key-value database…
Sep 21 15:50:27 hostname systemd[1]: Started Redis persistent key-value database.
\ displaying pulpcore-worker@1.service
● pulpcore-worker@1.service - Pulp Worker
Loaded: loaded (/etc/systemd/system/pulpcore-worker@.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2021-09-21 15:50:27 CDT; 22min ago
Main PID: 1417 (pulpcore-worker)
CGroup: /system.slice/system-pulpcore\x2dworker.slice/pulpcore-worker@1.service
└─1417 /usr/bin/python3 /usr/bin/pulpcore-worker

Sep 21 15:50:27 hostname systemd[1]: Started Pulp Worker.
Sep 21 15:50:49 hostname pulpcore-worker-1[1417]: pulp [None]: pulpcore.tasking.entrypoint:INFO: Starting distributed type worker
Sep 21 15:50:50 hostname pulpcore-worker-1[1417]: pulp [None]: pulpcore.tasking.worker_watcher:INFO: Worker β€˜1417@hostname’ is back online.
\ displaying pulpcore-worker@2.service
● pulpcore-worker@2.service - Pulp Worker
Loaded: loaded (/etc/systemd/system/pulpcore-worker@.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2021-09-21 15:50:27 CDT; 22min ago
Main PID: 1427 (pulpcore-worker)
CGroup: /system.slice/system-pulpcore\x2dworker.slice/pulpcore-worker@2.service
└─1427 /usr/bin/python3 /usr/bin/pulpcore-worker

Sep 21 15:50:27 hostname systemd[1]: Started Pulp Worker.
Sep 21 15:50:49 hostname pulpcore-worker-2[1427]: pulp [None]: pulpcore.tasking.entrypoint:INFO: Starting distributed type worker
Sep 21 15:50:50 hostname pulpcore-worker-2[1427]: pulp [None]: pulpcore.tasking.worker_watcher:INFO: New worker β€˜1427@hostname’ discovered
\ displaying pulpcore-worker@3.service
● pulpcore-worker@3.service - Pulp Worker
Loaded: loaded (/etc/systemd/system/pulpcore-worker@.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2021-09-21 15:50:27 CDT; 22min ago
Main PID: 1418 (pulpcore-worker)
CGroup: /system.slice/system-pulpcore\x2dworker.slice/pulpcore-worker@3.service
└─1418 /usr/bin/python3 /usr/bin/pulpcore-worker

Sep 21 15:50:27 hostname systemd[1]: Started Pulp Worker.
Sep 21 15:50:49 hostname pulpcore-worker-3[1418]: pulp [None]: pulpcore.tasking.entrypoint:INFO: Starting distributed type worker
Sep 21 15:50:50 hostname pulpcore-worker-3[1418]: pulp [None]: pulpcore.tasking.worker_watcher:INFO: Worker β€˜1418@hostname’ is back online.
\ displaying pulpcore-worker@4.service
● pulpcore-worker@4.service - Pulp Worker
Loaded: loaded (/etc/systemd/system/pulpcore-worker@.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2021-09-21 15:50:27 CDT; 22min ago
Main PID: 1422 (pulpcore-worker)
CGroup: /system.slice/system-pulpcore\x2dworker.slice/pulpcore-worker@4.service
└─1422 /usr/bin/python3 /usr/bin/pulpcore-worker

Sep 21 15:50:27 hostname systemd[1]: Started Pulp Worker.
Sep 21 15:50:49 hostname pulpcore-worker-4[1422]: pulp [None]: pulpcore.tasking.entrypoint:INFO: Starting distributed type worker
Sep 21 15:50:50 hostname pulpcore-worker-4[1422]: pulp [None]: pulpcore.tasking.worker_watcher:INFO: New worker β€˜1422@hostname’ discovered
\ displaying pulpcore-worker@5.service
● pulpcore-worker@5.service - Pulp Worker
Loaded: loaded (/etc/systemd/system/pulpcore-worker@.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2021-09-21 15:50:27 CDT; 22min ago
Main PID: 1426 (pulpcore-worker)
CGroup: /system.slice/system-pulpcore\x2dworker.slice/pulpcore-worker@5.service
└─1426 /usr/bin/python3 /usr/bin/pulpcore-worker

Sep 21 15:50:27 hostname systemd[1]: Started Pulp Worker.
Sep 21 15:50:49 hostname pulpcore-worker-5[1426]: pulp [None]: pulpcore.tasking.entrypoint:INFO: Starting distributed type worker
Sep 21 15:50:50 hostname pulpcore-worker-5[1426]: pulp [None]: pulpcore.tasking.worker_watcher:INFO: Worker β€˜1426@hostname’ is back online.
\ displaying pulpcore-worker@6.service
● pulpcore-worker@6.service - Pulp Worker
Loaded: loaded (/etc/systemd/system/pulpcore-worker@.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2021-09-21 15:50:27 CDT; 22min ago
Main PID: 1415 (pulpcore-worker)
CGroup: /system.slice/system-pulpcore\x2dworker.slice/pulpcore-worker@6.service
└─1415 /usr/bin/python3 /usr/bin/pulpcore-worker

Sep 21 15:50:27 hostname systemd[1]: Started Pulp Worker.
Sep 21 15:50:49 hostname pulpcore-worker-6[1415]: pulp [None]: pulpcore.tasking.entrypoint:INFO: Starting distributed type worker
Sep 21 15:50:50 hostname pulpcore-worker-6[1415]: pulp [None]: pulpcore.tasking.worker_watcher:INFO: Worker β€˜1415@hostname’ is back online.
\ displaying tomcat
● tomcat.service - Apache Tomcat Web Application Container
Loaded: loaded (/usr/lib/systemd/system/tomcat.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2021-09-21 16:05:26 CDT; 7min ago
Main PID: 4616 (java)
Tasks: 69
CGroup: /system.slice/tomcat.service
└─4616 /usr/lib/jvm/jre-11/bin/java -Xms1024m -Xmx4096m -Djava.security.auth.login.config=/usr/share/tomcat/conf/login.config -classpath /usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/share/java/commons-daemon.jar -Dcatalina.base=/usr/share/tomcat -Dcatalina.home=/usr/share/tomcat -Djava.endorsed.dirs= -Djava.io.tmpdir=/var/cache/tomcat/temp -Djava.util.logging.config.file=/usr/share/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager org.apache.catalina.startup.Bootstrap start

Sep 21 16:05:32 hostname server[4616]: Sep 21, 2021 4:05:32 PM com.google.inject.internal.ProxyFactory
Sep 21 16:05:32 hostname server[4616]: WARNING: Method [public void org.candlepin.model.ContentCurator.delete(org.candlepin.model.Persisted)] is synthetic and is being intercepted by [com.google.inject.persist.jpa.JpaLocalTxnInterceptor@77ad815]. This could indicate a bug. The method may be intercepted twice, or may not be intercepted at all.
Sep 21 16:05:32 hostname server[4616]: Sep 21, 2021 4:05:32 PM com.google.inject.internal.ProxyFactory
Sep 21 16:05:32 hostname server[4616]: WARNING: Method [public void org.candlepin.model.EntitlementCertificateCurator.delete(org.candlepin.model.Persisted)] is synthetic and is being intercepted by [com.google.inject.persist.jpa.JpaLocalTxnInterceptor@77ad815]. This could indicate a bug. The method may be intercepted twice, or may not be intercepted at all.
Sep 21 16:05:32 hostname server[4616]: Sep 21, 2021 4:05:32 PM com.google.inject.internal.ProxyFactory
Sep 21 16:05:32 hostname server[4616]: WARNING: Method [public java.lang.Iterable org.candlepin.resource.OwnerResource.createBatchContent(java.lang.String,java.util.List)] is synthetic and is being intercepted by [com.google.inject.persist.jpa.JpaLocalTxnInterceptor@77ad815]. This could indicate a bug. The method may be intercepted twice, or may not be intercepted at all.
Sep 21 16:05:40 hostname server[4616]: Sep 21, 2021 4:05:40 PM org.apache.catalina.startup.HostConfig deployDirectory
Sep 21 16:05:40 hostname server[4616]: INFO: Deployment of web application directory /var/lib/tomcat/webapps/candlepin has finished in 13,320 ms
Sep 21 16:05:40 hostname server[4616]: Sep 21, 2021 4:05:40 PM org.apache.catalina.startup.Catalina start
Sep 21 16:05:40 hostname server[4616]: INFO: Server startup in 13383 ms
\ displaying dynflow-sidekiq@orchestrator
● dynflow-sidekiq@orchestrator.service - Foreman jobs daemon - orchestrator on sidekiq
Loaded: loaded (/usr/lib/systemd/system/dynflow-sidekiq@.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2021-09-21 15:52:03 CDT; 20min ago
Docs: https://theforeman.org
Main PID: 1473 (sidekiq)
CGroup: /system.slice/system-dynflow\x2dsidekiq.slice/dynflow-sidekiq@orchestrator.service
└─1473 sidekiq 5.2.7 [0 of 1 busy]

Sep 21 15:50:27 hostname systemd[1]: Starting Foreman jobs daemon - orchestrator on sidekiq…
Sep 21 15:50:36 hostname dynflow-sidekiq@orchestrator[1473]: 2021-09-21T20:50:36.357Z 1473 TID-gl INFO: GitLab reliable fetch activated!
Sep 21 15:50:36 hostname dynflow-sidekiq@orchestrator[1473]: 2021-09-21T20:50:36.434Z 1473 TID-h5 INFO: Booting Sidekiq 5.2.7 with redis options {:id=>β€œSidekiq-server-PID-1473”, :url=>β€œredis://localhost:6379/0”}
Sep 21 15:50:53 hostname dynflow-sidekiq@orchestrator[1473]: API controllers newer than Apipie cache! Run apipie:cache rake task to regenerate cache.
Sep 21 15:51:01 hostname dynflow-sidekiq@orchestrator[1473]: /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_puppet-1.0.1/lib/foreman_puppet/register.rb:141: warning: already initialized constant Foreman::Plugin::RbacSupport::AUTO_EXTENDED_ROLES
Sep 21 15:51:01 hostname dynflow-sidekiq@orchestrator[1473]: /usr/share/foreman/app/registries/foreman/plugin/rbac_support.rb:5: warning: previous definition of AUTO_EXTENDED_ROLES was here
Sep 21 15:51:26 hostname dynflow-sidekiq@orchestrator[1473]: User with login admin already exists, not seeding as admin.
Sep 21 15:52:03 hostname systemd[1]: Started Foreman jobs daemon - orchestrator on sidekiq.
\ displaying foreman
● foreman.service - Foreman
Loaded: loaded (/usr/lib/systemd/system/foreman.service; enabled; vendor preset: disabled)
Drop-In: /etc/systemd/system/foreman.service.d
└─installer.conf
Active: active (running) since Tue 2021-09-21 15:51:25 CDT; 21min ago
Docs: https://theforeman.org
Main PID: 1483 (ruby)
Tasks: 176
CGroup: /system.slice/foreman.service
β”œβ”€1483 puma 5.3.2 (unix:///run/foreman.sock) [foreman]
β”œβ”€3154 puma: cluster worker 0: 1483 [foreman]
β”œβ”€3157 puma: cluster worker 1: 1483 [foreman]
β”œβ”€3162 puma: cluster worker 2: 1483 [foreman]
β”œβ”€3168 puma: cluster worker 3: 1483 [foreman]
β”œβ”€3174 puma: cluster worker 4: 1483 [foreman]
β”œβ”€3176 puma: cluster worker 5: 1483 [foreman]
β”œβ”€3177 puma: cluster worker 6: 1483 [foreman]
β”œβ”€3181 puma: cluster worker 7: 1483 [foreman]
└─3187 puma: cluster worker 8: 1483 [foreman]

Sep 21 15:51:25 hostname foreman[1483]: [1483] - Worker 7 (PID: 3181) booted in 0.24s, phase: 0
Sep 21 15:51:25 hostname foreman[1483]: [1483] - Worker 6 (PID: 3177) booted in 0.27s, phase: 0
Sep 21 15:51:25 hostname foreman[1483]: [1483] - Worker 5 (PID: 3176) booted in 0.28s, phase: 0
Sep 21 15:51:25 hostname systemd[1]: Started Foreman.
Sep 21 16:05:24 hostname foreman[1483]: warning: broker sent EOF, and connection not reliable
Sep 21 16:05:24 hostname foreman[1483]: #<Thread:0x0000000013ed5c98 /opt/theforeman/tfm/root/usr/share/gems/gems/logging-2.3.0/lib/logging/diagnostic_context.rb:471 run> terminated with exception (report_on_exception is true):
Sep 21 16:05:24 hostname foreman[1483]: /opt/theforeman/tfm/root/usr/share/gems/gems/stomp-1.4.9/lib/client/utils.rb:198:in block (2 levels) in start_listeners': Received message is nil, and connection not reliable (Stomp::Error::NilMessageError) Sep 21 16:05:24 hostname foreman[1483]: from /opt/theforeman/tfm/root/usr/share/gems/gems/stomp-1.4.9/lib/client/utils.rb:194:in loop’
Sep 21 16:05:24 hostname foreman[1483]: from /opt/theforeman/tfm/root/usr/share/gems/gems/stomp-1.4.9/lib/client/utils.rb:194:in block in start_listeners' Sep 21 16:05:24 hostname foreman[1483]: from /opt/theforeman/tfm/root/usr/share/gems/gems/logging-2.3.0/lib/logging/diagnostic_context.rb:474:in block in create_with_logging_context’
\ displaying httpd
● httpd.service - The Apache HTTP Server
Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2021-09-21 15:50:27 CDT; 22min ago
Docs: man:httpd(8)
man:apachectl(8)
Main PID: 1467 (httpd)
Status: β€œTotal requests: 0; Current requests/sec: 0; Current traffic: 0 B/sec”
Tasks: 12
CGroup: /system.slice/httpd.service
β”œβ”€1467 /usr/sbin/httpd -DFOREGROUND
β”œβ”€1657 /usr/sbin/httpd -DFOREGROUND
β”œβ”€1658 /usr/sbin/httpd -DFOREGROUND
β”œβ”€1659 /usr/sbin/httpd -DFOREGROUND
β”œβ”€1660 /usr/sbin/httpd -DFOREGROUND
β”œβ”€1661 /usr/sbin/httpd -DFOREGROUND
β”œβ”€1662 /usr/sbin/httpd -DFOREGROUND
β”œβ”€1663 /usr/sbin/httpd -DFOREGROUND
β”œβ”€1664 /usr/sbin/httpd -DFOREGROUND
β”œβ”€3549 /usr/sbin/httpd -DFOREGROUND
β”œβ”€3551 /usr/sbin/httpd -DFOREGROUND
└─3552 /usr/sbin/httpd -DFOREGROUND

Sep 21 15:50:27 hostname systemd[1]: Starting The Apache HTTP Server…
Sep 21 15:50:27 hostname systemd[1]: Started The Apache HTTP Server.
\ displaying puppetserver
● puppetserver.service - puppetserver Service
Loaded: loaded (/usr/lib/systemd/system/puppetserver.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2021-09-21 15:51:08 CDT; 21min ago
Process: 1377 ExecStart=/opt/puppetlabs/server/apps/puppetserver/bin/puppetserver start (code=exited, status=0/SUCCESS)
Main PID: 1827 (java)
Tasks: 52 (limit: 4915)
CGroup: /system.slice/puppetserver.service
└─1827 /usr/bin/java -Xms2G -Xmx2G -Djruby.logger.class=com.puppetlabs.jruby_utils.jruby.Slf4jLogger -XX:ReservedCodeCacheSize=512m -XX:OnOutOfMemoryError=β€œkill -9 %p” -XX:ErrorFile=/var/log/puppetlabs/puppetserver/puppetserver_err_pid%p.log -cp /opt/puppetlabs/server/apps/puppetserver/puppet-server-release.jar:/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/facter.jar:/opt/puppetlabs/server/data/puppetserver/jars/* clojure.main -m puppetlabs.trapperkeeper.main --config /etc/puppetlabs/puppetserver/conf.d --bootstrap-config /etc/puppetlabs/puppetserver/services.d/,/opt/puppetlabs/server/apps/puppetserver/config/services.d/ --restart-file /opt/puppetlabs/server/data/puppetserver/restartcounter

Sep 21 15:50:27 hostname systemd[1]: Starting puppetserver Service…
Sep 21 15:51:08 hostname systemd[1]: Started puppetserver Service.
\ displaying dynflow-sidekiq@worker-1
● dynflow-sidekiq@worker-1.service - Foreman jobs daemon - worker-1 on sidekiq
Loaded: loaded (/usr/lib/systemd/system/dynflow-sidekiq@.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2021-09-21 15:51:29 CDT; 21min ago
Docs: https://theforeman.org
Main PID: 1481 (sidekiq)
CGroup: /system.slice/system-dynflow\x2dsidekiq.slice/dynflow-sidekiq@worker-1.service
└─1481 sidekiq 5.2.7 [0 of 5 busy]

Sep 21 15:50:27 hostname systemd[1]: Starting Foreman jobs daemon - worker-1 on sidekiq…
Sep 21 15:50:36 hostname dynflow-sidekiq@worker-1[1481]: 2021-09-21T20:50:36.363Z 1481 TID-gt INFO: GitLab reliable fetch activated!
Sep 21 15:50:36 hostname dynflow-sidekiq@worker-1[1481]: 2021-09-21T20:50:36.435Z 1481 TID-gx INFO: Booting Sidekiq 5.2.7 with redis options {:id=>β€œSidekiq-server-PID-1481”, :url=>β€œredis://localhost:6379/0”}
Sep 21 15:50:53 hostname dynflow-sidekiq@worker-1[1481]: API controllers newer than Apipie cache! Run apipie:cache rake task to regenerate cache.
Sep 21 15:51:01 hostname dynflow-sidekiq@worker-1[1481]: /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_puppet-1.0.1/lib/foreman_puppet/register.rb:141: warning: already initialized constant Foreman::Plugin::RbacSupport::AUTO_EXTENDED_ROLES
Sep 21 15:51:01 hostname dynflow-sidekiq@worker-1[1481]: /usr/share/foreman/app/registries/foreman/plugin/rbac_support.rb:5: warning: previous definition of AUTO_EXTENDED_ROLES was here
Sep 21 15:51:24 hostname dynflow-sidekiq@worker-1[1481]: User with login admin already exists, not seeding as admin.
Sep 21 15:51:29 hostname systemd[1]: Started Foreman jobs daemon - worker-1 on sidekiq.
\ displaying dynflow-sidekiq@worker-hosts-queue-1
● dynflow-sidekiq@worker-hosts-queue-1.service - Foreman jobs daemon - worker-hosts-queue-1 on sidekiq
Loaded: loaded (/usr/lib/systemd/system/dynflow-sidekiq@.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2021-09-21 15:51:27 CDT; 21min ago
Docs: https://theforeman.org
Main PID: 1478 (sidekiq)
CGroup: /system.slice/system-dynflow\x2dsidekiq.slice/dynflow-sidekiq@worker-hosts-queue-1.service
└─1478 sidekiq 5.2.7 [0 of 5 busy]

Sep 21 15:50:27 hostname systemd[1]: Starting Foreman jobs daemon - worker-hosts-queue-1 on sidekiq…
Sep 21 15:50:36 hostname dynflow-sidekiq@worker-hosts-queue-1[1478]: 2021-09-21T20:50:36.399Z 1478 TID-gi INFO: GitLab reliable fetch activated!
Sep 21 15:50:36 hostname dynflow-sidekiq@worker-hosts-queue-1[1478]: 2021-09-21T20:50:36.448Z 1478 TID-ha INFO: Booting Sidekiq 5.2.7 with redis options {:id=>β€œSidekiq-server-PID-1478”, :url=>β€œredis://localhost:6379/0”}
Sep 21 15:50:53 hostname dynflow-sidekiq@worker-hosts-queue-1[1478]: API controllers newer than Apipie cache! Run apipie:cache rake task to regenerate cache.
Sep 21 15:51:01 hostname dynflow-sidekiq@worker-hosts-queue-1[1478]: /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_puppet-1.0.1/lib/foreman_puppet/register.rb:141: warning: already initialized constant Foreman::Plugin::RbacSupport::AUTO_EXTENDED_ROLES
Sep 21 15:51:01 hostname dynflow-sidekiq@worker-hosts-queue-1[1478]: /usr/share/foreman/app/registries/foreman/plugin/rbac_support.rb:5: warning: previous definition of AUTO_EXTENDED_ROLES was here
Sep 21 15:51:21 hostname dynflow-sidekiq@worker-hosts-queue-1[1478]: User with login admin already exists, not seeding as admin.
Sep 21 15:51:27 hostname systemd[1]: Started Foreman jobs daemon - worker-hosts-queue-1 on sidekiq.
\ displaying foreman-proxy
● foreman-proxy.service - Foreman Proxy
Loaded: loaded (/usr/lib/systemd/system/foreman-proxy.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2021-09-21 15:50:30 CDT; 21min ago
Main PID: 1378 (ruby)
Tasks: 5
CGroup: /system.slice/foreman-proxy.service
└─1378 ruby /usr/share/foreman-proxy/bin/smart-proxy --no-daemonize

Sep 21 15:50:27 hostname systemd[1]: Starting Foreman Proxy…
Sep 21 15:50:29 hostname smart-proxy[1378]: /usr/share/foreman-proxy is not writable.
Sep 21 15:50:29 hostname smart-proxy[1378]: Bundler will use `/tmp/bundler20210921-1378-10jmmex1378’ as your home directory temporarily.
Sep 21 15:50:29 hostname smart-proxy[1378]: Your Gemfile lists the gem rsec (< 1) more than once.
Sep 21 15:50:29 hostname smart-proxy[1378]: You should probably keep only one of them.
Sep 21 15:50:29 hostname smart-proxy[1378]: Remove any duplicate entries and specify the gem only once.
Sep 21 15:50:29 hostname smart-proxy[1378]: While it’s not a problem now, it could cause errors if you change the version of one of them later.
Sep 21 15:50:30 hostname systemd[1]: Started Foreman Proxy.
Sep 21 16:05:21 hostname smart-proxy[1378]: 10.220.146.89 - - [21/Sep/2021:16:05:21 CDT] β€œGET /features HTTP/1.1” 200 73
Sep 21 16:05:21 hostname smart-proxy[1378]: - β†’ /features
\ All services are running [OK]

I guess selinux is blocking something, we have an other server in a different environment, that was upgraded to Katello 4.2 with Foreman 3.0 with selinux disabled and that server is working perfectly fine without any issues, wonder why this server have issues with selinux disabled Katello 4.2 upgrade?
also with selinux enabled with enforcing have issues after the upgrade?