Fresh Install w/firefox error: SEC_ERROR_BAD_SIGNATURE

Problem:
Web UI wont load. In Firefox the error is “Error code: SEC_ERROR_BAD_SIGNATURE”

Expected outcome:
Web UI loads without issue after clean fresh install

Foreman and Proxy versions:
2.2

Foreman and Proxy plugin versions:

Distribution and version:
2.2

Other relevant data:

Some logs:

[root@foreman httpd]# ll -rt
total 16
-rw-r–r-- 1 root root 0 Feb 23 18:15 foreman_error.log
-rw-r–r-- 1 root root 0 Feb 23 18:15 crane_error_ssl.log
-rw-r–r-- 1 root root 0 Feb 23 18:15 crane_access_ssl.log
-rw-r–r-- 1 root root 760 Feb 23 18:15 error_log
-rw-r–r-- 1 root root 858 Feb 23 18:25 foreman-ssl_error_ssl.log
-rw-r–r-- 1 root root 2097 Feb 23 18:26 foreman-ssl_access_ssl.log
-rw-r–r-- 1 root root 146 Feb 23 18:29 foreman_access.log
[root@foreman httpd]# tail -30 foreman-ssl_error_ssl.log
[Tue Feb 23 18:25:48.646546 2021] [ssl:warn] [pid 11170] [client 192.168.100.178:35474] AH02227: Failed to set r->user to ‘SSL_CLIENT_S_DN_CN’
[Tue Feb 23 18:25:49.558052 2021] [ssl:warn] [pid 11169] [client 192.168.100.178:35486] AH02227: Failed to set r->user to ‘SSL_CLIENT_S_DN_CN’
[Tue Feb 23 18:25:52.633566 2021] [ssl:warn] [pid 11171] [client 192.168.100.178:35500] AH02227: Failed to set r->user to ‘SSL_CLIENT_S_DN_CN’
[Tue Feb 23 18:25:52.809276 2021] [ssl:warn] [pid 11166] [client 192.168.100.178:35508] AH02227: Failed to set r->user to ‘SSL_CLIENT_S_DN_CN’
[Tue Feb 23 18:25:53.007415 2021] [ssl:warn] [pid 11167] [client 192.168.100.178:35516] AH02227: Failed to set r->user to ‘SSL_CLIENT_S_DN_CN’
[Tue Feb 23 18:25:53.143420 2021] [ssl:warn] [pid 11170] [client 192.168.100.178:35522] AH02227: Failed to set r->user to ‘SSL_CLIENT_S_DN_CN’
[root@foreman httpd]# tail -30 foreman-ssl_access_ssl.log
192.168.100.178 - - [23/Feb/2021:18:25:44 -0500] “GET /api/v2/smart_proxies?search=name=%22foreman.rogue.com%22 HTTP/1.1” 200 134 “-” “OAuth gem v0.5.1”
192.168.100.178 - - [23/Feb/2021:18:25:48 -0500] “GET /pulp/api/v3/status/ HTTP/1.1” 200 494 “-” “Ruby”
192.168.100.178 - - [23/Feb/2021:18:25:49 -0500] “GET /pulp/api/v3/status/ HTTP/1.1” 200 495 “-” “Ruby”
192.168.100.178 - - [23/Feb/2021:18:25:48 -0500] “POST /api/v2/smart_proxies HTTP/1.1” 201 2190 “-” “OAuth gem v0.5.1”
192.168.100.178 - - [23/Feb/2021:18:25:52 -0500] “GET /api/v2/smart_proxies?search=name=%22foreman.rogue.com%22 HTTP/1.1” 200 830 “-” “OAuth gem v0.5.1”
192.168.100.178 - - [23/Feb/2021:18:25:52 -0500] “GET /pulp/api/v3/status/ HTTP/1.1” 200 496 “-” “Ruby”
192.168.100.178 - - [23/Feb/2021:18:25:52 -0500] “GET /pulp/api/v3/status/ HTTP/1.1” 200 496 “-” “Ruby”
192.168.100.178 - - [23/Feb/2021:18:25:53 -0500] “GET /pulp/api/v3/status/ HTTP/1.1” 200 496 “-” “Ruby”
192.168.100.178 - - [23/Feb/2021:18:25:53 -0500] “GET /pulp/api/v3/status/ HTTP/1.1” 200 496 “-” “Ruby”
192.168.100.178 - - [23/Feb/2021:18:25:52 -0500] “PUT /api/v2/smart_proxies/1/refresh HTTP/1.1” 200 794 “-” “OAuth gem v0.5.1”
192.168.100.178 - - [23/Feb/2021:18:26:00 -0500] “GET /node/foreman.rogue.com?format=yml HTTP/1.1” 404 - “-” “Ruby”
192.168.100.178 - - [23/Feb/2021:18:26:03 -0500] “POST /api/hosts/facts HTTP/1.1” 201 1083 “-” “Ruby”
192.168.100.178 - - [23/Feb/2021:18:26:16 -0500] “GET /node/foreman.rogue.com?format=yml HTTP/1.1” 200 925 “-” “Ruby”
192.168.100.178 - - [23/Feb/2021:18:26:17 -0500] “POST /api/config_reports HTTP/1.1” 201 775 “-” “Ruby”
192.168.100.178 - admin [23/Feb/2021:18:26:27 -0500] “GET /pulp/api/v2/consumers/ HTTP/1.1” 200 2 “-” “rest-client/2.0.2 (linux-gnu x86_64) ruby/2.5.5p157”
192.168.100.178 - admin [23/Feb/2021:18:26:29 -0500] “GET /pulp/api/v2/events/ HTTP/1.1” 200 2 “-” “rest-client/2.0.2 (linux-gnu x86_64) ruby/2.5.5p157”
192.168.100.178 - admin [23/Feb/2021:18:26:30 -0500] “POST /pulp/api/v2/content/units/rpm/search/ HTTP/1.1” 200 2 “-” “rest-client/2.0.2 (linux-gnu x86_64) ruby/2.5.5p157”
[root@foreman httpd]# tail -30 foreman_access.log
192.168.100.1 - - [23/Feb/2021:18:29:29 -0500] “GET / HTTP/1.1” 301 92 “-” “Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0”
[root@foreman httpd]#

[root@foreman foreman-proxy]# ll
total 12
-rw-r–r-- 1 foreman-proxy foreman-proxy 8266 Feb 23 18:25 proxy.log
-rw-r–r-- 1 foreman-proxy foreman-proxy 0 Feb 23 18:23 smart_proxy_dynflow_core.log
[root@foreman foreman-proxy]# tail -50 proxy.log
4f:f3:95:76:87:83:c8:da:f2:69:de:85:fe:3d:d9:cf:a7:5a:
0b:48:19:00:db:f4:9b:9c:a9:61:79:da:37:85:f2:20:39:31:
1e:41:a3:e1:40:a5:13:15:7d:e9:a9:7b:2b:ef:4e:25:d2:6a:
9a:1c:d8:d7:7f:ac:fd:4e:87:c1:24:20:93:a1:24:d1:5f:00:
14:d5:51:09:9e:05:c0:60:64:85:30:9a:67:6e:2a:45:f3:60:
fa:da:83:b4:21:0b:91:0f:55:3b:c0:92:d4:5b:5d:aa:2b:9a:
82:a1:b9:38:f7:91:07:14:77:99:84:ab:e1:a7:11:80:05:fe:
d8:1e:d1:0c:2f:7d:e7:d8:e5:68:37:65:a1:12:3b:d7:5a:fa:
b1:ae:4b:9b:a4:ff:04:f2:e4:ec:5c:5b:5c:e5:39:b8:0f:81:
be:45:73:6d:93:a3:f0:6e:b0:39:41:0c:82:d2:93:23:b5:a0:
a6:08:79:6e:95:ef:bd:63:3e:73:05:09:3f:f1:5c:f0:62:11:
66:a0:59:64:9d:f8:a2:4f:30:4c:81:1f:5a:eb:8e:45:8f:26:
0c:0e:3c:b0:fe:57:8e:68:97:2c:bb:5c:8c:8b:ee:62:91:e6:
0b:6a:6a:fd:76:fa:bf:43:6c:13:bf:5e:71:41:7c:10:a5:32:
d9:30:6e:71:e7:02:07:a1:40:9e:ac:83:8b:ce:56:c4:1a:aa:
7a:72:08:95:d7:d6:14:64:1f:0a:59:18:5f:e9:5e:e4:df:5b:
73:8f:bd:67:a8:05:43:c5:79:07:05:57:be:b7:d0:76:5b:f2:
ad:c0:64:46:a7:93:b5:03:2e:b3:88:d7:1f:cd:4e:5f:fc:77:
a8:d0:a0:ec:97:e3:9e:ee:84:0c:3a:83:84:be:26:a6:c3:82:
07:e7:b9:8e:bf:2f:52:ff:15:cd:47:f8:78:14:77:ac:5c:55:
7e:eb:de:1f:b8:d1:73:0a:3e:85:e4:cf:00:51:a5:d2:f6:11:
86:b4:ae:62:d6:a5:ea:60:80:b4:3c:0a:c9:37:f7:6d:90:f0:
18:0a:fe:42:dc:60:67:b3:57:9f:c9:c4:1e:e9:70:aa:88:04:
b0:a0:bf:85:e7:9c:16:bf:f7:59:c2:f5:d1:69:c2:6e:06:32:
5d:e4:a8:29:b1:56:c9:77:41:54:0e:c1:55:96:71:58:a7:99:
fd:9b:9d:8e:80:3f:a2:9c:05:ed:1c:2a:e5:8a:33:cf:6b:7b:
4d:49:7a:13:1c:5b:95:7c:d6:91:ff:87:42:4b:1a:11:2a:5a:
bc:c2:59:b1:44:92:79:72:51:47:ac:22:c9:8c:72:dc:cd:39:
85:9c:05:02:bd:38:74:6e

2021-02-23T18:25:43 [I] WEBrick::HTTPServer#start: pid=13746 port=9090
2021-02-23T18:25:43 [I] Smart proxy has launched on 1 socket(s), waiting for requests
2021-02-23T18:25:48 85612339 [I] Started GET /v2/features
2021-02-23T18:25:48 85612339 [I] Finished GET /v2/features with 200 (362.19 ms)
2021-02-23T18:25:49 85612339 [I] Started GET /ssh/pubkey
2021-02-23T18:25:49 85612339 [I] Finished GET /ssh/pubkey with 200 (0.84 ms)
2021-02-23T18:25:49 85612339 [I] Started GET /v2/features
2021-02-23T18:25:49 85612339 [I] Finished GET /v2/features with 200 (24.31 ms)
2021-02-23T18:25:52 35069b78 [I] Started GET /v2/features
2021-02-23T18:25:52 35069b78 [I] Finished GET /v2/features with 200 (23.84 ms)
2021-02-23T18:25:52 35069b78 [I] Started GET /ssh/pubkey
2021-02-23T18:25:52 35069b78 [I] Finished GET /ssh/pubkey with 200 (0.29 ms)
2021-02-23T18:25:52 35069b78 [I] Started GET /v2/features
2021-02-23T18:25:52 35069b78 [I] Finished GET /v2/features with 200 (31.67 ms)
2021-02-23T18:25:52 35069b78 [I] Started GET /pulp/status/puppet
2021-02-23T18:25:52 35069b78 [I] Finished GET /pulp/status/puppet with 200 (0.52 ms)
2021-02-23T18:25:52 35069b78 [I] Started GET /v2/features
2021-02-23T18:25:53 35069b78 [I] Finished GET /v2/features with 200 (34.73 ms)
2021-02-23T18:25:53 35069b78 [I] Started GET /v2/features
2021-02-23T18:25:53 35069b78 [I] Finished GET /v2/features with 200 (25.51 ms)
[root@foreman foreman-proxy]#

[root@foreman log]# tail -50 messages
Feb 23 18:25:53 foreman systemd: [/etc/systemd/system/pulpcore-content.service:7] Unknown lvalue ‘type’ in section ‘Service’
Feb 23 18:25:53 foreman systemd: [/etc/systemd/system/pulpcore-api.service:7] Unknown lvalue ‘type’ in section ‘Service’
Feb 23 18:25:53 foreman systemd: Reloading.
Feb 23 18:25:53 foreman systemd: [/etc/systemd/system/pulpcore-content.service:7] Unknown lvalue ‘type’ in section ‘Service’
Feb 23 18:25:53 foreman systemd: [/etc/systemd/system/pulpcore-api.service:7] Unknown lvalue ‘type’ in section ‘Service’
Feb 23 18:25:53 foreman systemd: Reloading.
Feb 23 18:25:53 foreman systemd: [/etc/systemd/system/pulpcore-content.service:7] Unknown lvalue ‘type’ in section ‘Service’
Feb 23 18:25:53 foreman systemd: [/etc/systemd/system/pulpcore-api.service:7] Unknown lvalue ‘type’ in section ‘Service’
Feb 23 18:25:56 foreman puppet-agent[13888]: Starting Puppet client version 6.21.1
Feb 23 18:25:58 foreman su: (to foreman) root on none
Feb 23 18:26:00 foreman puppet-agent[13959]: Unable to fetch my node definition, but the agent run will continue:
Feb 23 18:26:00 foreman puppet-agent[13959]: Error 500 on SERVER: Server Error: Failed to find foreman.rogue.com via exec: Execution of ‘/etc/puppetlabs/puppet/node.rb foreman.rogue.com’ returned 1:
Feb 23 18:26:16 foreman puppet-agent[13959]: Applied catalog in 0.18 seconds
Feb 23 18:26:28 foreman mongod.27017[4372]: [conn77] command pulp_database.$cmd command: createIndexes { createIndexes: “users”, indexes: [ { background: false, name: “roles_-1”, key: { roles: -1 } } ] } numYields:0 reslen:149 locks:{ Global: { acquireCount: { r: 1, w: 1 } }, Database: { acquireCount: { W: 1 } } } protocol:op_query 416ms
Feb 23 18:26:29 foreman mongod.27017[4372]: [conn77] build index on: pulp_database.consumers properties: { v: 2, unique: true, key: { id: -1 }, name: “id_-1”, ns: “pulp_database.consumers”, background: true }
Feb 23 18:26:29 foreman mongod.27017[4372]: [conn77] build index done. scanned 0 total records. 0 secs
Feb 23 18:26:29 foreman mongod.27017[4372]: [conn77] command pulp_database.$cmd command: createIndexes { createIndexes: “consumers”, indexes: [ { unique: true, background: true, name: “id_-1”, key: { id: -1 } } ] } numYields:0 reslen:113 locks:{ Global: { acquireCount: { r: 1, w: 1 } }, Database: { acquireCount: { w: 1, W: 2 }, acquireWaitCount: { W: 1 }, timeAcquiringMicros: { W: 626 } }, Collection: { acquireCount: { w: 1 } } } protocol:op_query 565ms
Feb 23 18:26:29 foreman mongod.27017[4372]: [conn77] build index on: pulp_database.consumers properties: { v: 2, key: { notes: -1 }, name: “notes_-1”, ns: “pulp_database.consumers”, background: true }
Feb 23 18:26:29 foreman mongod.27017[4372]: [conn77] build index done. scanned 0 total records. 0 secs
Feb 23 18:26:29 foreman mongod.27017[4372]: [conn77] command pulp_database.$cmd command: createIndexes { createIndexes: “consumers”, indexes: [ { unique: false, background: true, name: “notes_-1”, key: { notes: -1 } } ] } numYields:0 reslen:113 locks:{ Global: { acquireCount: { r: 1, w: 1 } }, Database: { acquireCount: { w: 1, W: 2 } }, Collection: { acquireCount: { w: 1 } } } protocol:op_query 140ms
Feb 23 18:26:29 foreman mongod.27017[4372]: [conn73] build index on: pulp_database.event_listeners properties: { v: 2, unique: true, key: { id: -1 }, name: “id_-1”, ns: “pulp_database.event_listeners”, background: true }
Feb 23 18:26:29 foreman mongod.27017[4372]: [conn73] build index done. scanned 0 total records. 0 secs
Feb 23 18:26:29 foreman mongod.27017[4372]: [conn73] command pulp_database.$cmd command: createIndexes { createIndexes: “event_listeners”, indexes: [ { unique: true, background: true, name: “id_-1”, key: { id: -1 } } ] } numYields:0 reslen:113 locks:{ Global: { acquireCount: { r: 1, w: 1 } }, Database: { acquireCount: { w: 1, W: 2 } }, Collection: { acquireCount: { w: 1 } } } protocol:op_query 328ms
Feb 23 18:26:41 foreman dhclient[3274]: DHCPREQUEST on eth0 to 192.168.100.1 port 67 (xid=0x1dd67254)
Feb 23 18:26:41 foreman dhclient[3274]: DHCPACK from 192.168.100.1 (xid=0x1dd67254)
Feb 23 18:26:41 foreman NetworkManager[3184]: [1614122801.1703] dhcp4 (eth0): address 192.168.100.178
Feb 23 18:26:41 foreman NetworkManager[3184]: [1614122801.1705] dhcp4 (eth0): plen 24 (255.255.255.0)
Feb 23 18:26:41 foreman NetworkManager[3184]: [1614122801.1708] dhcp4 (eth0): gateway 192.168.100.1
Feb 23 18:26:41 foreman NetworkManager[3184]: [1614122801.1712] dhcp4 (eth0): lease time 3600
Feb 23 18:26:41 foreman NetworkManager[3184]: [1614122801.1713] dhcp4 (eth0): hostname ‘foreman’
Feb 23 18:26:41 foreman NetworkManager[3184]: [1614122801.1713] dhcp4 (eth0): nameserver ‘192.168.100.1’
Feb 23 18:26:41 foreman NetworkManager[3184]: [1614122801.1713] dhcp4 (eth0): domain name ‘virt’
Feb 23 18:26:41 foreman NetworkManager[3184]: [1614122801.1714] dhcp4 (eth0): state changed bound → bound
Feb 23 18:26:41 foreman dhclient[3274]: bound to 192.168.100.178 – renewal in 1580 seconds.
Feb 23 18:26:41 foreman dbus[3180]: [system] Activating via systemd: service name=‘org.freedesktop.nm_dispatcher’ unit=‘dbus-org.freedesktop.nm-dispatcher.service’
Feb 23 18:26:41 foreman systemd: Starting Network Manager Script Dispatcher Service…
Feb 23 18:26:41 foreman dbus[3180]: [system] Successfully activated service ‘org.freedesktop.nm_dispatcher’
Feb 23 18:26:41 foreman systemd: Started Network Manager Script Dispatcher Service.
Feb 23 18:26:41 foreman nm-dispatcher: req:1 ‘dhcp4-change’ [eth0]: new request (4 scripts)
Feb 23 18:26:41 foreman nm-dispatcher: req:1 ‘dhcp4-change’ [eth0]: start running ordered scripts…
Feb 23 18:30:01 foreman systemd: Created slice User Slice of foreman.
Feb 23 18:30:01 foreman systemd: Started Session 25 of user foreman.
Feb 23 18:30:01 foreman systemd: Started Session 24 of user root.
Feb 23 18:30:13 foreman systemd: Removed slice User Slice of foreman.
Feb 23 18:31:23 foreman pulp: celery.beat:INFO: Scheduler: Sending due task download_deferred_content (pulp.server.controllers.repository.queue_download_deferred)
Feb 23 18:31:23 foreman pulp: celery.worker.strategy:INFO: Received task: pulp.server.controllers.repository.queue_download_deferred[f05dccbb-f31e-4217-be26-33aa9e095a40]
Feb 23 18:31:23 foreman pulp: celery.worker.strategy:INFO: Received task: pulp.server.controllers.repository.download_deferred[19b0d506-a802-4443-8b01-5837468fdfdc]
Feb 23 18:31:23 foreman pulp: celery.app.trace:INFO: [f05dccbb] Task pulp.server.controllers.repository.queue_download_deferred[f05dccbb-f31e-4217-be26-33aa9e095a40] succeeded in 0.0361468690007s: None
Feb 23 18:31:24 foreman pulp: celery.app.trace:INFO: [19b0d506] Task pulp.server.controllers.repository.download_deferred[19b0d506-a802-4443-8b01-5837468fdfdc] succeeded in 1.015186738s: None
Feb 23 18:40:01 foreman systemd: Started Session 26 of user root.
[root@foreman log]#

I ran the katello-certs-check and it fails on the bundle. Again this is a clean install with what was downloaded from and installed with these options. . .


Required Repositories
yum -y localinstall https://yum.theforeman.org/releases/2.2/el7/x86_64/foreman-release.rpm
yum -y localinstall https://fedorapeople.org/groups/katello/releases/yum/3.17/katello/el7/x86_64/katello-repos-latest.rpm
https://fedorapeople.org/groups/katello/releases/yum/3.9/katello/el7/x86_64/katello-repos-3.9.1-1.el7.noarch.rpm%20

yum -y localinstall https://yum.puppet.com/puppet6-release-el-7.noarch.rpm
yum -y install epel-release centos-release-scl-rh

Install Katello (will install foreman-installer) (long running command)
yum -y install katello

Install Foreman with Katello (long running command)

foreman-installer --scenario “katello” --foreman-initial-organization “Lab” --foreman-initial-location “Home” --foreman-proxy-dhcp “false” --foreman-proxy-dns “false” --foreman-proxy-tftp “true” --foreman-proxy-puppet “true” --foreman-proxy-puppetca “true” --puppet-runinterval “3600” --foreman-initial-admin-password “password” --enable-foreman-plugin-discovery --enable-foreman-proxy-plugin-discovery --foreman-proxy-plugin-discovery-install-images=true --enable-foreman-plugin-ansible --enable-foreman-proxy-plugin-ansible --enable-foreman-plugin-remote-execution --enable-foreman-proxy-plugin-remote-execution-ssh --foreman-proxy-plugin-discovery-source-url=http://downloads.theforeman.org/discovery/releases/3.7/


[root@foreman etc]# katello-certs-check -c foreman-proxy/foreman_ssl_cert.pem -k foreman-proxy/foreman_ssl_key.pem -b foreman-proxy/foreman_ssl_ca.pem
Checking server certificate encoding:
[OK]

Checking expiration of certificate:
[OK]

Checking expiration of CA bundle:
[OK]

Checking if server certificate has CA:TRUE flag
[OK]

Checking for private key passphrase:
[OK]

Checking to see if the private key matches the certificate:
[OK]

Checking CA bundle against the certificate file:
[FAIL]

The /etc/foreman-proxy/foreman_ssl_ca.pem does not verify the /etc/foreman-proxy/foreman_ssl_cert.pem
/etc/foreman-proxy/foreman_ssl_cert.pem: C = US, ST = North Carolina, O = FOREMAN, OU = FOREMAN_PROXY, CN = foreman.rogue.com
error 26 at 0 depth lookup:unsupported certificate purpose

Checking CA bundle size: 1
[OK]

Checking Subject Alt Name on certificate
[OK]

Checking if any Subject Alt Name on certificate matches the Subject CN
[OK]

Checking Key Usage extension on certificate for Key Encipherment
[OK]

Checking for use of shortname as CN
[OK]

I managed to get the option to accept the risk and login.
Curious, why the certs aren’t properly signed.

Solved (kinda?)