[Katello] Katello custom cert install fails

Edson_Manners · June 28, 2016, 7:08pm

I'm adding a custom wildcard cert to my existing production Katello 2.4
install on CentOS 7.2 using these
instructions: https://github.com/Katello/katello-installer#certificates
I've followed this other thread but he had an issue in his certs but mine
(GoDaddy) pass the test clean.

The katello-certs-check succeeded, but when I run the resulting
katello-installer command for a currently running Katello installation I
get this error:
Marking certificate /root/ssl-build/katello-server-ca for update
/Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]:
Failed to call refresh: /usr/sbin/foreman-rake db:seed returned 1 instead
of one of [0]
/Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]:
/usr/sbin/foreman-rake db:seed returned 1 instead of one of [0]

even though if I run foreman-rake db:seed independently it runs correctly:
[root@katello ~]# foreman-rake db:seed --trace
** Invoke db:seed (first_time)
** Execute db:seed
** Invoke db:abort_if_pending_migrations (first_time)
** Invoke environment (first_time)
** Execute environment
** Invoke db:load_config (first_time)
** Execute db:load_config
** Execute db:abort_if_pending_migrations
Seeding /usr/share/foreman/db/seeds.d/03-auth_sources.rb
Seeding /usr/share/foreman/db/seeds.d/03-permissions.rb
Seeding /usr/share/foreman/db/seeds.d/03-roles.rb
Seeding /usr/share/foreman/db/seeds.d/04-admin.rb
Seeding /usr/share/foreman/db/seeds.d/05-taxonomies.rb
Seeding /usr/share/foreman/db/seeds.d/06-architectures.rb
Seeding /usr/share/foreman/db/seeds.d/07-provisioning_templates.rb
Seeding /usr/share/foreman/db/seeds.d/08-partition_tables.rb
Seeding /usr/share/foreman/db/seeds.d/10-installation_media.rb
Seeding /usr/share/foreman/db/seeds.d/11-smart_proxy_features.rb
Seeding /usr/share/foreman/db/seeds.d/13-compute_profiles.rb
Seeding /usr/share/foreman/db/seeds.d/15-bookmarks.rb
Seeding /usr/share/foreman/db/seeds.d/16-mail_notifications.rb
Seeding
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.11/db/seeds.d/20-foreman_tasks_permissions.rb
Seeding
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_bootdisk-6.1.0/db/seeds.d/50-bootdisk_templates.rb
Seeding
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_discovery-4.1.2/db/seeds.d/50_discovery_templates.rb
Seeding
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.11/db/seeds.d/60-dynflow_proxy_feature.rb
Seeding
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_discovery-4.1.2/db/seeds.d/60_discovery_proxy_feature.rb
Seeding
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.11/db/seeds.d/61-foreman_tasks_bookmarks.rb
All seed files executed

The WebGUI certs seem to install correctly though as the web browser
recognizes it however the local capsule that runs puppet, puppetca, tftp
and bmc on the server no longer connects.
[root@katello katello-installer]# katello-installer --certs-server-cert
"/etc/pki/katello/certs/rcc.fsu.edu.cer"
> --certs-server-cert-req
"/etc/pki/katello/csr/rcc.fsu.edu.csr"
> --certs-server-key
"/etc/pki/katello/private/rcc.fsu.edu.key"
> --certs-server-ca-cert
"/etc/pki/katello/certs/ca.pem"
> --certs-update-server --certs-update-server-ca
Marking certificate
/root/ssl-build/katello.rcc.fsu.edu/katello.rcc.fsu.edu-apache for update
Marking certificate
/root/ssl-build/katello.rcc.fsu.edu/katello.rcc.fsu.edu-foreman-proxy for
update
Marking certificate /root/ssl-build/katello-server-ca for update
/Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]:
Failed to call refresh: /usr/sbin/foreman-rake db:seed returned 1 instead
of one of [0]
/Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]:
/usr/sbin/foreman-rake db:seed returned 1 instead of one of [0]
/Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[katello.rcc.fsu.edu]:
Could not evaluate: Proxy katello.rcc.fsu.edu cannot be registered (503
Service Unavailable): N/A
Installing Done
[100%] […]
Something went wrong! Check the log for ERROR-level output
The full log is at /var/log/katello-installer/katello-installer.log

···

------------------------------------------------------------------------------------

Here is the tails of the installer.lo

[ INFO 2016-06-28 14:01:40 main] All hooks in group post finished
[DEBUG 2016-06-28 14:01:40 main] Exit with status code: 6 (signal was 6)
[ERROR 2016-06-28 14:01:40 main] Repeating errors encountered during run:
[ERROR 2016-06-28 14:01:40 main]
/Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]:
Failed to call refresh: /usr/sbin/foreman-rake db:seed returned 1 instead
of one of [0]
[ERROR 2016-06-28 14:01:40 main]
/Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]:
/usr/sbin/foreman-rake db:seed returned 1 instead of one of [0]
[ERROR 2016-06-28 14:01:40 main]
/Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[katello.rcc.fsu.edu]:
Could not evaluate: Proxy katello.rcc.fsu.edu cannot be registered (503
Service Unavailable): N/A
[DEBUG 2016-06-28 14:01:40 main] Cleaning
/etc/katello-installer/d20160628-15904-mwu582
[DEBUG 2016-06-28 14:01:40 main] Cleaning /tmp/default_values.yaml

And the local proxy is in fact running:
[root@katello katello-installer]# systemctl status foreman-proxy
● foreman-proxy.service - Foreman Proxy
Loaded: loaded (/usr/lib/systemd/system/foreman-proxy.service; enabled;
vendor preset: disabled)
Active: active (running) since Tue 2016-06-28 10:59:09 EDT; 4h 6min ago
Process: 717 ExecStart=/usr/share/foreman-proxy/bin/smart-proxy
(code=exited, status=0/SUCCESS)
Main PID: 725 (ruby)
CGroup: /system.slice/foreman-proxy.service
└─725 ruby /usr/share/foreman-proxy/bin/smart-proxy

Jun 28 10:59:07 katello.rcc.fsu.edu systemd[1]: Stopping Foreman Proxy…
Jun 28 10:59:07 katello.rcc.fsu.edu systemd[1]: Starting Foreman Proxy…
Jun 28 10:59:09 katello.rcc.fsu.edu systemd[1]: PID file
/run/foreman-proxy/foreman-proxy.pid not readable (yet?) after start.
Jun 28 10:59:09 katello.rcc.fsu.edu systemd[1]: Started Foreman Proxy.

Any ideas. It’s a production machine and I tried to go back by regenerating
the self-signed certs but that gave the same error so I’m stuck trying to
make the custom certs work.

Edson_Manners · June 28, 2016, 8:11pm

SOLVED. There was a warning in the install log.

[root@katello ~]# grep WARN
/var/log/katello-installer/katello-installer.[12].log | grep Def
/var/log/katello-installer/katello-installer.1.log:[ WARN 2016-06-28
13:52:13 main]
/Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns:
Couldn't find Organization with name = Default Organization

Looks like you shouldn't remove the 'Default Organization'. Foreman
shouldn't let you remove it if it needs it for other commands. But I'd
rather be able to remove the Default Organization and not have commands
rely on it though. My $0.02.

···

On Tuesday, June 28, 2016 at 3:08:41 PM UTC-4, Edson Manners wrote: > > I'm adding a custom wildcard cert to my existing production Katello 2.4 > install on CentOS 7.2 using these instructions: > https://github.com/Katello/katello-installer#certificates > I've followed this other thread but he had an issue in his certs but mine > (GoDaddy) pass the test clean. > > The katello-certs-check succeeded, but when I run the resulting > katello-installer command for a currently running Katello installation I > get this error: > Marking certificate /root/ssl-build/katello-server-ca for update > /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: > Failed to call refresh: /usr/sbin/foreman-rake db:seed returned 1 instead > of one of [0] > /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: > /usr/sbin/foreman-rake db:seed returned 1 instead of one of [0] > > > even though if I run foreman-rake db:seed independently it runs correctly: > [root@katello ~]# foreman-rake db:seed --trace > ** Invoke db:seed (first_time) > ** Execute db:seed > ** Invoke db:abort_if_pending_migrations (first_time) > ** Invoke environment (first_time) > ** Execute environment > ** Invoke db:load_config (first_time) > ** Execute db:load_config > ** Execute db:abort_if_pending_migrations > Seeding /usr/share/foreman/db/seeds.d/03-auth_sources.rb > Seeding /usr/share/foreman/db/seeds.d/03-permissions.rb > Seeding /usr/share/foreman/db/seeds.d/03-roles.rb > Seeding /usr/share/foreman/db/seeds.d/04-admin.rb > Seeding /usr/share/foreman/db/seeds.d/05-taxonomies.rb > Seeding /usr/share/foreman/db/seeds.d/06-architectures.rb > Seeding /usr/share/foreman/db/seeds.d/07-provisioning_templates.rb > Seeding /usr/share/foreman/db/seeds.d/08-partition_tables.rb > Seeding /usr/share/foreman/db/seeds.d/10-installation_media.rb > Seeding /usr/share/foreman/db/seeds.d/11-smart_proxy_features.rb > Seeding /usr/share/foreman/db/seeds.d/13-compute_profiles.rb > Seeding /usr/share/foreman/db/seeds.d/15-bookmarks.rb > Seeding /usr/share/foreman/db/seeds.d/16-mail_notifications.rb > Seeding > /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.11/db/seeds.d/20-foreman_tasks_permissions.rb > Seeding > /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_bootdisk-6.1.0/db/seeds.d/50-bootdisk_templates.rb > Seeding > /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_discovery-4.1.2/db/seeds.d/50_discovery_templates.rb > Seeding > /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.11/db/seeds.d/60-dynflow_proxy_feature.rb > Seeding > /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_discovery-4.1.2/db/seeds.d/60_discovery_proxy_feature.rb > Seeding > /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.11/db/seeds.d/61-foreman_tasks_bookmarks.rb > All seed files executed > > The WebGUI certs seem to install correctly though as the web browser > recognizes it however the local capsule that runs puppet, puppetca, tftp > and bmc on the server no longer connects. > [root@katello katello-installer]# katello-installer --certs-server-cert > "/etc/pki/katello/certs/rcc.fsu.edu.cer"\ > > --certs-server-cert-req > "/etc/pki/katello/csr/rcc.fsu.edu.csr"\ > > --certs-server-key > "/etc/pki/katello/private/rcc.fsu.edu.key"\ > > --certs-server-ca-cert > "/etc/pki/katello/certs/ca.pem"\ > > --certs-update-server --certs-update-server-ca > Marking certificate /root/ssl-build/ > katello.rcc.fsu.edu/katello.rcc.fsu.edu-apache for update > Marking certificate /root/ssl-build/ > katello.rcc.fsu.edu/katello.rcc.fsu.edu-foreman-proxy for update > Marking certificate /root/ssl-build/katello-server-ca for update > /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: > Failed to call refresh: /usr/sbin/foreman-rake db:seed returned 1 instead > of one of [0] > /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: > /usr/sbin/foreman-rake db:seed returned 1 instead of one of [0] > /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[ > katello.rcc.fsu.edu]: Could not evaluate: Proxy katello.rcc.fsu.edu > cannot be registered (503 Service Unavailable): N/A > Installing Done > [100%] [...............] > Something went wrong! Check the log for ERROR-level output > The full log is at /var/log/katello-installer/katello-installer.log > > > ------------------------------------------------------------------------------------ > > Here is the tails of the installer.lo > > [ INFO 2016-06-28 14:01:40 main] All hooks in group post finished > [DEBUG 2016-06-28 14:01:40 main] Exit with status code: 6 (signal was 6) > [ERROR 2016-06-28 14:01:40 main] Repeating errors encountered during run: > [ERROR 2016-06-28 14:01:40 main] > /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: > Failed to call refresh: /usr/sbin/foreman-rake db:seed returned 1 instead > of one of [0] > [ERROR 2016-06-28 14:01:40 main] > /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: > /usr/sbin/foreman-rake db:seed returned 1 instead of one of [0] > [ERROR 2016-06-28 14:01:40 main] > /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[ > katello.rcc.fsu.edu]: Could not evaluate: Proxy katello.rcc.fsu.edu > cannot be registered (503 Service Unavailable): N/A > [DEBUG 2016-06-28 14:01:40 main] Cleaning > /etc/katello-installer/d20160628-15904-mwu582 > [DEBUG 2016-06-28 14:01:40 main] Cleaning /tmp/default_values.yaml > > And the local proxy is in fact running: > [root@katello katello-installer]# systemctl status foreman-proxy > ● foreman-proxy.service - Foreman Proxy > Loaded: loaded (/usr/lib/systemd/system/foreman-proxy.service; enabled; > vendor preset: disabled) > Active: active (running) since Tue 2016-06-28 10:59:09 EDT; 4h 6min ago > Process: 717 ExecStart=/usr/share/foreman-proxy/bin/smart-proxy > (code=exited, status=0/SUCCESS) > Main PID: 725 (ruby) > CGroup: /system.slice/foreman-proxy.service > └─725 ruby /usr/share/foreman-proxy/bin/smart-proxy > > Jun 28 10:59:07 katello.rcc.fsu.edu systemd[1]: Stopping Foreman Proxy... > Jun 28 10:59:07 katello.rcc.fsu.edu systemd[1]: Starting Foreman Proxy... > Jun 28 10:59:09 katello.rcc.fsu.edu systemd[1]: PID file > /run/foreman-proxy/foreman-proxy.pid not readable (yet?) after start. > Jun 28 10:59:09 katello.rcc.fsu.edu systemd[1]: Started Foreman Proxy. > > Any ideas. It's a production machine and I tried to go back by > regenerating the self-signed certs but that gave the same error so I'm > stuck trying to make the custom certs work. >

Edson_Manners · June 29, 2016, 5:38pm

Ok. I found actual errors. Running this I get:

wget http://katello.rcc.fsu.edu:9090/puppet
HTTP request sent, awaiting response… Read error (Connection reset by
peer) in headers.
Retrying.

···

-- E, [2016-06-29T13:36:38.867482 #33667] ERROR -- : OpenSSL::SSL::SSLError: SSL_accept returned=1 errno=0 state=SSLv2/v3 read client hello A: http request /usr/share/ruby/openssl/ssl.rb:226:in `accept'

On Tuesday, June 28, 2016 at 3:08:41 PM UTC-4, Edson Manners wrote:

I’m adding a custom wildcard cert to my existing production Katello 2.4
install on CentOS 7.2 using these instructions:
https://github.com/Katello/katello-installer#certificates
I’ve followed this other thread but he had an issue in his certs but mine
(GoDaddy) pass the test clean.

The katello-certs-check succeeded, but when I run the resulting
katello-installer command for a currently running Katello installation I
get this error:
Marking certificate /root/ssl-build/katello-server-ca for update
/Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]:
Failed to call refresh: /usr/sbin/foreman-rake db:seed returned 1 instead
of one of [0]
/Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]:
/usr/sbin/foreman-rake db:seed returned 1 instead of one of [0]

even though if I run foreman-rake db:seed independently it runs correctly:
[root@katello ~]# foreman-rake db:seed --trace
** Invoke db:seed (first_time)
** Execute db:seed
** Invoke db:abort_if_pending_migrations (first_time)
** Invoke environment (first_time)
** Execute environment
** Invoke db:load_config (first_time)
** Execute db:load_config
** Execute db:abort_if_pending_migrations
Seeding /usr/share/foreman/db/seeds.d/03-auth_sources.rb
Seeding /usr/share/foreman/db/seeds.d/03-permissions.rb
Seeding /usr/share/foreman/db/seeds.d/03-roles.rb
Seeding /usr/share/foreman/db/seeds.d/04-admin.rb
Seeding /usr/share/foreman/db/seeds.d/05-taxonomies.rb
Seeding /usr/share/foreman/db/seeds.d/06-architectures.rb
Seeding /usr/share/foreman/db/seeds.d/07-provisioning_templates.rb
Seeding /usr/share/foreman/db/seeds.d/08-partition_tables.rb
Seeding /usr/share/foreman/db/seeds.d/10-installation_media.rb
Seeding /usr/share/foreman/db/seeds.d/11-smart_proxy_features.rb
Seeding /usr/share/foreman/db/seeds.d/13-compute_profiles.rb
Seeding /usr/share/foreman/db/seeds.d/15-bookmarks.rb
Seeding /usr/share/foreman/db/seeds.d/16-mail_notifications.rb
Seeding
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.11/db/seeds.d/20-foreman_tasks_permissions.rb
Seeding
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_bootdisk-6.1.0/db/seeds.d/50-bootdisk_templates.rb
Seeding
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_discovery-4.1.2/db/seeds.d/50_discovery_templates.rb
Seeding
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.11/db/seeds.d/60-dynflow_proxy_feature.rb
Seeding
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_discovery-4.1.2/db/seeds.d/60_discovery_proxy_feature.rb
Seeding
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.11/db/seeds.d/61-foreman_tasks_bookmarks.rb
All seed files executed

The WebGUI certs seem to install correctly though as the web browser
recognizes it however the local capsule that runs puppet, puppetca, tftp
and bmc on the server no longer connects.
[root@katello katello-installer]# katello-installer --certs-server-cert
"/etc/pki/katello/certs/rcc.fsu.edu.cer"\
                  --certs-server-cert-req 
“/etc/pki/katello/csr/rcc.fsu.edu.csr”\
                  --certs-server-key 
“/etc/pki/katello/private/rcc.fsu.edu.key”\
                  --certs-server-ca-cert 
“/etc/pki/katello/certs/ca.pem”\
                  --certs-update-server --certs-update-server-ca
Marking certificate /root/ssl-build/
katello.rcc.fsu.edu/katello.rcc.fsu.edu-apache for update
Marking certificate /root/ssl-build/
katello.rcc.fsu.edu/katello.rcc.fsu.edu-foreman-proxy for update
Marking certificate /root/ssl-build/katello-server-ca for update
/Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]:
Failed to call refresh: /usr/sbin/foreman-rake db:seed returned 1 instead
of one of [0]
/Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]:
/usr/sbin/foreman-rake db:seed returned 1 instead of one of [0]
/Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[
katello.rcc.fsu.edu]: Could not evaluate: Proxy katello.rcc.fsu.edu
cannot be registered (503 Service Unavailable): N/A
Installing Done
[100%] […]
Something went wrong! Check the log for ERROR-level output
The full log is at /var/log/katello-installer/katello-installer.log

Here is the tails of the installer.lo

[ INFO 2016-06-28 14:01:40 main] All hooks in group post finished
[DEBUG 2016-06-28 14:01:40 main] Exit with status code: 6 (signal was 6)
[ERROR 2016-06-28 14:01:40 main] Repeating errors encountered during run:
[ERROR 2016-06-28 14:01:40 main]
/Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]:
Failed to call refresh: /usr/sbin/foreman-rake db:seed returned 1 instead
of one of [0]
[ERROR 2016-06-28 14:01:40 main]
/Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]:
/usr/sbin/foreman-rake db:seed returned 1 instead of one of [0]
[ERROR 2016-06-28 14:01:40 main]
/Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[
katello.rcc.fsu.edu]: Could not evaluate: Proxy katello.rcc.fsu.edu
cannot be registered (503 Service Unavailable): N/A
[DEBUG 2016-06-28 14:01:40 main] Cleaning
/etc/katello-installer/d20160628-15904-mwu582
[DEBUG 2016-06-28 14:01:40 main] Cleaning /tmp/default_values.yaml

And the local proxy is in fact running:
[root@katello katello-installer]# systemctl status foreman-proxy
● foreman-proxy.service - Foreman Proxy
Loaded: loaded (/usr/lib/systemd/system/foreman-proxy.service; enabled;
vendor preset: disabled)
Active: active (running) since Tue 2016-06-28 10:59:09 EDT; 4h 6min ago
Process: 717 ExecStart=/usr/share/foreman-proxy/bin/smart-proxy
(code=exited, status=0/SUCCESS)
Main PID: 725 (ruby)
CGroup: /system.slice/foreman-proxy.service
└─725 ruby /usr/share/foreman-proxy/bin/smart-proxy

Jun 28 10:59:07 katello.rcc.fsu.edu systemd[1]: Stopping Foreman Proxy…
Jun 28 10:59:07 katello.rcc.fsu.edu systemd[1]: Starting Foreman Proxy…
Jun 28 10:59:09 katello.rcc.fsu.edu systemd[1]: PID file
/run/foreman-proxy/foreman-proxy.pid not readable (yet?) after start.
Jun 28 10:59:09 katello.rcc.fsu.edu systemd[1]: Started Foreman Proxy.

Any ideas. It’s a production machine and I tried to go back by
regenerating the self-signed certs but that gave the same error so I’m
stuck trying to make the custom certs work.

Edson_Manners · June 28, 2016, 8:22pm

Nevermind it's not the installer built clean but my local capsule still
gives a foreman-proxy error:
E, [2016-06-28T16:20:59.236274 #38281] ERROR – : OpenSSL::SSL::SSLError:
SSL_accept returned=1 errno=0 state=SSLv3 read client certificate A: tlsv1
alert unknown ca
/usr/share/ruby/openssl/ssl.rb:226:in `accept'

···

On Tuesday, June 28, 2016 at 4:11:52 PM UTC-4, Edson Manners wrote: > > SOLVED. There was a warning in the install log. > > [root@katello ~]# grep WARN > /var/log/katello-installer/katello-installer.[12].log | grep Def > /var/log/katello-installer/katello-installer.1.log:[ WARN 2016-06-28 > 13:52:13 main] > /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns: > Couldn't find Organization with name = Default Organization > > Looks like you shouldn't remove the 'Default Organization'. Foreman > shouldn't let you remove it if it needs it for other commands. But I'd > rather be able to remove the Default Organization and not have commands > rely on it though. My $0.02. > > On Tuesday, June 28, 2016 at 3:08:41 PM UTC-4, Edson Manners wrote: >> >> I'm adding a custom wildcard cert to my existing production Katello 2.4 >> install on CentOS 7.2 using these instructions: >> https://github.com/Katello/katello-installer#certificates >> I've followed this other thread but he had an issue in his certs but mine >> (GoDaddy) pass the test clean. >> >> The katello-certs-check succeeded, but when I run the resulting >> katello-installer command for a currently running Katello installation I >> get this error: >> Marking certificate /root/ssl-build/katello-server-ca for update >> /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: >> Failed to call refresh: /usr/sbin/foreman-rake db:seed returned 1 instead >> of one of [0] >> /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: >> /usr/sbin/foreman-rake db:seed returned 1 instead of one of [0] >> >> >> even though if I run foreman-rake db:seed independently it runs correctly: >> [root@katello ~]# foreman-rake db:seed --trace >> ** Invoke db:seed (first_time) >> ** Execute db:seed >> ** Invoke db:abort_if_pending_migrations (first_time) >> ** Invoke environment (first_time) >> ** Execute environment >> ** Invoke db:load_config (first_time) >> ** Execute db:load_config >> ** Execute db:abort_if_pending_migrations >> Seeding /usr/share/foreman/db/seeds.d/03-auth_sources.rb >> Seeding /usr/share/foreman/db/seeds.d/03-permissions.rb >> Seeding /usr/share/foreman/db/seeds.d/03-roles.rb >> Seeding /usr/share/foreman/db/seeds.d/04-admin.rb >> Seeding /usr/share/foreman/db/seeds.d/05-taxonomies.rb >> Seeding /usr/share/foreman/db/seeds.d/06-architectures.rb >> Seeding /usr/share/foreman/db/seeds.d/07-provisioning_templates.rb >> Seeding /usr/share/foreman/db/seeds.d/08-partition_tables.rb >> Seeding /usr/share/foreman/db/seeds.d/10-installation_media.rb >> Seeding /usr/share/foreman/db/seeds.d/11-smart_proxy_features.rb >> Seeding /usr/share/foreman/db/seeds.d/13-compute_profiles.rb >> Seeding /usr/share/foreman/db/seeds.d/15-bookmarks.rb >> Seeding /usr/share/foreman/db/seeds.d/16-mail_notifications.rb >> Seeding >> /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.11/db/seeds.d/20-foreman_tasks_permissions.rb >> Seeding >> /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_bootdisk-6.1.0/db/seeds.d/50-bootdisk_templates.rb >> Seeding >> /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_discovery-4.1.2/db/seeds.d/50_discovery_templates.rb >> Seeding >> /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.11/db/seeds.d/60-dynflow_proxy_feature.rb >> Seeding >> /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_discovery-4.1.2/db/seeds.d/60_discovery_proxy_feature.rb >> Seeding >> /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.11/db/seeds.d/61-foreman_tasks_bookmarks.rb >> All seed files executed >> >> The WebGUI certs seem to install correctly though as the web browser >> recognizes it however the local capsule that runs puppet, puppetca, tftp >> and bmc on the server no longer connects. >> [root@katello katello-installer]# katello-installer --certs-server-cert >> "/etc/pki/katello/certs/rcc.fsu.edu.cer"\ >> > --certs-server-cert-req >> "/etc/pki/katello/csr/rcc.fsu.edu.csr"\ >> > --certs-server-key >> "/etc/pki/katello/private/rcc.fsu.edu.key"\ >> > --certs-server-ca-cert >> "/etc/pki/katello/certs/ca.pem"\ >> > --certs-update-server --certs-update-server-ca >> Marking certificate /root/ssl-build/ >> katello.rcc.fsu.edu/katello.rcc.fsu.edu-apache for update >> Marking certificate /root/ssl-build/ >> katello.rcc.fsu.edu/katello.rcc.fsu.edu-foreman-proxy for update >> Marking certificate /root/ssl-build/katello-server-ca for update >> /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: >> Failed to call refresh: /usr/sbin/foreman-rake db:seed returned 1 instead >> of one of [0] >> /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: >> /usr/sbin/foreman-rake db:seed returned 1 instead of one of [0] >> /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[ >> katello.rcc.fsu.edu]: Could not evaluate: Proxy katello.rcc.fsu.edu >> cannot be registered (503 Service Unavailable): N/A >> Installing Done >> [100%] [...............] >> Something went wrong! Check the log for ERROR-level output >> The full log is at /var/log/katello-installer/katello-installer.log >> >> >> ------------------------------------------------------------------------------------ >> >> Here is the tails of the installer.lo >> >> [ INFO 2016-06-28 14:01:40 main] All hooks in group post finished >> [DEBUG 2016-06-28 14:01:40 main] Exit with status code: 6 (signal was 6) >> [ERROR 2016-06-28 14:01:40 main] Repeating errors encountered during run: >> [ERROR 2016-06-28 14:01:40 main] >> /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: >> Failed to call refresh: /usr/sbin/foreman-rake db:seed returned 1 instead >> of one of [0] >> [ERROR 2016-06-28 14:01:40 main] >> /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: >> /usr/sbin/foreman-rake db:seed returned 1 instead of one of [0] >> [ERROR 2016-06-28 14:01:40 main] >> /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[ >> katello.rcc.fsu.edu]: Could not evaluate: Proxy katello.rcc.fsu.edu >> cannot be registered (503 Service Unavailable): N/A >> [DEBUG 2016-06-28 14:01:40 main] Cleaning >> /etc/katello-installer/d20160628-15904-mwu582 >> [DEBUG 2016-06-28 14:01:40 main] Cleaning /tmp/default_values.yaml >> >> And the local proxy is in fact running: >> [root@katello katello-installer]# systemctl status foreman-proxy >> ● foreman-proxy.service - Foreman Proxy >> Loaded: loaded (/usr/lib/systemd/system/foreman-proxy.service; >> enabled; vendor preset: disabled) >> Active: active (running) since Tue 2016-06-28 10:59:09 EDT; 4h 6min ago >> Process: 717 ExecStart=/usr/share/foreman-proxy/bin/smart-proxy >> (code=exited, status=0/SUCCESS) >> Main PID: 725 (ruby) >> CGroup: /system.slice/foreman-proxy.service >> └─725 ruby /usr/share/foreman-proxy/bin/smart-proxy >> >> Jun 28 10:59:07 katello.rcc.fsu.edu systemd[1]: Stopping Foreman Proxy... >> Jun 28 10:59:07 katello.rcc.fsu.edu systemd[1]: Starting Foreman Proxy... >> Jun 28 10:59:09 katello.rcc.fsu.edu systemd[1]: PID file >> /run/foreman-proxy/foreman-proxy.pid not readable (yet?) after start. >> Jun 28 10:59:09 katello.rcc.fsu.edu systemd[1]: Started Foreman Proxy. >> >> Any ideas. It's a production machine and I tried to go back by >> regenerating the self-signed certs but that gave the same error so I'm >> stuck trying to make the custom certs work. >> >

iNecas · June 30, 2016, 9:04am

Do you use all-in-one setup, or do you have some external
proxy/capsule in place?

– Ivan

···

On Wed, Jun 29, 2016 at 7:38 PM, Edson Manners wrote: > Ok. I found actual errors. Running this I get: > > > wget http://katello.rcc.fsu.edu:9090/puppet > HTTP request sent, awaiting response... Read error (Connection reset by > peer) in headers. > Retrying. > -- > E, [2016-06-29T13:36:38.867482 #33667] ERROR -- : OpenSSL::SSL::SSLError: > SSL_accept returned=1 errno=0 state=SSLv2/v3 read client hello A: http > request > /usr/share/ruby/openssl/ssl.rb:226:in `accept' > > On Tuesday, June 28, 2016 at 3:08:41 PM UTC-4, Edson Manners wrote: >> >> I'm adding a custom wildcard cert to my existing production Katello 2.4 >> install on CentOS 7.2 using these instructions: >> https://github.com/Katello/katello-installer#certificates >> I've followed this other thread but he had an issue in his certs but mine >> (GoDaddy) pass the test clean. >> >> The katello-certs-check succeeded, but when I run the resulting >> katello-installer command for a currently running Katello installation I get >> this error: >> Marking certificate /root/ssl-build/katello-server-ca for update >> >> /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: >> Failed to call refresh: /usr/sbin/foreman-rake db:seed returned 1 instead of >> one of [0] >> >> /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: >> /usr/sbin/foreman-rake db:seed returned 1 instead of one of [0] >> >> >> even though if I run foreman-rake db:seed independently it runs correctly: >> [root@katello ~]# foreman-rake db:seed --trace >> ** Invoke db:seed (first_time) >> ** Execute db:seed >> ** Invoke db:abort_if_pending_migrations (first_time) >> ** Invoke environment (first_time) >> ** Execute environment >> ** Invoke db:load_config (first_time) >> ** Execute db:load_config >> ** Execute db:abort_if_pending_migrations >> Seeding /usr/share/foreman/db/seeds.d/03-auth_sources.rb >> Seeding /usr/share/foreman/db/seeds.d/03-permissions.rb >> Seeding /usr/share/foreman/db/seeds.d/03-roles.rb >> Seeding /usr/share/foreman/db/seeds.d/04-admin.rb >> Seeding /usr/share/foreman/db/seeds.d/05-taxonomies.rb >> Seeding /usr/share/foreman/db/seeds.d/06-architectures.rb >> Seeding /usr/share/foreman/db/seeds.d/07-provisioning_templates.rb >> Seeding /usr/share/foreman/db/seeds.d/08-partition_tables.rb >> Seeding /usr/share/foreman/db/seeds.d/10-installation_media.rb >> Seeding /usr/share/foreman/db/seeds.d/11-smart_proxy_features.rb >> Seeding /usr/share/foreman/db/seeds.d/13-compute_profiles.rb >> Seeding /usr/share/foreman/db/seeds.d/15-bookmarks.rb >> Seeding /usr/share/foreman/db/seeds.d/16-mail_notifications.rb >> Seeding >> /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.11/db/seeds.d/20-foreman_tasks_permissions.rb >> Seeding >> /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_bootdisk-6.1.0/db/seeds.d/50-bootdisk_templates.rb >> Seeding >> /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_discovery-4.1.2/db/seeds.d/50_discovery_templates.rb >> Seeding >> /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.11/db/seeds.d/60-dynflow_proxy_feature.rb >> Seeding >> /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_discovery-4.1.2/db/seeds.d/60_discovery_proxy_feature.rb >> Seeding >> /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.11/db/seeds.d/61-foreman_tasks_bookmarks.rb >> All seed files executed >> >> The WebGUI certs seem to install correctly though as the web browser >> recognizes it however the local capsule that runs puppet, puppetca, tftp and >> bmc on the server no longer connects. >> [root@katello katello-installer]# katello-installer --certs-server-cert >> "/etc/pki/katello/certs/rcc.fsu.edu.cer"\ >> > --certs-server-cert-req >> > "/etc/pki/katello/csr/rcc.fsu.edu.csr"\ >> > --certs-server-key >> > "/etc/pki/katello/private/rcc.fsu.edu.key"\ >> > --certs-server-ca-cert >> > "/etc/pki/katello/certs/ca.pem"\ >> > --certs-update-server --certs-update-server-ca >> Marking certificate >> /root/ssl-build/katello.rcc.fsu.edu/katello.rcc.fsu.edu-apache for update >> Marking certificate >> /root/ssl-build/katello.rcc.fsu.edu/katello.rcc.fsu.edu-foreman-proxy for >> update >> Marking certificate /root/ssl-build/katello-server-ca for update >> >> /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: >> Failed to call refresh: /usr/sbin/foreman-rake db:seed returned 1 instead of >> one of [0] >> >> /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: >> /usr/sbin/foreman-rake db:seed returned 1 instead of one of [0] >> >> /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[katello.rcc.fsu.edu]: >> Could not evaluate: Proxy katello.rcc.fsu.edu cannot be registered (503 >> Service Unavailable): N/A >> Installing Done >> [100%] [...............] >> Something went wrong! Check the log for ERROR-level output >> The full log is at /var/log/katello-installer/katello-installer.log >> >> >> ------------------------------------------------------------------------------------ >> >> Here is the tails of the installer.lo >> >> [ INFO 2016-06-28 14:01:40 main] All hooks in group post finished >> [DEBUG 2016-06-28 14:01:40 main] Exit with status code: 6 (signal was 6) >> [ERROR 2016-06-28 14:01:40 main] Repeating errors encountered during run: >> [ERROR 2016-06-28 14:01:40 main] >> /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: >> Failed to call refresh: /usr/sbin/foreman-rake db:seed returned 1 instead of >> one of [0] >> [ERROR 2016-06-28 14:01:40 main] >> /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: >> /usr/sbin/foreman-rake db:seed returned 1 instead of one of [0] >> [ERROR 2016-06-28 14:01:40 main] >> /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[katello.rcc.fsu.edu]: >> Could not evaluate: Proxy katello.rcc.fsu.edu cannot be registered (503 >> Service Unavailable): N/A >> [DEBUG 2016-06-28 14:01:40 main] Cleaning >> /etc/katello-installer/d20160628-15904-mwu582 >> [DEBUG 2016-06-28 14:01:40 main] Cleaning /tmp/default_values.yaml >> >> And the local proxy is in fact running: >> [root@katello katello-installer]# systemctl status foreman-proxy >> ● foreman-proxy.service - Foreman Proxy >> Loaded: loaded (/usr/lib/systemd/system/foreman-proxy.service; enabled; >> vendor preset: disabled) >> Active: active (running) since Tue 2016-06-28 10:59:09 EDT; 4h 6min ago >> Process: 717 ExecStart=/usr/share/foreman-proxy/bin/smart-proxy >> (code=exited, status=0/SUCCESS) >> Main PID: 725 (ruby) >> CGroup: /system.slice/foreman-proxy.service >> └─725 ruby /usr/share/foreman-proxy/bin/smart-proxy >> >> Jun 28 10:59:07 katello.rcc.fsu.edu systemd[1]: Stopping Foreman Proxy... >> Jun 28 10:59:07 katello.rcc.fsu.edu systemd[1]: Starting Foreman Proxy... >> Jun 28 10:59:09 katello.rcc.fsu.edu systemd[1]: PID file >> /run/foreman-proxy/foreman-proxy.pid not readable (yet?) after start. >> Jun 28 10:59:09 katello.rcc.fsu.edu systemd[1]: Started Foreman Proxy. >> >> Any ideas. It's a production machine and I tried to go back by >> regenerating the self-signed certs but that gave the same error so I'm stuck >> trying to make the custom certs work. > > -- > You received this message because you are subscribed to the Google Groups > "Foreman users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to foreman-users+unsubscribe@googlegroups.com. > To post to this group, send email to foreman-users@googlegroups.com. > Visit this group at https://groups.google.com/group/foreman-users. > For more options, visit https://groups.google.com/d/optout.

iNecas · June 29, 2016, 6:26am

Hi, might be related to [1]. Could you try running

for i in $(ls /etc/pki/katello-certs-tools/certs/*)
do
rpm -e $(rpm -qf $i)
done

and then running the installer again.

Btw. I have an fix proposed for the installer if you wanted to give it
a try https://github.com/Katello/puppet-certs/pull/91

[1] - https://bugzilla.redhat.com/show_bug.cgi?id=1218251

···

On Tue, Jun 28, 2016 at 10:22 PM, Edson Manners wrote: > Nevermind it's not the installer built clean but my local capsule still > gives a foreman-proxy error: > E, [2016-06-28T16:20:59.236274 #38281] ERROR -- : OpenSSL::SSL::SSLError: > SSL_accept returned=1 errno=0 state=SSLv3 read client certificate A: tlsv1 > alert unknown ca > /usr/share/ruby/openssl/ssl.rb:226:in `accept' > > > On Tuesday, June 28, 2016 at 4:11:52 PM UTC-4, Edson Manners wrote: >> >> SOLVED. There was a warning in the install log. >> >> [root@katello ~]# grep WARN >> /var/log/katello-installer/katello-installer.[12].log | grep Def >> /var/log/katello-installer/katello-installer.1.log:[ WARN 2016-06-28 >> 13:52:13 main] >> /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns: >> Couldn't find Organization with name = Default Organization >> >> Looks like you shouldn't remove the 'Default Organization'. Foreman >> shouldn't let you remove it if it needs it for other commands. But I'd >> rather be able to remove the Default Organization and not have commands rely >> on it though. My $0.02. >> >> On Tuesday, June 28, 2016 at 3:08:41 PM UTC-4, Edson Manners wrote: >>> >>> I'm adding a custom wildcard cert to my existing production Katello 2.4 >>> install on CentOS 7.2 using these instructions: >>> https://github.com/Katello/katello-installer#certificates >>> I've followed this other thread but he had an issue in his certs but mine >>> (GoDaddy) pass the test clean. >>> >>> The katello-certs-check succeeded, but when I run the resulting >>> katello-installer command for a currently running Katello installation I get >>> this error: >>> Marking certificate /root/ssl-build/katello-server-ca for update >>> >>> /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: >>> Failed to call refresh: /usr/sbin/foreman-rake db:seed returned 1 instead of >>> one of [0] >>> >>> /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: >>> /usr/sbin/foreman-rake db:seed returned 1 instead of one of [0] >>> >>> >>> even though if I run foreman-rake db:seed independently it runs >>> correctly: >>> [root@katello ~]# foreman-rake db:seed --trace >>> ** Invoke db:seed (first_time) >>> ** Execute db:seed >>> ** Invoke db:abort_if_pending_migrations (first_time) >>> ** Invoke environment (first_time) >>> ** Execute environment >>> ** Invoke db:load_config (first_time) >>> ** Execute db:load_config >>> ** Execute db:abort_if_pending_migrations >>> Seeding /usr/share/foreman/db/seeds.d/03-auth_sources.rb >>> Seeding /usr/share/foreman/db/seeds.d/03-permissions.rb >>> Seeding /usr/share/foreman/db/seeds.d/03-roles.rb >>> Seeding /usr/share/foreman/db/seeds.d/04-admin.rb >>> Seeding /usr/share/foreman/db/seeds.d/05-taxonomies.rb >>> Seeding /usr/share/foreman/db/seeds.d/06-architectures.rb >>> Seeding /usr/share/foreman/db/seeds.d/07-provisioning_templates.rb >>> Seeding /usr/share/foreman/db/seeds.d/08-partition_tables.rb >>> Seeding /usr/share/foreman/db/seeds.d/10-installation_media.rb >>> Seeding /usr/share/foreman/db/seeds.d/11-smart_proxy_features.rb >>> Seeding /usr/share/foreman/db/seeds.d/13-compute_profiles.rb >>> Seeding /usr/share/foreman/db/seeds.d/15-bookmarks.rb >>> Seeding /usr/share/foreman/db/seeds.d/16-mail_notifications.rb >>> Seeding >>> /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.11/db/seeds.d/20-foreman_tasks_permissions.rb >>> Seeding >>> /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_bootdisk-6.1.0/db/seeds.d/50-bootdisk_templates.rb >>> Seeding >>> /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_discovery-4.1.2/db/seeds.d/50_discovery_templates.rb >>> Seeding >>> /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.11/db/seeds.d/60-dynflow_proxy_feature.rb >>> Seeding >>> /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_discovery-4.1.2/db/seeds.d/60_discovery_proxy_feature.rb >>> Seeding >>> /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.11/db/seeds.d/61-foreman_tasks_bookmarks.rb >>> All seed files executed >>> >>> The WebGUI certs seem to install correctly though as the web browser >>> recognizes it however the local capsule that runs puppet, puppetca, tftp and >>> bmc on the server no longer connects. >>> [root@katello katello-installer]# katello-installer --certs-server-cert >>> "/etc/pki/katello/certs/rcc.fsu.edu.cer"\ >>> > --certs-server-cert-req >>> > "/etc/pki/katello/csr/rcc.fsu.edu.csr"\ >>> > --certs-server-key >>> > "/etc/pki/katello/private/rcc.fsu.edu.key"\ >>> > --certs-server-ca-cert >>> > "/etc/pki/katello/certs/ca.pem"\ >>> > --certs-update-server --certs-update-server-ca >>> Marking certificate >>> /root/ssl-build/katello.rcc.fsu.edu/katello.rcc.fsu.edu-apache for update >>> Marking certificate >>> /root/ssl-build/katello.rcc.fsu.edu/katello.rcc.fsu.edu-foreman-proxy for >>> update >>> Marking certificate /root/ssl-build/katello-server-ca for update >>> >>> /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: >>> Failed to call refresh: /usr/sbin/foreman-rake db:seed returned 1 instead of >>> one of [0] >>> >>> /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: >>> /usr/sbin/foreman-rake db:seed returned 1 instead of one of [0] >>> >>> /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[katello.rcc.fsu.edu]: >>> Could not evaluate: Proxy katello.rcc.fsu.edu cannot be registered (503 >>> Service Unavailable): N/A >>> Installing Done >>> [100%] [...............] >>> Something went wrong! Check the log for ERROR-level output >>> The full log is at /var/log/katello-installer/katello-installer.log >>> >>> >>> ------------------------------------------------------------------------------------ >>> >>> Here is the tails of the installer.lo >>> >>> [ INFO 2016-06-28 14:01:40 main] All hooks in group post finished >>> [DEBUG 2016-06-28 14:01:40 main] Exit with status code: 6 (signal was 6) >>> [ERROR 2016-06-28 14:01:40 main] Repeating errors encountered during run: >>> [ERROR 2016-06-28 14:01:40 main] >>> /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: >>> Failed to call refresh: /usr/sbin/foreman-rake db:seed returned 1 instead of >>> one of [0] >>> [ERROR 2016-06-28 14:01:40 main] >>> /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: >>> /usr/sbin/foreman-rake db:seed returned 1 instead of one of [0] >>> [ERROR 2016-06-28 14:01:40 main] >>> /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[katello.rcc.fsu.edu]: >>> Could not evaluate: Proxy katello.rcc.fsu.edu cannot be registered (503 >>> Service Unavailable): N/A >>> [DEBUG 2016-06-28 14:01:40 main] Cleaning >>> /etc/katello-installer/d20160628-15904-mwu582 >>> [DEBUG 2016-06-28 14:01:40 main] Cleaning /tmp/default_values.yaml >>> >>> And the local proxy is in fact running: >>> [root@katello katello-installer]# systemctl status foreman-proxy >>> ● foreman-proxy.service - Foreman Proxy >>> Loaded: loaded (/usr/lib/systemd/system/foreman-proxy.service; >>> enabled; vendor preset: disabled) >>> Active: active (running) since Tue 2016-06-28 10:59:09 EDT; 4h 6min >>> ago >>> Process: 717 ExecStart=/usr/share/foreman-proxy/bin/smart-proxy >>> (code=exited, status=0/SUCCESS) >>> Main PID: 725 (ruby) >>> CGroup: /system.slice/foreman-proxy.service >>> └─725 ruby /usr/share/foreman-proxy/bin/smart-proxy >>> >>> Jun 28 10:59:07 katello.rcc.fsu.edu systemd[1]: Stopping Foreman Proxy... >>> Jun 28 10:59:07 katello.rcc.fsu.edu systemd[1]: Starting Foreman Proxy... >>> Jun 28 10:59:09 katello.rcc.fsu.edu systemd[1]: PID file >>> /run/foreman-proxy/foreman-proxy.pid not readable (yet?) after start. >>> Jun 28 10:59:09 katello.rcc.fsu.edu systemd[1]: Started Foreman Proxy. >>> >>> Any ideas. It's a production machine and I tried to go back by >>> regenerating the self-signed certs but that gave the same error so I'm stuck >>> trying to make the custom certs work. > > -- > You received this message because you are subscribed to the Google Groups > "Foreman users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to foreman-users+unsubscribe@googlegroups.com. > To post to this group, send email to foreman-users@googlegroups.com. > Visit this group at https://groups.google.com/group/foreman-users. > For more options, visit https://groups.google.com/d/optout.

Edson_Manners · July 5, 2016, 2:39pm

I have one external proxy. It does DNS and DHCP. This was causing service
disruption and so I've gone through the painful process of reverting from
an old backup.
I've rebuilt a new empty katello-test VM and will be testing this process
more thoroughly to better understand where it all went wrong. I really
appreciated your help Ivan.

···

On Thursday, June 30, 2016 at 5:05:18 AM UTC-4, Ivan Necas wrote: > > Do you use all-in-one setup, or do you have some external > proxy/capsule in place? > > -- Ivan > > On Wed, Jun 29, 2016 at 7:38 PM, Edson Manners > wrote: > > Ok. I found actual errors. Running this I get: > > > > > > wget http://katello.rcc.fsu.edu:9090/puppet > > HTTP request sent, awaiting response... Read error (Connection reset by > > peer) in headers. > > Retrying. > > -- > > E, [2016-06-29T13:36:38.867482 #33667] ERROR -- : > OpenSSL::SSL::SSLError: > > SSL_accept returned=1 errno=0 state=SSLv2/v3 read client hello A: http > > request > > /usr/share/ruby/openssl/ssl.rb:226:in `accept' > > > > On Tuesday, June 28, 2016 at 3:08:41 PM UTC-4, Edson Manners wrote: > >> > >> I'm adding a custom wildcard cert to my existing production Katello 2.4 > >> install on CentOS 7.2 using these instructions: > >> https://github.com/Katello/katello-installer#certificates > >> I've followed this other thread but he had an issue in his certs but > mine > >> (GoDaddy) pass the test clean. > >> > >> The katello-certs-check succeeded, but when I run the resulting > >> katello-installer command for a currently running Katello installation > I get > >> this error: > >> Marking certificate /root/ssl-build/katello-server-ca for update > >> > >> > /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: > > >> Failed to call refresh: /usr/sbin/foreman-rake db:seed returned 1 > instead of > >> one of [0] > >> > >> > /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: > > >> /usr/sbin/foreman-rake db:seed returned 1 instead of one of [0] > >> > >> > >> even though if I run foreman-rake db:seed independently it runs > correctly: > >> [root@katello ~]# foreman-rake db:seed --trace > >> ** Invoke db:seed (first_time) > >> ** Execute db:seed > >> ** Invoke db:abort_if_pending_migrations (first_time) > >> ** Invoke environment (first_time) > >> ** Execute environment > >> ** Invoke db:load_config (first_time) > >> ** Execute db:load_config > >> ** Execute db:abort_if_pending_migrations > >> Seeding /usr/share/foreman/db/seeds.d/03-auth_sources.rb > >> Seeding /usr/share/foreman/db/seeds.d/03-permissions.rb > >> Seeding /usr/share/foreman/db/seeds.d/03-roles.rb > >> Seeding /usr/share/foreman/db/seeds.d/04-admin.rb > >> Seeding /usr/share/foreman/db/seeds.d/05-taxonomies.rb > >> Seeding /usr/share/foreman/db/seeds.d/06-architectures.rb > >> Seeding /usr/share/foreman/db/seeds.d/07-provisioning_templates.rb > >> Seeding /usr/share/foreman/db/seeds.d/08-partition_tables.rb > >> Seeding /usr/share/foreman/db/seeds.d/10-installation_media.rb > >> Seeding /usr/share/foreman/db/seeds.d/11-smart_proxy_features.rb > >> Seeding /usr/share/foreman/db/seeds.d/13-compute_profiles.rb > >> Seeding /usr/share/foreman/db/seeds.d/15-bookmarks.rb > >> Seeding /usr/share/foreman/db/seeds.d/16-mail_notifications.rb > >> Seeding > >> > /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.11/db/seeds.d/20-foreman_tasks_permissions.rb > > >> Seeding > >> > /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_bootdisk-6.1.0/db/seeds.d/50-bootdisk_templates.rb > > >> Seeding > >> > /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_discovery-4.1.2/db/seeds.d/50_discovery_templates.rb > > >> Seeding > >> > /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.11/db/seeds.d/60-dynflow_proxy_feature.rb > > >> Seeding > >> > /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_discovery-4.1.2/db/seeds.d/60_discovery_proxy_feature.rb > > >> Seeding > >> > /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.7.11/db/seeds.d/61-foreman_tasks_bookmarks.rb > > >> All seed files executed > >> > >> The WebGUI certs seem to install correctly though as the web browser > >> recognizes it however the local capsule that runs puppet, puppetca, > tftp and > >> bmc on the server no longer connects. > >> [root@katello katello-installer]# katello-installer --certs-server-cert > >> "/etc/pki/katello/certs/rcc.fsu.edu.cer"\ > >> > --certs-server-cert-req > >> > "/etc/pki/katello/csr/rcc.fsu.edu.csr"\ > >> > --certs-server-key > >> > "/etc/pki/katello/private/rcc.fsu.edu.key"\ > >> > --certs-server-ca-cert > >> > "/etc/pki/katello/certs/ca.pem"\ > >> > --certs-update-server --certs-update-server-ca > >> Marking certificate > >> /root/ssl-build/katello.rcc.fsu.edu/katello.rcc.fsu.edu-apache for > update > >> Marking certificate > >> /root/ssl-build/katello.rcc.fsu.edu/katello.rcc.fsu.edu-foreman-proxy > for > >> update > >> Marking certificate /root/ssl-build/katello-server-ca for update > >> > >> > /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: > > >> Failed to call refresh: /usr/sbin/foreman-rake db:seed returned 1 > instead of > >> one of [0] > >> > >> > /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: > > >> /usr/sbin/foreman-rake db:seed returned 1 instead of one of [0] > >> > >> /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[ > katello.rcc.fsu.edu]: > >> Could not evaluate: Proxy katello.rcc.fsu.edu cannot be registered > (503 > >> Service Unavailable): N/A > >> Installing Done > >> [100%] [...............] > >> Something went wrong! Check the log for ERROR-level output > >> The full log is at /var/log/katello-installer/katello-installer.log > >> > >> > >> > ------------------------------------------------------------------------------------ > > >> > >> Here is the tails of the installer.lo > >> > >> [ INFO 2016-06-28 14:01:40 main] All hooks in group post finished > >> [DEBUG 2016-06-28 14:01:40 main] Exit with status code: 6 (signal was > 6) > >> [ERROR 2016-06-28 14:01:40 main] Repeating errors encountered during > run: > >> [ERROR 2016-06-28 14:01:40 main] > >> > /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: > > >> Failed to call refresh: /usr/sbin/foreman-rake db:seed returned 1 > instead of > >> one of [0] > >> [ERROR 2016-06-28 14:01:40 main] > >> > /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: > > >> /usr/sbin/foreman-rake db:seed returned 1 instead of one of [0] > >> [ERROR 2016-06-28 14:01:40 main] > >> /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[ > katello.rcc.fsu.edu]: > >> Could not evaluate: Proxy katello.rcc.fsu.edu cannot be registered > (503 > >> Service Unavailable): N/A > >> [DEBUG 2016-06-28 14:01:40 main] Cleaning > >> /etc/katello-installer/d20160628-15904-mwu582 > >> [DEBUG 2016-06-28 14:01:40 main] Cleaning /tmp/default_values.yaml > >> > >> And the local proxy is in fact running: > >> [root@katello katello-installer]# systemctl status foreman-proxy > >> ● foreman-proxy.service - Foreman Proxy > >> Loaded: loaded (/usr/lib/systemd/system/foreman-proxy.service; > enabled; > >> vendor preset: disabled) > >> Active: active (running) since Tue 2016-06-28 10:59:09 EDT; 4h 6min > ago > >> Process: 717 ExecStart=/usr/share/foreman-proxy/bin/smart-proxy > >> (code=exited, status=0/SUCCESS) > >> Main PID: 725 (ruby) > >> CGroup: /system.slice/foreman-proxy.service > >> └─725 ruby /usr/share/foreman-proxy/bin/smart-proxy > >> > >> Jun 28 10:59:07 katello.rcc.fsu.edu systemd[1]: Stopping Foreman > Proxy... > >> Jun 28 10:59:07 katello.rcc.fsu.edu systemd[1]: Starting Foreman > Proxy... > >> Jun 28 10:59:09 katello.rcc.fsu.edu systemd[1]: PID file > >> /run/foreman-proxy/foreman-proxy.pid not readable (yet?) after start. > >> Jun 28 10:59:09 katello.rcc.fsu.edu systemd[1]: Started Foreman Proxy. > >> > >> Any ideas. It's a production machine and I tried to go back by > >> regenerating the self-signed certs but that gave the same error so I'm > stuck > >> trying to make the custom certs work. > > > > -- > > You received this message because you are subscribed to the Google > Groups > > "Foreman users" group. > > To unsubscribe from this group and stop receiving emails from it, send > an > > email to foreman-user...@googlegroups.com . > > To post to this group, send email to forema...@googlegroups.com > . > > Visit this group at https://groups.google.com/group/foreman-users. > > For more options, visit https://groups.google.com/d/optout. >

Edson_Manners · June 29, 2016, 5:32pm

>
> Ivan,

The RPM removal and re-run of the installer got the local capsule back to
connecting properly and everything looks like it should be fine. But for
some reason everything is now sluggish and clients won't connect but there
aren't any Certificate errors.

[root@katello emanners]# puppet agent -t
Warning: Unable to fetch my node definition, but the agent run will
continue:
Warning: Net::ReadTimeout
Info: Retrieving pluginfacts
Info: Retrieving plugin
Error: Could not retrieve catalog from remote server: Error 400 on SERVER:
Could not find class classX for katello.xxx.xxx.xxx on node
katello.xxx.xxx.xxx
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

So now I'm not so sure what's wrong. This is the only actual error I see in
the logs:
Jun 29 13:32:05 katello puppet-master[13486]: Report processor failed:
Could not send report to Foreman at
https://katello.rcc.fsu.edu/api/reports: Net::ReadTimeout