Upgrade katello 3.6 -> 3.7: smartproxy ssl certificate verification failed

Wouter · March 12, 2019, 2:46pm

Problem:
I am trying to upgrade Katello from version 3.6 to 3.7 with the command: foreman-installer --scenario katello --upgrade

The update fails with the error:

/Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[bewdk-infpmgm01.netwerk.hessenoordnatie.com]: Could not evaluate: Exception SSL_connect returned=1 errno=0 state=error: certificate verify failed in get request to: https://bewdk-infpmgm01.netwerk.hessenoordnatie.com/api/v2/smart_proxies?search=name=%22bewdk-infpmgm01.netwerk.hessenoordnatie.com%22

Searched the forum and Satellite documentation but couldn’t find a solution.
Relevant /var/log/foreman-installer/katello.log added below.

Expected outcome:
Successful upgrade

Foreman and Proxy versions:
Upgrade Foreman 1.17.1 to Foreman 1.18.3

Foreman and Proxy plugin versions:

Other relevant data:

[ INFO 2019-03-12T15:37:25 main] Executing hooks in group post
[DEBUG 2019-03-12T15:37:25 main] Hook /usr/share/katello-installer-base/hooks/post/10-post_install.rb returned nil
[ERROR 2019-03-12T15:37:25 main] Upgrade failed during the installation phase. Fix the error and re-run the upgrade.
[DEBUG 2019-03-12T15:37:25 main] Hook /usr/share/katello-installer-base/hooks/post/30-upgrade.rb returned [#<Logging::Logger:0x0000000276f1b8 @name="main", @parent=#<Logging::RootLogger:0x00000002b05ac0 @name="root", @appenders=[], @additive=false, @caller_tracing=false, @level=0>, @appenders=[#<Logging::Appenders::RollingFile:0x0000000277a2c0 @roller=#<Logging::Appenders::RollingFile::Roller:0x0000000277a298 @fn="/var/log/foreman-installer/katello{{.%d}}.log", @roll_by=:number, @filename="/var/log/foreman-installer/katello.log", @roll=false, @keep=nil, @copy_file="/var/log/foreman-installer/katello.log._copy_", @glob="/var/log/foreman-installer/katello.*.log", @number_rgxp=/\/var\/log\/foreman-installer\/katello.(\d+).log/, @format="/var/log/foreman-installer/katello.%d.log">, @size=nil, @age_fn="/var/log/foreman-installer/katello.log.age", @age_fn_mtime=nil, @age=nil, @encoding=#<Encoding:UTF-8>, @mode="a+:UTF-8", @io=#<File:/var/log/foreman-installer/katello.log>, @close_method=:close, @buffer=[], @immediate=[], @auto_flushing=1, @async=false, @async_flusher=nil, @flush_period=nil, @name="configure", @closed=false, @filters=[], @mutex=#<ReentrantMutex:0x00000002779a78 @locker=nil>, @layout=#<Logging::Layouts::Pattern:0x00000002b0ced8 @obj_format=:string, @backtrace=true, @utc_offset=nil, @cause_depth=8, @created_at=2019-03-12 15:33:55 +0100, @date_pattern="%Y-%m-%dT%H:%M:%S", @date_method=nil, @pattern="[%5l %d %c] %m\n", @color_scheme=nil>, @level=0, @write_size=500>], @additive=true, @level=0, @caller_tracing=false>, #<Logging::Logger:0x00000002ad3f20 @name="fatal", @parent=#<Logging::RootLogger:0x00000002b05ac0 @name="root", @appenders=[], @additive=false, @caller_tracing=false, @level=0>, @appenders=[#<Logging::Appenders::Stderr:0x00000002ada618 @io=#<IO:<STDERR>>, @close_method=:close, @buffer=[], @immediate=[], @auto_flushing=1, @async=false, @async_flusher=nil, @flush_period=nil, @name="stderr", @closed=false, @filters=[], @mutex=#<ReentrantMutex:0x00000002ada2a8 @locker=nil>, @layout=#<Logging::Layouts::Pattern:0x00000002b03018 @obj_format=:string, @backtrace=true, @utc_offset=nil, @cause_depth=8, @created_at=2019-03-12 15:33:55 +0100, @date_pattern="%Y-%m-%dT%H:%M:%S", @date_method=nil, @pattern="[%5l %d %c] %m\n", @color_scheme=#<Logging::ColorScheme:0x00000002b039f0 @scheme={"date"=>"\e[34m", "logger"=>"\e[36m", "line"=>"\e[33m", "file"=>"\e[33m", "method"=>"\e[33m", "info"=>"\e[32m", "warn"=>"\e[33m", "error"=>"\e[31m", "fatal"=>"\e[37m\e[41m"}, @lines=false, @levels=true>, @name_map_0=["DEBUG", "\e[32m INFO\e[0m", "\e[33m WARN\e[0m", "\e[31mERROR\e[0m", "\e[37m\e[41mFATAL\e[0m"]>, @level=0, @encoding=nil, @write_size=500>], @additive=true, @level=4, @caller_tracing=false>]
[ INFO 2019-03-12T15:37:25 main] All hooks in group post finished
[DEBUG 2019-03-12T15:37:25 main] Exit with status code: 6 (signal was 6)
[ERROR 2019-03-12T15:37:25 main] Errors encountered during run:
[ERROR 2019-03-12T15:37:25 main]  /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[bewdk-infpmgm01.netwerk.hessenoordnatie.com]: Could not evaluate: Exception SSL_connect returned=1 errno=0 state=error: certificate verify failed in get request to: https://bewdk-infpmgm01.netwerk.hessenoordnatie.com/api/v2/smart_proxies?search=name=%22bewdk-infpmgm01.netwerk.hessenoordnatie.com%22
[ERROR 2019-03-12T15:37:25 main] /usr/share/foreman-installer/modules/foreman/lib/puppet/provider/foreman_resource/rest_v3.rb:89:in `rescue in request'
[ERROR 2019-03-12T15:37:25 main] /usr/share/foreman-installer/modules/foreman/lib/puppet/provider/foreman_resource/rest_v3.rb:71:in `request'
[ERROR 2019-03-12T15:37:25 main] /usr/share/foreman-installer/modules/foreman/lib/puppet/provider/foreman_smartproxy/rest_v3.rb:6:in `proxy'
[ERROR 2019-03-12T15:37:25 main] /usr/share/foreman-installer/modules/foreman/lib/puppet/provider/foreman_smartproxy/rest_v3.rb:13:in `id'
[ERROR 2019-03-12T15:37:25 main] /usr/share/foreman-installer/modules/foreman/lib/puppet/provider/foreman_smartproxy/rest_v3.rb:17:in `exists?'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/property/ensure.rb:81:in `retrieve'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/type.rb:1070:in `retrieve'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/type.rb:1098:in `retrieve_resource'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:300:in `from_resource'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:20:in `evaluate'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:233:in `apply'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:249:in `eval_resource'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:163:in `call'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:163:in `block (2 levels) in evaluate'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:507:in `block in thinmark'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/benchmark.rb:294:in `realtime'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:506:in `thinmark'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:163:in `block in evaluate'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:118:in `traverse'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:154:in `evaluate'
[ERROR 2019-03-12T15:37:25 main] /usr/share/gems/gems/kafo-2.1.0/modules/kafo_configure/lib/puppet/parser/functions/add_progress.rb:30:in `evaluate_with_trigger'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:222:in `block in apply'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/log.rb:155:in `with_destination'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/report.rb:146:in `as_logging_destination'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:221:in `apply'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:171:in `block in apply_catalog'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:224:in `block in benchmark'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/benchmark.rb:294:in `realtime'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:223:in `benchmark'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:170:in `apply_catalog'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:343:in `run_internal'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:221:in `block in run'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:306:in `override'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:195:in `run'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:348:in `apply_catalog'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:274:in `block in main'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:306:in `override'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:225:in `main'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:170:in `run_command'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:375:in `block in run'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:662:in `exit_on_fail'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:375:in `run'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:132:in `run'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:72:in `execute'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/bin/puppet:5:in `<main>'
[ERROR 2019-03-12T15:37:25 main] Wrapped exception:
[ERROR 2019-03-12T15:37:25 main] SSL_connect returned=1 errno=0 state=error: certificate verify failed
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/net/http.rb:923:in `connect'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/net/http.rb:923:in `block in connect'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/timeout.rb:90:in `block in timeout'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/timeout.rb:100:in `call'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/timeout.rb:100:in `timeout'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/net/http.rb:923:in `connect'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/net/http.rb:863:in `do_start'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/net/http.rb:852:in `start'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/net/http.rb:1375:in `request'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/gems/oauth-0.5.1/lib/oauth/consumer.rb:161:in `request'
[ERROR 2019-03-12T15:37:25 main] /usr/share/foreman-installer/modules/foreman/lib/puppet/provider/foreman_resource/rest_v3.rb:76:in `request'
[ERROR 2019-03-12T15:37:25 main] /usr/share/foreman-installer/modules/foreman/lib/puppet/provider/foreman_smartproxy/rest_v3.rb:6:in `proxy'
[ERROR 2019-03-12T15:37:25 main] /usr/share/foreman-installer/modules/foreman/lib/puppet/provider/foreman_smartproxy/rest_v3.rb:13:in `id'
[ERROR 2019-03-12T15:37:25 main] /usr/share/foreman-installer/modules/foreman/lib/puppet/provider/foreman_smartproxy/rest_v3.rb:17:in `exists?'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/property/ensure.rb:81:in `retrieve'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/type.rb:1070:in `retrieve'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/type.rb:1098:in `retrieve_resource'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:300:in `from_resource'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:20:in `evaluate'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:233:in `apply'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:249:in `eval_resource'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:163:in `call'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:163:in `block (2 levels) in evaluate'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:507:in `block in thinmark'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/benchmark.rb:294:in `realtime'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:506:in `thinmark'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:163:in `block in evaluate'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:118:in `traverse'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:154:in `evaluate'
[ERROR 2019-03-12T15:37:25 main] /usr/share/gems/gems/kafo-2.1.0/modules/kafo_configure/lib/puppet/parser/functions/add_progress.rb:30:in `evaluate_with_trigger'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:222:in `block in apply'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/log.rb:155:in `with_destination'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/report.rb:146:in `as_logging_destination'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:221:in `apply'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:171:in `block in apply_catalog'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:224:in `block in benchmark'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/benchmark.rb:294:in `realtime'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:223:in `benchmark'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:170:in `apply_catalog'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:343:in `run_internal'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:221:in `block in run'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:306:in `override'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:195:in `run'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:348:in `apply_catalog'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:274:in `block in main'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:306:in `override'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:225:in `main'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:170:in `run_command'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:375:in `block in run'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:662:in `exit_on_fail'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:375:in `run'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:132:in `run'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:72:in `execute'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/bin/puppet:5:in `<main>'
[ERROR 2019-03-12T15:37:25 main]  /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[bewdk-infpmgm01.netwerk.hessenoordnatie.com]: Failed to call refresh: Exception SSL_connect returned=1 errno=0 state=error: certificate verify failed in get request to: https://bewdk-infpmgm01.netwerk.hessenoordnatie.com/api/v2/smart_proxies?search=name=%22bewdk-infpmgm01.netwerk.hessenoordnatie.com%22
[ERROR 2019-03-12T15:37:25 main]  /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[bewdk-infpmgm01.netwerk.hessenoordnatie.com]: Exception SSL_connect returned=1 errno=0 state=error: certificate verify failed in get request to: https://bewdk-infpmgm01.netwerk.hessenoordnatie.com/api/v2/smart_proxies?search=name=%22bewdk-infpmgm01.netwerk.hessenoordnatie.com%22
[ERROR 2019-03-12T15:37:25 main] /usr/share/foreman-installer/modules/foreman/lib/puppet/provider/foreman_resource/rest_v3.rb:89:in `rescue in request'
[ERROR 2019-03-12T15:37:25 main] /usr/share/foreman-installer/modules/foreman/lib/puppet/provider/foreman_resource/rest_v3.rb:71:in `request'
[ERROR 2019-03-12T15:37:25 main] /usr/share/foreman-installer/modules/foreman/lib/puppet/provider/foreman_smartproxy/rest_v3.rb:6:in `proxy'
[ERROR 2019-03-12T15:37:25 main] /usr/share/foreman-installer/modules/foreman/lib/puppet/provider/foreman_smartproxy/rest_v3.rb:13:in `id'
[ERROR 2019-03-12T15:37:25 main] /usr/share/foreman-installer/modules/foreman/lib/puppet/provider/foreman_smartproxy/rest_v3.rb:17:in `exists?'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/property/ensure.rb:81:in `retrieve'
[ERROR 2019-03-12T15:37:25 main] /usr/share/foreman-installer/modules/foreman/lib/puppet/type/foreman_smartproxy.rb:72:in `refresh'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:147:in `process_callback'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:34:in `block in process_events'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:119:in `block in queued_events'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:118:in `each'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:118:in `queued_events'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:33:in `process_events'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:250:in `eval_resource'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:163:in `call'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:163:in `block (2 levels) in evaluate'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:507:in `block in thinmark'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/benchmark.rb:294:in `realtime'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:506:in `thinmark'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:163:in `block in evaluate'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:118:in `traverse'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:154:in `evaluate'
[ERROR 2019-03-12T15:37:25 main] /usr/share/gems/gems/kafo-2.1.0/modules/kafo_configure/lib/puppet/parser/functions/add_progress.rb:30:in `evaluate_with_trigger'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:222:in `block in apply'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/log.rb:155:in `with_destination'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/report.rb:146:in `as_logging_destination'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:221:in `apply'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:171:in `block in apply_catalog'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:224:in `block in benchmark'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/benchmark.rb:294:in `realtime'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:223:in `benchmark'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:170:in `apply_catalog'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:343:in `run_internal'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:221:in `block in run'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:306:in `override'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:195:in `run'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:348:in `apply_catalog'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:274:in `block in main'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:306:in `override'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:225:in `main'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:170:in `run_command'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:375:in `block in run'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:662:in `exit_on_fail'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:375:in `run'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:132:in `run'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:72:in `execute'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/bin/puppet:5:in `<main>'
[ERROR 2019-03-12T15:37:25 main] Wrapped exception:
[ERROR 2019-03-12T15:37:25 main] SSL_connect returned=1 errno=0 state=error: certificate verify failed
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/net/http.rb:923:in `connect'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/net/http.rb:923:in `block in connect'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/timeout.rb:90:in `block in timeout'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/timeout.rb:100:in `call'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/timeout.rb:100:in `timeout'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/net/http.rb:923:in `connect'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/net/http.rb:863:in `do_start'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/net/http.rb:852:in `start'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/net/http.rb:1375:in `request'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/gems/oauth-0.5.1/lib/oauth/consumer.rb:161:in `request'
[ERROR 2019-03-12T15:37:25 main] /usr/share/foreman-installer/modules/foreman/lib/puppet/provider/foreman_resource/rest_v3.rb:76:in `request'
[ERROR 2019-03-12T15:37:25 main] /usr/share/foreman-installer/modules/foreman/lib/puppet/provider/foreman_smartproxy/rest_v3.rb:6:in `proxy'
[ERROR 2019-03-12T15:37:25 main] /usr/share/foreman-installer/modules/foreman/lib/puppet/provider/foreman_smartproxy/rest_v3.rb:13:in `id'
[ERROR 2019-03-12T15:37:25 main] /usr/share/foreman-installer/modules/foreman/lib/puppet/provider/foreman_smartproxy/rest_v3.rb:17:in `exists?'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/property/ensure.rb:81:in `retrieve'
[ERROR 2019-03-12T15:37:25 main] /usr/share/foreman-installer/modules/foreman/lib/puppet/type/foreman_smartproxy.rb:72:in `refresh'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:147:in `process_callback'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:34:in `block in process_events'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:119:in `block in queued_events'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:118:in `each'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:118:in `queued_events'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:33:in `process_events'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:250:in `eval_resource'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:163:in `call'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:163:in `block (2 levels) in evaluate'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:507:in `block in thinmark'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/benchmark.rb:294:in `realtime'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:506:in `thinmark'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:163:in `block in evaluate'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:118:in `traverse'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:154:in `evaluate'
[ERROR 2019-03-12T15:37:25 main] /usr/share/gems/gems/kafo-2.1.0/modules/kafo_configure/lib/puppet/parser/functions/add_progress.rb:30:in `evaluate_with_trigger'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:222:in `block in apply'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/log.rb:155:in `with_destination'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/report.rb:146:in `as_logging_destination'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:221:in `apply'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:171:in `block in apply_catalog'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:224:in `block in benchmark'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/benchmark.rb:294:in `realtime'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:223:in `benchmark'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:170:in `apply_catalog'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:343:in `run_internal'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:221:in `block in run'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:306:in `override'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:195:in `run'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:348:in `apply_catalog'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:274:in `block in main'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:306:in `override'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:225:in `main'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:170:in `run_command'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:375:in `block in run'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:662:in `exit_on_fail'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:375:in `run'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:132:in `run'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:72:in `execute'
[ERROR 2019-03-12T15:37:25 main] /opt/puppetlabs/puppet/bin/puppet:5:in `<main>'
[ERROR 2019-03-12T15:37:25 main] Upgrade failed during the installation phase. Fix the error and re-run the upgrade.
[DEBUG 2019-03-12T15:37:25 main] Cleaning /tmp/kafo_puppet20190312-19998-pmkzpl.conf
[DEBUG 2019-03-12T15:37:25 main] Cleaning /tmp/kafo_hiera20190312-19998-1b8kxlh
[DEBUG 2019-03-12T15:37:25 main] Cleaning /tmp/kafo_puppet20190312-19998-c6ur4s.conf
[DEBUG 2019-03-12T15:37:25 main] Cleaning /tmp/default_values.yaml
[ INFO 2019-03-12T15:37:25 main] Installer finished in 201.451157516 seconds

Wouter · June 19, 2019, 10:39am

I still haven’t been able to upgrade my Foreman server.

If I do a curl to the same url as in the error I do not get a ssl certificate verification failed error:

[root@bewdk-infpmgm01 ~]# curl https://admin:xxxxxxxx@bewdk-infpmgm01.netwerk.hessenoordnatie.com/api/v2/smart_proxies?search=name=“bewdk-infpmgm01.netwerk.hessenoordnatie.com”
{
“total”: 1,
“subtotal”: 1,
“page”: 1,
“per_page”: 20,
“search”: “name=“bewdk-infpmgm01.netwerk.hessenoordnatie.com””,
“sort”: {
“by”: null,
“order”: null
},
“results”: [{“created_at”:“2018-07-11 09:36:11 UTC”,“updated_at”:“2018-07-11 09:36:11 UTC”,“name”:“bewdk-infpmgm01.netwerk.hessenoordnatie.com”,“id”:1,“url”:“https://bewdk-infpmgm01.netwerk.hessenoordnatie.com:9090”,“features”:[{“name”:“Templates”,“id”:1},{“name”:“Pulp”,“id”:2},{“name”:“TFTP”,“id”:4},{“name”:“Puppet”,“id”:7},{“name”:“Puppet CA”,“id”:8},{“name”:“Logs”,“id”:12},{“name”:“Openscap”,“id”:14}]}]
}

Wouter · June 19, 2019, 1:17pm

My colleague resolved the issue.

Apparently the environment variables HTTP_PROXY and HTTPS_PROXY were still set.
The foreman-installer does a check for http_proxy and https_proxy but not HTTP_PROXY and HTTPS_PROXY.

I unset the environment variables and the issue was resolved.

iballou · July 1, 2019, 3:32pm

This issue was addressed here: https://github.com/theforeman/foreman-installer/pull/370

leer · January 21, 2022, 6:10pm

Hi, Do you mind show me how you unset the environment variables ? I has same error when I upgrade from Katello 4.21 to 4.3. Thanks.

RL