Katello 3.15 upgrade issues with postgresql

Problem:
Multiple installer sections complain that they can’t connect to postgresql sockets.
e.g.

 /Stage[main]/Foreman::Plugin::Puppetdb/Foreman_config_entry[puppetdb_enabled]/value: change from '' to 'true' failed: Execution of '/usr/sbin/foreman-rake -- config -k 'puppetdb_enabled' -v 'true'' returned 1: rake aborted!
PG::ConnectionBad: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?

Rubocop not loaded.
 Execution of '/usr/sbin/foreman-rake -- config -k 'remote_execution_cockpit_url' -v '/webcon/=%{host}'' returned 1: rake aborted!
PG::ConnectionBad: could not connect to server: No such file or directory
        Is the server running locally and accepting

It appears that postgresql has been successfully upgraded from 9.2 to 12, and it is running and the socket files exist. I’ve tried with selinux in permissive for giggles but still no joy there.

Expected outcome:
Installer Runs succesfully

Foreman and Proxy versions:
upgrading to katello 3.15 & foreman 2

Distribution and version:
CentOS Linux release 7.8.2003

Installer Log

[ INFO 2020-06-02T15:27:02 main]  /Stage[main]/Foreman::Plugin::Puppetdb/Foreman_config_entry[puppetdb_enabled]: Starting to evaluate the resource (1045 of 2446)
[DEBUG 2020-06-02T15:27:02 main]  Executing with uid=foreman gid=foreman: '/usr/sbin/foreman-rake -- config -k 'puppetdb_enabled''
[DEBUG 2020-06-02T15:27:14 main]  Executing with uid=foreman gid=foreman: '/usr/sbin/foreman-rake -- config -k 'puppetdb_enabled' -v 'true''
[ERROR 2020-06-02T15:27:25 main]  Execution of '/usr/sbin/foreman-rake -- config -k 'puppetdb_enabled' -v 'true'' returned 1: rake aborted!
[ERROR 2020-06-02T15:27:25 main] PG::ConnectionBad: could not connect to server: No such file or directory
[ERROR 2020-06-02T15:27:25 main]        Is the server running locally and accepting
[ERROR 2020-06-02T15:27:25 main]        connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/pg-1.1.4/lib/pg.rb:56:in `initialize'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/pg-1.1.4/lib/pg.rb:56:in `new'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/pg-1.1.4/lib/pg.rb:56:in `connect'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:684:in `connect'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:215:in `initialize'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:40:in `new'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:40:in `postgresql_connection'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:809:in `new_connection'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:853:in `checkout_new_connection'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:832:in `try_to_checkout_new_connection'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:793:in `acquire_connection'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:521:in `checkout'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:380:in `connection'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:1008:in `retrieve_connection'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_handling.rb:118:in `retrieve_connection'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_handling.rb:90:in `connection'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/schema_migration.rb:22:in `table_exists?'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/schema_migration.rb:26:in `create_table'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1187:in `initialize'
[ERROR 2020-06-02T15:27:25 main] /usr/share/foreman/app/registries/foreman/plugin.rb:343:in `new'
[ERROR 2020-06-02T15:27:25 main] /usr/share/foreman/app/registries/foreman/plugin.rb:343:in `pending_migrations'
[ERROR 2020-06-02T15:27:25 main] /usr/share/foreman/app/registries/foreman/plugin.rb:284:in `permission'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-1.1.1/lib/foreman_tasks/engine.rb:52:in `block (3 levels) in <class:Engine>'
[ERROR 2020-06-02T15:27:25 main] /usr/share/foreman/app/registries/foreman/plugin.rb:268:in `instance_eval'
[ERROR 2020-06-02T15:27:25 main] /usr/share/foreman/app/registries/foreman/plugin.rb:268:in `security_block'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-1.1.1/lib/foreman_tasks/engine.rb:51:in `block (2 levels) in <class:Engine>'
[ERROR 2020-06-02T15:27:25 main] /usr/share/foreman/app/registries/foreman/plugin.rb:76:in `instance_eval'
[ERROR 2020-06-02T15:27:25 main] /usr/share/foreman/app/registries/foreman/plugin.rb:76:in `register'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-1.1.1/lib/foreman_tasks/engine.rb:36:in `block in <class:Engine>'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/railties-5.2.1/lib/rails/initializable.rb:32:in `instance_exec'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/railties-5.2.1/lib/rails/initializable.rb:32:in `run'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/railties-5.2.1/lib/rails/initializable.rb:61:in `block in run_initializers'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/railties-5.2.1/lib/rails/initializable.rb:60:in `run_initializers'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/railties-5.2.1/lib/rails/application.rb:361:in `initialize!'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/railties-5.2.1/lib/rails/railtie.rb:190:in `public_send'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/railties-5.2.1/lib/rails/railtie.rb:190:in `method_missing'
[ERROR 2020-06-02T15:27:25 main] /usr/share/foreman/config/environment.rb:5:in `<top (required)>'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/polyglot-0.3.5/lib/polyglot.rb:65:in `require'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/dependencies.rb:287:in `block in require'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/dependencies.rb:253:in `load_dependency'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/dependencies.rb:287:in `require'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/railties-5.2.1/lib/rails/application.rb:337:in `require_environment!'
[ERROR 2020-06-02T15:27:25 main] /opt/theforeman/tfm/root/usr/share/gems/gems/railties-5.2.1/lib/rails/application.rb:520:in `block in run_tasks_blocks'
[ERROR 2020-06-02T15:27:25 main] /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/exe/rake:27:in `<top (required)>'
[ERROR 2020-06-02T15:27:25 main] Tasks: TOP => config => environment
[ERROR 2020-06-02T15:27:25 main] (See full trace by running task with --trace)
[ERROR 2020-06-02T15:27:25 main] Rubocop not loaded.

Looks like we’ll need a similar fix as:

I saw the cockpit issue and have tried the proposed solution of running the installer a couple of times. No joy there I’m afraid.

So is this a bug with puppetdb too? I’d rather not go build an external puppet server and puppetdb just now but I guess that would get me round this problem. Running the installer with --no-enable-puppetdb would also fix it for now but I still get the cockpit error.

Is there any plan to fix?