Problem: container_gateway service does not start occasionally or after server reboot on foreman smart proxy
Expected outcome: All foreman smart proxy services start smoothly after reboot
Foreman and Proxy versions: 3.12/4.14
Foreman and Proxy plugin versions:
Distribution and version: RHEL9
Other relevant data: The service is probably starting earlier than postgres db see logs:
2025-02-09T20:29:45 [I] Successfully initialized 'ansible'
2025-02-09T20:29:45 [E] Couldn't enable 'container_gateway'
2025-02-09T20:29:45 [W] Error details for Couldn't enable 'container_gateway': <Sequel::DatabaseConnectionError>: PG::ConnectionBad: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
/usr/share/gems/gems/pg-1.5.7/lib/pg/connection.rb:836:in `connect_start'
/usr/share/gems/gems/pg-1.5.7/lib/pg/connection.rb:836:in `connect_to_hosts'
/usr/share/gems/gems/pg-1.5.7/lib/pg/connection.rb:772:in `new'
/usr/share/gems/gems/sequel-5.83.1/lib/sequel/adapters/postgres.rb:231:in `connect'
/usr/share/gems/gems/sequel-5.83.1/lib/sequel/database/connecting.rb:245:in `new_connection'
/usr/share/gems/gems/sequel-5.83.1/lib/sequel/connection_pool.rb:163:in `make_new'
/usr/share/gems/gems/sequel-5.83.1/lib/sequel/connection_pool/threaded.rb:225:in `assign_connection'
/usr/share/gems/gems/sequel-5.83.1/lib/sequel/connection_pool/threaded.rb:139:in `acquire'
/usr/share/gems/gems/sequel-5.83.1/lib/sequel/connection_pool/threaded.rb:91:in `hold'
/usr/share/gems/gems/sequel-5.83.1/lib/sequel/database/connecting.rb:283:in `synchronize'
/usr/share/gems/gems/sequel-5.83.1/lib/sequel/database/connecting.rb:291:in `test_connection'
/usr/share/gems/gems/sequel-5.83.1/lib/sequel/database/misc.rb:200:in `initialize'
/usr/share/gems/gems/sequel-5.83.1/lib/sequel/database/connecting.rb:54:in `new'
/usr/share/gems/gems/sequel-5.83.1/lib/sequel/database/connecting.rb:54:in `connect'
/usr/share/gems/gems/sequel-5.83.1/lib/sequel/core.rb:124:in `connect'
/usr/share/gems/gems/smart_proxy_container_gateway-3.1.0/lib/smart_proxy_container_gateway/database.rb:8:in `initialize'
/usr/share/gems/gems/smart_proxy_container_gateway-3.1.0/lib/smart_proxy_container_gateway/container_gateway.rb:39:in `new'
/usr/share/gems/gems/smart_proxy_container_gateway-3.1.0/lib/smart_proxy_container_gateway/container_gateway.rb:39:in `block (2 levels) in <class:Plugin>'
/usr/share/foreman-proxy/lib/proxy/dependency_injection.rb:14:in `initialize'
/usr/share/foreman-proxy/lib/proxy/dependency_injection.rb:48:in `new'
/usr/share/foreman-proxy/lib/proxy/dependency_injection.rb:48:in `singleton_dependency'
/usr/share/gems/gems/smart_proxy_container_gateway-3.1.0/lib/smart_proxy_container_gateway/container_gateway.rb:29:in `block in <class:Plugin>'
/usr/share/foreman-proxy/lib/proxy/pluggable.rb:153:in `load_dependency_injection_wirings'
/usr/share/foreman-proxy/lib/proxy/plugin_initializer.rb:249:in `block in wire_up_dependencies'
/usr/share/foreman-proxy/lib/proxy/plugin_initializer.rb:248:in `each'
/usr/share/foreman-proxy/lib/proxy/plugin_initializer.rb:248:in `wire_up_dependencies'
/usr/share/foreman-proxy/lib/proxy/plugin_initializer.rb:238:in `configure_plugin'
/usr/share/foreman-proxy/lib/proxy/plugin_initializer.rb:109:in `block in configure'
/usr/share/foreman-proxy/lib/proxy/plugin_initializer.rb:109:in `each'
/usr/share/foreman-proxy/lib/proxy/plugin_initializer.rb:109:in `configure'
/usr/share/foreman-proxy/lib/proxy/plugin_initializer.rb:187:in `each'
/usr/share/foreman-proxy/lib/proxy/plugin_initializer.rb:187:in `initialize_plugins'
/usr/share/foreman-proxy/lib/launcher.rb:129:in `launch'
/usr/share/foreman-proxy/bin/smart-proxy:6:in `<main>'
2025-02-09T20:29:45 [E] Disabling all modules in the group ['container_gateway'] due to a failure in one of them: PG::ConnectionBad: could not connect to server: No such file or directory