Foreman 3.1 install failure on Ubuntu 20.04 LTS

blackknight36 · March 17, 2022, 2:26pm

The foreman install command fails when run on Ubuntu 18.04 with an error as follows.

2022-03-17 07:11:42 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns:
2022-03-17 07:11:42 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: Caused by:
2022-03-17 07:11:42 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: ActiveRecord::StatementInvalid: PG::UndefinedTable: ERROR:  relation "environments" does not exist

I have verified that the database exists and is accessible by foreman however there is no “environments” table as mentioned in the error message. Is there a way to manually create this table? Is there anything else I need to do before the install will work?

Foreman and Proxy versions:
3.1 release.

Foreman and Proxy plugin versions:
3.1 release

Distribution and version:
Ubuntu 20.04 LTS

evgeni · March 17, 2022, 6:11pm

Can you please provide more context from the installer log? It should be visible which migration triggers this issue.

blackknight36 · March 17, 2022, 7:11pm

Sure. Here are the log entries regarding db:migrate.

2022-03-17 07:28:56 [DEBUG ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]: Starting to evaluate the resource (1013 of 1072)
2022-03-17 07:28:56 [DEBUG ] [configure] Exec[foreman-rake-db:migrate](provider=posix): Executing check '/usr/sbin/foreman-rake db:abort_if_pending_migrations'
2022-03-17 07:29:01 [DEBUG ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/unless: rake aborted!
2022-03-17 07:29:01 [DEBUG ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/unless: Errno::EEXIST: File exists @ dir_s_mkdir - /usr/share/foreman/log
2022-03-17 07:29:01 [DEBUG ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/unless: /usr/share/foreman/lib/foreman/logging.rb:135:in `ensure_log_directory'
2022-03-17 07:29:01 [DEBUG ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/unless: /usr/share/foreman/lib/foreman/logging.rb:15:in `configure'
2022-03-17 07:29:01 [DEBUG ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/unless: /usr/share/foreman/config/application.rb:230:in `<class:Application>'
2022-03-17 07:29:01 [DEBUG ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/unless: /usr/share/foreman/config/application.rb:98:in `<module:Foreman>'
2022-03-17 07:29:01 [DEBUG ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/unless: /usr/share/foreman/config/application.rb:96:in `<top (required)>'
2022-03-17 07:29:01 [DEBUG ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/unless: /usr/share/foreman/Rakefile:1:in `require'
2022-03-17 07:29:01 [DEBUG ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/unless: /usr/share/foreman/Rakefile:1:in `<top (required)>'
2022-03-17 07:29:01 [DEBUG ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/unless: /usr/share/foreman/vendor/ruby/2.7.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
2022-03-17 07:29:01 [DEBUG ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/unless: (See full trace by running task with --trace)
2022-03-17 07:29:01 [DEBUG ] [configure] Exec[foreman-rake-db:migrate](provider=posix): Executing '/usr/sbin/foreman-rake db:migrate'
2022-03-17 07:29:01 [DEBUG ] [configure] Executing with uid=foreman: '/usr/sbin/foreman-rake db:migrate'
2022-03-17 07:29:07 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: rake aborted!
2022-03-17 07:29:07 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: Errno::EEXIST: File exists @ dir_s_mkdir - /usr/share/foreman/log
2022-03-17 07:29:07 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/foreman/lib/foreman/logging.rb:135:in `ensure_log_directory
2022-03-17 07:29:07 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/foreman/lib/foreman/logging.rb:135:in `ensure_log_directory'
2022-03-17 07:29:07 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/foreman/lib/foreman/logging.rb:15:in `configure'
2022-03-17 07:29:07 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/foreman/config/application.rb:230:in `<class:Application>'
2022-03-17 07:29:07 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/foreman/config/application.rb:98:in `<module:Foreman>'
2022-03-17 07:29:07 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/foreman/config/application.rb:96:in `<top (required)>'
2022-03-17 07:29:07 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/foreman/Rakefile:1:in `require'
2022-03-17 07:29:07 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/foreman/Rakefile:1:in `<top (required)>'
2022-03-17 07:29:07 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/foreman/vendor/ruby/2.7.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
2022-03-17 07:29:07 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: (See full trace by running task with --trace)
2022-03-17 07:29:07 [ERROR ] [configure] '/usr/sbin/foreman-rake db:migrate' returned 1 instead of one of [0]
2022-03-17 07:29:07 [ERROR ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: change from 'notrun' to ['0'] failed: '/usr/sbin/foreman-rake db:migrate' returned 1 instead of one of [0]
2022-03-17 07:29:07 [DEBUG ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]: Evaluated in 10.46 seconds
2022-03-17 07:29:07 [DEBUG ] [configure] Foreman::Rake[db:migrate]: Starting to evaluate the resource (1014 of 1072)
2022-03-17 07:29:07 [DEBUG ] [configure] Foreman::Rake[db:migrate]: Resource is being skipped, unscheduling all events
2022-03-17 07:29:07 [DEBUG ] [configure] Foreman::Rake[db:migrate]: Evaluated in 0.00 seconds

The install command I used is as follows.

cmd = 'foreman-installer --enable-foreman-plugin-tasks --enable-foreman-proxy-plugin-chef --foreman-proxy-plugin-chef-server-url="https://chef.it.example.com/organizations/default" --foreman-proxy-plugin-chef-client-name="pivotal" --foreman-proxy-plugin-chef-private-key="/etc/opscode/pivotal.pem" --no-enable-puppet --foreman-server-ssl-crl='

evgeni · March 18, 2022, 7:39am

That doesn’t look like the same installer invocation and a different error?

You should have logs of the previous runs in /var/log/foreman-installer

blackknight36 · March 23, 2022, 8:58pm

The second command posted was run manually, the first one was done by chef. The error appears to be the same however, the environments table is missing. Is there an SQL file I can run to create this table?
`

blackknight36 · March 24, 2022, 3:07pm

I’ve done some more testing and it appears that the rake db:migrate task is failing due to a missing configuration file. There are two broken symlinks under /usr/share/foreman as shown in the attached screenshot.

Reinstalling the foreman package did not change anything.

blackknight36 · March 24, 2022, 3:17pm

Another issue. After creating the missing configuration files Apache fails to start due to an error in the configuration.

2022-03-24 08:13:00 [ERROR ] [configure] Mar 24 08:13:00 VMWARE47AA1C-VM systemd[1]: Failed to start The Apache HTTP Server.
2022-03-24 08:13:00 [ERROR ] [configure] /Stage[main]/Apache::Service/Service[httpd]/ensure: change from 'stopped' to 'running' failed: Systemd start for apache2 failed!
2022-03-24 08:13:00 [ERROR ] [configure] journalctl log for apache2:
2022-03-24 08:13:00 [ERROR ] [configure] -- Logs begin at Wed 2022-03-23 07:06:09 PDT, end at Thu 2022-03-24 08:13:00 PDT. --
2022-03-24 08:13:00 [ERROR ] [configure] Mar 24 08:13:00 VMWARE47AA1C-VM systemd[1]: Starting The Apache HTTP Server...
2022-03-24 08:13:00 [ERROR ] [configure] Mar 24 08:13:00 VMWARE47AA1C-VM apachectl[2823498]: AH00526: Syntax error on line 69 of /etc/apache2/sites-enabled/05-foreman-ssl.conf:
2022-03-24 08:13:00 [ERROR ] [configure] Mar 24 08:13:00 VMWARE47AA1C-VM apachectl[2823498]: SSLCertificateChainFile: file '/etc/puppetlabs/puppet/ssl/certs/ca.pem' does not exist or is empty
2022-03-24 08:13:00 [ERROR ] [configure] Mar 24 08:13:00 VMWARE47AA1C-VM apachectl[2823495]: Action 'start' failed.
2022-03-24 08:13:00 [ERROR ] [configure] Mar 24 08:13:00 VMWARE47AA1C-VM apachectl[2823495]: The Apache error log may have more information.
2022-03-24 08:13:00 [ERROR ] [configure] Mar 24 08:13:00 VMWARE47AA1C-VM systemd[1]: apache2.service: Control process exited, code=exited, status=1/FAILURE
2022-03-24 08:13:00 [ERROR ] [configure] Mar 24 08:13:00 VMWARE47AA1C-VM systemd[1]: apache2.service: Failed with result 'exit-code'.
2022-03-24 08:13:00 [ERROR ] [configure] Mar 24 08:13:00 VMWARE47AA1C-VM systemd[1]: Failed to start The Apache HTTP Server.
2022-03-24 08:15:28 [NOTICE] [configure] System configuration has finished.

Of course there is no CA file because we do not use puppet.

I was able to resolve this after manually copying the /etc/ssl/certs/ca-certificates.crt file over to /etc/puppetlabs/puppet/ssl/certs/ca.pem.

evgeni · March 25, 2022, 2:16pm

Yeah, if you don’t enable Puppet, you need to provide certs by some other means. If you don’t things explode.