Foreman 3.11 upgrade - pgsql 13 database fails to start

Problem:

During Foreman 3.11 upgrade, postgresql 13 database fails to start.

Expected outcome:

Successful completion of the Forman 3.10 to 3.11 upgrade

Foreman and Proxy versions:

Foreman 3.10

Foreman and Proxy plugin versions:

Standard installation, no special plugins

Distribution and version:

Rocky Linux 8.10

System specs:

Disk: 60 GB (44 GB free)
vCPUs: 4
RAM: 10 GB

Other relevant data:

In the Upgrade to 3.11 documentation, step 3A, ‘foreman-rake db:migrate’ fails:

foreman-rake db:migrate
rake aborted!
ActiveRecord::ConnectionNotEstablished: could not connect to server: No such file or directory

Investigation reveals that service postgresql is not running:

systemctl status postgresql
● postgresql.service - PostgreSQL database server
   Loaded: loaded (/usr/lib/systemd/system/postgresql.service; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/postgresql.service.d
           └─postgresql.conf
   Active: failed (Result: exit-code) since Sun 2024-06-30 00:17:44 EDT; 11s ago
  Process: 32462 ExecStartPre=/usr/libexec/postgresql-check-db-dir postgresql (code=exited, status=1/FAILURE)
 Main PID: 773 (code=exited, status=0/SUCCESS)

Jun 30 00:17:44 foreman2.intranet.psfc.coop systemd[1]: Starting PostgreSQL database server...
Jun 30 00:17:44 foreman2.intranet.psfc.coop postgresql-check-db-dir[32462]: An old version of the database format was found.
Jun 30 00:17:44 foreman2.intranet.psfc.coop postgresql-check-db-dir[32462]: Use 'postgresql-setup --upgrade' to upgrade to version '13'
Jun 30 00:17:44 foreman2.intranet.psfc.coop postgresql-check-db-dir[32462]: See /usr/share/doc/postgresql/README.rpm-dist for more information.
Jun 30 00:17:44 foreman2.intranet.psfc.coop systemd[1]: postgresql.service: Control process exited, code=exited status=1
Jun 30 00:17:44 foreman2.intranet.psfc.coop systemd[1]: postgresql.service: Failed with result 'exit-code'.
Jun 30 00:17:44 foreman2.intranet.psfc.coop systemd[1]: Failed to start PostgreSQL database server.

Running ‘postgresql-setup --upgrade’, as suggested in the output above, is successful in upgrading the database, but two additional steps are necessary. Here are the four (4) additional steps I performed to upgrade the database after postgresql 13 packages were installed (dnf module switch-to postgresql:13) per documentation:

  1. Delete, or comment out, line 756 in file /var/lib/pgsql/data/postgresql.conf to resolve an error while running “postgresql-setup --upgrade”
   grep -n '/var/lib/pgsql/data' /var/lib/pgsql/data/postgresql.conf
   > 756:data_directory = '/var/lib/pgsql/data'
  1. Install the postgresql-upgrade package:

    dnf install postgresql-upgrade

  2. Run the postgresql database upgrade:

    postgresql-setup --upgrade

  3. Start the postgresql database service:

    systemctl start postgresql.service

After performing these steps, I was able to continue with the Foreman 3.11 upgrade (foreman-rake db:migrate, etc.) and I now have a fully functional Foreman 3.11 system.

If verified, can these steps be added to the upgrade documentation?

Best regards,

-jm

3 Likes

@Griffin-Sullivan and @ekohl any thoughts on this?

Yes, that does indeed look like an oversight. When using the installer to upgrade we take care of these steps for you. That was added here:

This should be added to the manual upgrade documentation.