Error updating vom 2.3.1 on 2.3.2 on a katello 3.18.1

Problem:
Because I want to test Bug #31662: Foreman content proxy doesn't listen on port 80 anymore - Installer - Foreman and 2.3.2 was just released I wanted to update from 2.3.1 to 2.3.2 on my katello 3.18.1 before testing the patch. However, the foreman-installer run quickly failed with an error:

2021-01-20 06:45:50 [ERROR ] [root] Target /var/lib/pulp/media already exists. Unable to migrate Pulp media root from /var/lib/pulp/docroot.

Ensure /var/lib/pulp/docroot is not a directory or /var/lib/pulp/media does not exist.

Both directories exist and both are empty.

Foreman and Proxy versions:
Foreman 2.3.2 updated from 2.3.1
Katello 3.18.1

Distribution and version:
CentOS 7.9.2009

Other relevant data:
During the update the following rpms where updated:

Updating:
 foreman                 noarch 2.3.2-1.el7   ORG_foreman_2_3_el7_x86_64  35 M
 foreman-cli             noarch 2.3.2-1.el7   ORG_foreman_2_3_el7_x86_64  23 k
 foreman-debug           noarch 2.3.2-1.el7   ORG_foreman_2_3_el7_x86_64  29 k
 foreman-dynflow-sidekiq noarch 2.3.2-1.el7   ORG_foreman_2_3_el7_x86_64  26 k
 foreman-installer       noarch 1:2.3.2-1.el7 ORG_foreman_2_3_el7_x86_64 1.7 M
 foreman-installer-katello
                         noarch 1:2.3.2-1.el7 ORG_foreman_2_3_el7_x86_64  74 k
 foreman-postgresql      noarch 2.3.2-1.el7   ORG_foreman_2_3_el7_x86_64  24 k
 foreman-proxy           noarch 2.3.2-1.el7   ORG_foreman_2_3_el7_x86_64 153 k
 foreman-release         noarch 2.3.2-1.el7   ORG_foreman_2_3_el7_x86_64 8.4 k
 foreman-selinux         noarch 2.3.2-1.el7   ORG_foreman_2_3_el7_x86_64  52 k
 foreman-service         noarch 2.3.2-1.el7   ORG_foreman_2_3_el7_x86_64  26 k
 python3-pulp-2to3-migration
                         noarch 0.6.0-2.el7   ORG_katello_pulpcore_3_7_el7_x86_64
                                                                          176 k

if /var/lib/pulp/media is truly empty, you can remove that directory and re-run the installer.

@ekohl, we wanted to have some guidance for users that run into this situation – did we create any yet?

1 Like

Just noticed another error message during yum update on the content proxy, i.e. update from foreman 2.3.1 to 2.3.2 on a smart proxy for content on a katello 3.18.1 installation:

 (1/5): foreman-debug-2.3.2-1.el7.noarch.rpm                                                                                                                                                                 |  29 kB  00:00:00     
(2/5): foreman-installer-2.3.2-1.el7.noarch.rpm                                                                                                                                                             | 1.7 MB  00:00:00     
(3/5): foreman-installer-katello-2.3.2-1.el7.noarch.rpm                                                                                                                                                     |  74 kB  00:00:00     
(4/5): foreman-selinux-2.3.2-1.el7.noarch.rpm                                                                                                                                                               |  52 kB  00:00:00     
(5/5): foreman-proxy-2.3.2-1.el7.noarch.rpm                                                                                                                                                                 | 153 kB  00:00:00     
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                              5.3 MB/s | 2.0 MB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Updating   : foreman-debug-2.3.2-1.el7.noarch                                                                                                                                                                               1/10 
  Updating   : 1:foreman-installer-2.3.2-1.el7.noarch                                                                                                                                                                         2/10 
  Updating   : 1:foreman-installer-katello-2.3.2-1.el7.noarch                                                                                                                                                                 3/10 
warning: /etc/foreman-installer/scenarios.d/katello-answers.yaml created as /etc/foreman-installer/scenarios.d/katello-answers.yaml.rpmnew
/etc/foreman-installer/scenarios.d/katello.migrations/200605154320-dont-use-pulpcore-rpm-on-upgrades.rb:9:in `block (2 levels) in load_migrations': undefined method `[]' for true:TrueClass (NoMethodError)
	from /opt/theforeman/tfm/root/usr/share/gems/gems/kafo-6.1.2/lib/kafo/migrations.rb:25:in `instance_eval'
	from /opt/theforeman/tfm/root/usr/share/gems/gems/kafo-6.1.2/lib/kafo/migrations.rb:25:in `block (2 levels) in load_migrations'
	from /opt/theforeman/tfm/root/usr/share/gems/gems/kafo-6.1.2/lib/kafo/migration_context.rb:10:in `instance_eval'
	from /opt/theforeman/tfm/root/usr/share/gems/gems/kafo-6.1.2/lib/kafo/migration_context.rb:10:in `execute'
	from /opt/theforeman/tfm/root/usr/share/gems/gems/kafo-6.1.2/lib/kafo/migrations.rb:38:in `block in run'
	from /opt/theforeman/tfm/root/usr/share/gems/gems/kafo-6.1.2/lib/kafo/migrations.rb:35:in `each'
	from /opt/theforeman/tfm/root/usr/share/gems/gems/kafo-6.1.2/lib/kafo/migrations.rb:35:in `run'
	from /opt/theforeman/tfm/root/usr/share/gems/gems/kafo-6.1.2/lib/kafo/configuration.rb:324:in `run_migrations'
	from /opt/theforeman/tfm/root/usr/share/gems/gems/kafo-6.1.2/lib/kafo/kafo_configure.rb:115:in `initialize'
	from /opt/theforeman/tfm/root/usr/share/gems/gems/clamp-1.1.2/lib/clamp/command.rb:132:in `new'
	from /opt/theforeman/tfm/root/usr/share/gems/gems/clamp-1.1.2/lib/clamp/command.rb:132:in `run'
	from /opt/theforeman/tfm/root/usr/share/gems/gems/kafo-6.1.2/lib/kafo/kafo_configure.rb:50:in `run'
	from /sbin/foreman-installer:8:in `<main>'
warning: %post(foreman-installer-katello-1:2.3.2-1.el7.noarch) scriptlet failed, exit status 1
Non-fatal POSTIN scriptlet failure in rpm package 1:foreman-installer-katello-2.3.2-1.el7.noarch
  Updating   : foreman-proxy-2.3.2-1.el7.noarch                                                                                                                                                                               4/10 
  Updating   : foreman-selinux-2.3.2-1.el7.noarch                                                                                                                                                                             5/10 
  Cleanup    : 1:foreman-installer-katello-2.3.1-1.el7.noarch                                                                                                                                                                 6/10 
  Cleanup    : foreman-proxy-2.3.1-1.el7.noarch                                                                                                                                                                               7/10 
  Cleanup    : foreman-debug-2.3.1-1.el7.noarch                                                                                                                                                                               8/10 
  Cleanup    : 1:foreman-installer-2.3.1-1.el7.noarch                                                                                                                                                                         9/10 
  Cleanup    : foreman-selinux-2.3.1-1.el7.noarch                                                                                                                                                                            10/10 
Uploading Package Profile
Loaded plugins: fastestmirror, product-id, subscription-manager
Loaded plugins: fastestmirror, product-id, subscription-manager
Uploading Tracer Profile
  Verifying  : 1:foreman-installer-2.3.2-1.el7.noarch                                                                                                                                                                         1/10 
  Verifying  : foreman-debug-2.3.2-1.el7.noarch                                                                                                                                                                               2/10 
  Verifying  : foreman-proxy-2.3.2-1.el7.noarch                                                                                                                                                                               3/10 
  Verifying  : foreman-selinux-2.3.2-1.el7.noarch                                                                                                                                                                             4/10 
  Verifying  : 1:foreman-installer-katello-2.3.2-1.el7.noarch                                                                                                                                                                 5/10 
  Verifying  : 1:foreman-installer-2.3.1-1.el7.noarch                                                                                                                                                                         6/10 
  Verifying  : foreman-debug-2.3.1-1.el7.noarch                                                                                                                                                                               7/10 
  Verifying  : foreman-proxy-2.3.1-1.el7.noarch                                                                                                                                                                               8/10 
  Verifying  : foreman-selinux-2.3.1-1.el7.noarch                                                                                                                                                                             9/10 
  Verifying  : 1:foreman-installer-katello-2.3.1-1.el7.noarch                                                                                                                                                                10/10 

Updated:
  foreman-debug.noarch 0:2.3.2-1.el7       foreman-installer.noarch 1:2.3.2-1.el7       foreman-installer-katello.noarch 1:2.3.2-1.el7       foreman-proxy.noarch 0:2.3.2-1.el7       foreman-selinux.noarch 0:2.3.2-1.el7      

Complete!
Uploading Enabled Repositories Report
Loaded plugins: fastestmirror, product-id, subscription-manager

Is there any guidance on how to work past this? I ran into this on a test instance today. I can rebuild it if I need to but if there’s an easy workaround I would prefer it.

As mentioned before:

Thanks. That doesn’t apply to my situation. I was asking for the other scenario - if the directory isn’t empty.

Does anybody have any suggestions on how to get past this? I just tested again with 3.18.1 and it’s the same situation.

/var/lib/pulp/media is not empty on this particular system so i can’t use the posted workaround.

What do you have in those two directories?

[root@foremandev docroot]# ls
artifact

[root@foremandev media]# ls
artifact

And what is inside the artifact directories? The same content?

The directory structures look very similar but I have 49GB in docroot and 8GB in media

I’m pretty sure those numbers correlate to my deb repos in pulp2 (49GB) and my SuSE repos in pulp3. When this instance was built - deb repos were pulp2 only.

In case anybody is in a similar situation and wondering what happened here - I gave up and reloaded it.