Failed running forman-maintain: Duplicate declaration: Package[katello] is already declared

zzz · September 12, 2024, 3:29am

Problem:
When running the following, it fails: “foreman-maintain packages install libvirt-client” Can anyone please point me in the direction on what I need to do to resolve this? The same issue appears when I run other commands. i.e. (foreman-installer --enable-foreman-proxy-plugin-remote-execution-scrip)

Thx Chris

Expected outcome:
I expect to be able to install the capability to build KVM hosts

Foreman and Proxy versions:
Foreman Proxy version: foreman-proxy-3.10.0-1.el8.noarch

Foreman and Proxy plugin versions:
foreman-postgresql-3.10.0-1.el8.noarch

rubygem-hammer_cli_foreman_remote_execution-0.3.0-1.fm3_10.el8.noarch

rubygem-foreman_discovery-24.0.1-1.fm3_10.el8.noarch

foreman-selinux-3.10.0-1.el8.noarch

rubygem-hammer_cli_foreman-3.10.0-1.el8.noarch

rubygem-hammer_cli_foreman_tasks-0.0.20-1.fm3_10.el8.noarch

rubygem-foreman_maintain-1.5.1-1.el8.noarch

rubygem-foreman_remote_execution-13.0.0-1.fm3_11.el8.noarch

foreman-installer-3.10.0-1.el8.noarch

foreman-cli-3.10.0-1.el8.noarch

foreman-redis-3.10.0-1.el8.noarch

rubygem-foreman-tasks-9.1.1-1.fm3_11.el8.noarch

foreman-dynflow-sidekiq-3.10.0-1.el8.noarch

foreman-installer-katello-3.10.0-1.el8.noarch

foreman-release-3.10.0-1.el8.noarch

foreman-3.10.0-1.el8.noarch

foreman-service-3.10.0-1.el8.noarch

Distribution and version:
Red Hat Enterprise Linux release 8.10 (Ootpa)

Other relevant data:

--------------------------------------------------------------------------------
Procedures::Installer::Upgrade: 2024-09-12 13:04:45 [WARN  ] [boot] <Array> ["Unsetting environment variable 'http_proxy' for the duration of the install."]
2024-09-12 13:04:45 [WARN  ] [boot] <Array> ["Unsetting environment variable 'https_proxy' for the duration of the install."]
2024-09-12 13:04:47 [NOTICE] [root] Loading installer configuration. This will take some time.
2024-09-12 13:04:53 [NOTICE] [root] Running installer with log based terminal output at level NOTICE.
2024-09-12 13:04:53 [NOTICE] [root] Use -l to set the terminal output log level to ERROR, WARN, NOTICE, INFO, or DEBUG. See --full-help for definitions.
2024-09-12 13:05:02 [NOTICE] [configure] Starting system configuration.
2024-09-12 13:05:11 [ERROR ] [configure] Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Resource Statement, Duplicate declaration: Package[katello] is already declared at (file: /usr/share/foreman-installer/modules/katello/manifests/init.pp, line: 63); cannot redeclare (file: /usr/share/foreman-installer/modules/foreman/manifests/cli/plugin.pp, line: 25) (file: /usr/share/foreman-installer/modules/foreman/manifests/cli/plugin.pp, line: 25, column: 3) (file: /usr/share/foreman-installer/modules/foreman/manifests/cli/katello.pp, line: 8) on node utils.meerkat.mcri.edu.au
2024-09-12 13:05:11 [NOTICE] [configure] System configuration has finished.

evgeni · September 12, 2024, 6:03am

The failing line is puppet-foreman/manifests/cli/plugin.pp at master · theforeman/puppet-foreman · GitHub

Which only would install the katello package if foreman::cli::params::hammer_plugin_prefix is empty (which it shouldn’t be: puppet-foreman/manifests/cli/params.pp at master · theforeman/puppet-foreman · GitHub)

Would you mind sharing the output of grep -r hammer_plugin_prefix /etc/foreman-installer/ /usr/share/foreman-installer/ from the affected system?

zzz · September 12, 2024, 6:31am

Thx so much. Here is the output

[root@utils dynamic]# grep -r hammer_plugin_prefix /etc/foreman-installer/ /usr/share/foreman-installer/
/usr/share/foreman-installer/modules/foreman/manifests/cli/globals.pp:# @param hammer_plugin_prefix
/usr/share/foreman-installer/modules/foreman/manifests/cli/globals.pp:  Optional[String] $hammer_plugin_prefix = undef,
/usr/share/foreman-installer/modules/foreman/manifests/cli/params.pp:      $_hammer_plugin_prefix = 'rubygem-hammer_cli_'
/usr/share/foreman-installer/modules/foreman/manifests/cli/params.pp:      $_hammer_plugin_prefix = 'ruby-hammer-cli-'
/usr/share/foreman-installer/modules/foreman/manifests/cli/params.pp:      $_hammer_plugin_prefix = undef
/usr/share/foreman-installer/modules/foreman/manifests/cli/params.pp:      $_hammer_plugin_prefix = undef
/usr/share/foreman-installer/modules/foreman/manifests/cli/params.pp:      $_hammer_plugin_prefix = undef
/usr/share/foreman-installer/modules/foreman/manifests/cli/params.pp:  $hammer_plugin_prefix = pick($foreman::cli::globals::hammer_plugin_prefix, $_hammer_plugin_prefix)
/usr/share/foreman-installer/modules/foreman/manifests/cli/plugin.pp:  String $package = "${foreman::cli::params::hammer_plugin_prefix}${title}",

evgeni · September 12, 2024, 6:39am

Okay, that looks exactly as it should. You’re not overriding it anywhere (or have old files declaring it).

May I assume that when you call facter os.family the output is RedHat?
And when you run rpm -V foreman-installer foreman-installer-katello the only listed files are in /etc/foreman-installer?

zzz · September 12, 2024, 6:47am

Hi Here is the output. and yep redhat 8.9

[root@utils dynamic]# facter os.family
RedHat
[root@utils dynamic]# rpm -V foreman-installer foreman-installer-katello
S.5....T.  c /etc/foreman-installer/scenarios.d/foreman-answers.yaml
S.5....T.  c /etc/foreman-installer/scenarios.d/foreman-migrations-applied
S.5....T.  c /etc/foreman-installer/scenarios.d/foreman.yaml
S.5....T.  c /etc/foreman-installer/scenarios.d/foreman-proxy-content-answers.yaml
S.5....T.  c /etc/foreman-installer/scenarios.d/foreman-proxy-content-migrations-applied
S.5....T.  c /etc/foreman-installer/scenarios.d/foreman-proxy-content.yaml
S.5....T.  c /etc/foreman-installer/scenarios.d/katello-answers.yaml
S.5....T.  c /etc/foreman-installer/scenarios.d/katello-migrations-applied
S.5....T.  c /etc/foreman-installer/scenarios.d/katello.yaml

evgeni · September 12, 2024, 7:12am

Well, damn.

Can you create an sos report please (dnf install sos && sos report) and share the resulting file with us? (The tool tries clean up passwords and other really sensitive information, but the report will contain hostnames etc – probably best to share it via DM or mail to evgeni at redhat dot com)

zzz · September 12, 2024, 7:19am

Will do. I’ll send it (10:00 am AEST) tomorrow. Thx

zzz · September 12, 2024, 10:58am

Dumb question but how do I DM you on this forum. Not seeing a button. I have the sosreport but it will be too big to email.

evgeni · September 12, 2024, 11:12am

You can also drop it in Google Drive or something and just share the link.

Anyway, you go to Profile - evgeni - TheForeman and there is “Message” button

ekohl · September 12, 2024, 11:48am

Do you by any chance have missed the --foreman-cli true parameter to enable the base CLI?

zzz · September 12, 2024, 12:10pm

Another error when trying this.

[root@utils ~]# **foreman-installer --enable-foreman-cli**
2024-09-12 22:00:38 [WARN  ] [boot] <Array> ["Unsetting environment variable 'http_proxy' for the duration of the install."]
2024-09-12 22:00:38 [WARN  ] [boot] <Array> ["Unsetting environment variable 'https_proxy' for the duration of the install."]
2024-09-12 22:00:40 [NOTICE] [root] Loading installer configuration. This will take some time.
2024-09-12 22:00:45 [NOTICE] [root] Running installer with log based terminal output at level NOTICE.
2024-09-12 22:00:45 [NOTICE] [root] Use -l to set the terminal output log level to ERROR, WARN, NOTICE, INFO, or DEBUG. See --full-help for definitions.
2024-09-12 22:00:54 [NOTICE] [configure] Starting system configuration.
2024-09-12 22:01:20 [NOTICE] [configure] 250 configuration steps out of 1820 steps complete.
2024-09-12 22:01:23 [NOTICE] [configure] 500 configuration steps out of 1889 steps complete.
2024-09-12 22:01:24 [NOTICE] [configure] 750 configuration steps out of 1891 steps complete.
2024-09-12 22:01:34 [NOTICE] [configure] 1000 configuration steps out of 1898 steps complete.
2024-09-12 22:01:35 [NOTICE] [configure] 1250 configuration steps out of 1903 steps complete.
2024-09-12 22:02:09 [NOTICE] [configure] 1500 configuration steps out of 1903 steps complete.
2024-09-12 22:03:07 [NOTICE] [configure] 1750 configuration steps out of 1903 steps complete.
2024-09-12 22:03:07 [ERROR ] [configure] /Stage[main]/Foreman::Register/Foreman_host[foreman-utils.blahi.edu.au]: Could not evaluate: Exception SSL_connect returned=1 errno=0 state=error: certificate verify failed (self signed certificate in certificate chain) in get request to: https://utils.blah.edu.au/api/v2/hosts?search=name%3D%22utils.blah.edu.au%22
2024-09-12 22:03:07 [ERROR ] [configure] Wrapped exception:
2024-09-12 22:03:07 [ERROR ] [configure] SSL_connect returned=1 errno=0 state=error: certificate verify failed (self signed certificate in certificate chain)
2024-09-12 22:04:10 [NOTICE] [configure] System configuration has finished.

Error 1: Puppet Foreman_host resource 'foreman-utils.blah.edu.au' failed. Logs:
  /Stage[main]/Foreman::Register/Foreman_host[foreman-utils.blah.edu.au]
    Adding autorequire relationship with Anchor[foreman::service]
    Adding autorequire relationship with Anchor[foreman::providers::oauth]
    Starting to evaluate the resource (1783 of 1903)
    Could not evaluate: Exception SSL_connect returned=1 errno=0 state=error: certificate verify failed (self signed certificate in certificate chain) in get request to: https://utils.blah.edu.au/api/v2/hosts?search=name%3D%22utils.blah.edu.au%22
Wrapped exception:
SSL_connect returned=1 errno=0 state=error: certificate verify failed (self signed certificate in certificate chain)
    Evaluated in 0.02 seconds
  Foreman_host[foreman-utils.blah.edu.au](provider=rest_v3)
    Making get request to https://utils.blah.edu.au/api/v2/hosts?search=name%3D%22utils.blah.edu.au%22

1 error was detected during installation.
Please address the errors and re-run the installer to ensure the system is properly configured.
Failing to do so is likely to result in broken functionality.type or paste code here

evgeni · September 12, 2024, 12:37pm

Got the sosreport, will try to look into it tomorrow morning.

zzz · September 12, 2024, 12:43pm

Many thanks.

Regards,

Christopher Welsh
Scientific Computing (Cluster/HPC)
IT Infrastructure Team
Information Technology

Murdoch Children’s Research Institute
The Royal Children’s Hospital, 50 Flemington Road
Parkville, Victoria 3052 Australia

zzz · September 13, 2024, 3:22am

Hi mate, did you get a chance to look over the sos report? Thx.

evgeni · September 13, 2024, 6:41am

Hey good morning.

Yeah… How did you set up this machine?

Your installer answers file has

foreman: false
foreman::cli: false

Which is roughly “do not install anything Foreman related”, but then Foreman packages are installed and services are running.

In this state, I would not expect the installer to work properly

zzz · September 13, 2024, 7:10am

I believe the following way I’d set it up. I am sure I missed some commands though.

dnf install https://yum.theforeman.org/releases/3.10/el9/x86_64/foreman-release.rpm
dnf install https://yum.theforeman.org/katello/4.12/katello/el9/x86_64/katello-repos-latest.rpm
dnf install https://yum.puppet.com/puppet7-release-el-9.noarch.rpm
dnf update
dnf install foreman-installer-katello
foreman-installer --scenario katello
foreman-installer \
--foreman-proxy-dns true \
--foreman-proxy-dns-managed true \
--foreman-proxy-dns-interface eno4 \
--foreman-proxy-dns-zone mydomain \
--foreman-proxy-dns-reverse 10.10.in-adr.arpa \
--foreman-proxy-dhcp true \
--foreman-proxy-dhcp-managed true \
--foreman-proxy-dhcp-interface eno4 \
--foreman-proxy-dhcp-range "10.10.255.5 10.10.255.250" \
--foreman-proxy-dhcp-gateway 10.10.112.6 \
--foreman-proxy-dhcp-nameservers 10.10.112.6 \
--foreman-proxy-tftp true \
--foreman-proxy-tftp-managed true \
--foreman-proxy-tftp-servername 10.10.112.6

type or paste code here

Can you recommend any corrective steps to get it working again? I have spent so many hours configuring this and most of what I need works.

In terms of remediation:
i.e. would I need to edit the file to set these to true? Or run a foreman command magic formula?

BTY, Thx so much for your quick response as I’m new to this build.

evgeni · September 13, 2024, 7:19am

You should not ever edit that file by hand, no.

The above steps should not have gotten you into this situation (but they are for EL9, your sosreport is from EL8).
At some point the installer would need to have been run with --no-enable-foreman.

You can try fixing this by running with --enable-foreman, but you better have backups (well, you better have backups either way, right?)

zzz · September 13, 2024, 9:37am

Thx mate,

The server is backed up but only recently

Christopher Welsh,
Scientific Computing (Cluster/HPC)
IT Infrastructure Team
Information Technology

Murdoch Childrens Research Institute
The Royal Childrens Hospital, 50 Flemington Rd,
Parkville, Victoria, 3052 Australia

zzz · October 22, 2024, 6:06am

Hi Evgeni,

Just back from leave but wanted to thank you Evgeni and the broader community for the effort you put in to resolving my issue. I don’t know if you can imagine the pressure this take off us. For me it was huge. Thx

For the record it turned out I needed the certs moved from puppet to Martello. Here is the remediation.

From evgeni:

Edit the answers file:
vim /etc/foreman-installer/scenarios.d/katello-answers.yaml
At line 36, you should see a “foreman” section starting.
Around line 59 you should see various server_ssl_* parameters being
set, especially server_ssl_ca, server_ssl_chain, server_ssl_cert,
server_ssl_key.
They’d be using “/etc/puppetlabs/puppet/” paths right now.
Change them to read as following:
server_ssl_ca: “/etc/pki/katello/certs/katello-default-ca.crt”
server_ssl_chain: “/etc/pki/katello/certs/katello-server-ca.crt”
server_ssl_cert: “/etc/pki/katello/certs/katello-apache.crt”
server_ssl_key: “/etc/pki/katello/private/katello-apache.key”

If your file does NOT contain the old puppetlabs values, DON’T EDIT.
Show me lines 35-60 please.

If you did edit, run the installer, show me the output