That’s an odd one - it can’t find the kafo_configure module. Can you verify /opt/theforeman/tfm/root/usr/share/gems/gems/kafo-4.1.0/modules does exist and lists the kafo_configure module?
It’s there:
ls -l /opt/theforeman/tfm/root/usr/share/gems/gems/kafo-4.1.0/modules
total 4
drwxr-xr-x 5 root root 4096 Jul 6 18:15 kafo_configure
I had started chasing this down, but sadly I just don’t know enough about how these are being called to work it out. It looked to me like puppet_version_semver IS a defined resource type, so I didn’t understand why it wasn’t found:
ls -l /opt/theforeman/tfm/root/usr/share/gems/gems/kafo-4.1.0/modules/kafo_configure/manifests/
total 12
-rw-r–r-- 1 root root 763 Apr 6 14:35 dump_values.pp
-rw-r–r-- 1 root root 526 Apr 6 14:35 init.pp
-rw-r–r-- 1 root root 281 Apr 6 14:35 puppet_version_semver.pp
cat /opt/theforeman/tfm/root/usr/share/gems/gems/kafo-
4.1.0/modules/kafo_configure/manifests/init.pp
# This class is called from the kafo configure script
# and expects a yaml file to exist at either:
# optional $answers class parameter
# $modulepath/config/answers.yaml
# /etc/kafo-configure/answers.yaml
#
# @param add_progress
# Whether to add a progress bar. Only works on Puppet < 6.
class kafo_configure(
Boolean $add_progress = $::kafo_add_progress,
) {
if $add_progress and SemVer($facts[‘puppetversion’]) =~ SemVerRange(‘< 6.0.0’) {
add_progress()
}
lookup(‘classes’, {merge => unique}).include
}
cat /opt/theforeman/tfm/root/usr/share/gems/gems/kafo-
4.1.0/modules/kafo_configure/manifests/puppet_version_semver.pp
define
kafo_configure::puppet_version_semver($requirement) {
unless SemVer($facts[‘puppetversion’]) =~ SemVerRange($requirement) {
fail(“kafo_configure::puppet_version_failure: Puppet ${facts[‘puppetversion’]} does not meet
requirements for ${title} (${requirement})”)
}
}
Are you by any chance using RVM or other similar technologies?
I’m not using RVM
Any possibility this is a path problem? I see semantic_puppet on the system, but it seems like it isn’t being found.
find / -name semantic_puppet
/opt/puppetlabs/server/data/puppetserver/vendored-jruby-gems/gems/semantic_puppet-1.0.2/lib/semantic_puppet
/opt/puppetlabs/server/data/puppetserver/vendored-jruby-gems/gems/semantic_puppet-1.0.2/spec/unit/semantic_puppet
/opt/puppetlabs/puppet/lib/ruby/vendor_gems/gems/semantic_puppet-1.0.2/lib/semantic_puppet
/opt/puppetlabs/puppet/lib/ruby/vendor_gems/gems/semantic_puppet-1.0.2/spec/unit/semantic_puppet
OK, I’ve managed to get a little further down the path. I looked through some of the older logs from successful upgrades, and it looks to me like it’s trying to use a puppet.conf file that isn’t being written out for some reason. I can find older kafo directories in /tmp, but none for any of the runs that I have done since installing the rpms for foreman 2.1 and Katello 3.16rc3
I’ve pinned down the problem, and it’s due to the GEM_PATH configuration in the tfm scl environment. As it is currently configured, puppet is unable to find its built-in gems once the tfm environment is enabled. Adding /opt/puppetlabs/puppet/lib/ruby/vendor_gems to GEM_PATH got the installer to run once more.
Since this is being set by a wrapper, I updated the /opt/theforeman/tfm/enable file to include the additional path. The file now looks like this:
. scl_source enable rh-ruby25
if [ -e /opt/theforeman/tfm/enable_assets ]; then
. /opt/theforeman/tfm/enable_assets
fi
export PATH=/opt/theforeman/tfm/root/usr/bin${PATH:+:${PATH}}
export LIBRARY_PATH=/opt/theforeman/tfm/root/usr/lib64${LIBRARY_PATH:+:${LIBRARY_PATH}}
export LD_LIBRARY_PATH=/opt/theforeman/tfm/root/usr/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
export MANPATH=/opt/theforeman/tfm/root/usr/share/man:${MANPATH}
export CPATH=/opt/theforeman/tfm/root/usr/include${CPATH:+:${CPATH}}
export PKG_CONFIG_PATH=/opt/theforeman/tfm/root/usr/lib64/pkgconfig${PKG_CONFIG_PATH:+:${PKG_CONFIG_PATH}}
export GEM_PATH=/opt/theforeman/tfm/root/usr/share/gems:/opt/puppetlabs/puppet/lib/ruby/vendor_gems:${GEM_PATH:+${GEM_PATH}}${GEM_PATH:-scl enable rh-ruby25 -- ruby -e "print Gem.path.join(':')"}
export GEM_HOME=“/opt/theforeman/tfm/root/usr/share/gems”
This should not be needed. Foreman should never need any gems from /opt/puppetlabs/puppet/lib/ruby/vendor_gem. Puppet should be isolated not use anything from tfm.
We have done some fixes around this in 2.1.0 GA. I think 3.16 RC4 is also on the mirrors (but not yet officially announced). Would you mind testing with that?
I set the enable file back to tfm-runtime default, updated to katello 3.16rc4, and am back to “Could not get default values, check log file at /var/log/foreman-installer/katello.log for more information”
I did a couple of additional tests to make some comparisons:
[root@myserver ~]# puppet --version
6.16.0
[root@myserver ~]# scl enable rh-ruby25 bash
[root@myserver ~]# puppet --version
6.16.0
[root@myserver ~]# exit
[root@myserver ~]# scl enable tfm bash
[root@myserver ~]# puppet --version
Ignoring bcrypt-3.1.12 because its extensions are not built. Try: gem pristine bcrypt --version 3.1.12
Ignoring ffi-1.12.2 because its extensions are not built. Try: gem pristine ffi --version 1.12.2
Ignoring nio4r-2.5.2 because its extensions are not built. Try: gem pristine nio4r --version 2.5.2
Ignoring nokogiri-1.10.9 because its extensions are not built. Try: gem pristine nokogiri --version 1.10.9
Ignoring passenger-4.0.18 because its extensions are not built. Try: gem pristine passenger --version 4.0.18
Ignoring pg-1.1.4 because its extensions are not built. Try: gem pristine pg --version 1.1.4
Ignoring rkerberos-0.1.5 because its extensions are not built. Try: gem pristine rkerberos --version 0.1.5
Ignoring ruby-libvirt-0.7.1 because its extensions are not built. Try: gem pristine ruby-libvirt --version 0.7.1
Ignoring sqlite3-1.3.13 because its extensions are not built. Try: gem pristine sqlite3 --version 1.3.13
Ignoring unf_ext-0.0.7.2 because its extensions are not built. Try: gem pristine unf_ext --version 0.0.7.2
Ignoring unicode-0.4.4.4 because its extensions are not built. Try: gem pristine unicode --version 0.4.4.4
Ignoring websocket-driver-0.7.1 because its extensions are not built. Try: gem pristine websocket-driver --version 0.7.1
Ignoring bcrypt-3.1.12 because its extensions are not built. Try: gem pristine bcrypt --version 3.1.12
Ignoring ffi-1.12.2 because its extensions are not built. Try: gem pristine ffi --version 1.12.2
Ignoring nio4r-2.5.2 because its extensions are not built. Try: gem pristine nio4r --version 2.5.2
Ignoring nokogiri-1.10.9 because its extensions are not built. Try: gem pristine nokogiri --version 1.10.9
Ignoring passenger-4.0.18 because its extensions are not built. Try: gem pristine passenger --version 4.0.18
Ignoring pg-1.1.4 because its extensions are not built. Try: gem pristine pg --version 1.1.4
Ignoring rkerberos-0.1.5 because its extensions are not built. Try: gem pristine rkerberos --version 0.1.5
Ignoring ruby-libvirt-0.7.1 because its extensions are not built. Try: gem pristine ruby-libvirt --version 0.7.1
Ignoring sqlite3-1.3.13 because its extensions are not built. Try: gem pristine sqlite3 --version 1.3.13
Ignoring unf_ext-0.0.7.2 because its extensions are not built. Try: gem pristine unf_ext --version 0.0.7.2
Ignoring unicode-0.4.4.4 because its extensions are not built. Try: gem pristine unicode --version 0.4.4.4
Ignoring websocket-driver-0.7.1 because its extensions are not built. Try: gem pristine websocket-driver --version 0.7.1
cannot load such file – semantic_puppet
Puppet is not expected to function with the SCL enabled. The installer should clear the environment prior to calling it.
It would be really useful if you shared the actual installer log. Without it, it’s really hard to say anything.
After reading another post I got a clearer idea:
It is executing /usr/local/bin/puppet which breaks our de-SCL code. As a workaround, could you try to rename that, run the installer and see if it passes?
@ehelms opened Bug #30365: Puppet bin scripts that exist outside AIO environment as symlinks to AIO puppet bin files break installation in SCL - Kafo - Foreman to track this
Thank you, I have not had a chance to try out the upgrade yet. Should I try it with the patches from your PR, or just remove the symlinks?
If you’d be willing to test the Kafo patch (https://github.com/theforeman/kafo/pull/259) then that’d be greatly appreciated. If it doesn’t, removing the symlinks is a quick workaround. After running the installer, you can place them again.
Unfortunately, the issue persists, even with the patch. I’ll try it with the link removed.
Removing the symlinks also did not work. This is how puppet looks on my system:
#!/bin/sh
# We keep around any paths in LD_LIBRARY_PATH that start with /opt/rh/. Those paths are
# from redhat software collections packages and are required for things like SCL python
# to work. See PUP-8351.
STRIP_LDLYP_COMMAND=" \
if ENV['LD_LIBRARY_PATH']; \
print ENV['LD_LIBRARY_PATH'].split(':', -1).keep_if { |path| path.start_with?('/opt/rh/') }.join(':') \
end"
LD_LIBRARY_PATH=`/opt/puppetlabs/puppet/bin/ruby -e "$STRIP_LDLYP_COMMAND"`
export LD_LIBRARY_PATH
unset LD_PRELOAD
# If $PATH does not match a regex for /opt/puppetlabs/bin
if [ `expr "${PATH}" : '.*/opt/puppetlabs/bin'` -eq 0 ]; then
# Add /opt/puppetlabs/bin to a possibly empty $PATH
PATH="${PATH:+${PATH}:}/opt/puppetlabs/bin"
export PATH
fi
COMMAND=`basename "${0}"`
exec /opt/puppetlabs/puppet/bin/${COMMAND} "$@"My mistake, I had removed all sets of symlinks, but once I got a look at puppet_strings_module_parser.rb I better understood how the environment is being cleaned up when calling puppet. It looks like removing just the symlinks in /usr/local/bin worked as you described. It has revealed another issue, but I think that’s a matter for a seperate thread.
Thank you for all of your help!
I had the same issue. I removed the /usr/bin/puppet symlink and now running the “foreman-installer --help” works. I was doing a fresh install of foreman 2.1 on a CentOS 7.7 VM. It was very useful finding this posting. It saved me a lot of troubleshooting time. Thanks.
Keith