I have a Foreman 3.31 server provisioning CentOS 7 servers

Support
1

**Problem:VM looks to be building with PXE but once it reboots it just goes back into PXE again no option to boot into hard disk

**Expected outcome: Finishes the build and boots up into the VM.

**Foreman and Proxy versions:.3.31

I am relatively new to Foreman and had some success with these build in 1.7 but then I realized I am way outdated and tried to upgrade. Got all the way to 3.31 before it needed an el8 server.

So now I have this set up in 3.31 using smart proxy DHCP and DNS and it will preform the build but it just continuously reboots and preforms the PXE build over and over…

Any assistance or insight would be greatly appreciated.

2

Just a wild guess, but this sounds like your VM is unable to tell Foreman it finished the installation.
Maybe you are missing something in your template. If you are using custom templates, any changes since 1.7 are probably not in there.
Take a look at your provisioning (kickstart) template and check if you have something like this at the end: https://github.com/theforeman/foreman/blob/develop/app/views/unattended/provisioning_templates/provision/kickstart_default.erb#L390-L396
There should be some call to the “built” snippet or at least some form of curl/wget to the Foreman built endpoint like in the snippet.

3

So what I did was took the Kickstart default provisioning template and modified it to install the packages I need and time zones etc… So it does have that call in the end:

if test -f /tmp/foreman_built; then
  echo "calling home: build is done!"
  <%= indent(2, skip1: true) { snippet('built', :variables => { :endpoint => 'built', :method => 'POST', :body_file => '/root/install.post.log' }) } -%>
else
  echo "calling home: build failed!"
  <%= indent(2, skip1: true) { snippet('built', :variables => { :endpoint => 'failed', :method => 'POST', :body_file => '/root/install.post.log' }) } -%>
fi

Where can I find that snippet to varify it’s all there?

4

Looking through the logs on the server. I had to disable the NIC to get it to boot to the vm:
/var/log/anaconda/anaconda.log:

15:22:49,685 INFO anaconda: Creating users
15:22:49,686 INFO anaconda: Configuring addons
15:22:49,696 INFO anaconda: Configuring addons
15:22:49,696 INFO anaconda: Generating initramfs
15:23:28,767 INFO anaconda: Generating initramfs
15:23:28,767 INFO anaconda: Running post-installation scripts
15:23:28,768 INFO anaconda: Running kickstart %%post script(s)
15:29:33,303 ERR anaconda: Error code 1 running the kickstart script at line 188

In the kickstart line 188 is:

186: <% if (is_fedora && os_major >= 28) || (rhel_compatible && os_major > 7) -%>
187: authselect --useshadow --passalgo=<%= @host.operatingsystem.password_hash.downcase || 'sha256' %> --kickstart
188: <% else -%>
189: authconfig --useshadow --passalgo=<%= @host.operatingsystem.password_hash.downcase || 'sha256' %> --kickstart
190: <% end -%>
191: timezone --utc <%= host_param('time-zone') || 'America/Los_Angeles' %> <%= host_param('ntp-server') ? "--ntpservers #{host_param('ntp-server')}" : '' %>
5

journal.log

Feb 15 15:29:29 xor-q-fmtest01.xpressbetonline.com dracut[112808]: *** Creating initramfs image file '/boot/initramfs-3.10.0-1160.el7.x86_64.img' done ***
Feb 15 15:29:33 xor-q-fmtest01.xpressbetonline.com program[1555]: Return code: 0
Feb 15 15:29:33 xor-q-fmtest01.xpressbetonline.com program[1555]: Running... /bin/sh /tmp/ks-script-eIcvka
Feb 15 15:29:33 xor-q-fmtest01.xpressbetonline.com program[1555]: cp: cannot stat '/tmp/*.pre.*.log': No such file or directory
Feb 15 15:29:33 xor-q-fmtest01.xpressbetonline.com program[1555]: Return code: 1
Feb 15 15:29:33 xor-q-fmtest01.xpressbetonline.com anaconda[1555]: Error code 1 running the kickstart script at line 188
Feb 15 15:29:33 xor-q-fmtest01.xpressbetonline.com program[1555]: Running... /bin/sh /tmp/ks-script-PVuoYc
Feb 15 15:30:24 xor-q-fmtest01.xpressbetonline.com program[1555]: calling home: build is done!
Feb 15 15:30:24 xor-q-fmtest01.xpressbetonline.com program[1555]: Return code: 0
Feb 15 15:30:24 xor-q-fmtest01.xpressbetonline.com program[1555]: Running... /bin/sh /tmp/ks-script-aZzojx
Feb 15 15:30:24 xor-q-fmtest01.xpressbetonline.com program[1555]: Return code: 0
Feb 15 15:30:24 xor-q-fmtest01.xpressbetonline.com program[1555]: Running... /bin/sh /tmp/ks-script-7w8cIY

ks-script-eIcvka.log

cp: cannot stat '/tmp/*.pre.*.log': No such file or directory

packaging.log

15:16:46,430 INFO packaging: Error downloading '.treeinfo': [Errno 14] HTTPS Error 404 - Not Found
15:16:46,431 INFO packaging: Trying to download 'treeinfo'
15:16:46,476 INFO packaging: Error downloading 'treeinfo': [Errno 14] HTTPS Error 404 - Not Found
15:16:46,477 ERR packaging: got HTTP Error 404 when downloading [.]treeinfo files

seeing this a lot in syslog:

15:25:38,037 NOTICE kernel:type=1400 audit(1708010738.035:115): avc:  denied  { create } for  pid=1513 comm="in:imjournal" name="imjournal.state.tmp" scontext=system_u:system_r:syslogd_t:s0 tcontext=system_u:object_r:root_t:s0 tclass=file permissive=1
15:25:38,037 NOTICE kernel:type=1400 audit(1708010738.035:116): avc:  denied  { write open } for  pid=1513 comm="in:imjournal" path="/imjournal.state.tmp" dev="dm-0" ino=658 scontext=system_u:system_r:syslogd_t:s0 tcontext=system_u:object_r:root_t:s0 tclass=file permissive=1
15:25:38,037 NOTICE kernel:type=1400 audit(1708010738.035:117): avc:  denied  { getattr } for  pid=1513 comm="in:imjournal" path="/imjournal.state.tmp" dev="dm-0" ino=658 scontext=system_u:system_r:syslogd_t:s0 tcontext=system_u:object_r:root_t:s0 tclass=file permissive=1
15:25:38,037 NOTICE kernel:type=1400 audit(1708010738.035:118): avc:  denied  { rename } for  pid=1513 comm="in:imjournal" name="imjournal.state.tmp" dev="dm-0" ino=658 scontext=system_u:system_r:syslogd_t:s0 tcontext=system_u:object_r:root_t:s0 tclass=file permissive=1
15:25:38,037 NOTICE kernel:type=1400 audit(1708010738.035:119): avc:  denied  { unlink } for  pid=1513 comm="in:imjournal" name="imjournal.state" dev="dm-0" ino=659 scontext=system_u:system_r:syslogd_t:s0 tcontext=system_u:object_r:root_t:s0 tclass=file permissive=1
6

Here is the Kickstart in it’s entirety:

<%#
kind: provision
name: Kickstart default
model: ProvisioningTemplate
oses:
- AlmaLinux
- CentOS
- CentOS_Stream
- Fedora
- RedHat
- Rocky
test_on:
- host4dhcp
- host6dhcp
- host4and6dhcp
- host4static
- host6static
description: |
  The provisioning template for kickstart based distributions. The output is fetched by Anaconda installer during
  the network based installation. To customize the installation, modify the host parameters.

  This template accepts the following parameters:
  - lang: string (default="en_US.UTF-8")
  - selinux-mode: string (default="disabled")
  - keyboard: string (default="us")
  - time-zone: string (default="America/Los_Angeles")
  - http-proxy: string (default="")
  - http-proxy-port: string (default="")
  - force-puppet: boolean (default=false)
  - enable-epel: boolean (default=true)
  - enable-puppetlabs-repo: boolean (default=false)
  - enable-puppetlabs-puppet5-repo: boolean (default=false)
  - enable-puppetlabs-puppet6-repo: boolean (default=false)
  - skip-puppet-setup: boolean (default=false)
  - salt_master: string (default=undef)
  - ntp-server: string (default="time.qa.xpressbetonline.com")
  - bootloader-append: string (default="ipv6.disable=1 net.ifnames=0 crashkernel=auto nofb quiet splash=quiet")
  - disable-firewall: boolean (default=true)
  - package_upgrade: boolean (default=true)
  - disable-uek: boolean (default=false)
  - use-ntp: boolean (default depends on OS release)
  - fips_enabled: boolean (default=false)
  - encrypt_grub: boolean (default=false)

  Reference links:
  - https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/installation_guide/s1-kickstart2-options
  - https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/installation_guide/sect-kickstart-syntax
  - https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/performing_an_advanced_rhel_installation/kickstart-commands-and-options-reference_installing-rhel-as-an-experienced-user
-%>
# This kickstart file was rendered from the Foreman provisioning template "<%= @template_name %>".

<%
  rhel_compatible = @host.operatingsystem.family == 'Redhat' && @host.operatingsystem.name != 'Fedora'
  is_fedora = @host.operatingsystem.name == 'Fedora'
  os_major = @host.operatingsystem.major.to_i
  os_minor = @host.operatingsystem.minor.to_i
  realm_compatible = (@host.operatingsystem.name == 'Fedora' && os_major >= 20) || (rhel_compatible && os_major >= 7)
  # safemode renderer does not support unary negation
  proxy_uri = host_param('http-proxy') ? "http://#{host_param('http-proxy')}:#{host_param('http-proxy-port')}" : nil
  proxy_string = proxy_uri ? " --proxy=#{proxy_uri}" : ''
  puppet_enabled = !host_param_true?('skip-puppet-setup') && (host_puppet_server.present? || host_param_true?('force-puppet'))
  salt_enabled = host_param('salt_master') ? true : false
  chef_enabled = @host.respond_to?(:chef_proxy) && @host.chef_proxy
  section_end = (rhel_compatible && os_major <= 5) ? '' : '%end'
  use_ntp = host_param_true?('use-ntp', (is_fedora && os_major < 16) || (rhel_compatible && os_major <= 7))
  iface = @host.provision_interface
  appstream_present = false
-%>

<% if (is_fedora && os_major < 29) || (rhel_compatible && os_major <= 7) -%>
install
<% end -%>
<%
if host_param('kickstart_liveimg')
  img_name = host_param('kickstart_liveimg')
  liveimg_url = if host_param('kt_activation_keys')
    repository_url(img_name, 'isos')
  else
    if img_name.match(%r|^([\w\-\+]+)://|)
      img_name
    else
      "#{medium_uri}/#{img_name}"
    end
  end
%>
liveimg --url=<%= liveimg_url %> <%= proxy_string %>
<% else %>
<%= @mediapath %><%= proxy_string %>
<% @additional_media.each do |medium| -%>
repo --name <%= medium[:name] %> --baseurl <%= medium[:url] %> <%= medium[:install] ? ' --install' : '' %><%= proxy_string %>
<% end -%>
<%= snippet_if_exists(template_name + " custom repositories") %>
<% end %>
lang <%= host_param('lang') || 'en_US.UTF-8' %>
selinux --<%= host_param('selinux-mode') || host_param('selinux') || 'disabled' %>
keyboard <%= host_param('keyboard') || 'us' %>
skipx

<%
  network_options = []
  nameservers = []
  subnet4 = iface.subnet
  subnet6 = iface.subnet6

  # device and hostname
  if iface.bond? && rhel_compatible && os_major >= 6
    network_options.push("--device=#{iface.identifier}")
  else
    network_options.push("--device=#{iface.mac || iface.identifier}")
  end
  network_options.push("--hostname #{@host.name}")

  # single stack
  if subnet4 && !subnet6
    network_options.push("--noipv6")
  elsif !subnet4 && subnet6
    network_options.push("--noipv4")
  end

  # dual stack MTU check
  raise("IPv4 and IPv6 subnets have different MTU") if subnet4 && subnet6 && subnet4.mtu.present? && subnet6.mtu.present? && subnet4.mtu != subnet6.mtu

  # IPv4
  if (subnet4 && !subnet4.dhcp_boot_mode?) || @static
    network_options.push("--bootproto static")
    network_options.push("--ip=#{iface.ip}")
    network_options.push("--netmask=#{subnet4.mask}")
    network_options.push("--gateway=#{subnet4.gateway}")
  elsif subnet4 && subnet4.dhcp_boot_mode?
    network_options.push("--bootproto dhcp")
  end
  if subnet4
    nameservers.concat(subnet4.dns_servers)
    network_options.push("--mtu=#{subnet4.mtu}") if subnet4.mtu.present?
  end

  # IPv6
  if rhel_compatible && os_major >= 6
    if (subnet6 && !subnet6.dhcp_boot_mode?) || @static6
      network_options.push("--ipv6=#{iface.ip6}/#{subnet6.cidr}")
      network_options.push("--ipv6gateway=#{subnet6.gateway}")
    elsif subnet6 && subnet6.dhcp_boot_mode?
      if host_param_true?('use-slaac')
        network_options.push("--ipv6 auto")
      else
        network_options.push("--ipv6 dhcp")
      end
    end
    if subnet6
      nameservers.concat(subnet6.dns_servers)
      network_options.push("--mtu=#{subnet6.mtu}") if subnet6.mtu.present?
    end
  end

  # bond
  if iface.bond? && rhel_compatible && os_major >= 6
    bond_slaves = iface.attached_devices_identifiers.join(',')
    network_options.push("--bondslaves=#{bond_slaves}")
    network_options.push("--bondopts=mode=#{iface.mode},#{iface.bond_options.tr(' ', ',')}")
  end

  # VLAN (only on physical is recognized)
  if iface.virtual? && iface.tag.present? && iface.attached_to.present?
    if rhel_compatible && os_major == 6
      network_options.push("--vlanid=#{iface.tag}")
    else
      network_options.push("--interfacename=vlan#{iface.tag}")
    end
  end

  # DNS
  if nameservers.size > 0
    network_options.push("--nameserver=#{nameservers.join(',')}")
  else
    network_options.push("--nodns")
  end
-%>
network <%= network_options.join(' ') %>

rootpw --iscrypted <%= root_pass %>
<% if host_param_true?('disable-firewall') -%>
firewall --disable
<% else -%>
firewall --<%= os_major >= 6 ? 'service=' : '' %>ssh
<% end -%>
<% if (is_fedora && os_major >= 28) || (rhel_compatible && os_major > 7) -%>
authselect --useshadow --passalgo=<%= @host.operatingsystem.password_hash.downcase || 'sha256' %> --kickstart
<% else -%>
authconfig --useshadow --passalgo=<%= @host.operatingsystem.password_hash.downcase || 'sha256' %> --kickstart
<% end -%>
timezone --utc <%= host_param('time-zone') || 'America/Los_Angeles' %> <%= host_param('ntp-server') ? "--ntpservers #{host_param('ntp-server')}" : '' %>

<% if rhel_compatible -%>
services --disabled gpm,NetworkManager,sendmail,cups,pcmcia,isdn,rawdevices,hpoj,bluetooth,openibd,avahi-daemon,avahi-dnsconfd,hidd,hplip,pcscd
<% end -%>

<% if realm_compatible && host_enc['parameters']['realm'] && @host.realm && @host.realm.realm_type == 'Active Directory' -%>
# One-time password will be requested at install time. Otherwise, $HOST[OTP] is used as a placeholder value.
realm join --one-time-password='<%= @host.otp || "$HOST[OTP]" %>' <%= @host.realm %>
<% end -%>

<% if @host.operatingsystem.name == 'OracleLinux' && os_major == 7 && os_minor < 5 -%>
repo --name="Server-Mysql"
<% end -%>

<% if @host.operatingsystem.name == 'Fedora' && os_major <= 16 -%>
# Bootloader exception for Fedora 16:
bootloader --append="<%= host_param('bootloader-append') || 'ipv6.disable=1 net.ifnames=0 crashkernel=auto nofb quiet splash=quiet' %> <%= ks_console %>" <%= grub_pass %>
part biosboot --fstype=biosboot --size=1
<% else -%>
bootloader --location=mbr --append="<%= host_param('bootloader-append') || 'ipv6.disable=1 net.ifnames=0 crashkernel=auto nofb quiet splash=quiet' %>" <%= grub_pass %>
<% if os_major == 5 -%>
key --skip
<% end -%>
<% end -%>

<% if @dynamic -%>
%include /tmp/diskpart.cfg
<% else -%>
<%= @host.diskLayout %>
<% end -%>

text
reboot

%packages
@base
@development
@legacy-unix
bind-utils
chrony
expect
firewalld
kexec-tools
net-snmp
net-tools
nfs-utils
ntp
perl-libwww-perl
wget
-avahi
-cyrus-sasl-sql
-yum-rhn-plugin
-yum-security
<% if os_major >= 6 -%>
redhat-lsb-core
<% end -%>
<%= section_end %>

<% if @dynamic -%>
%pre --log=/tmp/install.pre.dynamic.log
<%= snippet_if_exists(template_name + " custom pre") %>
<%= @host.diskLayout %>
<%= section_end %>
<% end -%>

user --name=xbadmin --homedir=/localhome/xbadmin --iscrypted --password=*password* --shell=/bin/bash

%post --nochroot
exec < /dev/tty3 > /dev/tty3
chvt 3
(

<% if host_param_false?('no-resolv-override') -%>
cp -va /etc/resolv.conf /mnt/sysimage/etc/resolv.conf
<% end -%>

<%= snippet_if_exists(template_name + " custom postnochroot") -%>

chvt 1
) 2>&1 | tee /mnt/sysimage/root/install.postnochroot.log
<%= section_end %>

<%#
Main post script, if it fails the last post is still executed.
%>
%post
exec < /dev/tty3 > /dev/tty3
chvt 3
(
logger "Starting anaconda <%= @host %> postinstall"
<%= snippet 'kickstart_networking_setup' %>

<%= snippet 'ntp' %>

<%= snippet 'yum_proxy' %>

<% if rhel_compatible && host_param_true?('enable-epel') -%>
<%= snippet 'epel' -%>
<% end -%>

<%= snippet 'redhat_register' %>

<% if host_enc['parameters']['realm'] && @host.realm && (@host.realm.realm_type == 'FreeIPA' || @host.realm.realm_type == 'Red Hat Identity Management') -%>
<%= snippet 'freeipa_register' %>
<% end -%>

<% unless host_param_false?('package_upgrade') -%>
# update all the base packages from the updates repository
if [ -f /usr/bin/dnf ]; then
  dnf -y update
else
  yum -t -y update
fi
<% end -%>

<%= snippet('remote_execution_ssh_keys') %>

<%= snippet "blacklist_kernel_modules" %>

<% if chef_enabled %>
<%= snippet 'chef_client' %>
<% end -%>

<% if puppet_enabled %>
<% if host_param_true?('enable-puppetlabs-repo') || host_param_true?('enable-official-puppet7-repo') || host_param_true?('enable-puppetlabs-puppet6-repo')|| host_param_true?('enable-puppetlabs-puppet5-repo') -%>
<%= snippet 'puppetlabs_repo' %>
<% end -%>
<%= snippet 'puppet_setup' %>
<% end -%>

<% if salt_enabled %>
<%= snippet 'saltstack_setup' %>
<% end -%>

<% if @host.operatingsystem.name == 'OracleLinux' && host_param_true?('disable-uek') -%>
# Uninstall the Oracle Unbreakable Kernel packages
yum -t -y remove kernel-uek*
sed -e 's/DEFAULTKERNEL=kernel-uek/DEFAULTKERNEL=kernel/g' -i /etc/sysconfig/kernel
<% end -%>

<%= snippet('ansible_provisioning_callback') %>

<%= snippet 'efibootmgr_netboot' %>

<%= snippet_if_exists(template_name + " custom post") %>

<% if host_param_true?('host_registration_insights') -%>
<%= snippet 'insights' -%>
<% end -%>

touch /tmp/foreman_built

chvt 1
) 2>&1 | tee /root/install.post.log
<%= section_end %>

# copy %pre log files into chroot
%post --nochroot
cp -vf /tmp/*.pre.*.log /mnt/sysimage/root/
<%= section_end %>

<%#
The last post section halts Anaconda to prevent endless loop in case HTTP request fails
%>
<% if (is_fedora && os_major < 20) || (rhel_compatible && os_major < 7) -%>
%post
<% else -%>
%post --erroronfail --log=/root/install-callhome.post.log
<% end -%>

<%= snippet 'eject_cdrom' -%>

if test -f /tmp/foreman_built; then
  echo "calling home: build is done!"
  <%= indent(2, skip1: true) { snippet('built', :variables => { :endpoint => 'built', :method => 'POST', :body_file => '/root/install.post.log' }) } -%>
else
  echo "calling home: build failed!"
  <%= indent(2, skip1: true) { snippet('built', :variables => { :endpoint => 'failed', :method => 'POST', :body_file => '/root/install.post.log' }) } -%>
fi

sync
<%= section_end %>
7

So actually I just used the Kickstart default and it also just continuously boots into PXE after the build.

8

Hm, that’s definetly weird. Does your host’s build status change “installed” in Foreman? Do you see any calls to the /unattended/built endpoint in /var/log/foreman/production.log? If so, are there any possibly related error messages in there?

9

These error are (in my experience) somewhat normal. For whatever reason dnf is always looking for treeinfo files, regardless of if they are needed. From what I understood when I looked this up in the past, these files are only required for atomic hosts.