Again in the pxe boot trap

After completely new installing Foreman / Katello i’ve now fallen into the PXE boot trap again. After it worked with my previous installation. As far as I can see all configuration files appear to be the same… different this time is that it appears to load grubx64.efi but then breaks and goes into the grub 2 “Minimal bash-like line editing” mode.

This is my dhcpd.conf :

#
# DHCP Server Configuration file.
#   see /usr/share/doc/dhcp*/dhcpd.conf.example
#   see dhcpd.conf(5) man page
#
DHCPDARGS="eth0";
allow booting;
allow bootp;
set vendorclass = option vendor-class-identifier;
option pxe-system-type code 93 = unsigned integer 16;
set pxetype = option pxe-system-type;

option domain-name "md80.ch";

subnet 192.168.178.0 netmask 255.255.255.0 {
        option subnet-mask 255.255.255.0;
        option domain-name-servers 192.168.178.41;
        option routers 192.168.178.1;
        default-lease-time 14400;
        max-lease-time 28800;
  if substring(vendorclass, 0, 9)="PXEClient" {
    if pxetype=00:06 or pxetype=00:07 {
        filename "grub2/grubx64.efi";
    } else {
        filename "pxelinux.0";
    }
  }
  pool {
    range 192.168.178.101 192.168.178.200;
  }
}

host svmqhub2.md80.ch {
 hardware ethernet 00:15:5d:01:27:0d;
 fixed-address 192.168.178.102;
 next-server 192.168.178.22;
}

This is the tftp config in xinetd.d

# This file is being maintained by Puppet.
# DO NOT EDIT

service tftp
{
        port            = 69
        disable         = no
        socket_type     = dgram
        protocol        = udp
        wait            = yes
        user            = root
        group           = root
        groups          = yes
        server          = /usr/sbin/in.tftpd
        server_args     = -v -s /var/lib/tftpboot -m /etc/tftpd.map
        per_source      = 11
        cps             = 100 2
        flags           = IPv4
        instances       = UNLIMITED
}

in the /var/log/messages file i see:

Feb 24 11:59:37 katello dhcpd: DHCPDISCOVER from 00:15:5d:01:27:0b via eth0
Feb 24 11:59:38 katello dhcpd: DHCPOFFER on 192.168.178.102 to 00:15:5d:01:27:0b via eth0
Feb 24 11:59:41 katello dhcpd: DHCPREQUEST for 192.168.178.102 (192.168.178.22) from 00:15:5d:01:27:0b via eth0
Feb 24 11:59:41 katello dhcpd: DHCPACK on 192.168.178.102 to 00:15:5d:01:27:0b via eth0
Feb 24 11:59:41 katello in.tftpd[214275]: RRQ from 192.168.178.102 filename grub2/grubx64.efi
Feb 24 11:59:41 katello in.tftpd[214275]: Error code 8: User aborted the transfer
Feb 24 11:59:41 katello in.tftpd[214276]: RRQ from 192.168.178.102 filename grub2/grubx64.efi
Feb 24 11:59:41 katello in.tftpd[214276]: Client 192.168.178.102 finished grub2/grubx64.efi

selinux is currently set to permissive and firewalld is currently stopped. I can tftp grubx64.efi to another system.
What else can I look for ?

I forgot the system details:
Centos 7.6
Foreman 1.20.2
Katello 3.10

After installing Snagit, I managed to caputre the screen that is displayed very shortly after the booting starts.

After that it imediately goes to the grub> prompt.

Even stranger now. I tried a different VM which has the exact same settings and that one boots. All i did was to add the dhcpd.conf and create the system in Foreman.
When the successfull one boot it looks like this in /var/log/messages:

Feb 24 12:55:09 katello dhcpd: DHCPREQUEST for 192.168.178.103 (192.168.178.22) from 00:15:5d:ee:f9:02 via eth0
Feb 24 12:55:09 katello dhcpd: DHCPACK on 192.168.178.103 to 00:15:5d:ee:f9:02 via eth0
Feb 24 12:55:09 katello in.tftpd[8659]: RRQ from 192.168.178.103 filename grub2/grubx64.efi
Feb 24 12:55:09 katello in.tftpd[8659]: Error code 8: User aborted the transfer
Feb 24 12:55:09 katello in.tftpd[8660]: RRQ from 192.168.178.103 filename grub2/grubx64.efi
Feb 24 12:55:10 katello in.tftpd[8660]: Client 192.168.178.103 finished grub2/grubx64.efi
Feb 24 12:55:10 katello in.tftpd[8664]: RRQ from 192.168.178.103 filename /grub2/grub.cfg-01-00-15-5d-ee-f9-02
Feb 24 12:55:10 katello in.tftpd[8664]: Client 192.168.178.103 finished /grub2/grub.cfg-01-00-15-5d-ee-f9-02
Feb 24 12:55:10 katello in.tftpd[8665]: RRQ from 192.168.178.103 filename grub2/x86_64-efi/command.lst
Feb 24 12:55:10 katello in.tftpd[8665]: Client 192.168.178.103 File not found grub2/x86_64-efi/command.lst
Feb 24 12:55:10 katello in.tftpd[8666]: RRQ from 192.168.178.103 filename grub2/x86_64-efi/fs.lst
Feb 24 12:55:10 katello in.tftpd[8666]: Client 192.168.178.103 File not found grub2/x86_64-efi/fs.lst
Feb 24 12:55:10 katello in.tftpd[8667]: RRQ from 192.168.178.103 filename grub2/x86_64-efi/crypto.lst
Feb 24 12:55:10 katello in.tftpd[8667]: Client 192.168.178.103 File not found grub2/x86_64-efi/crypto.lst
Feb 24 12:55:10 katello in.tftpd[8668]: RRQ from 192.168.178.103 filename grub2/x86_64-efi/terminal.lst
Feb 24 12:55:10 katello in.tftpd[8668]: Client 192.168.178.103 File not found grub2/x86_64-efi/terminal.lst
Feb 24 12:55:10 katello in.tftpd[8669]: RRQ from 192.168.178.103 filename /grub2/grub.cfg-01-00-15-5d-ee-f9-02
Feb 24 12:55:10 katello in.tftpd[8669]: Client 192.168.178.103 finished /grub2/grub.cfg-01-00-15-5d-ee-f9-02
Feb 24 12:55:20 katello in.tftpd[8681]: RRQ from 192.168.178.103 filename boot/centos-mirror-qVbSBrznIWMc-vmlinuz
Feb 24 12:55:22 katello in.tftpd[8681]: Client 192.168.178.103 finished boot/centos-mirror-qVbSBrznIWMc-vmlinuz
Feb 24 12:55:22 katello in.tftpd[8682]: RRQ from 192.168.178.103 filename boot/centos-mirror-qVbSBrznIWMc-initrd.img
Feb 24 12:55:38 katello in.tftpd[8682]: Client 192.168.178.103 finished boot/centos-mirror-qVbSBrznIWMc-initrd.img

Hello,

If I understand correctly pxe is working after creating system on Foreman, if I see the message for non working system then it shows client system was able to download file,

Feb 24 11:59:41 katello in.tftpd[214276]: RRQ from 192.168.178.102 filename grub2/grubx64.efi
Feb 24 11:59:41 katello in.tftpd[214276]: Client 192.168.178.102 finished grub2/grubx64.efi

but later on for working system its actually getting grub.cfg-mac-addr file which gets created when we create system on Foreman,

/grub2/grub.cfg-01-00-15-5d-ee-f9-02

If this is the case then there should be grub.cfg file with mac address to go for next stage.