Foreman discovery IP

tania · November 28, 2019, 3:24pm

Hi Foreman support,

Problem:
I am currently unable to discover hosts. The system boots into the discovery image fine, but then gives me the error message 1001: getaddrinfo: Name or service not known, body: N/A. The discovery image seems to fail to achieve to get an IP address, saying Waiting for IP4 address to generate SSL cert and is therefore unable to supply the fact.

If I interrupt the process in the discovery image to manually set network setup, an IP address is provided and then I am to continue to build the machine.

Expected outcome:
Automatically get IP Adress

Foreman and Proxy versions:
Foreman: 1.23.1

Foreman and Proxy plugin versions:
foreman_discovery 15.1.0
foreman_discovery image: 3.5.7

Distribution and version:
Centos 7.7.1908

Thanks,
Tania

lzap · November 29, 2019, 9:24am

Tania,

can you boot a discovered node with fdi.rootpw=redhat and after it gets busted with this change to console 3+ and log in and perform:

nmcli -w 1 -t -f IP4.ADDRESS con show primary

This is exactly what SSL cert generation script (which is a pre-launch script for foreman-proxy) does - in a loop it waits until this gives actually an IP4 address.

Do you use IPv6? That’s actually completely ignored, Foreman does not support IPv6 provisioning yet so discovery does not support it either.

Anyway, the amount of seconds discovery will wait until IPv4 is reported by network manager is 120 by default and you can override via fdi.ipwait=10. However as I said, HTTPS endpoint will not have proper SSL certificate so you will get SSL error when you attempt to provision it.

Next year we are dropping HTTPS communication from discovery and everything will be based on SSH so this code will disappear.

tania · November 29, 2019, 1:58pm

Hi lzap,

Thank you for the quick response. When running the command I get:

nmcli -w 1 -t -f IP4.ADDRESS con show primary

ERROR: primary - no such connection profile

I tried increasing the wait time, but it did not solve the issue. I also ran:

nmcli connection show
name UUID Type Device
secondary-mac ethernet –

I can confirm that we are currently only using IPv4. Any other ideas would be much appreciated.

Thanks,
Tania

lzap · November 29, 2019, 3:34pm

NM profile “primary” is being created by the service /etc/systemd/system/nm-prepare.service which calls utility nm-configure which prepares it in /etc/NetworkManager/system-connections/primary. Service /usr/lib/systemd/system/foreman-proxy.service depends on it and it’s the one that waits for the IPv4 address.

Investigate why the nm-prepare service (thus nm-configure script) did not configure primary network manager profile.

Is this PXE mode or are you booting from USB stick?

tania · December 2, 2019, 11:11am

Hi lzap,

Looking at the nm-configure script, the BOOTMAC isn’t being populated by the /usr/share/fdi/commonfunc.sh script. When running the commands in commonfunc.sh I notice that if I run cat /proc/cmdline the BOOTIF={mac} is blank, giving me BOOTIF=01------. Is there anything I can check to ensure the mac is passed correctly via PXE.

In the /etc/NetworkManager/system-connections/ I notice two files. primary which does not contain a mac and secondary-{mac} that does.

Thanks,
Tania

tania · December 2, 2019, 1:45pm

Extra information

My grub looks like:

To load MAC-based config explicitly, a MAC address separated by dash chars is needed.

Also due to bug in RHEL 7.4 files are loaded with an extra “:” character at the end.

This workarounds both cases, make sure “regexp.mod” file is present on the TFTP.

For more info see: https://bugzilla.redhat.com/show_bug.cgi?id=1370642#c70

insmod regexp
regexp --set=1:m1 --set=2:m2 --set=3:m3 --set=4:m4 --set=5:m5 --set=6:m6 ‘^([0-9a-f]{1,2}):([0-9a-f]{1,2}):([0-9a-f]{1,2}):([0-9a-f]{1,2}):([0-9a-f]{1,2}):([0-9a-f]{1,2})’ “$net_default_mac”
mac=${m1}-${m2}-${m3}-${m4}-${m5}-${m6}

Only grub2 from redhat has MAC-based config loading patch, load explicitly

configfile=/grub2/grub.cfg-01-$mac
source “$configfile”

And if that fails render chain and discovery menu

menuentry ‘Foreman Discovery Image efi’ --id discoveryefi {
linuxefi boot/fdi-image/vmlinuz0 rootflags=loop root=live:/fdi.iso rootfstype=auto ro rd.live.image acpi=force rd.luks=0 rd.md=0 rd.dm=0 rd.lvm=0 rd.bootif=0 rd.neednet=0 nokaslr nomodeset proxy.url=https://foreman.thomac.net proxy.type=foreman BOOTIF=01-$mac fdi.zips=boot/snipe_extension.zip
initrdefi boot/fdi-image/initrd0.img
}

and tftp logs looks like:

RRQ from filename grub2/shim.efi
Error code 8: User aborted the transfer
RRQ from filename grub2/shim.efi
Client finished grub2/shim.efi
RRQ from filename grub2/grubx64.efi
Client finished grub2/grubx64.efi
RRQ from filename /grub2/grub.cfg-{mac}
Client File not found /grub2/grub.cfg-{mac}
RRQ from filename /grub2/grub.cfg-0A172C4A
Client File not found /grub2/grub.cfg-0A172C4A
RRQ from filename /grub2/grub.cfg-0A172C4
Client File not found /grub2/grub.cfg-0A172C4
RRQ from filename /grub2/grub.cfg-0A172C
Client File not found /grub2/grub.cfg-0A172C
RRQ from filename /grub2/grub.cfg-0A172
Client File not found /grub2/grub.cfg-0A172
RRQ from filename /grub2/grub.cfg-0A17
Client File not found /grub2/grub.cfg-0A17
RRQ from filename /grub2/grub.cfg-0A1
Client File not found /grub2/grub.cfg-0A1
RRQ from filename /grub2/grub.cfg-0A
Client File not found /grub2/grub.cfg-0A
RRQ from filename /grub2/grub.cfg-0
Client File not found /grub2/grub.cfg-0
RRQ from filename /grub2/grub.cfg
Client finished /grub2/grub.cfg
RRQ from filename /EFI/centos/x86_64-efi/command.lst
Client File not found /EFI/centos/x86_64-efi/command.lst
RRQ from filename /EFI/centos/x86_64-efi/fs.lst
Client File not found /EFI/centos/x86_64-efi/fs.lst
RRQ from filename /EFI/centos/x86_64-efi/crypto.lst
Client File not found /EFI/centos/x86_64-efi/crypto.lst
RRQ from filename /EFI/centos/x86_64-efi/terminal.lst
Client File not found /EFI/centos/x86_64-efi/terminal.lst
RRQ from filename /grub2/grub.cfg
Client finished /grub2/grub.cfg
RRQ from filename /EFI/centos/x86_64-efi/regexp.mod
Client File not found /EFI/centos/x86_64-efi/regexp.mod
RRQ from filename /grub2/grub.cfg-01------
Client File not found /grub2/grub.cfg-01------
RRQ from filename boot/fdi-image/vmlinuz0
Client finished boot/fdi-image/vmlinuz0
RRQ from filename boot/fdi-image/initrd0.img

tania · December 2, 2019, 3:54pm

Hi lzap,

I found a fix by creating:

/var/lib/tftpboot/EFI/centos/x86_64-efi

and copying /usr/lib/grub/x86_64-efi/regexp.mod to the directory, and setting the forman-proxy permissions. I tried a symlink but the foreman server didn’t seem to like the permissions.

Thanks,
Tania

lzap · December 3, 2019, 8:17am

Oh in that case FDI puts “MAC address was not provided or detected, leaving unconfigured” error message in logs. We require BOOTIF argument to be present.

We actually got rid of this in 1.24 as smart-proxy TFTP module now creates both configurations. After you upgrade your instance this will not be needed anymore.

tania · December 3, 2019, 9:00am

Excellent, I shall wait till 1.24 becomes stable.

Thanks for your help.
Tania

lzap · December 3, 2019, 1:23pm

Well if you’ve fixed it there is no reason to wait.