Pulp issues with downloading squashfs.img

So, that truckload of HP Elitedesk 800 G6 Tower PCs just arrived and I wanted to provision a handful of them using our Foreman/Katello server … that turned out to be little less straightforward then I anticipated:
PXE boot loads the Foreman Discovery Image (3.8) but when booting into discovery, things get stuck - unable to connect to the Foreman Server. Reason: no network interface detected.
Reason: those PCs have a newer release for the Intel i219-LM NIC which is not supported by the FDI. G. directed me to elrepo and I managed to build a custom FDI that includes the right driver. Hooray: FDI boot now connects to the server and the machine appears under the discovered hosts in the GUI (I can share the image if someone needs it).
As I expected, the next step - provisioning CentOS7 - failed too since it also lacks the driver. Getting a signed driver into the base image looked a bit too complicated so I decided to go for the inst.dd boot option and use a USB key with the driver. Struggled a bit getting that working - mainly due to kernel versions but once I got the USB key created correctly, the base image recognized the NIC and downloaded the kickstart file (I can share that USB image too).
Next problem: the kickstart process bombs out with this message:

dracut-initqueue[876]:  losetup: /tmp/curl_fetch_url1/squashfs.img: warning: file does fit into a 512-byte sector the end of the file will be ignored.
SQUASHFS error: unable to read xattr id index table
dracut-initqueue[876]: mount: wrong fs type, bad option, bad superblock on /dev/loop0,
dracut-initqueue[876]: missing codepage or helper program, or other error
dracut-initqueue[876]: ....

and this is where I’m stuck right now.
I realize this might not be ‘all Foreman’ but I hope some of you very wise guys can shed a little light on this and put me on the right track …

well some further research pointed me to this remark in some fedora forum:

hi there, just an update. i figured out the problem. in the install.img not all of the newest drivers are included, so during installation, when the switch from pxe to the img occures, the client looses connection. the error message is somewhat misleading… andreas

Sounds like a reasonable explanation - so looks like one more location to squeeze in the new driver …

Well, forget about that last remark - I managed to fix it but not in a way I like.

Problem is/was with squashfs.img which is downloaded at boot time form https:///pulp/content//Library/custom/CentOS_7/base-x86_64/LiveOS/squashfs.img
I noticed that the file size was around 760MB while the file on the CentOS mirror is something like 497MB. When I downloaded the 760MB file I could not unsquash it - which does work for the original 497MB file. That explains the error thrown by anaconda.

So I took it a bit further and created another base repo - just the same CentOS 7 one but with a different name. Synced it and examined squashfs.img after the sync. Big surprise 1.1GB !! I have no clue what is causing this - any suggestions anyone?

So how did I fix things? I searched /var/lib/pulp/media/artifact for a file with exactly the same name (all files under that dir have meaningless name looking like a uid). Made a backup copy and replaced it by the squashfs.img I downloaded from a CentOIS mirror. Tried the https:///pulp/content//Library/custom/CentOS_7/base-x86_64/LiveOS/squashfs.img in my browser and now it says the download size is 497 MB (just to prove I replaced the right file).
Tried kickstart again - WORKS :slight_smile: (using the driver from the USB stick).

Big question: how come that squashfs.img is not correct in the pulp repository …

I renamed the title so it’s more relevant, it really looks like na issue with @katello or Pulp. I am not sure what is wrong.

1 Like

An update: the weekly repo sync again corrupted squashfs.img - it jumped again to something like 800MB which resulted again in the kickstart bombing out when trying to unsquash it.
Even more strange: this does not apply to the proxies - there, the squashfs.img is correct and stays correct