Network based provision with own media

Support
1

Problem:
I’m getting those errors (in foreman-proxy logs) while machine is booting:

ERROR – : [2150] 2018-06-05 00:37:48 URL: ftp://192.168.122.127/pub/ubuntu/dists/xenial/main/installer-amd64/current/images/netboot/ubuntu-installer/amd64/linux [7015896] -> “/var/lib/tftpboot/boot/Ubuntu-16.04-x86_64-linux” [1]

ERROR – : [2157] 2018-06-05 00:37:48 URL: ftp://192.168.122.127/pub/ubuntu/dists/xenial/main/installer-amd64/current/images/netboot/ubuntu-installer/amd64/initrd.gz [37401443] -> “/var/lib/tftpboot/boot/Ubuntu-16.04-x86_64-initrd.gz” [1]

Expected outcome:

Foreman and Proxy versions: 1.17

Foreman and Proxy plugin versions:

Other relevant data:
I’ve tried to speed up network based provision (using default mirrors takes really long) and to do so i made ftp server and copied there files from the mirror. Except for above errors in the booted machine i’m getting:
“The file needed for preconfiguration could not be retrieved from …”
What’s more when i try to review templates for this host, all of them return 404 pages. So:

  1. What am i missing here?
  2. Whats the best and fastest way to provision new hosts? I’ve used image based however i read thats a really bad idea to use the same image for multiple hosts. And i kinda want to automate provisioning, so i dont really like the idea of cloning image every time i want to create new host.
    Dont know if its relevant but i use libvirt and ubuntu OS.
2

If you click on the ftp link above, will it download?

And if you do the same but via wget on the host you run TFTP Proxy on, will it download as well?

My guess is that you mirrored it incorrectly or there are some (permission?) issues?

This is how it works when new host is created:

  • Foreman figures out kernel/initramdisk URL from OS/Install Media
  • Foreman determines which TFTP Smart Proxy to use from associated Subnet
  • Foreman asks TFTP Proxy to download both files
  • Smart proxy performs HTTP download of the two files and places them in TFTP directory
3

On I think Smart Proxy no longer supports non-http(s) installation media URLs. We used to use wget to download the files, newer versions now use pure Ruby HTTP client. You will need to publish your mirror via HTTP.

Looks like a regression to me? @ohadlevy ?

4

Yeah i can download files via wget + normal browser. Will check with http instead of ftp and will come back with results. Thanks for the help so far.
BTW what about my 2nd question?I know it prolly depends on specific use-case, but whats the fastet way (that doesnt cause concurrency problems like image based)? Installing ubuntu from official mirror takes like 30mins.

5

Fastest will be always image-based provisioning. Not sure what you mean by concurrency issue, but if you have problems with many servers overloading your download site, you can try our mass bare-metal image-based provisioning using discovery:

https://github.com/theforeman/foreman-discovery-image/pull/100

The workflow is based on https://www.udpcast.linux.lu/ tool:

  • Boot your servers into discovered image
  • Start udpcast listeners
  • On your server with image, send the image to one or multiple nodes
  • Reboot the servers

We want to allow remote execution plugin on discovered servers to automate this, you can do this ad-hoc for now.

Also A/B image-based provisioning is very fast, that’s what CoreOS updater does. You can achieve the same by creating two volumes and mirroring the other partition from your server using HTTP or udpcast or bittorrent or even rsync/casync.

6

If the smart proxy can’t download over https it sounds like a regression as it used to ignore certs and still download. I guess it’s not a common usage case hence went undetected

7

It’s actually from FTP, we used wget which might be able to download those. Now it’s HTTP ruby.

8

For the record …

PXE installations of Ubuntu have always been slow in my experience - even without Foreman, I’ve seen it take 45 mins to install Ubuntu from a netboot ISO. I don’t think it’s something on the user side, it’s more likely something inherent to Ubuntu - Debian installs on the same network take far less time.

9

By concurrency i meant there might be some problems with using the same image for starting multiple machines, or am i wrong here ?
Also, changing ftp to http worked.

1 Like