Autoinstalling ubuntu-server 20.04.3 from the live-iso

Tutorials
,
30

Hi @langesmalle @acittlau @upeter @Zedeau :slight_smile:

I have created a draft PR to document the procedure to provision hosts running Ubuntu 22.04. I will keep you posted once this is merged.

4 Likes
31

Hi,

I’ve followed langesmalle initial instructions on provisoning Ubuntu 20.04.3. I may be mistaken but based on this line in the “Preseed default PXELinux Autoinstall” template …

APPEND ip=dhcp url=http://<%= @preseed_server %><%= @preseed_path %> autoinstall ds=nocloud-net;s=http://<%= @preseed_server %>/pub/preseed/autoinstall/<%= @host.shortname %>/ root=/dev/ram0 ramdisk_size=1500000 fsck.mode=skip

… the following 2 settings …

  • url=http://<%= @preseed_server %><%= @preseed_path %>
  • s=http://<%= @preseed_server %>/pub/preseed/autoinstall/<%= @host.shortname %>/

… dictates where the iso file is and where the meta-data and user-data files are. The provisioning process depends on these 3 files, irrespective of what is configured in the following templates:

  • Preseed Autoinstall cloud-init user data
  • preseed_netplan_generic_interface
  • preseed_netplan_setup

I’ve installed Ubuntu 20.04.4 in a VM manually setting all the required configurations, copied its autoinstall-user-data file as the user-data file for provisioning and have provisiond an Ubuntu 20.04.4 VM successfully (fully unattended) through Foreman’s discovery process using a Host Group.

But I’m trying to find a way to update the user-data file before the Build process (namely static ip address) but without the Provisioning template coming into play, I lose the ability to override host parameters in the “Preseed Autoinstall cloud-init user data” template.

Unless I’ve mis-interpreted the contents in this thread as it has a lot of information and I could have done the provisioning process all wrong.

Thanks.

1 Like
32

Hey @tlauwk!
In order to re-enable the Provisioning template, you can make use of the userdata interface of Foreman. Have a look at these changes. Then, you have to define a userdata template for the OS and select it for the host. This gives you the option to make use of provisioning templates as usually. This line must be changed accordingly:

APPEND ip=dhcp url=http://<%= @preseed_server %><%= preseed_path %> autoinstall ds=nocloud-net;s=http://<%= userdata_server %>/userdata/ root=/dev/ram0 ramdisk_size=1500000 fsck.mode=skip

It references the userdata interface instead of pointing to a static file. The variable userdata_server points to the corresponding smart proxy and must be initialized as followed:

userdata_server = foreman_url('').sub(/.*:\/\//, '').split('/').first
userdata_server = @preseed_server if userdata_server.split(':').first == foreman_server_fqdn

Be aware that this won’t be the final state of the template - the PR is still in review and might change, but it should fix your problem for now.

1 Like
33

Hi @bastian-src,

Thanks for the heads up, it works a treat.

I had to make a slight adjustment as I’ve got a remote site with a Smart Proxy installed (DHCP, TFTP, Discovery, Templates and Logs). The VM at the remote site resolves …

http://<%= userdata_server %>/userdata/

… to …

http://remote-smart-proxy:8000/userdata/

… which doesn’t work. I had to swap <%= userdata_server %> with <%= @preseed_server %> and the build would continue successfully as it now resolves to …

http://foreman-master:80/userdata/

All good so far. What I would like though if for the VMs to download the iso from their respective site smart proxy.

Something like …

http://<%= @smart-proxy %><%= port 80 %><%= preseed_path %>

… so it doesn’t have to cross the WAN.

And if possible get the user-data template from their respective site proxy as well:

http://<%= @smart-proxy %><%= port 80 %>/userdata/

I was wondering if this was possible with a different foreman variable in the paths?

Thanks.

1 Like
34

The @preseed_server resolves to the domain defined in the corresponding Installation Media of the host. So, I’d suggest creating an Installation Media (or even better, also an additional Host Group) for every of your Smart Proxies, and use the @preseed_server variable to reference the iso image in the template. Then, you can just associate the corresponding host group with the installation media & the hosts are gonna use that respective proxy (aka preseed_server) in the template.

In order to get the userdata from the respective Smart Proxy as well, I’d recommend using the userdata_server variable as described earlier. Depending on your Foreman version, your Smart Proxies forwards http://remote-smart-proxy:8000/userdata/ requests to http://foreman-master:80/userdata/ - Therefore, you have to check that:

  • a) port 8000 is open on your Smart Proxy and that you have /etc/foreman-proxy/settings.d/templates.yml is configured accordingly
  • b) you configured :trusted_proxies on your foreman

In order to configure the trusted proxies, run the following command on your foreman:

# foreman-installer --foreman-trusted-proxies "127.0.0.1" --foreman-trusted-proxies "::1" --foreman-trusted-proxies "<Your_Proxy_IP>"

It adds a field trusted_proxies in your /etc/foreman/settings.yaml. Let me know if this works for you!

2 Likes
35

creating an Installation Media (or even better, also an additional Host Group) for every of your Smart Proxies

I was hoping to avoid doing the above as I actually have 2 remote proxies (may be more later) so that would mean 3 installation medias and 3 host groups. So triple the chance of my engineers selecting the wrong option during deployment. I’ll have to think about this … iso download time vs administrative simplicity.

Port 8000 isn’t listening on the remote smart proxies. Just to clarify on the remote smart proxies, I should run foreman-installer -i and set the following:

5. Set http, current value: true
6. Set http_port, current value: 8000

On the foreman-master, the foreman-installer -i says:

55. Set trusted_proxies, current value: []
localhost addresses (127.0.0.1/8, ::1) need to be in trusted_proxies

Is the /8 mask required for configuring localhost addresses in foreman-trusted-proxies?

Thanks.

36

Ah okay, yeah makes sense. Maybe, you can limit the host groups visible to your engineers by adapting their access rights via location and organization?

Yep, that’s right. You should also check your firewall settings to enable port 8000.

That description is wrong, it must be an address and not a subnet. So, the mask shouldn’t be here.

37

Thanks bastian-src.

Its all working well now using the variable userdata_server.

Cheers.

1 Like
38

Has anyone been able to get an autoinstall working via UEFI?

I have cloned the “Preseed default PXEGrub2” template and replaced the menuentry value with the following, based on the working “Preseed default PXELinux Autoinstall” template:

menuentry '<%= template_name %>' {
  linux<%= efi_suffix %>  ubuntu/focal/vmlinuz ip=dhcp url=http://<%= @preseed_server %><%= @preseed_path %> autoinstall ds=nocloud-net\;s=http://<%= @preseed_server %>/userdata/ cloud-config-url=/dev/null root=/dev/ram0 ramdisk_size=1500000 fsck.mode=skip
  initrd<%= efi_suffix %> ubuntu/focal/initrd
}

This seems to work, right up until it should be loading the userdata and starting the automated cloud-init process. Instead, it starts asking questions beginning with language and keyboard layout, which are answered in the user-data file.

Looking at my webserver logs, it is requesting the user-data and meta-data files, but it seems to be ignoring them and asking for information instead.

Any ideas, or anyone have a working UEFI PXE template I can copy? We have newer hardware which won’t boot in legacy BIOS mode, and so we have to use UEFI to provision them.

39

This is an issue that also affects my environment. I have not yet found a solution.

40

My issue turned out to be a missing interface identifier in Foreman, which was causing the user-data YAML to be invalid:

network:
  version: 2
  ethernets:
    : # <-- Note the missing hash key
      dhcp4: true
      dhcp6: false

Didn’t catch it on my other test nodes because those were rebuilds of existing machines which already had the interface values populated via Puppet. This was a new build, and previously our builds didn’t require setting a specific interface name so we’ve left it blank. Time to update our docs!

Just to note, filling out the IP Address is required as well, because Foreman uses that to authenticate requests to the /userdata/ endpoints, based off the IP of the request, so the host entry IP and the IP of the machine PXE booting have to match. If you have a reverse proxy in front of Foreman, be sure you are correctly setting the X-Forwarded-For header, and are passing that to Foreman correctly. We had to add the mod_remoteip module to Apache and configure it to trust our load balancer.

All working now!

3 Likes
41

Make sure to file a PR to our templates if you spot a bug. THanks!

42

@ncstate-mafields ,

I have been able to make it work for UEFI systems by having this in my PXEGrub2 template:

Note, this is still with the solution I have worked out on our bootstrap system, so pobably some variables needs to be changed.

menuentry "Install Ubuntu Server <%= @host.operatingsystem.release_name %>" {
    set gfxpayload=keep
    linux ubuntu/<%= @host.operatingsystem.release_name %>/vmlinuz ip=dhcp url=http://<%= @preseed_server %><%= @preseed_path %> autoinstall ds=nocloud-net\;s=http://<%= @preseed_server %>/pub/preseed/autoinstall/<%= @host.shortname %>/ root=/dev/ram0 ramdisk_size=1500000 fsck.mode=skip
    initrd ubuntu/<%= @host.operatingsystem.release_name %>/initrd
}

As you can see it contains the extra line “set gfxpayload=keep”, and the line starting with linux does not contain “cloud-config-url=/dev/null”.

Hope this will help you further.

43

Hi,

I’ve ensured my template points to ds=nocloud-net;s=http://<%= @preseed_server %>/pub/preseed/autoinstall/<%= @host.shortname %>/ and created the pub directory on my foreman server as I do not have katello installed.

However I still do not see the <host.shortname> with userdata being generated in the pub directory. Should I be able to generate the userdata file without Katello installed?

Many Thanks,
Tania

44

@tania ,
This is indeed the part which is not yet available to be created by forman (we are still running foreman 3.2/katello 4.4)>
In order to have these files, I copy the content of the provisioning-template (which you can view from the host) into th efile user-data.
The file meta-data alsways contains the same content.

45

Hi,

Thank you for the response. I figured I could copy and paste the user-data and that works fine, however with potentially 100 machines to build in a year, this would be rather labour intensive. Do you know if it is planned to introduce this functionality with just foreman?

Many Thanks,
Tania

46

Hi,
i wonder how you all succeeded with having the finish script download and execution in the “user-data”/“runcmd” section as it is delivered in the templates with Foreman 3.3. I had install loops, since it was not executed before the reboot. I shifted it to the “late-commands” section with “curtin in-target” prefixes.
Am i the only one? Is my setup so special?

47

@langesmalle @tania The static userdata endpoint in the pub directory shouldn’t be necessary in Foremen 3.2.

It is capable of the http://my.foreman.com/userdata/ API endpoint. So, you can create a userdata template according to the current upstream template (see here). Then, you have to associate the template with the corresponding operating system first and choose the template in that operating system configuration afterwards.

You can see in this post how the PXELinux (similar for UEFI) template can then point to the corresponding address.

Does this help you/make it easier to configure?

48

Hey @timdeluxe!

No, your setup is probably not so special. I did also struggle with the runcmd section - does it work when you shift the section to curtin in-target?

I’m deploying hosts with the latest version of the template without any problems. You can see it here. You gotta pay attention to change the download directory to /target/tmp/finish.sh when using the in-target commands afterwards.

49

@ncstate-mafields regarding your issue with the missing network identifier:

My issue turned out to be a missing interface identifier in Foreman, which was causing the user-data YAML to be invalid:

I’ve currently this PR open which originated from @acittlau’s post here. It implements a MAC-based network device assignment which is more reliable.