Autoinstalling ubuntu-server 20.04.3 from the live-iso

ashishsn6 · November 22, 2023, 2:38pm

Yes tried it, still the same issue.

Don’t think the image is getting downloaded because I get the error in 20 seconds when the device is booted from network.

I can see log “Started Get “/userdata/” for foreman-ip” however it ends with error “ActionController::RoutingError”

yes, it gives me the option to choose n/w static or dhcp, then few network setting & URL to download image which is starting with “archive.ubuntu.com”. As per my understanding this means the endpoint is not able to pick the image from my hosted ISO.

bastian-src · November 22, 2023, 3:35pm

I just figured there is an issue with your URL here. It points to the directory, not the ISO image as it is supposed to.

The path comes from the corresponding Installation Media and the template should turn the directory path to an .iso suffix:

github.com

theforeman/foreman/blob/8fbfbca34b694cdb701f3e33b1f6c89203a62b68/app/views/unattended/provisioning_templates/PXELinux/preseed_default_pxelinux_autoinstall.erb#L37


      
              options << "auto=true"
              options << "domain=#{@host.domain}"
            else
              options << 'console-setup/ask_detect=false console-setup/layout=USA console-setup/variant=USA keyboard-configuration/layoutcode=us localechooser/translation/warn-light=true localechooser/translation/warn-severe=true'
            end
            if @host.provision_interface.vlanid.present?
              options << "netcfg/use_vlan=true netcfg/vlan_id=#{@host.provision_interface.vlanid}"
            end
            options << "locale=#{host_param('lang') || 'en_US'}"
            options = options.join(' ')
            image_path = @preseed_path.sub(/\/?$/, '.iso')
          -%>
          #
          # WARNING
          #
          # Foreman will not download the kernel/initramdisk to PXE automatically. Please follow
          # the official Ubuntu documentation and extract the files from the LiveCD (DVD) manually
          # and optionally update the KERNEL and INITRD lines in this template.
          #
          DEFAULT linux cloud-init autoinstall
          LABEL linux cloud-init autoinstall

Check your Installation Media, the PXELinux template, and maybe test it with a static path to verify that’s the issue.

EDIT

Also, check the links mentioned here in case you didn’t already:

ashishsn6 · November 22, 2023, 3:54pm

So I don’t need to extract the ISO image on my apache server?
At present my ISO is extracted and I have removed the ‘.iso’ from the template as I was not getting the extracted ISO structure in the browser while checking.

bastian-src · November 22, 2023, 4:00pm

You need both.

Since you want to deploy an Ubuntu 20.04 host with Autoinstall (so, an Ubuntu 20.04.3+ host), you just follow the 22.04 setup as described in the docs (for Ubuntu 20.04.0/1/2, it’s been different):

https://docs.theforeman.org/nightly/Provisioning_Hosts/index-foreman-el.html#Creating_an_Installation_Medium_for_Ubuntu_22_04_provisioning

Thereby, Foreman needs the extracted iso image in order to download the boot files and the Ubuntu host needs the unextracted iso image for its installation.

We’re working on automating this process such that you only need to provide the iso image in the future and Foreman extracts the boot files on its own.

I hope this helps

ashishsn6 · November 22, 2023, 4:38pm

Will try & get back
Thanks

ashishsn6 · November 24, 2023, 2:31pm

Hey @bastian-src

Tried with the above suggestion. The good news is Ubuntu ISO is now getting downloaded & starting the installation however now while installing it gets into GUI Desktop mode.

The last log I see on the screen before going into GUI is " Started session 1 of user ubuntu", at this stage non of the files like /etc/hosts or /etc/resolv.conf is update when I am checking on the terminal.

Sharing the below info if it helps:
The initrd & vmlinuz are still not getting downloaded, I have to manually download them.
The ISO file I am using is of Ubuntu Desktop.

jamesps-ebi · November 24, 2023, 2:40pm

Which ubuntu version are you installing? I believe Ubuntu only added autoinstall support for the Desktop version with 23.04+

So if you’re trying this with 22.04, autoinstall is not going to work unless you use the server ISO

ashishsn6 · November 24, 2023, 4:36pm

Ooh is it… I am using ubuntu desktop 20.04.6 version ISO.

Does that mean if I use ubuntu desktop 23.04 I can use the same templates
for provisioning & it should work?

arikraz · November 26, 2023, 9:11am

hi , thanks for this article
im not sure regarding <hostname.short >
as i dont know the name prior to provisioning ,
and it implies that i should edit it per provisioning
(or i dindt get it right )
can you share example ?

also regarding the templates , i use Satellite 6.12 , and i can locate the mentioned preseed templates , but im not sure under what section i should associate them :
ie :
under “host initial configuration template " : ???
under " user data template” : ???
and so on for the rest of the template types

Thanks

Arik

jamesps-ebi · November 27, 2023, 8:30am

Yeh, they switched the desktop installer to the same subiquity (server) installer in 23.04.
Best to just skip to 23.10.1 now though.

The templates should work the same for desktop autoinstall, however, I think currently they only support ‘semi-automated’ install for the desktop installer. You will have to include at least 1 interactive-section in your user-data template.
Although you can try fully automated and see what happens.

jamesps-ebi · November 27, 2023, 8:32am

P.S if you really need LTS desktop install right now, you could try doing just a standard automated server installation, then just install ubuntu-desktop package on top.

There might be some other minor tweaks needed as well. See here for a decent guide: GitHub - canonical/autoinstall-desktop

ashishsn6 · November 28, 2023, 3:50pm

Thanks @jamesps-ebi & @bastian-src for the help.

After using the ubuntu server 20.04.6 ISO, I was able to install the OS.
However there are still few issues:

The initrd & vmlinuz are still not getting downloaded.
The machine reboots after installation but still gets into PXE boot again for installation which is unlike Red Hat/Cent OS provisioning. I have to manually change the boot sequence.
It seems the root password is not getting properly updated as I am not able to login via root after the installations.

Any suggestions on the above pls…

bastian-src · November 29, 2023, 8:04am

Sorry, I missed this when you said which Foreman version you are on: Foreman 3.3 cannot detect the boot files automatically since it doesn’t know the adjusted directory path. Foreman 3.4 can actually find the boot files in /casper.
So, I think you would have to upgrade Foreman to solve this issue.

This sounds like your finish template is not executed successfully. Do you see in your Foreman log that the client downloads the template? Or in your Ubuntu installation log, can you see that something similar to “curtin wget <template URL>” (you can see the actual URL when you look at the rendered userdata template) is executed?

“2.” and “3.” could occur due to issues in the userdata template. At the time of the 3.3 release, Ubuntu Autoinstall has been in an early stage for us.
I think in Foreman 3.3, there has been an issue with the finish template not being downloaded and you couldn’t create a root user. Both was fixed with this commit:

github.com/theforeman/foreman

Fixes #34700 - autoinstall template enable root user

committed 12:04PM - 30 May 22 UTC

bastian-src

+34 -22

The `identity` field doesn't allow to add/edit root as a user. Therefore, the us…er-data field must be used. Compare the Ubuntu Autoinstall documentation: https://ubuntu.com/server/docs/install/autoinstall-reference#identity and the cloud-init documentation: https://cloudinit.readthedocs.io/en/latest/topics/modules.html#users-and-groups * Switch finish template to late-commands * Replace identitiy with user-data * Apply template changes to snapshot Co-authored-by: Oleh Fedorenko <ofedoren@redhat.com>

The commit is part of Foreman 3.4.

Even though you might be able to fix 2. & 3. by using the latest version of the template, I’d recommend you upgrading your Foreman in order to get the latest changes & fixes.

I hope this helps

gabriel22x · January 16, 2024, 3:20pm

Hey together I’ve been reading through this and a lot of other posts the past days. I’m trying to implement Foreman at my company and installed Foreman with a running Discovery Image for UEFI and Legacy clients so far. My Testclients can boot into the Discovery Image and get recognised by Foreman. Im stuck at the part of starting the provisioning of discovered Hosts atm. I’m a beginner with Foreman and I’m overwhelmed by all the different posts.

My goal is to get Ubuntu 20.04 and Ubuntu 22.04 Provisioning to work after Hosts have been discovered.

Foreman 3.7.0 is installed on Ubuntu 20.04
Active Features:
Discovery
HTTPBoot
Logs
Puppet
Puppet CA
TFTP

Following this thread I’m not sure which steps are still needed and which are not. Is there a cleaner description for the needed steps 2024? It seems for me as some steps the creator of this Thread did are not needed anymore as they have been implemented in Default Templates by Foreman Devs themselves?

Which Templates are needed minimum to get provisioning with autoinstall to work? Do I need Linux host_init_config default or Preseed Default? Pretty sure I saw both versions in different posts till now.

Do I have to use iPXE or can I just use PXE?

Also can I follow the instructions inside the manual for RH with adjusted paths? Provisioning Hosts

I can’t find up to date instructions for Debian regarding this case inside the Manual, correct me if I’m wrong…

If my provided Info isn’t enough let me know, I would gladly share more to get some guidance!

Kind Regards
Gabriel

quartsize · January 16, 2024, 6:44pm

I also struggled with which templates were needed, but ended up using Preseed default finish, Preseed default PXEGrub2 Autoinstall, and Preseed Autoinstall cloud-init user data.

foreman_user264 · January 30, 2024, 8:47pm

@lzap I am having issues with getting Ubuntu 22.04 provisioned throught Foreman and I have described my issue with the user-data file in post Need help with PXE booting Ubuntu 22.04 on Foreman v 3.5 (nginx), getting 404 Not Found on userdata.

I am using dhcp for the host IP so I do not know the host IP in advance when I provision the host, I am getting 404 on the user-data when I try to provision a host with the following PXELinux line for userdata: ds=nocloud;s=http://<%= foreman_request_addr %>/userdata/.

I can get the provisioning working if I do specify the IP address in the interface menu on Foreman, but cannot get it working with just the MAC address - I am confused if Foreman still does not have the feature to source user-data file with just the MAC address?

I can although get it working if I use - ds=nocloud;s=http://<%= foreman_request_addr %>/userdata/<%= @host.mac %>/ - Is it ok to use the @host.mac in the APPEND line in this case???

Any help is appreciated, Thank you!

lzap · February 7, 2024, 10:08am

Preseed and user-data both use REMOTE_IP to detect the host which is calling home. It is designed this way, therefore, you must allocate the IP in advance. I do not think there is any workaround as far as I am aware.

For some workflows, REMOTE_IP might be just a fallback mechanism, I am not sure if tokens are used for any of these.

github.com

theforeman/foreman/blob/develop/app/controllers/userdata_controller.rb

class UserdataController < ApplicationController
  include ::Foreman::Controller::TemplateRendering

  layout false

  skip_before_action :require_login, :check_user_enabled, :session_expiry, :update_activity_time, :set_taxonomy, :authorize, :verify_authenticity_token

  before_action :set_admin_user
  before_action :skip_secure_headers
  before_action :skip_session
  before_action :find_host

  def userdata
    render_userdata_template
  end

  def metadata
    data = {
      :'instance-id' => "i-#{Digest::SHA1.hexdigest(@host.id.to_s)[0..17]}",
      :hostname => @host.name,

This file has been truncated. show original

ashishsn6 · February 22, 2024, 2:31pm

Thanks everyone for the support, I was able to successfully provision the Ubuntu OS.

Dirk · February 22, 2024, 3:59pm

As I stumbled over some minutes ago, can someone confirm that the following changes are required with 22.04.3 in the template “Preseed Autoinstall cloud-init user data”.

changing hashed_password to password
removing the else with the empty array for ssh_authorized_keys

Until I changed this I got some schema errors and with the changes 22.04.2 and 22.04.3 work fine for me. But I want someone to confirm this as I want to be 100% sure before make a PR to change the template.

bastian-src · March 12, 2024, 8:25am

Hey Dirk,

sorry for the late response. But yes, you’re absolutely right about that. The issues have been reported by a user in this issue:

and were fixed by this PR:

github.com/theforeman/foreman

Fixes #37011 - Apply latest Ubuntu Autoinstall userdata changes

theforeman:develop ← ATIX-AG:fix_37011_update_userdata_template

opened 09:39AM - 14 Feb 24 UTC

bastian-src

+3 -5

As mentioned in the ticket, the latest version of Ubuntu Autoinstall introduces …some changes. Therefore, the following adaptions are made: * Add explicit fallback for offline installation * Replace deprecated lock-passwd with lock_passwd * Remove empty ssh keys array if none provided Still, Autoinstall deployment will fail at the moment, due to #10036

Still, there is currently an issue with the netplan which affects the Autointall template too. We’re working on it as part of this PR:

github.com/theforeman/foreman

Refs #37010 - Ensure correct yaml for netplan

theforeman:develop ← ATIX-AG:refs_37010_fix_netplan

opened 02:02PM - 02 Feb 24 UTC

bastian-src

+8 -8

The changes in #9967 introduce the save navigation operator. Unfortunately, the …corresponding [snippet](https://github.com/theforeman/foreman/blob/e87de8d1a30aeba3da6d0cb2518d94aa450db55e/app/views/unattended/provisioning_templates/snippet/preseed_netplan_generic_interface.erb#L18) relies on the passed arguments being `true`/`false` instead of `nil`. @ekohl should we add a safety check to the `preseed_netplan_generic_interface` that turns `nil` into `false` or revert the save navigator change in this snippet? Or am I overseeing something here?