Autoinstalling ubuntu-server 20.04.3 from the live-iso

ashishsn6 · November 24, 2023, 4:36pm

Ooh is it… I am using ubuntu desktop 20.04.6 version ISO.

Does that mean if I use ubuntu desktop 23.04 I can use the same templates
for provisioning & it should work?

arikraz · November 26, 2023, 9:11am

hi , thanks for this article
im not sure regarding <hostname.short >
as i dont know the name prior to provisioning ,
and it implies that i should edit it per provisioning
(or i dindt get it right )
can you share example ?

also regarding the templates , i use Satellite 6.12 , and i can locate the mentioned preseed templates , but im not sure under what section i should associate them :
ie :
under “host initial configuration template " : ???
under " user data template” : ???
and so on for the rest of the template types

Thanks

Arik

jamesps-ebi · November 27, 2023, 8:30am

Yeh, they switched the desktop installer to the same subiquity (server) installer in 23.04.
Best to just skip to 23.10.1 now though.

The templates should work the same for desktop autoinstall, however, I think currently they only support ‘semi-automated’ install for the desktop installer. You will have to include at least 1 interactive-section in your user-data template.
Although you can try fully automated and see what happens.

jamesps-ebi · November 27, 2023, 8:32am

P.S if you really need LTS desktop install right now, you could try doing just a standard automated server installation, then just install ubuntu-desktop package on top.

There might be some other minor tweaks needed as well. See here for a decent guide: GitHub - canonical/autoinstall-desktop

ashishsn6 · November 28, 2023, 3:50pm

Thanks @jamesps-ebi & @bastian-src for the help.

After using the ubuntu server 20.04.6 ISO, I was able to install the OS.
However there are still few issues:

The initrd & vmlinuz are still not getting downloaded.
The machine reboots after installation but still gets into PXE boot again for installation which is unlike Red Hat/Cent OS provisioning. I have to manually change the boot sequence.
It seems the root password is not getting properly updated as I am not able to login via root after the installations.

Any suggestions on the above pls…

bastian-src · November 29, 2023, 8:04am

Sorry, I missed this when you said which Foreman version you are on: Foreman 3.3 cannot detect the boot files automatically since it doesn’t know the adjusted directory path. Foreman 3.4 can actually find the boot files in /casper.
So, I think you would have to upgrade Foreman to solve this issue.

This sounds like your finish template is not executed successfully. Do you see in your Foreman log that the client downloads the template? Or in your Ubuntu installation log, can you see that something similar to “curtin wget <template URL>” (you can see the actual URL when you look at the rendered userdata template) is executed?

“2.” and “3.” could occur due to issues in the userdata template. At the time of the 3.3 release, Ubuntu Autoinstall has been in an early stage for us.
I think in Foreman 3.3, there has been an issue with the finish template not being downloaded and you couldn’t create a root user. Both was fixed with this commit:

github.com/theforeman/foreman

Fixes #34700 - autoinstall template enable root user

committed 12:04PM - 30 May 22 UTC

bastian-src

+34 -22

The `identity` field doesn't allow to add/edit root as a user. Therefore, the us…er-data field must be used. Compare the Ubuntu Autoinstall documentation: https://ubuntu.com/server/docs/install/autoinstall-reference#identity and the cloud-init documentation: https://cloudinit.readthedocs.io/en/latest/topics/modules.html#users-and-groups * Switch finish template to late-commands * Replace identitiy with user-data * Apply template changes to snapshot Co-authored-by: Oleh Fedorenko <ofedoren@redhat.com>

The commit is part of Foreman 3.4.

Even though you might be able to fix 2. & 3. by using the latest version of the template, I’d recommend you upgrading your Foreman in order to get the latest changes & fixes.

I hope this helps

gabriel22x · January 16, 2024, 3:20pm

Hey together I’ve been reading through this and a lot of other posts the past days. I’m trying to implement Foreman at my company and installed Foreman with a running Discovery Image for UEFI and Legacy clients so far. My Testclients can boot into the Discovery Image and get recognised by Foreman. Im stuck at the part of starting the provisioning of discovered Hosts atm. I’m a beginner with Foreman and I’m overwhelmed by all the different posts.

My goal is to get Ubuntu 20.04 and Ubuntu 22.04 Provisioning to work after Hosts have been discovered.

Foreman 3.7.0 is installed on Ubuntu 20.04
Active Features:
Discovery
HTTPBoot
Logs
Puppet
Puppet CA
TFTP

Following this thread I’m not sure which steps are still needed and which are not. Is there a cleaner description for the needed steps 2024? It seems for me as some steps the creator of this Thread did are not needed anymore as they have been implemented in Default Templates by Foreman Devs themselves?

Which Templates are needed minimum to get provisioning with autoinstall to work? Do I need Linux host_init_config default or Preseed Default? Pretty sure I saw both versions in different posts till now.

Do I have to use iPXE or can I just use PXE?

Also can I follow the instructions inside the manual for RH with adjusted paths? Provisioning Hosts

I can’t find up to date instructions for Debian regarding this case inside the Manual, correct me if I’m wrong…

If my provided Info isn’t enough let me know, I would gladly share more to get some guidance!

Kind Regards
Gabriel

quartsize · January 16, 2024, 6:44pm

I also struggled with which templates were needed, but ended up using Preseed default finish, Preseed default PXEGrub2 Autoinstall, and Preseed Autoinstall cloud-init user data.

foreman_user264 · January 30, 2024, 8:47pm

@lzap I am having issues with getting Ubuntu 22.04 provisioned throught Foreman and I have described my issue with the user-data file in post Need help with PXE booting Ubuntu 22.04 on Foreman v 3.5 (nginx), getting 404 Not Found on userdata.

I am using dhcp for the host IP so I do not know the host IP in advance when I provision the host, I am getting 404 on the user-data when I try to provision a host with the following PXELinux line for userdata: ds=nocloud;s=http://<%= foreman_request_addr %>/userdata/.

I can get the provisioning working if I do specify the IP address in the interface menu on Foreman, but cannot get it working with just the MAC address - I am confused if Foreman still does not have the feature to source user-data file with just the MAC address?

I can although get it working if I use - ds=nocloud;s=http://<%= foreman_request_addr %>/userdata/<%= @host.mac %>/ - Is it ok to use the @host.mac in the APPEND line in this case???

Any help is appreciated, Thank you!

lzap · February 7, 2024, 10:08am

Preseed and user-data both use REMOTE_IP to detect the host which is calling home. It is designed this way, therefore, you must allocate the IP in advance. I do not think there is any workaround as far as I am aware.

For some workflows, REMOTE_IP might be just a fallback mechanism, I am not sure if tokens are used for any of these.

github.com

theforeman/foreman/blob/develop/app/controllers/userdata_controller.rb

class UserdataController < ApplicationController
  include ::Foreman::Controller::TemplateRendering

  layout false

  skip_before_action :require_login, :check_user_enabled, :session_expiry, :update_activity_time, :set_taxonomy, :authorize, :verify_authenticity_token

  before_action :set_admin_user
  before_action :skip_secure_headers
  before_action :skip_session
  before_action :find_host

  def userdata
    render_userdata_template
  end

  def metadata
    data = {
      :'instance-id' => "i-#{Digest::SHA1.hexdigest(@host.id.to_s)[0..17]}",
      :hostname => @host.name,

This file has been truncated. show original

ashishsn6 · February 22, 2024, 2:31pm

Thanks everyone for the support, I was able to successfully provision the Ubuntu OS.

Dirk · February 22, 2024, 3:59pm

As I stumbled over some minutes ago, can someone confirm that the following changes are required with 22.04.3 in the template “Preseed Autoinstall cloud-init user data”.

changing hashed_password to password
removing the else with the empty array for ssh_authorized_keys

Until I changed this I got some schema errors and with the changes 22.04.2 and 22.04.3 work fine for me. But I want someone to confirm this as I want to be 100% sure before make a PR to change the template.

bastian-src · March 12, 2024, 8:25am

Hey Dirk,

sorry for the late response. But yes, you’re absolutely right about that. The issues have been reported by a user in this issue:

and were fixed by this PR:

github.com/theforeman/foreman

Fixes #37011 - Apply latest Ubuntu Autoinstall userdata changes

theforeman:develop ← ATIX-AG:fix_37011_update_userdata_template

opened 09:39AM - 14 Feb 24 UTC

bastian-src

+3 -5

As mentioned in the ticket, the latest version of Ubuntu Autoinstall introduces …some changes. Therefore, the following adaptions are made: * Add explicit fallback for offline installation * Replace deprecated lock-passwd with lock_passwd * Remove empty ssh keys array if none provided Still, Autoinstall deployment will fail at the moment, due to #10036

Still, there is currently an issue with the netplan which affects the Autointall template too. We’re working on it as part of this PR:

github.com/theforeman/foreman

Refs #37010 - Ensure correct yaml for netplan

theforeman:develop ← ATIX-AG:refs_37010_fix_netplan

opened 02:02PM - 02 Feb 24 UTC

bastian-src

+8 -8

The changes in #9967 introduce the save navigation operator. Unfortunately, the …corresponding [snippet](https://github.com/theforeman/foreman/blob/e87de8d1a30aeba3da6d0cb2518d94aa450db55e/app/views/unattended/provisioning_templates/snippet/preseed_netplan_generic_interface.erb#L18) relies on the passed arguments being `true`/`false` instead of `nil`. @ekohl should we add a safety check to the `preseed_netplan_generic_interface` that turns `nil` into `false` or revert the save navigator change in this snippet? Or am I overseeing something here?

dmgeurts · January 7, 2025, 10:05am

I’ve gotten as far as deploying both Ubuntu 22.04 and 24.04. But I’m stumped about how the subscription-manager is meant to be installed? I see the installation instructions in the templates, but not how the repo is added. What am I missing, where in the templates are the directives for how the repo should be added?

OS deployment hangs without ever finishing part way through, if I try to modify a cloned copy of “Preseed Autoinstall cloud-init user data” in order to add an apt repo if the following way:

...
#cloud-config
autoinstall:
  version: 1
  apt:
    geoip: false
    preserve_sources_list: false
    primary:
      - arches: [amd64]
        uri: http://archive.ubuntu.com/ubuntu
      - arches: [default]
        uri: http://ports.ubuntu.com/ubuntu-ports
    sources:
      client.list:
        source: "deb https://oss.atix.de/Ubuntu24LTS/ stable main"
        filename: foreman_client.list
        keyid: CA56 F29E 0006 0B63 D089 0D0F 5BD9 6651 DF50 FFFB
        key: |
          -----BEGIN PGP PUBLIC KEY BLOCK-----
          Version: GnuPG v2.0.22 (GNU/Linux)
          
          mQINBFW/GuoBEADO3XQhv0IG2Ea4L4LklilkjhqxNy0CeL53dHH2/vfb7UE6JW6w
          rT7ksSA+0AGwAhAbALAxx5gAgyJmDF3LEC9EjAY6jtrGmEVdUQve7na651wpvCfG
          ...

I’m not certain this is meant to be done, but I can’t find any detail about how the Atix or a local version of the subscription-manager repo is added at the end of an autonomous deployment.

bastian-src · January 17, 2025, 8:50am

Hey @dmgeurts, thanks for your message!

Just in case you don’t know: The autoinstall template that you posted is not the “official” one, check here to get the latest template that we deliver with Foreman (please use the template version according to your foreman version): foreman/app/views/unattended/provisioning_templates/user_data/preseed_autoinstall_cloud_init.erb at develop · theforeman/foreman · GitHub

In my setup, I usually do not use the repos (which are set up in the autoinstall template) at all. More specifically, I’m explicitly disabling any updates during the installation process by setting host params accordingly (check here in the template). The last step of the autoinstall template runs the host’s finish template. I’d recommend using the finish template to integrate an install routine for the subscription-manager (see this snippet).

However, Foreman does not provide specific documentation on how to install the subscription-manager as far as I know.

dmgeurts · January 20, 2025, 11:25pm

@bastian-src Thank you for your reply and suggestions.

How does one troubleshoot the subscription_manager_setup template, I take it this is the finish template you’re referring to?

In fact Global Registration has provision for adding a repo. But I think these parameters are only available when using the Register Host feature from the Foreman UI. Not during host deployment, which is unfortunate.

I tried to add the following into a cloned copy of subsctiption_manager_setup, but this didn’t make any difference (with or without the check whether subscription-manager is installed):

      ...
    elif [ -f /etc/debian_version ]; then
      # Add subscription-manager repo if package is unavailable
      #if ! sudo dpkg -l subscription-manager ; then
      curl --silent --show-error --output /etc/apt/trusted.gpg.d/client.asc https://oss.atix.de/atix_gpg.pub
      echo "deb https://oss.atix.de/Ubuntu22LTS/ stable main" > /etc/apt/sources.list.d/client.list
      $PKG_MANAGER update
      #fi
      $PKG_MANAGER_INSTALL subscription-manager
    fi

finish.sh being run is shown in the logs. But no output of this script is logged as far as I’ve been able to find, probably due to the script being run in-target

dmgeurts · January 21, 2025, 9:08pm

Hang on, isn’t Global Registration only used when manually registering a host (Hosts >> Register Host)?

The Preseed default finish template installs freeipa-client, sets up the rex user, and calls unattended/built before trying to run subscription-manager facts --update. So I starting looking at this template, and noticed the standard version calls Preseed default finish custom post if it exists.

With this knowledge I managed to add the subscription-manager repository as part of finish.sh:

<%#
kind: snippet
name: Preseed default finish custom post
model: ProvisioningTemplate
snippet: true
description: |
  This snippet adds the subscription client repo to Debian/Ubuntu systems
-%>
<%
os_major = @host.operatingsystem.major.to_i
-%>

##
## Subscription manager
##
<% if plugin_present?('katello') -%>
echo "Adding Subscription manager repo"
if [ -f /etc/debian_version ]; then
  # Add subscription-manager repo if package is unavailable
  if ! [[ $(apt-cache search --names-only '^subscription-managr') ]] ; then
    curl --silent --show-error --output /etc/apt/trusted.gpg.d/client.asc https://oss.atix.de/atix_gpg.pub
    echo "deb https://oss.atix.de/Ubuntu<%= os_major %>LTS/ stable main" > /etc/apt/sources.list.d/client.list
    apt-get update
  fi
  #apt-get install subscription-manager -y
fi
<% end -%>

However, installing subscription-manager causes the unattended installation to bomb out. This is probably because Preseed default finish tries to run subscription-manager facts --updates which fails because the host hasn’t been registered yet. So I’m now trying to work out if there’s a snippet I can reuse for that.

And redhat_register looks like a good starting point for creating something like this for Debian/Ubuntu.

bastian-src · January 22, 2025, 8:51am

Running the registration before sending facts sounds good to me.

However, I think the topic got out of the scope of “autoinstalling ubuntu-server” and I’ve no further knowledge on the subscription-manager setup. So, I think you could start a new thread or get back to this one:

Hope this helps you!

dmgeurts · January 22, 2025, 9:04am

Sure thing, apologies for the hijack.