Ubuntu 22.04 provisioning error - cloud config schema errors

Support
,
1

Problem:
During the provisioning process, I receive the following error:

waiting for cloud-init...
finish: subiquity/Userdata/load_autoinstall_data: Cloud config schema errors: autoinstall.user-data.users.0: {'gecos': 'root', 'hashed_passwd': '<snipped>', 'lock-passwd': False, 'name':'root', 'ssh_authorized_keys': []} is not valid under any of the given schemas
An error occurred. Press enter to start a shell

Expected outcome:
The provisioning process completes without errors.

Foreman and Proxy versions:
3.9.1

Foreman and Proxy plugin versions:

  • foreman-tasks: 9.0.0
  • foreman_bootdisk: 21.2.1
  • foreman_remote_execution: 12.0.2
  • katello: 4.11.0

Distribution and version:
AlmaLinux 8.9

Other relevant data:

  • The host is being PXE-less booted, using a Full host image.
  • The host is a VMware VM (managed via vSphere)
  • The Operating System config:
    • Partition Table: preseed default autoinstall
    • Templates:
      • Finish template: Preseed default finish
      • Host initial configuration template: Linux host_init_config default
      • iPXE Template: Preseed default iPXE Autoinstall
      • Provisioning Template: preseed default
      • PXEGrub2 template: Preseed default PXEGrub2 Autoinstall
      • PXELinux template: Preseed default PXELinux Autoinstall
      • User data template: Preseed Autoinstall cloud-init user data
  • Host config:
    • Operating System:
      • Partition Table: Preseed default autoinstall
      • PXE loader: PXELinux BIOS

These are all default Templates (in case the names weren’t obvious), I haven’t modified any of them apart from associating them with the Operating System I created for Ubuntu 22.04.3.

The provisioning boot process appears successful, until reaching the error shown above.

After receiving the error, I can start a shell and do some troubleshooting.

The last line of /var/log/cloud-init.log:

handlers.py[DEBUG]: finish: modules-final: SUCCESS: running modules for final

I can provide more of that file via screenshot, if that helps.

I can run

curl -D - http://<my-server-URL>/userdata/meta-data

and it returns an HTTP 200 response, with the correct hostname, mac and local-ipv4 info for the Host I am trying to provision.

2

More troubleshooting info:

if I run

cloud-init schema --system --annotate

it returns

Valid cloud-config: user-data

if I run

cloud-init status --long

it returns

status: done
boot_status_code: enabled-by-generator
last_update: <date_snipped>
detail:
DataSourceNoCloudNet [seed=cmdline,/var/lib/cloud/seed/nocloud,http://<foreman_server_snipped>/userdata/00:50:56:aa:e0:f9/] [dsmode=net]

I checked /run/cloud/ds-identify.log, and it ends with

datasource 'nocloud-net' specified.
[up 133.26s] returning 0

The last lines of /var/log/cloud-init-output.log are:

Cloud-init v. 23.2.1-0ubuntu0~22.04.1 running 'modules:config' at <date_snipped>. Up 166.11 seconds.
Cloud-init v. 23.2.1-0ubuntu0~22.04.1 running 'modules:final' at <date_snipped>. Up 166.11 seconds.
Cloud-init v. 23.2.1-0ubuntu0~22.04.1 finished at <date_snipped>. Datasource DataSourceNoCloudNet [seed=cmdline,/var/lib/cloud/seed/nocloud,http://<foreman_server_snipped>/userdata/00:50:56:aa:e0:f9/] [dsmode=net].  Up 166.81 seconds

I checked /var/log/httpd/foreman_access.log on the Foreman server, and found this:

142.207.14.70 - - [16/Jan/2024:10:17:38 -0800] "GET /pub/installation_media/ubuntu/22.04-x86_64.iso HTTP/1.1" 200 2133391360 "-" "Wget"
142.207.14.70 - - [16/Jan/2024:10:19:40 -0800] "GET /userdata/00:50:56:aa:e0:f9/meta-data HTTP/1.1" 200 152 "-" "Cloud-Init/23.2.1-0ubuntu0~22.04.1"
142.207.14.70 - - [16/Jan/2024:10:19:40 -0800] "GET /userdata/00:50:56:aa:e0:f9/user-data HTTP/1.1" 200 1390 "-" "Cloud-Init/23.2.1-0ubuntu0~22.04.1"
142.207.14.70 - - [16/Jan/2024:10:19:40 -0800] "GET /userdata/00:50:56:aa:e0:f9/vendor-data HTTP/1.1" 404 1564 "-" "Cloud-Init/23.2.1-0ubuntu0~22.04.1"
142.207.14.70 - - [16/Jan/2024:10:19:41 -0800] "GET /userdata/00:50:56:aa:e0:f9/vendor-data HTTP/1.1" 404 1564 "-" "Cloud-Init/23.2.1-0ubuntu0~22.04.1"
142.207.14.70 - - [16/Jan/2024:10:19:42 -0800] "GET /userdata/00:50:56:aa:e0:f9/vendor-data HTTP/1.1" 404 1564 "-" "Cloud-Init/23.2.1-0ubuntu0~22.04.1"
142.207.14.70 - - [16/Jan/2024:10:19:43 -0800] "GET /userdata/00:50:56:aa:e0:f9/vendor-data HTTP/1.1" 404 1564 "-" "Cloud-Init/23.2.1-0ubuntu0~22.04.1"
142.207.14.70 - - [16/Jan/2024:10:19:44 -0800] "GET /userdata/00:50:56:aa:e0:f9/vendor-data HTTP/1.1" 404 1564 "-" "Cloud-Init/23.2.1-0ubuntu0~22.04.1"
142.207.14.70 - - [16/Jan/2024:10:19:45 -0800] "GET /userdata/00:50:56:aa:e0:f9/vendor-data HTTP/1.1" 404 1564 "-" "Cloud-Init/23.2.1-0ubuntu0~22.04.1"
142.207.14.70 - - [16/Jan/2024:10:19:46 -0800] "GET /userdata/00:50:56:aa:e0:f9/vendor-data HTTP/1.1" 404 1564 "-" "Cloud-Init/23.2.1-0ubuntu0~22.04.1"
142.207.14.70 - - [16/Jan/2024:10:19:47 -0800] "GET /userdata/00:50:56:aa:e0:f9/vendor-data HTTP/1.1" 404 1564 "-" "Cloud-Init/23.2.1-0ubuntu0~22.04.1"
142.207.14.70 - - [16/Jan/2024:10:19:48 -0800] "GET /userdata/00:50:56:aa:e0:f9/vendor-data HTTP/1.1" 404 1564 "-" "Cloud-Init/23.2.1-0ubuntu0~22.04.1"
142.207.14.70 - - [16/Jan/2024:10:19:49 -0800] "GET /userdata/00:50:56:aa:e0:f9/vendor-data HTTP/1.1" 404 1564 "-" "Cloud-Init/23.2.1-0ubuntu0~22.04.1"
142.207.14.70 - - [16/Jan/2024:10:19:50 -0800] "GET /userdata/00:50:56:aa:e0:f9/vendor-data HTTP/1.1" 404 1564 "-" "Cloud-Init/23.2.1-0ubuntu0~22.04.1"

I’m at a real loss here. There doesn’t seem to be any errors in the logs or problems with the user-data cloud-config.

3

I have resolved the issue (for now) by making a clone of “Preseed Autoinstall cloud-init user data” Provisioning template, and replacing:

user-data:
    disable_root: false
    fqdn: <%= @host.name %>
    users:
    - name: <%= username_to_create %>
      gecos: <%= realname_to_create %>
      lock-passwd: false
      hashed_passwd: <%= password_to_create %>
<% if !host_param('remote_execution_ssh_keys').blank? -%>
<%   if host_param('remote_execution_ssh_keys').is_a?(String) -%>
      ssh_authorized_keys: [<%= host_param('remote_execution_ssh_keys') %>]
<%   else -%>
      ssh_authorized_keys: <%= host_param('remote_execution_ssh_keys') %>
<%   end -%>
<% else -%>
      ssh_authorized_keys: []
<% end -%>

with

user-data:
    disable_root: false
    fqdn: <%= @host.name %>
    chpasswd:
      expire: false
      list:
        - root:<%= password_to_create %>

The provisioning process completes, and I am able to sign in to the new host using the root password configured in Foreman.

I did see a message about the ‘lists’ parameter being deprecated, so perhaps this will stop working with a future version of cloud-init.