Foreman Installed on Ubuntu 20.04 provision 22.04 version

Problem:
Cloud-init problem with Ubuntu 22.04

Expected outcome:

Foreman and Proxy versions:
3.4.1
Foreman and Proxy plugin versions:

Distribution and version:

Other relevant data:

I have problem with Ubuntu22.04 provisioning through Foreman 3.4.1 installed on Ubuntu 20.04.
Installation stuck on cloud-init and getting below error, I highly suspect that this error is not in relaiton with foreman, but as I did not found solution anywhere just checking if anybody had anything similar

2022-12-23T10:32:20 [I] WEBrick::HTTPServer#start: pid=526 port=8443
2022-12-23T10:32:20 [I] Smart proxy has launched on 1 socket(s), waiting for requests
2022-12-23T11:03:58 21d02baf [I] Started GET /tftp/serverName
2022-12-23T11:03:58 21d02baf [I] Finished GET /tftp/serverName with 200 (6.29 ms)
2022-12-23T11:03:58 21d02baf [I] Started GET /dhcp/10.237.9.0/mac/56:6f:c0:c5:00:a8
2022-12-23T11:03:58 21d02baf [I] Finished GET /dhcp/10.237.9.0/mac/56:6f:c0:c5:00:a8 with 200 (1.45 ms)
2022-12-23T11:03:58 21d02baf [I] Started GET /dhcp/10.237.9.0/ip/10.237.9.132
2022-12-23T11:03:58 21d02baf [I] Finished GET /dhcp/10.237.9.0/ip/10.237.9.132 with 200 (0.92 ms)
2022-12-23T11:03:58 21d02baf [I] Started POST /tftp/PXELinux/56:6f:c0:c5:00:a8
2022-12-23T11:03:58 21d02baf [I] Finished POST /tftp/PXELinux/56:6f:c0:c5:00:a8 with 200 (12.11 ms)
2022-12-23T11:03:58 21d02baf [I] Started POST /tftp/PXEGrub2/56:6f:c0:c5:00:a8
2022-12-23T11:03:58 21d02baf [I] Finished POST /tftp/PXEGrub2/56:6f:c0:c5:00:a8 with 200 (3.83 ms)
2022-12-23T11:03:58 21d02baf [I] Started POST /tftp/PXEGrub/56:6f:c0:c5:00:a8
2022-12-23T11:03:58 21d02baf [I] Finished POST /tftp/PXEGrub/56:6f:c0:c5:00:a8 with 200 (2.66 ms)
2022-12-23T11:03:58 21d02baf [I] Started POST /tftp/iPXE/56:6f:c0:c5:00:a8
2022-12-23T11:03:58 21d02baf [I] Finished POST /tftp/iPXE/56:6f:c0:c5:00:a8 with 200 (0.81 ms)
2022-12-23T11:04:04 fe36c89c [I] Started GET /v2/features
2022-12-23T11:04:04 fe36c89c [I] Finished GET /v2/features with 200 (4.76 ms)
2022-12-23T11:04:07 0623ff53 [I] Started GET /tftp/serverName
2022-12-23T11:04:07 0623ff53 [I] Finished GET /tftp/serverName with 200 (0.49 ms)
2022-12-23T11:04:07 0623ff53 [I] Started GET /dhcp/10.237.9.0/mac/56:6f:c0:c5:00:a8
2022-12-23T11:04:07 0623ff53 [I] Finished GET /dhcp/10.237.9.0/mac/56:6f:c0:c5:00:a8 with 200 (1.31 ms)
2022-12-23T11:04:07 0623ff53 [I] Started GET /dhcp/10.237.9.0/ip/10.237.9.132
2022-12-23T11:04:07 0623ff53 [I] Finished GET /dhcp/10.237.9.0/ip/10.237.9.132 with 200 (1.41 ms)
2022-12-23T11:04:07 0623ff53 [I] Started GET /dhcp/10.237.9.0/mac/56:6f:c0:c5:00:a8
2022-12-23T11:04:07 0623ff53 [I] Finished GET /dhcp/10.237.9.0/mac/56:6f:c0:c5:00:a8 with 200 (0.54 ms)
2022-12-23T11:04:07 0623ff53 [I] Started POST /tftp/PXELinux/56:6f:c0:c5:00:a8
2022-12-23T11:04:07 0623ff53 [I] Finished POST /tftp/PXELinux/56:6f:c0:c5:00:a8 with 200 (0.76 ms)
2022-12-23T11:04:07 0623ff53 [I] Started POST /tftp/fetch_boot_file
2022-12-23T11:04:07 0623ff53 [I] Finished POST /tftp/fetch_boot_file with 200 (5.23 ms)
2022-12-23T11:04:07 0623ff53 [I] [2125] Started task /usr/bin/curl\ --silent\ --show-error\ --connect-timeout\ 10\ --retry\ 3\ --retry-delay\ 10\ --max-time\ 3600\ --remote-time\ --time-cond\ /srv/tftp/boot/ubuntu2204-yJLsSoEw1b0Y-vmlinuz\ --write-out\ Task\ done,\ result:\ %{http_code},\ size\ downloaded:\ %{size_download}b,\ speed:\ %{speed_download}b/s,\ time:\ %{time_total}ms\ --output\ /srv/tftp/boot/ubuntu2204-yJLsSoEw1b0Y-vmlinuz\ --location\ http://10.237.9.31/pub/installation_media/ubuntu/22.04-x86_64//casper/vmlinuz
2022-12-23T11:04:07 0623ff53 [I] Started POST /tftp/fetch_boot_file
2022-12-23T11:04:07 0623ff53 [I] Finished POST /tftp/fetch_boot_file with 200 (0.89 ms)
2022-12-23T11:04:07 0623ff53 [I] [2127] Started task /usr/bin/curl\ --silent\ --show-error\ --connect-timeout\ 10\ --retry\ 3\ --retry-delay\ 10\ --max-time\ 3600\ --remote-time\ --time-cond\ /srv/tftp/boot/ubuntu2204-yJLsSoEw1b0Y-initrd\ --write-out\ Task\ done,\ result:\ %{http_code},\ size\ downloaded:\ %{size_download}b,\ speed:\ %{speed_download}b/s,\ time:\ %{time_total}ms\ --output\ /srv/tftp/boot/ubuntu2204-yJLsSoEw1b0Y-initrd\ --location\ http://10.237.9.31/pub/installation_media/ubuntu/22.04-x86_64//casper/initrd

Hi @gol19

i just had the exact same problem and by going in the shell, i noticed a OOM for cloud-init
my VM had 4Gb, i upped it to 8Gb and it solved the issue

you could try that :slight_smile:

good luck

I did same, 5GB was enough, do you know maybe if there is any option for ubuntu installation with lower memory

not that i know, maybe by playing with some cloud-init options if that even possible…

(looks like ubuntu team if making every effort to change everything at every release, i hate this distribution)

i have another question of my own, do you know if we can get unattented installation with 22.04 and foreman ? the default preseed doesn’t seem to work…

thanks

Hi, I installed it without many problem after pass that cloud init issue.
can you just tell me what version of foreman you are using. on last version 3.4.1 i succeded it.

it’s also version 3.4.1 i’m using, but i guess i must have messed up the install somehow

first of all i reinstall foreman after having installing it once, and now the SSL certificates are no longer recognized, and more importantly i guess my issue is with the /userdata access from the pxelinux.cfg file

i have this :
APPEND ip=dhcp url=http://foreman.me-corp.lan:80/pub/installation_media/ubuntu/22.04-x86_64.iso autoinstall ds=nocloud-net;s=http://foreman.me-corp.lan/userdata/ root=/dev/ram0 ramdisk_size=1500000 fsck.mode=skip

but when i tried to reach this : http://foreman.me-corp.lan/userdata/ from a browser, i’m getting a 404 error.
ActionController::RoutingError (No route matches [GET] "/userdata"):
in the foreman production.log

i’m sure this is a stupid error but i can’t find it yet :slight_smile: if you have an idea i’ll take it :wink:

thanks

can you open in apache new website like below example and restart apache2
Also please follow guide for other steps
https://docs.orcharhino.com/or/docs/sources/guides/ubuntu/provisioning_hosts/configuring_provisioning_resources.html
Inform me if you stuck somewhere else

root@foreman:/etc/apache2# cat sites-available/06-pub.conf
Alias /pub /var/www/html/pub

<Location /pub>

PassengerEnabled off

Options +FollowSymLinks +Indexes
Require all granted

then enable the site, and the alias and autoindex apache2 mods

a2enmod alias
a2enmod autoindex
a2ensite 06-pub.conf

systemctl restart apache2

i already made the site for /pub which works fine (i can start the install ubuntu 22 with no issue)

my problem is the autoinstall doesn’t work, i arrive to the dialog like i was using the ISO manually…

I beleive you stuck at the point where it ask you for language, so if you use latest foreman 3.4.1
I am using below templates
Finish Template: Preseed default finish
PXELinux Template: Preseed default PXELinux Autoinstall
User data template: Preseed Autoinstall cloud-init

Also use autoinstall partition table. For me worked everything with this out of box. then you can tweak it to be good for your situation.
If this will not work try review what each template doing for specifed VM, send me if you stuck somewhere else
try replace foreman.me-corp.local in foreman settings with IP address

I agree it sounds like you could be missing User data template: Preseed Autoinstall cloud-init

When I tested this template it provisioned Ubuntu 22.04 perfectly for me.

I would make sure to add this template to your Ubuntu 22.04 Operating System, delete your host entry and then re-create it.

Another thing is make sure to have an IP assigned to your host. I think it will only generate the userdata if you have assigned an IP address. For a while I was only putting MAC address in and it would generate the pxe files but not the userdata.

Hi @gol19 @wrender

sadly i’m not missing anything you asked me to check

it seems the error i’m getting is that the host i’m trying to install doesn’t receive the template

i found on another thread this command that i tried to run from my host which indeed get stopped asking me the language (meaning not auto provisioning) :

curl -D - http://theforeman.xxxxxxx.xxx/userdata/meta-data

this command works fine on the foreman server itself and return this (meaning it finds the template i guess) :

HTTP/1.1 200 OK
Date: Mon, 23 Jan 2023 13:58:18 GMT
Server: Apache
Content-Type: text/plain; charset=utf-8
ETag: W/"97e82732b6a2aee20dae3492a5f20e8a"
Cache-Control: max-age=0, private, must-revalidate
X-Request-Id: 5d5c4ed3-f9a3-4a80-bd0c-b10056b67556
X-Runtime: 0.042104
Via: 1.1 theforeman.xxxxxxxx.xxx
Transfer-Encoding: chunked

instance-id: i-356a192b7913b04c54
hostname: theforeman.xxxxxxxx.xxx
mac: ea:36:9a:5a:3b:cc
local-ipv4: 10.155.6.201
local-hostname: theforeman.xxxxxxx.xxx

but on the host i’m trying to install (i open a shell just after seeing the language option), i’m getting a 404, so i guess the problem is there…
only problem is, i have no idea where to solve it.

root@ubuntu-server:~# curl -D - -H theforeman.xxxxx.xx http://theforeman.xxxxxxx.xxx/userdata/meta-data
HTTP/1.1 404 Not Found
Date: Mon, 23 Jan 2023 13:58:00 GMT
Server: Apache
Content-Type: text/plain; charset=utf-8
Cache-Control: no-cache
X-Request-Id: 92ae4fc4-eb54-4f7d-b260-eb352f631792
X-Runtime: 0.013141
Via: 1.1 theforeman.xxxxxxxx.xxx
Transfer-Encoding: chunked

Could not find host for request 10.155.6.50

i tried to add the smart proxy in the trusted proxy in /etc/foreman/settings.yml, it didn’t change anything :

:trusted_proxies:
- '10.155.6.201/32'

this is what i get in the production.log itself about this error :


2023-01-23T14:06:00 [I|app|7b274a99] Started GET "/userdata/meta-data" for 10.155.6.50 at 2023-01-23 14:06:00 +0000
2023-01-23T14:06:00 [I|app|7b274a99] Processing by UserdataController#metadata as TEXT
2023-01-23T14:06:00 [E|app|7b274a99] Could not find host for request 10.155.6.50
2023-01-23T14:06:00 [I|app|7b274a99]   Rendered text template (Duration: 0.0ms | Allocations: 2)
2023-01-23T14:06:00 [I|app|7b274a99] Filter chain halted as :find_host rendered or redirected
2023-01-23T14:06:00 [I|app|7b274a99] Completed 404 Not Found in 10ms (Views: 0.4ms | ActiveRecord: 2.7ms | Allocations: 2333)

thanks for your help, this is infuriating that you said it worked out of the box for you :sweat_smile:

can you add in trusted proxies also
:1 and 127.0.0.1

Can you review userdata if is generated correctly on template tab, and try create simple userdata template for test like below, to see if vm will pull it at all

#cloud-config
autoinstall:
version: 1
identity:
hostname: ubuntu-server
password: “$6$exDY1mhS4KUYCE/2$zmn9ToZwTKLhCw.b4/b.ZRTIZM30JZ4QrOQ2aOXJ8yk96xpcCof0kxKwuX1kqLG/ygbJ1f8wxED22bTL4F46P0”
username: ubuntu

i’ll try the simple userdata, but i’m not very optimistic, as in my opinion foreman doesn’t even communicate with the thing…
i’m also trying to add the trusted proxy but for some reason now foreman-installer detects issues in configuration…

anyway on some other point, what foreman-install options/scenarios did you use when you said you installed ubuntu 22.04 out of the box ?

thanks

Good lick with this,

I used DHCP BMC and TFTP, it was enough for me as we are using foreman only for provisioning

any chance you still have the foreman-installer command line ?

i’m considering reinstalling a third time…

on ubuntu is painful tshoot foreman-installer error

I do not have foreman installer, but add dhvp and tftp it should be no error,

add like this in trusted proxies
trusted_proxies:

  • 127.0.0.1
  • “::1”
  • 10.237.9.31 - your ip

it seems i got confused with the foreman-installer switches, using --foreman-proxy-trusted-hosts was not the same as using the line “trusted_proxies” directly in /etc/foreman-proxy/settings.yml

i’m not exactly sure what happened, after reinstalling foreman on centOS (as it seems to be preferred) and simply running foreman-installer, adding afterwards the config options i needed, also by not specifying/modifying the template URL in the preseed, it finally worked

thanks a lof for all your help, it seems the deciding key was the trusted_proxies in the settings, that you have to manually add
the documentation is not really clear on this point, and also i find it weird to have to specifically “trust” your own ip address where the foreman UI and the foreman-proxy resides… seems like an overkill

anyway, thanks :slight_smile:

replying to myself (cause i can’t edit twice), the trusted_proxies needs to be in foreman/settings.yaml, not foreman-proxy