Unable to use HTTP Proxy in provisioning

Support
1

Problem:
Curl error timeout when provisioning
Foreman and Proxy versions:
Foreman 3.1.2
Distribution and version:
RHEL 8.5
Other relevant data:
Greetings yet again,

I’m at the moment working into provisioning Debian 11 into remote hosts. Long story short, curl is not using the http_proxy, failing the with timeout, which makes sense. Down below is the relevant proxy.log:

2022-03-09T11:33:25 85812bbc [I] Started POST /tftp/PXELinux/00:0c:29:b2:e5:4c
2022-03-09T11:33:25 85812bbc [I] Finished POST /tftp/PXELinux/00:0c:29:b2:e5:4c with 200 (3.82 ms)
2022-03-09T11:33:25 85812bbc [I] Started POST /tftp/PXEGrub2/00:0c:29:b2:e5:4c
2022-03-09T11:33:25 85812bbc [I] Finished POST /tftp/PXEGrub2/00:0c:29:b2:e5:4c with 200 (2.81 ms)
2022-03-09T11:33:25 85812bbc [I] Started POST /tftp/iPXE/00:0c:29:b2:e5:4c
2022-03-09T11:33:25 85812bbc [I] Finished POST /tftp/iPXE/00:0c:29:b2:e5:4c with 200 (2.05 ms)
2022-03-09T11:33:25 85812bbc [I] Started POST /tftp/fetch_boot_file
2022-03-09T11:33:25 85812bbc [I] Finished POST /tftp/fetch_boot_file with 200 (4.4 ms)
2022-03-09T11:33:25 85812bbc [I] [181847] Started task /usr/bin/curl\ --silent\ --show-error\ --connect-timeout\ 10\ --retry\ 3\ --retry-delay\ 10\ --max-time\ 3600\ --remote-time\ --time-cond\ /var/lib/tftpboot/boot/debian-mirror-2SWNytzhr9B6-linux\ --write-out\ Task\ done,\ result:\ \%\{http_code\},\ size\ downloaded:\ \%\{size_download\}b,\ speed:\ \%\{speed_download\}b/s,\ time:\ \%\{time_total\}ms\ --output\ /var/lib/tftpboot/boot/debian-mirror-2SWNytzhr9B6-linux\ --location\ http://ftp.debian.org/debian/dists/bullseye/main/installer-amd64/current/images/netboot/debian-installer/amd64/linux
2022-03-09T11:33:25 85812bbc [I] Started POST /tftp/fetch_boot_file
2022-03-09T11:33:25 85812bbc [I] Finished POST /tftp/fetch_boot_file with 200 (1.9 ms)
2022-03-09T11:33:25 85812bbc [I] [181851] Started task /usr/bin/curl\ --silent\ --show-error\ --connect-timeout\ 10\ --retry\ 3\ --retry-delay\ 10\ --max-time\ 3600\ --remote-time\ --time-cond\ /var/lib/tftpboot/boot/debian-mirror-2SWNytzhr9B6-initrd.gz\ --write-out\ Task\ done,\ result:\ \%\{http_code\},\ size\ downloaded:\ \%\{size_download\}b,\ speed:\ \%\{speed_download\}b/s,\ time:\ \%\{time_total\}ms\ --output\ /var/lib/tftpboot/boot/debian-mirror-2SWNytzhr9B6-initrd.gz\ --location\ http://ftp.debian.org/debian/dists/bullseye/main/installer-amd64/current/images/netboot/debian-installer/amd64/initrd.gz
(...)
2022-03-09T11:34:35 85812bbc [W] [181847] curl: (28) Connection timed out after 10000 milliseconds

2022-03-09T11:34:35 85812bbc [W] [181851] curl: (28) Connection timed out after 10000 milliseconds

Now, yes it needs to be defined, and I did so in Administer > Settings > General > HTTP(S) proxy and also in Content > Default HTTP proxy(of course I had created the HTTP Proxy entry in Infrastructure > HTTP Proxies).

It’s mentioned throughout in the user interface itself that “System-wide proxies must be configured at the operating system level”, however, when I set it on /etc/environment, it doesn’t seem to pick up(but it does work on user terminal). While looking up here in the forum, I’ve found this answer that says to add the variable on /etc/sysconfig/dynflowd, but such file doesn’t exist in my Foreman instance, and even so I’d question whether would be the correct file. I’m afraid I also had no luck looking in the documentation either, but I could have overlooked precisely what I needed.

So finally, after telling you my dilemma and life story, I’d just like to know where the variable http_proxy should added in order to have this to work.

2

Maybe this could help: Unable to add an EC2 compute resource : Error 500. How to setup an http proxy in Foreman ? - #2 by mld

Or maybe this one: Setting a http_proxy for provisioning in foreman - #2 by Dominic_Cleal

1 Like
3

Hey Ron, thanks for the reply.

I’m afraid however that neither of these are what I’m looking for. For the first link, it’s similar to the answer I’ve found and mentioned above, and again, the /etc/sysconfig/httpd file doesn’t exist, nor I think it would make sense for this to be httpd(why would it be Apache’s launching this?). For the second link, he’s having problems with preseed(kickstart for deb and family), and I haven’t yet reach that point into the provisioning. But I’ll do however keep it in my mind, so I appreciate.

To be clear, as pointed in the logs, my issue is that Foreman isn’t downloading the linux kernel or the initrd.gz with curl, making impossible for the hosts to netboot into installation. The link is correct, but with no proxy, it times out.

What service launches the curl command? Perhaps it’s possible to add the http_proxy variable to the service’s systemd file.

4

Hey, Foreman proxy setting does not affect smart-proxy which is spawning curl to download the boot files. You need to configure curl yourself to use a proxy. Note its running under foreman-proxy effective user.

2 Likes
5

Hey Izap,

I assumed as much, but was lost in direction on what to do, and you cleared up on that sense. I have yet been able to test it(in the client’s environment, I do not have access to any host, only the server itself) but I’m confident it will work, as I copied over the same curl config file to root, did unset of http_proxy, ran curl to a random page, and it worked.

This is what I’ve done:

[root@server ~]# cat /etc/passwd | grep foreman-proxy
foreman-proxy:x:983:981:Foreman Proxy daemon user:/usr/share/foreman-proxy:/bin/false (to find foreman-proxy's home)
[root@server ~]# cd /usr/share/foreman-proxy
[root@server foreman-proxy]# touch .curlrc (create config)
[root@server foreman-proxy]# vim .curlrc
[root@server foreman-proxy]# cat .curlrc
proxy [IP]:[PORT]
[root@server foreman-proxy]#

I’ve followed this impressively well put curl documentation, more specifically here for the config file and here for the proxy entry.

Once I have confirmation that all is well on host side on this topic, I’ll mark the solution to close this forum thread. Thank you for your kind support and work.

1 Like
6

I see you are root, keep in mind these files will be created with root owner and perhaps permissions that might not allow foreman-proxy to access such file.

7

I still haven’t been able to have it tested, which is annoying, but it’s what it is. As to being root, I checked beforehand(just didn’t paste that as I figured it was not need for someone in the future going through the past ordeal) and most files in this folder are 644 root:root(and now so’s the .curlrc), so the user foreman-proxy should the very least be able to read. I’ll change though to foreman-proxy:root if it complaints, not see it happening though.

To not leave this topic hanging, I’ll mark this as solution. If it turns out that’s not, I’ll come back to bother y’all again :smiley:

But thank you for your kind help.

1 Like
8

Took them long enough to have it tested, but it works \o/

1 Like