Using cloud-init-templates with foreman-proxy (foreman-proxy-content)

Problem:
Using userdata/cloud-init installation with a foreman-proxy-content-proxy as template server

**Expected outcome: **
Same results as if the host would connect to the main foreman-server to get the user-data-/cloud-init-template

Foreman and Proxy versions:
Both on 1.24.3

Distribution and version:
CentOS 7

Other relevant data:
Hi,

I am currently trying to provision new VMware machines using image-based deployments. For this I want to use cloud-init & userdata and static IP-adresses.
This works fine with an foreman 1.24.3 if my hosts are connected directly to the foreman-server itself to get their userdata-template.

However, I have separated network with no access to other subnets. For this I use a foreman-content-proxy, which is also configured to be template-server. This proxy works fine for other deployment-methods but in case of the userdata-templates the URL http:///userdata/user-data throws an 404.

curl -D - -H 'X-Forwarded-For: 192.168.167.90' https://<proxy-content-url>/userdata/user-data
HTTP/1.1 404 Not Found
Date: Fri, 28 Aug 2020 08:33:54 GMT
Server: Apache
Content-Length: 216
Content-Type: text/html; charset=iso-8859-1

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL /userdata/user-data was not found on this server.</p>
</body></html>

Other than that I have not found any helpful entries in the logs.

I also tried to access it on different ports. Port 8000 gives an Internal Server Error, which also appears in the production.log.
If I try to use port 8443 I get a userdata-template but this is rendered for my proxy-content-host and not for the given IP or host I want to provision. Same effect if I use a squid-proxy to connect to the main foreman.

I found that this should be somehow implemented in the pullrequest: https://github.com/theforeman/smart-proxy/pull/606

But it seems only to work for the main-foreman-server.

Am I missing some extra configuration to enable this or is this feature just not available?

Cheers,
Jonas

Hello, support was added in 1.24 according to git (1.23 according to redmine probably incorrectly set flag): Feature #24885: templates proxy should proxy cloud-init nocloud templates - Smart Proxy - Foreman

This was added as part of “templates” feature, the moment you have it turned on on HTTP endpoint, you should be able to reach it on the HTTP port (8000 in your case) under the /userdata path. Doublecheck the smart proxy config in settings.d/templates.yml and that the module was not turned off during boot because of some error.

Hi,

thank you for the answer. But my templates.yml looks ok:

--
# Enable this if the Proxy should handle template requests on behalf of Foreman
# Can be true, false, or http/https to enable just one of the protocols
:enabled: true

# This plugin also requires that :foreman_url: be set in the main settings.yml
# This lets the plugin know how to obtain the templates from foreman.

# This allows the proxy to define how hosts that are being provisioned where to
# obtain the templates from. Most installers don't support https, so it's recommended
# to enable an http port listener in the main config file too, and use it in
# the url below
#
# :template_url is the URL the host should use to contact the proxy for a template.
# The default protocol is http on port 80 unless otherwise specified in the url.
# Examples:
# https://1.2.3.4:8443            # default proxy https port
# http://1.2.3.4:8000             # default proxy http port
# https://smart-proxy.example.com # assumes port 443
# http://smart-proxy.example.com  # assumes port 80
# smart-proxy.example.com:8080    # assumes http
:template_url: http://<proxyurl>:8000

Or do I need some extra configuration for the userdata?

But I can try it again with the port 8000 in the URL - maybe I missed something during the tests until it worked without the proxy.

Hi again,

I retried it - with port 8000 I get a user-data-template, BUT it is not the correct one.
I expect the template to render with the details of the host I provisioned but I get the template rendered for the smart-proxy I am using to register to.
It seems that the template-API is forwarded to the foreman but then it renders the template for the smart proxy and not for the host I want to register.

This follows to a new cloned host (with the correct provisioned static IP by the vSphere Customization) but then it runs cloud-init and has the same hostname as my smart-proxy (which then leads to errors within the subscription-manager and the puppet registration).

That sounds like a nasty bug indeed. Let me take a look.

So it’s a forgotten configuration step in our guide:

Perform this step to fix the issue.

Hello lzap,

I have the same problem as described above.

If I now add the remote addr setting I don’t get any template with curl anymore.
curl gives me this output:

Could not find host for request 127.0.0.1

What is missing here so that it can be determined where the request is coming from?

If i try it with port 8000 instead of 8443 i get
Failed to retrieve user-data userdata template for {“kind”=>“user-data”}: Error retrieving userdata/user-data for {“url”=>“http://< smart-proxy-fqdn >:8443”} from < foreman-fqdn >.

Got it, remote_addr was wrong.

Hi,
I am trying to deploy a server cloudinit-based via foreman-proxy as well (version is foreman 2.3.5). Unfortunately the parameter remote_adress no longer exists in this version and apparently has been replaced by the setting “trusted hosts”.
I tried to set the trusted hosts in the katello-answers.yaml on the foreman and foreman-proxy-content-answers.yaml on the foreman proxy (with subsequently running foreman-installer → value is being correctly set in setting.yml), but deployment remains impossible, as the proxy returns an internal server error 500.
When trying to curl the user-data this only works for foreman, the foreman-proxy returns There was an error rendering the CloudInit default demo template: undefined method 'full_path' for nil:NilClass
This has not been changed by the trusted hosts setting. Can possibly any one tell me the correct setting to deploy cloudinit-based via foreman proxy? Thanks!

Your problem seems to be with installation procedure, this thread is about a particular problem in content proxying. I suggest you to create a new thread.

Hi @lzap,
the issue is not really an problem with the installation procedure.

In the original case I had, the solution was to configure my smart proxy in the settings using the “remote_addr” parameter. However, this parameter does not exist any more since foreman 2.3 but is still the solution to this problem according to the manual.
So the question is: where is the “remote_addr” parameter found in releases >= 2.3 or how should the content proxying be configured.

The setting was removed in https://projects.theforeman.org/issues/30779

Correct me if I read the patch wrong, @ekohl, it looks like in our docs (VMWare section), instead of this:

hammer settings set --name remote_addr --value '(localhost(4|6|4to6)?|192.168.122.(1|2|3))'

We need to instruct the user to do this:

https://github.com/theforeman/foreman-documentation/pull/872

Thank you! I think this helps and should be adapted in the documentation.

Can this setting be set via foreman-installer?
Because otherwise it will be deleted after each installer-run in my experience if I edit the settings.yml directly.

I don’t see this in our documentation, @ekohl ?

Are you sure that it is a space-separated list?

:trusted_proxies: "127.0.0.1/8 ::1"

With this option in the settings.yaml the foreman and dynflow services cannot start anymore, while

:trusted_proxies: 
  - "127.0.0.1/8" 
  - "::1"

seems to work. When adding additional IPs of the proxies, I can also curl the user-data-template again.

1 Like

Thanks, I’ve updated the docs!

1 Like

PR for foreman-installer: https://github.com/theforeman/puppet-foreman/pull/1011