Foreman and SP on the same host with dual network attachement (one private)

We don’t have documented unfortunately, depends on services you plan to use. I can tell that for TFTP and DHCP you simply configure the server with hostname set to the interface which is Foreman-facing so installer generates HTTPS certificate for Foreman-Proxy communication and you give TFTP and DHCP services inteface which is client (server) facing.

But we have services which are talking to smart proxy hostname directly, for example template endpoint (provision template download). Foreman renders URL which you want to download from as the Foreman-facing hostname which will not work (resolves to wrong IP address). There are various workarounds, the best is I think changing the template and using host/hostgroup/subnet parameter to override the hostname. @sean797 ?

Since Foreman is not aware of multi-home setups, you are on your own here and you will run into issues you need to resolve. But it’s been done in the past, we can help you when you hit those. I recommend to do POC installation in clean network without multi-homing and learn about how Foreman works first.

All client communication is not via proxy by default, but you can configure this. If you had single NIC on proxy, installer can easily set you things up. But in case of multi-nic this is more complex problem. I am familiar with PXE provisioning, so in that case TFTP, DHCP, DNS and kickstart request can be all completely proxy only. This is same for Puppet - master runs on Proxy itself. @ekohl do we have working installer setups for multi-nic proxies?

To force clients to download kickstarts from proxy instead of foreman, you need to install module called “smart proxy template”. Here is a snippet I use to install PXE setup on my single-nic environment:

 foreman-installer -v --scenario $SCENARIO \
  --foreman-admin-password=$PASS \
  --foreman-organizations-enabled true \
  --foreman-initial-organization=$ORG \
  --foreman-locations-enabled true \
  --foreman-initial-location=$LOC \
  --enable-foreman-plugin-discovery \
  --enable-foreman-plugin-bootdisk \
  --enable-foreman-plugin-remote-execution --enable-foreman-proxy-plugin-remote-execution-ssh \
  --enable-foreman-plugin-ansible --enable-foreman-proxy-plugin-ansible \
  --foreman-proxy-http=true \
  --foreman-proxy-dns true \
  --foreman-proxy-dns-interface eth0 \
  --foreman-proxy-dns-forwarders 192.168.${NATLAN}.1 \
  --foreman-proxy-dns-zone nat.lan \
  --foreman-proxy-dns-reverse ${NATLAN}.168.192.in-addr.arpa \
  --foreman-proxy-dhcp true \
  --foreman-proxy-dhcp-interface eth0 \
  --foreman-proxy-dhcp-gateway=192.168.${NATLAN}.1 \
  --foreman-proxy-dhcp-range="192.168.${NATLAN}.10 192.168.${NATLAN}.109" \
  --foreman-proxy-dhcp-nameservers="192.168.${NATLAN}.${IP}" \
  --foreman-proxy-tftp true \
  --foreman-proxy-tftp-servername=192.168.${NATLAN}.${IP} \
  --foreman-proxy-puppet true \
  --foreman-proxy-puppetca true \
  --foreman-proxy-templates true \
  --foreman-proxy-logs true \
  --foreman-proxy-register-in-foreman true

The option you are looking for probably is “–foreman-proxy-templates true”.

I admit this is confusing and @ekohl correct me if I am wrong, but everytime you call foreman-installer command it adds new options to YAML and reruns puppet. So basically it “remembers” already overriden options.

Hmm question or kafo devs (foreman-installer framework), maybe @mbacovsky or @ekohl? I have no idea. If I mess up, I dig in shell history and “correct” all options by providing them again.

Not puppet expert, not sure why you don’t see dhcpd.config file. Define “to see” verb, like the file literally disappears?

Foreman is a beast! Really, it has hundreds of features, plugins, hidden features and (few) bugs :slight_smile: Absolutely do start with simple use case and learn.

Ok thanks for your time.
I’d try to mix all these ideas.


Thomas.

We’ve run into a similar issue before as we have a similar setup. The solution we arrived on was leaving foreman and the proxy configured entirely with it’s public facing name/domain, and allowing it to listen/act on the private interface. Two options for allowing foreman to function in this configuration:

  1. Use bind views to present the private IP address of the foreman server against the public FQDN. Not sure if the puppet-dns module (used by the foreman-installer script) is capable of configuring this through the foreman-installer or whether it’ll need to be a custom job. Anything on your private subnet will then be able to route to the private address of your foreman server.
  2. Use local name resolution (/etc/hosts) on the private nodes to resolve the public hostname/domain to the private address of the foreman server

Incrementally, the project is improving in this space but it’s not perfect yet…

Hello again,

I started my setup all over again. I’m still installing/running on a single host with 2 nics but I switched their order and set up the hostname, so as almost eveything resolves/points to the private subnet (the one I want to install clients on). It looks like this :

  • eth0 : private IP
  • eth1 : public IP
  • hostname : private hostname (foreman.dev.cluster.pasteur.fr) set via hostnamectl set-hostname --static (CentOS 7)
  • /etc/sysconfig/network : no hostname
  • /etc/sysconfig/network-scripts/ifcfg-eth0 : DOMAIN=“dev.cluster.pasteur.fr
  • /etc/sysconfig/network-scripts/ifcfg-eth1 : DOMAIN=“it.pasteur.fr

And I installed with :
# foreman-installer
–foreman-proxy-dhcp=true
–foreman-proxy-dhcp-interface=eth0
–foreman-proxy-dhcp-managed=true
–foreman-proxy-dhcp-subnets=“192.168.10.0/24”
–foreman-proxy-dhcp-gateway=“192.168.10.10”
–foreman-proxy-tftp=true
–foreman-proxy-tftp-managed=true
–foreman-proxy-tftp-servername=“192.168.10.10”
–foreman-proxy-dns=true
–foreman-proxy-dns-forwarders=“157.99.64.64”
–foreman-proxy-dns-managed=true
–foreman-proxy-dns-zone=dev.cluster.pasteur.fr
–foreman-proxy-bmc=true
–enable-foreman-proxy-plugin-ansible
–enable-foreman-plugin-ansible
–foreman-proxy-dns-reverse=10.168.192.in-addr.arpa

I created in the UI the needed objects to install a client. All in the same private domain or network.
So basically nothing foreman related states the public name or ip.

Thus I, as before, am unable to fetch the kickstart file after the PXE step.
As a matter of fact, I see this line in the named zone file :

$ORIGIN dev.cluster.pasteur.fr.
foreman                 A       157.99.101.21
$TTL 86400      ; 1 day
test-409                A       192.168.10.200

-> this seems to be the problem. Do you know where the public ip for foreman comes from here (instead of the private ip) ?

I think it could work if I put in the UI subnet form the foreman private ip as the gateway but the correct way to proceed seems to register the good IP in the zone file. Do you have any idea of how to do that ?

Thanks


TH

Two things to check:

  1. The unattended_url setting in Foreman (probably not the issue, but worth checking)
  2. The “tftp_servername” setting on the proxy (I think it’s in the dhcp.yml plugin config)

(2) is usually the issue for PXE problems. Try setting it to the IP you want to use for the TFTP server. If you change it, you’ll need to rewrite the lease for the test host, use the Rebuild Config option in the Foreman UI.

Thanks for your reply

This one is correct.

I explicitly passed this on the command line to the private ip I was interested in.

Could the reason be that the public nic is the only one to have a default gateway ?

It’s possible. I’ve re-read the post, and realised I misunderstood - I thought PXE was failing, but you’re getting past that.

If you can boot an installer, can you get a shell on it? I know the Debian installer has a few virtual terminals you can switch to, but I’m less familiar with the Anaconda ones. If you can, it might be worth checking the network config live, and seeing where the issue is.

17:58:37.552276 ARP, Reply 192.168.10.200 is-at 0c:c4:7a:1e:8a:5d (oui Unknown), length 46
17:58:37.555320 ARP, Request who-has foreman.dev.cluster.pasteur.fr tell 192.168.10.200, length 46
17:58:37.555340 ARP, Reply foreman.dev.cluster.pasteur.fr is-at 00:50:56:8a:8b:09 (oui Unknown), length 28

17:58:37.555526 IP 192.168.10.200.44340 > foreman.dev.cluster.pasteur.fr.domain: 30597+ A? foreman.dev.cluster.pasteur.fr. (48)
17:58:37.555547 IP 192.168.10.200.44340 > foreman.dev.cluster.pasteur.fr.domain: 61248+ AAAA? foreman.dev.cluster.pasteur.fr. (48)
17:58:37.558082 IP foreman.dev.cluster.pasteur.fr.domain > 192.168.10.200.44340: 61248* 0/1/0 (89)
17:58:37.558101 IP foreman.dev.cluster.pasteur.fr.domain > 192.168.10.200.44340: 30597* 1/1/0 A 157.99.101.21 (78)

This tcpdump snippet shows to me that the client gets the right dns address but the wrong answer ? Which matches what’s in the zone db file of named ?

Sorry, it was my mistake : for some reason, in the answers.yml file was a foreman-proxy-dns-interface set to the wrong interface…
It echoes however my question above about how to completely reset a foreman config and if puppet just adds/change features provided on the installer of the command line…

Thanks anyway…

I’m almost there :

  • PXE works
  • kickstart file download works

given that I provide the private non-routed ip/name of foreman in /etc/hosts.

For the same reason (I’m installing clients on a private non-routed subnet) I also add a nic in that subnet on my media host (vmware http CentOS mirror) and the corresponding entry in /etc/hosts.

but now anaconda wants to access this host : can I add this host entry in the foreman managed dns via the UI or do I have to externally (nsupdate for instance) add it to the dynamic zone ?

Thanks


TH
but now

For this one : I’m not really sure why it’s needed : as a matter of fact, the next-server in the generated dhcpd.conf file is an IP address, not a name.
However, I notice that if I don’t provide this command line option, the “Services” tab for the Smart Proxies section in the UI shows only tftp “Version” but “false” for “tftp server”.

What am I missing ?

Thanks.


TH

Next server must be an IP this is required by PXE. When this is not set via tftp_servername then Foreman grabs smart proxy hostname and does reverse DNS PTR search for an IP and use that IP for the filename option.

It was irrelevant, I asked for it based on mis-reading the question. It only impacts PXE, which you say is working, so it’s not the problem.

As for your DNS, we don’t really have an interface for adding arbitrary records, AFAIK - only the host records that are automated when creating/updating hosts. So you’ll want to use nsupdate or whatever your favourite tool is for adding this (this irritates me every time I need to add some CNAMES…)

This explains my problem then (tftp works only when I provide this command line option). As a matter of fact, I can see a A record for forman in my forward dns zone but no ptr record in my reverse dns zone : is that normal ?

# grep -i foreman /etc/hosts
192.168.10.10	foreman.dev.cluster.pasteur.fr

# cat db.dev.cluster.pasteur.fr
$ORIGIN .
$TTL 10800	; 3 hours
dev.cluster.pasteur.fr	IN SOA	foreman.dev.cluster.pasteur.fr. root.dev.cluster.pasteur.fr. (
				26         ; serial
				86400      ; refresh (1 day)
				3600       ; retry (1 hour)
				604800     ; expire (1 week)
				3600       ; minimum (1 hour)
				)
			NS	foreman.dev.cluster.pasteur.fr.
$ORIGIN dev.cluster.pasteur.fr.
foreman			A	192.168.10.10
$TTL 600	; 10 minutes
mirrors			A	192.168.10.3

]# cat db.10.168.192.in-addr.arpa 
$ORIGIN .
$TTL 10800	; 3 hours
10.168.192.in-addr.arpa	IN SOA	foreman.dev.cluster.pasteur.fr. root.10.168.192.in-addr.arpa. (
				20         ; serial
				86400      ; refresh (1 day)
				3600       ; retry (1 hour)
				604800     ; expire (1 week)
				3600       ; minimum (1 hour)
				)
			NS	foreman.dev.cluster.pasteur.fr.
$ORIGIN 10.168.192.in-addr.arpa.
$TTL 600	; 10 minutes
3			PTR	mirrors.dev.cluster.pasteur.fr.

That’s how I added mirros in the above zones snippets.

Thanks.

I must add this is strange though since I can however see a correct next-server in my dhcpd.conf file

# Bootfile Handoff
next-server 192.168.10.10;

But again, I’m not really sure if it doesn’t come from a previous foreman-installer run with the --foreman-proxy-tftp-servername option set…Could that be possible ?

If this zone is managed by foreman, then this is possible. You associated Proxy for Domain but not for Subnet. In that case, Foreman manages only forward records. Associate both and you will get PTR as well.

This is probably set by our installer, yes. Plus it also sets the YAML configu for the TFTP module.

I started from a fresh install (I’m playing/experimenting installation scenarii) : I didn’t create any subnet yet in the UI indeed but neither did I create a domain.
Here was my command line

foreman-installer --foreman-proxy-dhcp=true --foreman-proxy-dhcp-interface=eth0 --foreman-proxy-dhcp-managed=true --foreman-proxy-dhcp-subnets="192.168.10.0/24" --foreman-proxy-tftp=true --foreman-proxy-tftp-managed=true --foreman-proxy-dns=true --foreman-proxy-dns-interface=eth0 --foreman-proxy-dns-forwarders="157.99.64.64" --foreman-proxy-dns-managed=true --foreman-proxy-dns-zone=dev.cluster.pasteur.fr --foreman-proxy-bmc=true --enable-foreman-proxy-plugin-ansible --enable-foreman-plugin-ansible --foreman-proxy-dns-reverse=10.168.192.in-addr.arpa

Our installer does not set A/PTR record for foreman itself, Foreman server should be in unmanaged zone. We assume your DNS record of Foreman FQDN is correctly set before the installation even starts. We don’t check it tho.

It’s strange, then since, from a fresh start, where I don’t even have named installed, I get the A record :

# hostname
foreman.dev.cluster.pasteur.fr

# ls -l /var/named
ls: cannot access /var/named: No such file or directory

#  foreman-installer --foreman-proxy-dhcp=true --foreman-proxy-dhcp-interface=eth0 --foreman-proxy-dhcp-managed=true --foreman-proxy-dhcp-subnets="192.168.10.0/24" --foreman-proxy-tftp=true --foreman-proxy-tftp-managed=true --foreman-proxy-dns=true --foreman-proxy-dns-interface=eth0 --foreman-proxy-dns-forwarders="157.99.64.64" --foreman-proxy-dns-managed=true --foreman-proxy-dns-zone=dev.cluster.pasteur.fr --foreman-proxy-bmc=true --enable-foreman-proxy-plugin-ansible --enable-foreman-plugin-ansible --foreman-proxy-dns-reverse=10.168.192.in-addr.arpa
[root@foreman hummel]# grep -i foreman /var/named/dynamic/db.dev.cluster.pasteur.fr 
@ IN SOA foreman.dev.cluster.pasteur.fr. root.dev.cluster.pasteur.fr. (
@ IN NS foreman.dev.cluster.pasteur.fr.
foreman.dev.cluster.pasteur.fr. IN A 192.168.10.10


But no PTR :

@ IN SOA foreman.dev.cluster.pasteur.fr. root.10.168.192.in-addr.arpa. (
@ IN NS foreman.dev.cluster.pasteur.fr.

Maybe I am just wrong, @ekohl ?