Integrating Foreman 1.23 and pfSense 2.4.x for VMWare hosts provisioning

Support
,
1

I’m trying to build automated, image based VM provisioning in virtualized network environment built around (virtual) pfSense firewall appliance. I would like to integrate Foreman with DHCP (isc_dhcpd) and DNS services (bind) provided by pfSense appliance. At the moment, image provisioning fails at the finish template phase because Foreman fails to SSH to a provisioned VM’s. The foreman-proxy service is integrated with pfSense DHCP server via OMAPI and SSHFS file system:

/var/dhcpd/etc/dhcpd.conf (on pfSense firewall)

omapi-port 7911;
key omapi_key {
algorithm HMAC-MD5;
secret "secret";
};

# ls -la /mnt/pfsense_dhcpd/etc/ (on Foreman server)

total 24
drwxr-xr-x 1 dhcpd _dhcp  512 Dec 13 23:39 .
drwxr-xr-x 1 root  root   512 Dec 12 03:48 ..
-rw-r--r-- 1 dhcpd _dhcp 2675 Dec 13 23:39 dhcpd.conf

# ls -la /mnt/pfsense_dhcpd/var/db/ (on Foreman server)

total 92
drwxr-xr-x 1 dhcpd _dhcp   512 Dec 14 19:39 .
drwxr-xr-x 1 dhcpd _dhcp   512 Nov  5 18:47 ..
-rw-r--r-- 1 dhcpd _dhcp 44493 Dec 14 20:26 dhcpd.leases

The “foreman-proxy” user can read and write from/to DHCPD config files

# sudo -u foreman-proxy grep 7911 /mnt/pfsense_dhcpd/etc/dhcpd.conf
omapi-port 7911;
# sudo -u foreman-proxy echo "#TEST" >> /mnt/pfsense_dhcpd/var/db/dhcpd.leases
# sudo -u foreman-proxy grep "#TEST" /mnt/pfsense_dhcpd/var/db/dhcpd.leases
#TEST

/etc/foreman-proxy/settings.d/dhcp.yml (on Foreman server)

:enabled: true
:use_provider: dhcp_isc
:server: 10.15.0.1

/etc/foreman-proxy/settings.d/dhcp_isc.yml (on Foreman server)

:config: /mnt/pfsense_dhcpd/etc/dhcpd.conf
:leases: /mnt/pfsense_dhcpd/var/db/dhcpd.leases
:key_name:  omapi_key
:key_secret: secret
:omapi_port: 7911

Problem:
My Foreman DHCP integration works BUT up to some extent so far… Foreman provides unassigned IP
for new VM, but no static DHCP mapping gets created on the DHCP server (verified many times). No errors are reported to foreman-proxy log (see below ). Therefore after a new VM starts - it gets a whatever random IP and Foreman fails to SSH to it.

2019-12-14T19:55:25 13a0a078 [I] Started GET /dhcp/10.20.0.0/unused_ip from=10.20.0.10&to=10.20.0.254
2019-12-14T19:55:27 13a0a078 [I] Finished GET /dhcp/10.20.0.0/unused_ip with 200 (2012.16 ms)
2019-12-14T20:01:41 17c2abd5 [I] Started POST /dhcp/10.20.0.0 
2019-12-14T20:01:41 17c2abd5 [I] Finished POST /dhcp/10.20.0.0 with 200 (6.68 ms)

Expected outcome:
A static DHCP mappings gets created

Foreman and Proxy versions:
Katello 3.13 - clean install, not upgrade

Foreman and Proxy plugin versions:
Katello 3.13 - clean install, not upgrade

Distribution and version:
Katello 3.13 - clean install, not upgrade

Other relevant data:
Relevant DHCPD log server lines

Dec 14 20:01:52 inf-pfsense-01.rbx.nc.supersolid.net dhcpd: DHCPREQUEST for 10.20.0.138 from 00:50:56:b7:3f:62 via vmx1.200
Dec 14 20:01:52 inf-pfsense-01.rbx.nc.supersolid.net dhcpd: DHCPREQUEST for 10.20.0.138 from 00:50:56:b7:3f:62 via vmx1.200
Dec 14 20:01:52 inf-pfsense-01.rbx.nc.supersolid.net dhcpd: DHCPACK on 10.20.0.138 to 00:50:56:b7:3f:62 (hostname) via vmx1.200
Dec 14 20:01:52 inf-pfsense-01.rbx.nc.supersolid.net dhcpd: DHCPACK on 10.20.0.138 to 00:50:56:b7:3f:62 (hostname) via vmx1.200
Dec 14 20:01:52 inf-pfsense-01.rbx.nc.supersolid.net dhcpd: reuse_lease: lease age 0 (secs) under 25% threshold, reply with unaltered, existing lease for 10.20.0.138
Dec 14 20:01:52 inf-pfsense-01.rbx.nc.supersolid.net dhcpd: reuse_lease: lease age 0 (secs) under 25% threshold, reply with unaltered, existing lease for 10.20.0.138
Dec 14 20:01:52 inf-pfsense-01.rbx.nc.supersolid.net dhcpd: DHCPDISCOVER from 00:50:56:b7:3f:62 (hostname) via vmx1.200
Dec 14 20:01:52 inf-pfsense-01.rbx.nc.supersolid.net dhcpd: DHCPDISCOVER from 00:50:56:b7:3f:62 (hostname) via vmx1.200
Dec 14 20:01:52 inf-pfsense-01.rbx.nc.supersolid.net dhcpd: DHCPOFFER on 10.20.0.138 to 00:50:56:b7:3f:62 (hostname) via vmx1.200
Dec 14 20:01:52 inf-pfsense-01.rbx.nc.supersolid.net dhcpd: DHCPOFFER on 10.20.0.138 to 00:50:56:b7:3f:62 (hostname) via vmx1.200
Dec 14 20:01:52 inf-pfsense-01.rbx.nc.supersolid.net dhcpd: DHCPREQUEST for 10.20.0.138 (10.20.0.1) from 00:50:56:b7:3f:62 (hostname) via vmx1.200
Dec 14 20:01:52 inf-pfsense-01.rbx.nc.supersolid.net dhcpd: DHCPREQUEST for 10.20.0.138 (10.20.0.1) from 00:50:56:b7:3f:62 (hostname) via vmx1.200
Dec 14 20:01:52 inf-pfsense-01.rbx.nc.supersolid.net dhcpd: DHCPACK on 10.20.0.138 to 00:50:56:b7:3f:62 (hostname) via vmx1.200
Dec 14 20:01:52 inf-pfsense-01.rbx.nc.supersolid.net dhcpd: DHCPACK on 10.20.0.138 to 00:50:56:b7:3f:62 (hostname) via vmx1.200
Dec 14 20:05:45 inf-pfsense-01.rbx.nc.supersolid.net dhcpd: DHCPREQUEST for 10.20.0.138 from 00:50:56:b7:3f:62 (hostname) via vmx1.200
Dec 14 20:05:45 inf-pfsense-01.rbx.nc.supersolid.net dhcpd: DHCPREQUEST for 10.20.0.138 from 00:50:56:b7:3f:62 (hostname) via vmx1.200
Dec 14 20:05:45 inf-pfsense-01.rbx.nc.supersolid.net dhcpd: DHCPACK on 10.20.0.138 to 00:50:56:b7:3f:62 (hostname) via vmx1.200
Dec 14 20:05:45 inf-pfsense-01.rbx.nc.supersolid.net dhcpd: DHCPACK on 10.20.0.138 to 00:50:56:b7:3f:62 (hostname) via vmx1.200
2

DHCPD version
[2.4.4-RELEASE][root@pfsense]/root: dhcpd --version
isc-dhcpd-4.3.6-P1

3

Pretty wild guess, but could pfSense somehow verify and prohibit creation of static mappings inside of the main DHCP pool as described here - https://docs.netgate.com/pfsense/en/latest/dhcp/static-mappings-inside-dhcp-pools.html ? The Foreman Proxy uses OMAPI - I have no idea how pfSense could enforce that… Since I can write directly into DHCPD leases file - Is there any way I could use a script to add a static mapping and workaround the issue?

4

Cheers, it’s good to hear back from you Capitan!

First, Foreman supports remote DHCP but since it must watch and read leases file using inotify, this will not work on NFS. I assume you export these via NFS or similar thing, then you must use “remote-isc” plugin which ships with Foreman. It does a different (slower) way of polling the leases file for changes. We don’t have any documentation about this for Foreman, however a good resource is Red Hat Satellite Installation guide for Capsule, chapter External DHCP:

But it looks like you are having issue with OMAPI updates not getting propagated. I don’t know anything about pfSense, however you must enable debug level on your smart proxy and see what exactly it replies. The whole ISC OMAPI communication is written into debug log, there will be probably something wrong. My wild bet - keys are not set properly. In that case follow the guide I’ve linked for proper key setup. Good luck!

1 Like
6

Thanks @lzap - debugging Foreman proxy was my next step!
pfSense is an extremely popular OpenSource WebGUI firewall based on FreeBSD - www.pfsense.org
it has ISC DHCPD installed by default and users can add BIND from pfSense packages.

I will let you know.

image
image779×391 25.4 KB
image
image761×526 36.9 KB