Thank you, Ondrej. Works as promised.
But now I have a bunch of hosts that need remediation. (I used the PCI policy.)
The reports provide Remediation Shell Scripts.
I am assuming that there is a way to have them run at scan time? Auto-run?
I’m sure a lot of this could be done via puppet, but that seems like a lot of work.
Perhaps someone has already done all of the heavy lifting. No need to reinvent that wheel …
Is there OpenSCAP support for Debian 9?
Shhhh . . . Don’t tell anyone, but this actually becoming fun.
I downloaded the scap-workbench on my desktop and created a Tailoring File.
Once I figured out that I needed to run the puppet agent again to push out the new SCAP content,
the scan produced the customized report. This might not sound like much progress,
but the pieces are beginning to fit together, and it’s making more sense. The fog is lifting . . .
I was pondering a moment ago the idea of “online workshop session(s)” ? No recording. No specific time restriction.
In addition, advising the component to be addressed, say OpenSCAP, a week or two before the session as to allow all to digest the available documentation and try it out first.
Only time zones being the biggest hurdle. Which has me contemplate weekends … 
I’m in the Pacific timezone (Los Angeles), but I have been known to keep odd hours …
We both currently got sun, only I’m one day ahead of ya ! 
What’s the weather like tomorrow?
32C sunny skies
Ahh, summer down under.
Seriously, I would be up for some online workshops, as long as we have someone who can actually answer questions. I know both of us will be on the asking side.
Until then, would the current best practice be a DHCPv4 / TFTP smart-proxy on the remote subnet as to provide the new host with a private IPv4 address, although being provisioned dual-stacked.
With the A & AAAA records being recorded in DNS the remote TFTP then provides iPXE to the new host.
iPXE references an HTTPS URL resolving to IPv6 to acquire the image to boot.
In essence, iPXE is not transferred over the public net and though the image is transported over the public net, it is via encrypted IPv6.
How many potholes do you foresee ?
Regarding remediation, we do not have that automated yet. Probably the easiest way to fix the failing rules is to take the remediation scripts and use remote execution plugin to run it on the target hosts. I would like to have much better solution in the future.
According to openscap docs, the scanner can be installed on Debian, so we would need to package foreman_scap_client for Debian and test it works as expected.
Oh many, I haven’t tried this myself, I have never seen this in action too.
Honestly, extending dnsmasq DHCP providers for IPv6 should be possible if you are willing to swap ISC DHCP with more modern dnsmasq:
But core API needs to be changed. It’s a lot of work.
First hurdle being production implementation. Thereafter I’d be able to through more resources at it. As you’re surely aware, IPv6 is high on my list.
In /etc/foreman-proxy/settings.yml
:http_port: 8000
Changing this port to 80 breaks the template service.
Yet, provisioning templates reference “foreman_url”, i.e:
url=<%= foreman_url('provision')%>
which expands to
url=http://foreman.domain.com:8000/unattended/provision?token=abcd1234-d1d2-d3d4-d5d6-abcdef123456
or
<%= foreman_url('iPXE') %>
expanding to
http://foreman.domain.com:8000/unattended/iPXE?token=abcd1234-d1d2-d3d4-d5d6-abcdef123456
or
<%= foreman_url('built') %>
expanding to
http://foreman.domain.com:8000/unattended/built?token=abcd1234-d1d2-d3d4-d5d6-abcdef123456
though these URL’s never resolve correctly, unless “:8000” is replaced with either nothing, “:80” (HTTP) or “:443” (HTTPS)
Under Administer > Settings > General Tab
"Foreman URL" is set to "http://foreman.domain.com"
Under Administer > Settings > Provioning Tab
"Unattended URL" is set to "http://foreman.domain.com"
The question being, where is the :8000 defined?
We see great activity in this thread and we are thinking introducing new section on Discourse called Tutorials where we could all share various howtos. If you like, give like: New forum category proposal: Tutorials
I had hoped that I’d by now be thanking @jmrice6640, @mason, @lzap, @Gwmngilfen, @Ondrej_Prazak, @ekohl, @tbrisker and all others involved in assisting with “Getting Started” …
Unfortunately, I have to report that I still cannot get my first auto-provisioned host working. It immediately comes to mind that someone still battling most probably does not understands DHCP options, TFTP (alas PXE Boot) and UEFI vs BIOS. For surely it’s only a few config changes to be on one’s merry way.
Do note that the main idea or goal hasn’t changed. A very basic uncluttered implementation by only installing / configuring the bare necessities as to have Foreman auto provision and config manage a physical and virtual host. A continuation on the Foreman configuration from [post:21, topic:12684].
Soooo, once again, I’ll herewith present my ongoing frustrations welcoming any comments, advise or corrections from anyone whom may still harbor any empathy.
The preamble: PXE Boot:
Host boots “network option”
DHCP server then provides:
IPv4 IPv6
1. IP address - optional (with SLAAC)
2. Gateway address - n/a
3. Name servers - advertised via radv
4. TFTP server - BootFile_URL
DHCPv4 option 66 - DHCPv6 option 59
IP address only - protocol + FQDN (or IP) + filename i.e.
http://foreman.domain.com/settings.yaml or tftp://foreman.domain.com/settings.yaml
5. Filename to boot - BootFile_Parameters
DHCPv4 option 67 - DHCPv6 option 60
filename only -
However, anything IPv6 related, be it PXE boot or UEFI HTTP boot is somehow broken somewhere in its implementations. iPXE (http://ipxe.org) to the rescue, being the only implementation worth the time and effort, as it simply works and also provides numerous alternate recompile options. (including IPv6)
Another sidenote being UEFI. It’s integration and features has me dumbfounded as to why anyone would still be flogging the legacy BIOS of a bygone era. This is also clear in the latest Intel hardware that does not support legacy BIOS any more, though intentionally cripple UEFI features…
As such, a host was auto-installed by providing iPXE (UEFI) to the host via PXE with either an embedded script pointing to a webserver or by utilising DHCP class options as to specify the config file and prevent DHCP boot loops. This basic logic transpires in the Foreman’s Provisioning Templates documentation where 3 “template kinds” are by default of interest, though should be 4 “template kinds” i.m.h.o, being:
1. PXE - Boot loader
Deployed to the TFTP server under /var/lib/tftpboot/ for
a. pxelinux.efi <- PXELinux
b. grub/ <- PXEGrub <- N/A - Legacy
c. grub2/grubx64.efi <- PXEGrub2
2. iPXE - iPXE config to boot
.... a key issue at present
3. Provision Template – The “answers” file
Preseed (for Debian) or Kickstart (for CentOS) deployed to the TFTP server under /var/lib/tftpboot/ (per machine)
a. pxelinux.cfg/{MAC}
or pxelinux,cfg/{MAC}.ipxe
b. grub/{MAC}
or grub/menu.list.{MAC}
c. grub2/grub.cfg-{MAC}
4. Finish – Post-install script
Also to advise Foreman on completion of the installation
a.
Foreman-installer only provides pxelinux.0 and grub2/grubx64.efi. pxelinux.0 being BIOS orientated and thus unfit for this use case. Other bootloaders such as iPXE.efi needs to be manually copied to the TFTP’s root folder (/var/lib/tftpboot) and renamed accordingly. An Operating System’s “PXE Loader” options being:
None - no filename passed (e.g. for HTTP booting via iPXE)
PXELinux BIOS - pxelinux.0
PXELinux UEFI - pxelinux.efi
Grub UEFI - grub/bootx64.efi
Grub2 UEFI - grub2/grubx64.efi
Grub2 UEFI SecureBoot - grub2/shim.efi
Grub2 UEFI HTTP - ? not documented
Grub2 UEFI HTTPS - ? not documented
Grub2 UEFI HTTPS SecureBoot - ? not documented
iPXE Embedded - ? not documented
iPXE UEFI HTTP - ? not documented
iPXE Chain BIOS - ? not documented
iPXE Chain UEFI - ? not documented
“PXELinux UEFI” thus being the NR 1 choice going forward. However, is “None” still relevant taking “iPXE UEFI HTTP” into account ?
With all the above knowledge, one would expect the road forward to be paved.
1. Download ipxe.efi from http://boot.ipxe.org/ipxe.efi into /var/lib/tftpboot
2. Copy or rename ipxe.efi to /var/lib/tftpboot/pxelinux.efi
3. Ensure pxelinux.efi ownership set to foreman-proxy:root and permissions being 644
4. OS in Foreman set to:
a. Provisioning template - Preseed Default
b. Finish template - Preseed default finish
c. PXELinux template - PXELinux chain iPXE
d. PXEGrub2 template - Preseed default PXEGrub2
e. iPXE template - Preseed default iPXE
5. Hosts’ “provisioning templates” resolves in Foreman to
a. Media - Debian Mirror
b. Partition Table - Preseed default
c. PXE Loader - PXELinux UEFI <-
---------------------------------------------------------------
d. PXEGrub2 template - Preseed default PXEGrub2
e. PXELinux template - PXELinux chain iPXE
f. iPXE template - Preseed default iPXE
g. Provisioning template - Preseed default
h. Finish template - Preseed default finish
Previewing the provisioning templates it’s clear the “foreman_url” expands to include port 8000 tacked on (i.e. http://foreman.domain.com:8000 ). Yet, under Administer > Settings > Provisioning the “Unattended URL” is specified without :8000. Neither is there any other URL in the GUI Settings page that reference port 8000.
Re-running the foreman-installer with “–foreman-proxy-template-url=https://foreman.domain.com” fixes all the provisioning templates by expanding “foreman_url” without port 8000 postfixed.
WARNING !!! Re-running the foreman-installer RESETS OTHER system configs NOT being specified. One specific example being DNS IPv6 reverse lookup zones being removed from /etc/named/zones.conf, though the zone files at /var/named/dynamic remains untouched.
Yet, once the templates plugin is installed, port 8000 then reflects as an open http-alt port on the foreman host… Is the templates plugin a dependency for provisioning ? Noted that at this point, I had not taken notice of “templates plugin” being a possible requirement. Is it acceptable to leave the –foreman-proxy-template-url as https://foreman.domain.com or should be it reverted back to https://foreman.domain.com:8000 to prevent other issues?
With port 8000 specified, encryption is not supported and there’s a difference in formatting:
http://foreman.domain.com/unattended/iPXE being rendered:
#!ipxe
set menu-default local
set menu-timeout 5000
:start
menu iPXE global boot menu
item --key l local Continue local boot
item shell Drop into iPXE shell
item reboot Reboot system
item
item --key d discovery Foreman Discovery
choose --timeout ${menu-timeout} --default ${menu-default} selected || goto cancel
set menu-timeout 0
goto ${selected}
:cancel
echo Menu canceled, dropping to shell
:shell
echo Use the command 'exit' to return to menu
shell
set menu-timeout 0
goto start
:failed
echo Boot failed, dropping to shell
goto shell
:reboot
reboot
:local
exit
:discovery
dhcp
kernel ${next-server}/boot/fdi-image/vmlinuz0 rootflags=loop root=live:/fdi.iso rootfstype=auto ro
rd.live.image acpi=force rd.luks=0 rd.md=0 rd.dm=0 rd.lvm=0 rd.bootif=0 rd.neednet=0 nomodeset
proxy.url=https://foreman.domain.com proxy.type=foreman BOOTIF=01-${net0/mac}
initrd ${next-server}/boot/fdi-image/initrd0.img
boot || goto failed
goto start
whereas http://foreman.domain.com:8000/unattended/iPXE results in:
#!ipxe set menu-default local set menu-timeout 5000 :start menu iPXE global boot menu item --key l local Continue local boot item shell Drop into iPXE shell item reboot Reboot system item item --key d discovery Foreman Discovery choose --timeout ${menu-timeout} --default ${menu-default} selected || goto cancel set menu-timeout 0 goto ${selected} :cancel echo Menu canceled, dropping to shell :shell echo Use the command 'exit' to return to menu shell set menu-timeout 0 goto start :failed echo Boot failed, dropping to shell goto shell :reboot reboot :local exit :discovery dhcp kernel ${next-server}/boot/fdi-image/vmlinuz0 rootflags=loop root=live:/fdi.iso rootfstype=auto ro rd.live.image acpi=force rd.luks=0 rd.md=0 rd.dm=0 rd.lvm=0 rd.bootif=0 rd.neednet=0 nomodeset proxy.url=https://foreman.domain.com proxy.type=foreman BOOTIF=01-${net0/mac} initrd ${next-server}/boot/fdi-image/initrd0.img boot || goto failed goto start
Though the “Using iPXE in Foreman” wiki topic sounds on par with this case, it unfortunately focuses only on BIOS implementations. Once updated to address UEFI, it should be integrated with an additional section clarifying the “PXE loader” options and bootloader files. i.e pxelinux.0 vs ipxe.efi vs ipxe.lkrn vs undionly-ipxe.0 in the manual.
This also raises the questions whether the “discovery and BIOS” limitation is still valid taking the following snippet from http://foreman.domain.com/unattended/iPXE into account:
:discovery
dhcp
kernel ${next-server}/boot/fdi-image/vmlinuz0 rootflags=loop root=live:/fdi.iso rootfstype=auto ro rd.live.image acpi=force rd.luks=0 rd.md=0 rd.dm=0 rd.lvm=0 rd.bootif=0 rd.neednet=0 nomodeset proxy.url=https://foreman.domain.com proxy.type=foreman BOOTIF=01-${net0/mac}
initrd ${next-server}/boot/fdi-image/initrd0.img
boot || goto failed
goto start
… or is this related to a BootDisk plugin requirement ?
At least with URLs in provisioning templates now resolving, a “Cancel build” followed by “Build” is done to ensure each machine’s config is rebuild to reflect correctly.
Starting the virtual host, it loads iPXE, yet goes into a boot loop. Breaking to the iPXE command line and executing “chain http://foreman.domain.com/unattended/iPXE” starts downloading the debian images from the debian mirror, though fails with
Failed to open file: initrd.img
Trying to load files to higher address
Failed to open file: initrd.img
Starting the physical host, it loads iPXE, yet also goes into a boot loop, yet one cannot break into the iPXE command line …
Reviewing /etc/dhcp/dhcpd.conf as to address the iPXE boot loop reflects the bootfile handoff commented as follows:
# Bootfile Handoff
next-server 10.0.0.2;
option architecture code 93 = unsigned integer 16 ;
if option architecture = 00:06 {
filename "grub2/shim.efi";
} elsif option architecture = 00:07 {
filename "grub2/shim.efi";
} elsif option architecture = 00:09 {
filename "grub2/shim.efi";
} else {
filename "pxelinux.0";
}
A quick change as follows does not address the DHCP boot loop isssue:
option architecture code 93 = unsigned integer 16;
if exists user-class and option user-class = "iPXE"; {
filename "http://foreman.domain.com/unattended/iPXE";
} elsif option architecture = 00:06 {
filename "grub2/shim.efi";
} elsif option architecture = 00:07 {
filename "grub2/shim.efi";
} elsif option architecture = 00:09 {
filename "grub2/shim.efi";
} else {
filename "pxelinux.0";
}
Notice only shim.efi and pxelinux.0 being referenced while “pxelinux.efi” isn’t specified at all in /etc/dhcp/dhcpd.conf (nor /etc/dhcp/dhcpd6.conf), yet the hardware successfully boots /var/lib/tftpboot/pxelinux.efi …
A bit of digging reveals “next-server”, “filename” and “hostname” written to /var/lib/dhcpd/dhcpd.leases as “superceded”. This being a new revelation. Question being, what is the correct approach now to break the iPXE boot loop?
Reviewing the PXELinux template - PXELinux chain iPXE – which expands in both scenarios to
DEFAULT linux
LABEL linux
KERNEL ipxe.lkrn
APPEND dhcp && chain http://foreman.domain.com/unattended/iPXE?token= 12345678-1234-1234-1234-123456789012
IPAPPEND 2
How did “KERNEL ipxe.lkrn” make it’s way into an iPXE.efi (UEFI) only configuration?
The iPXE template – Preseed default iPXE – expanding in both scenarios to
#!gpxe
kernel http://ftp.debian.org/debian/dists/stable/main/installer-amd64/current/images/netboot/debian-installer/amd64/linux initrd=initrd.img interface=auto url=http://foreman.domain.com/unattended/provision?token=12345678-1234-1234-1234-123456789012&static=yes ramdisk_size=10800 root=/dev/rd/0 rw auto netcfg/disable_dhcp=true BOOTIF=01-${netX/mac:hexhyp} hostname=test-01.domain.com auto=true domain=domain.com locale=en_US netcfg/get_ipaddress=${netX/ip} netcfg/get_netmask=${netX/netmask} netcfg/get_gateway=${netX/gateway} netcfg/get_nameservers=${dns} netcfg/confirm_static=true
initrd http://ftp.debian.org/debian/dists/stable/main/installer-amd64/current/images/netboot/debian-installer/amd64/initrd.gz
boot
Early days had shown Grub2 as the most eager to wanna work. Be it easiest to get going. Thus, changing only the “PXE loader” to “Grub2 UEFI” presents “grub2/grubx64.efi” as bootloader with “/grub2/grub.cfg-00-00-12-34-56-78-90” as the config to load. Once the Grub menu executes the single Grub menu entry labled “Preseed default PXEGrub2” on time-out, the debian-installer only terminates three quarters through the host’s installation with:
“Base system installation error
The debootstrap program exited with an error (return value 1).
Check /var/log/syslog or see virtual console 4 for the details.”
/var/log.syslog stating:
…
debootstrap: /usr/sbin/debootstrap –components=main –debian-installer –resolve-deps –keyring=/usr/share/keyrings/archive.gpg stable /target http://ftp.debian.org:80/debian
debootstrap: E: NOSCRIPT
debootstrap: EA: /usr/share/debootstrap/scripts/stable
debootstrap: EF: No such script: %s
Close, though still no sigar. Though Grub2’s implementation is easy to follow. The key files being the boot loader “grubx64.efi” located with a specific host’s config file “grub.cfg-00-00-12-34-56-78-90” under /var/lib/tftpboot. Both provided via DHCP (via dhcpd.leases) & TFTP aka PXE (UEFI) Booting.
Question being, how does this differ for iPXE with HTTP(S) ? … and this is where the headache starts …
“iPXE.efi” renamed to “pxelinux.efi” and saved under /var/lib/tftpboot. The config file being one directory deeper at /var/lib/tftpboot/pxelinux.cfg/00-00-12-34-56-78-90.ipxe. Yet once iPXE.efi loads, the associated config within 00-00-12-34-56-78-90.ipxe isn’t presented to allow iPXE to source the image from a URL.
Also, in the “PXELinux chain iPXE” template, does http://foreman.domain.com/unattended/ iPXE?token= 12345678-1234-1234-1234-12345678901 reflect to the contents of /var/lib/tftpboot/pxelinux.cfg/00-00-12-34-56-78-90.ipxe ? Where is this mapped? And if it is, is it not expected to rather be residing under /var/www/html/pub especially as TFTP could not be required…
If you feel exhausted reading this… I’m mentally drained, without any UEFI host having been iPXE provisioned as yet.
** Templates are being used stock standard as to reflect an OOB experience.
** As newcomers start with the bare minimum, plugin dependencies based on use cases or scenarios would be a great help allowing newcomers to FIRST get acquainted with those required plugins, technologies or methodologies, before trying to make sense of it through “debugging sessions” similar to this rant.
Beware, iPXE will only work on hardware that has an iPXE driver shipped. Not all hardware has this privilege. It can bite hard.
Anyway, by coincidence I am currently testing a custom hardware for a customer and they requested exactly what you are trying to achieve here. Let me tell you something: iPXE EFI provisioning works, I have just tested this for two days. I have to say this is fairly new code in Foreman and thus lots of this is undocumented.
The good news? I have all my notes and I am planning to do a full tutorial on how to set things up. You are very close I have to say, good work. Let me see if I can put together the content today or tomorrow. I will publish this on Discourse.
So stay tuned, I will link it ASAP.
Here it is:
Sorry @peek, that’s a bit above my pay grade at the moment.
After feeling triumphant about getting SCAP reports working, I took a step back and spent time with Puppet.
When one needs to learn everything, you have to start somewhere … Puppet Learning VM
My long list of potential uses for foreman/puppet and friends will likely include a control repository,
development cycle workflows, configuration management, and yes, host provisioning.
But I’ll have to come back to that one. By then, you experts will have blazed that trail.
This “Getting Started” thread has come a long way from whence it began.
When I joined in a month ago, I was completely overwhelmed, and had no idea where to start.
Reading back over it now, that same sense of befuddlement creeps back in again.
Thinking in terms of baby steps, an overview, some simple diagrams, and a guide.
These should have been presented long before the Installation Guide, IMHO.
OK, I have foreman/katello/puppet installed.
And now my question is …
What’s next?
Plugins? Is there a recommended set? A logical sequence?
I know, the customary response is, “That depends …”
This is where experience and wisdom could benefit us “Getting Started” folks the most.
It’s been a few days. This thread was cooling off. Thought I would share some progress…
I found a puppet module on GitHub for lucee.
But it was a little stale (2 years old), and installed nginx and apache as originally configured.
I modified it to do an unattended lucee installation, (version 5.2.9), without apache and nginx.
It uses a custom options file, a non-root run user, and also installs a MySQL client jar (RabbitMQ).
It modifies the firewall rules and starts the lucee service. Then I configured nginx to load balance
between two of these lucee VM’s.
This is only to say that I am finally beginning to understand some of the pieces and how they interact.
I still haven’t been able to explore how to provision the new VM’s directly, but I am using ESXi, not vSphere, and according to 5.2.9 VMware Notes, Only VMware clusters using vSphere are supported, not standalone ESX or ESXi servers (#1945).
So, that will have to wait for now.