I recently posted in foreman-users about a feature I'd like to see in
foreman_discovery. I talked a bit with lzap about the idea and have made
one pass at building a custom foreman_discovery image.
Here's summary of the current workflow I use / the problem:
- New servers arrive.
- Rack servers
- If the vendor provided me MAC addresses then I immediately add to
Foreman with primary PXE interface and BMC - Skip to step #5 - Get onboard MAC addresses either via BIOS or IPMI web interface
- Configure BIOS and set BMC static IP
- Provision system using Foreman.
- Infiniband Puppet module will configure Infiniband without an IP.
7a. Once rdma stack is loaded and the IB interface is up, with or without
an IP, the subnet manager issues a LID and a MAC address is available - Add infiniband interface to Foreman with newly created MAC address
- Re-run Puppet, applying IPoIB
- Reboot node as restarting all the necessary services that detect an IB
interface based on availability of IPoIB is numerous and faster to just
reboot. - Node back online, re-run Puppet to ensure in good state
- Node ready for production use in cluster
Fortunately we only receive 4-6 nodes every few months, but it seems like
steps 7-10 could be done away with if I had the ability to get the IB
interface MAC address before provisioning. So I looked at Foreman
Discovery, which seems like the perfect solution. I believe that the MAC
address is not coded into the device like standard Ethernet, but rather is
based on the LID issued by subnet manager, which should not change once the
subnet manager issues a LID.
I have not yet tested foreman_discovery Plugin as I've yet to update to
1.6.0, and want to try latest version. Initial attempts at using latest
RPM for the plugin failed with Foreman 1.5.1 because I'm using my patches
to enable 64-bit MAC address (back-ported into internal RPM), which break
variable used to refer to MAC address regex validation. My problem, but
explains why I haven't tested the plugin yet :).
So far I took the ovirt-node-plugin-foreman repo and tried building an
image without any changes. That failed. I was doing this on Ubuntu 12.04
(not my distro of choice by any means, but came with my laptop) and using
Libvirt. The failures I ran into were the following:
- Downgrade vagrant-libvirt plugin to 0.0.20 or would get issues with
invalid network syntax in XML generated for libvirt - Pass empty string to provision shell script "args" if SHELLARGS was empty
or vagrant would complain that "args" must be a string or an Array - Set the synced_folder to use rsync as NFS wasn't working, complained
about not being able to obtain lock. Not sure if the image pulled down by
the Vagrant file lacked some key program or what.
Once I got the vagrant image started I ran into these problems:
- python-lockfile was unable to be installed , I had to add "yum -y install
epel-release" towards the top of build_image.sh - Errors came up during build process that sudo required tty. I am not
sure why sudo was being used , so added "sed -r -i
's/^(Defaults\s+requiretty)/#\1/g' /etc/sudoers" to build_image.sh above
the "# build plugin" line.
I've pushed my changes to my fork [1]. They are by no means seen as
necessary, but it's to help show the issues I ran into and how I fixed them.
So right now I'm hoping I can be of some help to make the discovery image
capable of adding additional information to Foreman. Looking at the
ovirt-node-plugin-foreman project, it seems very complex but it's likely
I'm just not understanding the file layout, or where to start looking.