External IPAM core feature merged

Development
1

Hello all,

after a long journey, I have merged External IPAM support into Foreman core. This work made by @grizzthedj defines an IPAM Smart Proxy API which allows Foreman to allocate IPv4 and IPv6 addresses from an external authority. There is currently no module in Smart Proxy core, there is an “incubating” plugin and reference implementation for phpIPAM. Long term goal would be to have some implementations battle tested and then we can select some of them to be shipped with smart proxy - in that case we would do common extenralipam plugin and providers similarly to what we have today for other features.

To use this feature, install Smart Proxy phpIPAM (https://github.com/grizzthedj/smart_proxy_ipam - the project name is a bit off it should be smart_proxy_phpipam), refresh features of the proxy, select External IPAM when creating new Subnet, then pick External IPAM Smart Proxy on the Proxies tab.

That’s all, when creating new host, Foreman will request new IP through the API. Implementations must cache pre-allocated IP addresses in cache for some time to prevent race conditions, this code should move to common code in proxy so this don’t need to be duplicated.

There’s been some confusion around IPAM in Foreman, so I have put this into our documentation.

  • DHCP: the DHCP {SmartProxy} manages the assignment of IP addresses by finding the next available IP address starting from the first address of the range skipping all addresses that are reserved. Before assigning an IP address, {SmartProxy} sends an ICMP and TCP ping to verify whether the IP address is in use. {SmartProxy} DHCP module retains offered IP addresses for short period of time to prevent collisions during concurrent access, therefore temporary “holes” in the IP range can exist.

  • Internal DB: {Project} finds next available IP address from the Subnet range excluding all IP addresses from the {Project} database in sequence. Primary source of data is the database, not DHCP reservations. This IPAM is not safe when multiple hosts are being created in parallel, in that case use DHCP or Random DB IPAM instead.

  • Random DB: {Project} finds next available IP address from the Subnet range excluding all IP addresses from the {Project} database randomly. Primary source of data is the database, not DHCP reservations. This IPAM is safe to use with concurrent host creation as IP addresses are returned in random order minimizing chance of a conflict.

  • EUI-64: Extended Unique Identifier (EUI) 64bit IPv6 address generation, as per RFC2373, is obtained through the 48-bit MAC address.

  • External IPAM: Delegate IPAM to an external system through {SmartProxy} feature. {Project} currently does not ship with any external IPAM implementations, several plugins are in development.

We will show it how it works on the next demo. If you intend to create new implementation let us know so we can coordinate efforts to create a common smart proxy plugin just in time. Thanks to @grizzthedj for lot of work on this!

4 Likes
2

@grizzthedj I think one of the next steps could be:

  • Integrate foreman_ipam into Foreman core, unless you have some big plans. Current state only adds a dashboard widget and it’s quite useful. Since it’s use the common External IPAM API, there is no need to have that code in a plugin now.
  • Refactor smart proxy plugin into provider. We have a plugin-provider mechanism in smart proxy which we use in other features (e.g. dhcp, isc_dhcp). Some code could be shared across implementations (IP cache). Initially I wanted to keep the smart proxy plugin separately for some “incubation” phase but it looks there is already demand from our community and I think we can merge that into core since it’s simple enough and I have a working setup to help users with problems.
  • Installer - we need to add some code into puppet to be able to setup external ipam smart proxy plugin. This can be only done when we merge it into smart proxy codebase.
  • Documentation - feel free to contribute anything you think is important, specifically how to setup phpIPAM, it was a real pain for me until I realized I need to create API key with the same name as the username. Docs go here: https://github.com/theforeman/foreman-documentation
3

Great job @grizzthedj on pushing this forward and getting it to the finish line! And thanks @lzap and everyone else who was involved in reviewing and testing this. Can we please add this to the headline features section of the nightly manual?

2 Likes
4