Development of IPAM Plugins

Hi,
i want to look in to the external ipam feature and extend it with an provider for NetBox

im not sure whats the right approach to contribute the code. should i extend the existing smart_proxy_ipam from @grizzthedj with the stuff for netbox?
Or should i create a new smart_proxy_ipam_netbox ?

thanks Matthias

3 Likes

The right approach is to extend the existing smart_proxy_ipam, however, there needs to be a refactoring of that implementation before additional providers can be added effectively. What sort of timeline were you looking to implement Netbox as a provider?

2 Likes

ok cool. i will do this in the next upcoming weeks. it doenst seem to be a lot of work, the netbox api looks quite easy and functional.

hey @lzap & @grizzthedj
i have some questsions for you two :slight_smile:

  1. are we sure that this part in foreman itself is working?
    https://github.com/theforeman/foreman/blob/c4c097fab09588a886da113e0c598120644f5d34/lib/proxy_api/external_ipam.rb#L233

    when i try this out, the smart proxy correctly sends a 404 if a ip is not allocated in external ipam

    2020-07-31T16:05:00 0371b8ba [I] Finished GET /ipam/subnet/10.100.60.0/24/10.100.60.3 with 404 (413.5 ms)

    but foreman throws an error

    Creating IPv4 in External IPAM for norma-dolbin.whatever.internal task failed with the following error: ERF12-7024 [ProxyAPI::ProxyException]: Unable to obtain IP address for subnet_id 10.100.60.0/24 from External IPAM. ([RestClient::NotFound]: 404 Not Found) for proxy https://tfm-katello-t01.whatever.de:9090/ipam

  2. it would be good to have the possibility to enrich the ipam system with more Information than it an ip is in use and its mac address.
    the hosts fqdn or a link to the hosts page in foreman would be good.

  3. if i got this correctly the ip cache feature relies on mac addresses
    when creating a foreman host with vmware, the mac address is not available till the vm is deployed.
    right now, if i create two host parallel without a mac address filled in the race condition still applies

thanks Matthias

The error handling needs a major overhaul, both on the Foreman and Smart Proxy side. It is full of layer violations. Quite often the error from the backend system is sent all the way to Foreman. The point you found is a good example of that.

You already found my PR (https://github.com/grizzthedj/smart_proxy_ipam/pull/36) but in the end I came to the point where I wanted to do a complete rewrite from scratch and didn’t have the time to do so.

The downside is that it’s already been merged to Foreman, so making changes to the API is painful now.

This is very recent, there might be dragons. The plugin is not official yet.

Sure, work with @grizzthedj on extending it. Although I’d slightly prefer making everything more robust, creating 2nd implementation, settling down on the final API and moving this into theforeman github organization.

Interesting, yeah. It’s a bug. The API probably needs to be extended with some UUID flag and VMs would use VM UUID instead.

Hi, was any progress made?

Managed to smart_proxy_ipam working as a plugin after some fiddling, only to find out the Netbox API has changed so the interaction is currently broken.

@dmgeurts I’m also interested in this, but as you’ve already gone through the process of getting it up and running, maybe this plugin will do the trick:

Also, could you share your notes on how to make it actually work? I didn’t come further then the following:

yum install -y make gcc openssl-devel ruby-devel
gem install foreman_netbox
echo "gem 'foreman_netbox'" > /usr/share/foreman/bundler.d/foreman_netbox.rb
systemctl restart foreman

I attempted it on a Satellite 6.11 system in my lab, which uses Foreman 3.1 underneath, but Foreman doesn’t want to start afterwards because (it seems) it can’t find certain files.

Did you use EL 7 or 8? From the use of yum it suggests 7 and then you may have installed it using the system version rather than the SCL.

I’m running it on EL8 :slight_smile:

@dmgeurts Could you elaborate on what is broken or what has changed in the Netbox API?

As FYI, External IPAM features(with support for Netbox and phpIPAM) are in this PR awaiting to be merged into Smart Proxy core.

1 Like

I now think the wrong API is used as when I look at the code the Netbox API looks right to me, yet the error shows the other API used.

As far as I can tell my configuration is correct as I see Netbox mentioned in the logs, Yet the API calls are not compatible with Netbox.

I’m away from my computer at the moment so can’t give more specific details until a week from now.

Back at work now so can give faster responses. I’m on EL8 (CentOS Stream r8) and am finding that the queries made to Netbox are valid for phpIPAM, but not for Netbox. My config is set for Netbox and some of the logs show this as well, see Should installation of this plugin work on Foreman 3.3? · Issue #56 · grizzthedj/smart_proxy_ipam · GitHub

My prior suspicions about the Netbox API having changed are probably wrong. I now think the IPAM plugin wrongly tries to use the phpIPAM API when talking to Netbox.

Aug 15 00:04:46 fm smart-proxy[1273044]: 2022-08-15 00:04:46 - JSON::ParserError - 783: unexpected token at '<!DOCTYPE HTML>
[...]
Aug 15 00:04:47 fm smart-proxy[1273044]: #011/usr/local/share/gems/gems/smart_proxy_ipam-0.1.4/lib/smart_proxy_ipam/netbox/netbox_client.rb:48:in `get_ipam_subnet_by_cidr'
Aug 15 00:04:47 fm smart-proxy[1273044]: #011/usr/local/share/gems/gems/smart_proxy_ipam-0.1.4/lib/smart_proxy_ipam/netbox/netbox_client.rb:29:in `get_ipam_subnet'
Aug 15 00:04:47 fm smart-proxy[1273044]: #011/usr/local/share/gems/gems/smart_proxy_ipam-0.1.4/lib/smart_proxy_ipam/ipam_api.rb:100:in `block in <class:Api>'
[...]
Aug 15 00:04:47 fm smart-proxy[1273044]: 10.0.0.10 - - [15/Aug/2022:00:04:45 CEST] "GET /ipam/subnet/10.0.1.0/24?group= HTTP/1.1" 500 590509
Aug 15 00:04:47 fm smart-proxy[1273044]: - -> /ipam/subnet/10.0.1.0/24?group=

All I’ve been trying to do is add subnets known in Netbox to Foreman so that IPAM details can be kept in sync. We don’t register our hosts in Netbox but if Foreman can automate this then we may start doing so, but this is not an immediate need for us.

That said, it’s good to know that there’s work on integrating the CMDB aspect of Netbox into Foreman.

Thanks for the details @dmgeurts - I have posted a response to get more specific info in the github issue you opened. We can continue the conversation there, then report the fix back here once resolved.

1 Like

Has any thought been put into pushing the hostname of the machine being built to the IPAM solution? I have been working on adding support for Bluecat IPAM, but it requires the hostname to add a record.

Thanks

Some other users have also requested this. I think there is some value in having hostname, and potentially some other fields passed over as well.

Since the IPAM plugin integration is on its way into Smart Proxy core(the PR is above in this thread), I think it would make sense to wait until this is merged before doing anything.