New Plugin: foreman_phpipam

lzap · June 14, 2019, 9:28am

Frankly I am surprised that Foreman starts on MacOS. I know either smart proxy or foreman was not starting last time I tried.

You can either try start digging or just ignore these tests - your work will very unlikely affect them.

One more idea: try to migrate and reseed the database, admin is being assigned with some permissions which you might run twice?

grizzthedj · June 14, 2019, 3:24pm

I tried re-migrating and seeding the database, and still getting the same 4 failures.

Funny thing is that when I run the two roles and permissions test files individually, they are green, but fail when running the whole test suite.

bundle exec rails test test/controllers/api/v2/locations_controller_test.rb
bundle exec rails test test/controllers/api/v2/roles_controller_test.rb

Perhaps there are some tests that don’t clean up properly prior to these running?

I will try to dig a little more.

grizzthedj · June 17, 2019, 6:39pm

@lzap @tbrisker

I checked out the latest develop branch, and it seems the same 4 tests are failing there as well. I’m not sure if this has to do with my local setup or not, but I have proceeded with the PR anyway.

I have pushed the latest changes and submitted the PR for the phpIPAM Integration with Foreman in a new branch called plugin/foreman_ipam.

github.com/theforeman/foreman

Fixes #27067 - IPAM Integration with phpIPAM

theforeman:develop ← grizzthedj:plugin/foreman_ipam

opened 06:29PM - 17 Jun 19 UTC

grizzthedj

+542 -4

This PR is a patch to support of the use of plugins for external IPAM use, with …opensource provider phpIPAM(https://phpipam.net). The primary use case is to provide the next available IPv4 address from phpIPAM, for a given phpIPAM subnet. The functionality that Foreman will use for external IPAM is encompassed in the below plugins(which are both new). **foreman_ipam(v0.0.7)** **Git:** https://github.com/grizzthedj/foreman_ipam **Gem:** https://rubygems.org/gems/foreman_ipam **smart_proxy_ipam(v0.0.13)** **Git:** https://github.com/grizzthedj/smart_proxy_ipam **Gem:** https://rubygems.org/gems/smart_proxy_ipam The `foreman_ipam` plugin provides a very basic dashboard to display sections and subnets from phpIPAM, and the `smart_proxy_ipam` plugin communicates with the phpIPAM API. Features and functionality * Simple IPAM Dashboard page(Infrastructure => IPAM Dashboard) to view sections and subnets for phpIPAM. More features will be added here based on internal user feedback and from the community. * When creating a phpIPAM subnet(IPAM type = External IPAM) in Foreman, subnet/mask must exist in phpIPAM, or error will be thrown * When creating a new host in Foreman, auto suggested IP's from phpIPAM will be persisted back to phpIPAM on Host submit. * When updating the IP address on a NIC that uses a phpIPAM subnet, the old IP will be deleted from phpIPAM, and new one will be added. * Deleting a host in Foreman will delete all NIC ip's that used a phpIPAM subnet * Deleting a NIC from a host will also delete the IP from phpIPAM * Deleting a subnet in Foreman will NOT delete the associated subnet in phpIPAM. This could easily be added later if it makes sense to the community, however initial thoughts are to not delete it. NOTE: When creating a host in Foreman and adding multiple nics/interfaces, the same IP will be suggested from phpIPAM since it is not actually reserved in phpIPAM until you submit the host. The workaround for multiple nics is to use the suggested ip for the first interface(e.g. 10.0.0.1), then manually increment the IP for the second interface(e.g. 10.0.0.2). Errors will be thrown however if IP's are already taken. Prerequisites for testing: * Working Foreman environment. * A phpIPAM instance that the Forman Smart Proxy has network connectivity to. * The API must be enabled in the phpIPAM settings, and `Prettify Links` also enabled. * An API user in phpIPAM must be created. The credentials for this user are to be put in the `config/settings.d/external_ipam.yml` file in the Smart Proxy core

Here are the links to the Foreman Plugin repos, and published gems:

foreman_ipam(v0.0.3)
Git: GitHub - grizzthedj/foreman_ipam: Foreman plugin for integration with External IPAM providers
Gem: foreman_ipam | RubyGems.org | your community gem host

smart_proxy_ipam(v0.0.4)
Git: GitHub - grizzthedj/smart_proxy_ipam: Foreman Smart Proxy plugin for integration with External IPAM providers
Gem: smart_proxy_ipam | RubyGems.org | your community gem host

There are more details on the features and functionality in the PR itself, but had a few questions regarding next steps.

Questions:

How does one integrate the install/setup of these plugins with the foreman-installer?
How can we get this added to the list of supported Foreman/Proxy plugins?
In terms of integration testing, is there a project that has integration tests for Foreman? There are some tests, but none that test the actual phpIPAM integration.
Is there any available infrastructure(in the cloud or otherwise) where we can setup a small phpIPAM instance for integration testing purposes?

Feel free to reach out if I have missed anything here!

Cheers,

Chris

tbrisker · June 18, 2019, 8:06am

This is a two step process:

adding the plugins to the puppet-foreman and puppet-foreman_proxy modules. Examples can be seen here for foreman and here for proxy.
once the modules are updated, adding them as installer options requires adding them to the answers file and creating a migration. example here

List of Plugins - Foreman is a wiki page that can be edited to add it, if you don’t have permission to do that send me a private message with your redmine username and I’ll fix it.
If you wish to transfer the plugin to the official theforeman organization under github, please reach out to me or one of the other org admins (preferably @ekohl (ewoud on irc) or @ehelms, though @ohadlevy and @Gwmngilfen may also be able to help if they have time) on IRC to facilitate it. We can also open a project on redmine for tracking issues if you prefer that to github.

We have a jenkins instance that is used to test many projects. The configuration for it is maintained under https://github.com/theforeman/foreman-infra/tree/master/puppet/modules/jenkins_job_builder and should be hopefully not be too complicated to add jobs for the plugin based on examples from other plugins. @infra team should be able to help guide you here towards the best practices.

I’m not sure what it entails to spin up an instance, but we do have some tests that run vms as part of the testing process on our infra. If it’s something that can be automatically spun up in a vm or container as part of the testing process that’s likely possible. Deffering to @infra team again for further guidance.

lzap · June 18, 2019, 12:05pm

Well then it smells, if you checkout develop locally do you see the same four tests failing?

I’ve reviewed your initial patch, keep it going. You are defining brand new Smart Proxy API (feature) here which is great, we like this design as it really paied off so far.

grizzthedj · June 18, 2019, 1:36pm

I checked out the latest develop branch locally, and the same 4 tests are failing there as well. Not sure if it is related to my local environment or not.

Thanks @lzap and @tbrisker for the feedback on the PR, and for answering the other questions. I will get cracking on those changes!

lzap · June 18, 2019, 1:47pm

Probably a new thread with better subject could help, there are lot more devs here who might know.

grizzthedj · June 18, 2019, 5:50pm

lzap · March 20, 2020, 9:44am

For the record, the Foreman core part has been merged and will be in Foreman 2.1: