RFC: Drop parsing NIC interfaces

I have finished with step 1: getting some real-world JSON output from all major facter versions:

Step 2: create some integration tests from this and find out what is incorrectly parsed.

Now, here is the problem. A simple case - bridge.

Neither from facter 2.x nor facter 4.x you are able to tell if the bridge (br0) slave device is enp1s0 or enp7s0. Facter does not advertise relationships between NICs, we briefly touched extending it in another thread. What’s worse is that Facter 4.x does not even include slave devices, so there is less information available.

The same applies to all master-slave relationships like VLAN, bonds or teams and combinations.

All we do these days is “heuristic”, some device names like eth0.13 implies this “could” be probably a VLAN with ID 13. But that’s wild guessing.

My proposal then: let’s drop parsing alltogether and write a custom fact that adds those relationships, only if this fact is found then we have enough info to reconstruct the NICs. I suggest to start small and expand on top of this.

No, I don’t think. Facter (v3.11) reports bond like any other virtual or physical interface like following example (another oVirt HV, with vlan on bond and where XXX is an oVirt VM network, and em1,em2 are physical ifaces and bond slaves):

  "XXX": {
    "mac": "11:6a:1a:db:d1:45",
    "mtu": 1496
  },
  "bond0": {
    "mac": "11:6a:1a:db:d1:45",
    "mtu": 1500
  },
  "bond0.42": {
    "mac": "11:6a:1a:db:d1:45",
    "mtu": 1500
  },
  "em1": {
    "mac": "11:6a:1a:db:d1:45",
    "mtu": 1500
  },
  "em2": {
    "mac": "11:6a:1a:db:d1:46",
    "mtu": 1500
  },

Maybe rather than split network interface facts reporting by physical/virtual; because it seems it is complicated to know which one is physical/virtual; maybe reporting only interface with assigned IPs (all ifaces with IPs ? only ifaces with IPs corresponding to a network already present in foreman ?) will be more reliable compromise ?

I know it’s a little off topic but, if we keep the ‘netwok ifaces facts importer’ functionality (with added tests around it like @lzap said) , maybe it will be nice to be able to whitelist network ifaces to import/update instead of blacklisting (like it’s now possible in settings). It is sometime easier to manage (eg, in my case a whitelist that will be something like ‘eth[0-9].* bond[0-9].* p[0-9]p[0-9].* em[0-9].* ovitmgmt […]’). Or instead of an array of blacklist/whitelist, just a bare regex string that will permit the user to do what he want ?

Why a custom fact instead of submitting it upstream to facter?

I think that’s a fair point, if we do not get the data we can’t process it. How do the other config management tools handle this?

I’d rather start with a custom fact that can be of course contributed upstream once it turns out reliable. We will need it anyway for older versions.

I believe the best in business is Ubuntu MAAS which is based around IPMI/Moonshot/KVM. It sends out architecture and MAC address and pairs that with IPMI. It’s a different approach than ours since we boot OS installers while MAAS is solely image-based provisioning approach:

I am not aware of other config management tools which parse NICs, show me.

Thanks, I meant Salt, Ansible, …

Why would those tools parse network interfaces just like we do?

Sorry again, I meant what data do they provide. Foreman does parse interface data provided by e.g. ansible, doesn’t it?

For example in Ansible, we only parse basic information: regular interface (IP and MAC address).

I am currently looking into how to get additional information about interfaces in Linux and I am afraid this can’t be easily done in Ruby:

http://man7.org/linux/man-pages/man7/netdevice.7.html

To get extra flags, ioctl must be done with union within struct as a parameter. I think it could be easier to write a simple command line application which would give required data and call that from facter.

So I was able to write a small utility which talks to kernel via netlink and acquires NIC link data. Two regular interfaces:

link.enp1s0.mac => 52:54:00:aa:bb:cc
link.enp1s0.type => device
link.enp7s0.mac => 52:54:00:dd:ee:ff
link.enp7s0.type => device
link.lo.type => device

A simple bond:

link.bond0.bond.mode => active-backup
link.bond0.mac => 52:54:00:aa:bb:cc
link.bond0.type => bond
link.enp1s0.mac => 52:54:00:aa:bb:cc
link.enp1s0.master => bond0
link.enp1s0.slave => bond
link.enp1s0.type => device
link.enp7s0.mac => 52:54:00:aa:bb:cc
link.enp7s0.master => bond0
link.enp7s0.slave => bond
link.enp7s0.type => device
link.lo.type => device

Team:

link.enp1s0.mac => 52:54:00:aa:bb:cc
link.enp1s0.master => team0
link.enp1s0.type => device
link.enp7s0.mac => 52:54:00:aa:bb:cc
link.enp7s0.master => team0
link.enp7s0.type => device
link.lo.type => device
link.team0.mac => 52:54:00:aa:bb:cc
link.team0.type => team

Bridge with two slaves:

link.br0.mac => 52:54:00:aa:bb:cc
link.br0.type => bridge
link.enp1s0.mac => 52:54:00:aa:bb:cc
link.enp1s0.master => br0
link.enp1s0.type => device
link.enp7s0.mac => 52:54:00:dd:ee:ff
link.enp7s0.master => br0
link.enp7s0.type => device
link.lo.type => device

Bridge and regular:

link.br0.mac => 52:54:00:dd:ee:ff
link.br0.type => bridge
link.enp1s0.mac => 52:54:00:aa:bb:cc
link.enp1s0.type => device
link.enp7s0.mac => 52:54:00:dd:ee:ff
link.enp7s0.master => br0
link.enp7s0.type => device
link.lo.type => device

VLAN (output is currently dot-separated JSON is planned):

link.enp1s0.10.mac => 52:54:00:aa:bb:cc
link.enp1s0.10.parent => enp1s0
link.enp1s0.10.type => vlan
link.enp1s0.10.vlan.id => 10
link.enp1s0.10.vlan.protocol => 802.1q
link.enp1s0.mac => 52:54:00:aa:bb:cc
link.enp1s0.type => device
link.enp7s0.mac => 52:54:00:dd:ee:ff
link.enp7s0.type => device
link.lo.type => device

Finally bond with VLAN:

link.bond0.10.mac => 52:54:00:aa:bb:cc
link.bond0.10.parent => bond0
link.bond0.10.type => vlan
link.bond0.10.vlan.id => 10
link.bond0.10.vlan.protocol => 802.1q
link.bond0.bond.mode => active-backup
link.bond0.mac => 52:54:00:aa:bb:cc
link.bond0.type => bond
link.enp1s0.mac => 52:54:00:aa:bb:cc
link.enp1s0.master => bond0
link.enp1s0.slave => bond
link.enp1s0.type => device
link.enp7s0.mac => 52:54:00:aa:bb:cc
link.enp7s0.master => bond0
link.enp7s0.slave => bond
link.enp7s0.type => device
link.lo.type => device

Please take a look on this output and let me know if we are able to reconstruct all the types from this data. To me, it looks like this is enough! When confirmed, I am able to write a custom fact that runs this tool and reports the data back via facter.

At first glance it looks correct to reconstruct the NIC setup. What I really like about this is that it allows users to set their own NIC names (external, lan) which are actually bonds or even a VLAN on top of a bond. As you correctly concluded the current heuristics would never allow this. Looking forward to seeing this.

Let me know if you need help with shaping this into a custom fact.

1 Like

Thanks, I tried pure ruby but the netlink wrapper library did not work and it was very low-level anyway (too much of an effort to implement this). Also not sure is this kind of dependency would be accepted upstream.

In this design, the custom fact can simply search for /usr/bin/link_tool_whatever and if it is installed, facts will be provided. If not, it can be simply skipped. That should work fine.

Perhaps it should be implemented as an external fact.

1 Like

I think this conversation is about different feature though. I agree we should be able to deploy. But what’s disucssed here is, whether we want to reverse engineer the datacenter. E.g. if I deploy Foreman to brownfield, do I just deploy facter and get the overview of my bonds and vlans? Or do I have to manually setup everything in Foreman forms.

It is and that’s why the code is so complicated. If there’s no simple way and the current implementation shows it’s cause more problems than good, let’s get rid of it.

I’m afraid this won’t solve the main issue. We’d import subset of interfaces, but they would still be bond/vlans or whatever you have without us being able to tell. Therefore all validations like MAC uniqueness would cause issues.

yeah I think that is a good idea, instead of “I see hundred irelevant NICs”, we’ll be getting “I don’t see my vlan called custom01” which is definitely better for the user

I believe this is what we should aim for. The tool looks great and could be integrated in all kinds of config management or fact reporting solutions. Would we package it as a rpm and deploy it in the foreman-client repo?

Yeah, since this looks as reliable way to detect all information we need, I wouldn’t be against keeping parsing in. We’d still need to offer whitelisting as @bagasse suggested. However I’d like to better understand how much work it is to build and support that tool in all platforms we support in foreman-client. I agree it would be nice to integrate it with all facters, so the logic does not have to be implemented in every facter.

Would this even work in subscription-manager?

I think it makes a lot of sense to build on top of facter, due to history of Foreman. Therefore we should:

  • make a decision to build on top of facter as the defacto standard for all facts in Foreman
  • extend facter 2, 3 and NG with network link information
  • define minimum set of facts Foreman understand and document it
  • create transformation core API so other fact parsers can easily turn their format into our stanard

Let me go one after the other points now.

Although I was a bit against facter, I think it makes a lot of sense to build on top of it. I think we all agree here.

Extending with network link info (bond, vlans) is already in the works, I have a prototype external facts, currently working on rough edges and want to start on Foreman part soon.

Defining minimum set of facts Foreman understand is the most important one I think. Even facter NG (next generation) is not perfect, I don’t like how it throws all facts into one heap - volatile ones (e.g. free memory, uptime) with stable ones (available memory, boot time). It also reports huge amount of facts, we should cherry pick them throwing irrelevant away (these could be enabled by user via settings perhaps - whitelisting mentioned above). We should define minimum set we support (based on facter 3/NG facts plus some extra - e.g. network link we discussed above), we should define which facts are considered stable and which volatile and then refactor our parsing code in a clean way.

A good API for plugin authors is viable to get this into the right direction. Huge advantage is that user is presented with just one set of facts e.g. networking.interfaces.eth0.bindings6.ipaddress and this remains the same no matter which client reports this: Ansible, Facter or RHSM.