Ansible Callback not importing facts for Solaris hosts with ipmp interfaces as default_ipv4

lentferj · September 30, 2020, 8:21am

Problem:
Import of ansible_facts via callback fails for Solaris systems where ipmp interfaces are the default route.
Ansible fills ansible_default_ipv4 by scanning the routing table. If the determined interface on Solaris systems happens to be ipmp (e.g. ipmp1) ansible will return ansible_default_ipv4.macaddress=unkown, because, well - it is
On Solaris “ifconfig -a” does not output the mac address of the underlying physical interface as e.g. Linux does.

$ ifconfig -a
lo0: flags=2000000849<UP,LOOPBACK,RUNNING,MULTICAST,VIRTUAL> mtu 8232
        inet 127.0.0.1 netmask ff000000 
ipmp1: flags=108000000843<UP,BROADCAST,RUNNING,MULTICAST,IPMP,PHYSRUNNING> mtu 1500
        inet 10.x.x.x netmask ffff0000 broadcast 10.5.255.255
        groupname ipmp1
net0: flags=100000000843<UP,BROADCAST,RUNNING,MULTICAST,PHYSRUNNING> mtu 1500
        inet 0.0.0.0 netmask ff000000 broadcast 0.255.255.255
        groupname ipmp1
net1: flags=100000000843<UP,BROADCAST,RUNNING,MULTICAST,PHYSRUNNING> mtu 1500
        inet 0.0.0.0 netmask ff000000 broadcast 0.255.255.255
        groupname ipmp1

According to this

github.com

ansible/ansible/blob/devel/lib/ansible/module_utils/facts/network/sunos.py#L101


      
                  current_if = {'device': device, 'ipv4': [], 'ipv6': [], 'type': 'unknown'}
              else:
                  current_if = interfaces[device]
              flags = self.get_options(words[1])
              v = 'ipv4'
              if 'IPv6' in flags:
                  v = 'ipv6'
              if 'LOOPBACK' in flags:
                  current_if['type'] = 'loopback'
              current_if[v].append({'flags': flags, 'mtu': words[3]})
              current_if['macaddress'] = 'unknown'    # will be overwritten later
              return current_if
          
          # Solaris displays single digit octets in MAC addresses e.g. 0:1:2:d:e:f
          # Add leading zero to each octet where needed.
          def parse_ether_line(self, words, current_if, ips):
              macaddress = ''
              for octet in words[1].split(':'):
                  octet = ('0' + octet)[-2:None]
                  macaddress += (octet + ':')
              current_if['macaddress'] = macaddress[0:-1]

this is expected behaviour for ansible_facts.

The the problem is that uploading the facts via ansible foreman callback fails with “500 Internal Server Error” because it seems Foreman expects a MAC for ansible_default_ipv4 or at least not “unknown”.

Expected outcome:
Updating ansible_facts in Foreman via ansbile -m setup works

Foreman and Proxy versions:
Foreman and Proxy plugin versions:
$ rpm -qa | grep tfm
tfm-rubygem-tilt-2.0.8-4.el7.noarch
tfm-rubygem-facter-2.4.0-7.el7.x86_64
vm-tfm-prod.hive.dpdit.de-tomcat-1.0-1.noarch
tfm-rubygem-pg-1.1.4-3.el7.x86_64
vm-tfm-prod.hive.dpdit.de-qpid-client-cert-1.0-1.noarch
tfm-rubygem-unicode-display_width-1.0.5-4.el7.noarch
vm-tfm-prod.hive.dpdit.de-puppet-client-1.0-1.noarch
tfm-rubygem-sshkey-1.9.0-4.el7.noarch
tfm-rubygem-erubi-1.9.0-1.el7.noarch
tfm-rubygem-powerbar-2.0.1-2.el7.noarch
tfm-rubygem-activemodel-6.0.3.1-1.el7.noarch
tfm-rubygem-scoped_search-4.1.8-1.el7.noarch
tfm-rubygem-roadie-3.4.0-3.el7.noarch
tfm-rubygem-fog-core-2.1.0-3.el7.noarch
tfm-rubygem-optimist-3.0.0-2.el7.noarch
tfm-rubygem-faraday-0.15.4-2.el7.noarch
tfm-rubygem-crass-1.0.6-1.el7.noarch
tfm-rubygem-actiontext-6.0.3.1-1.el7.noarch
tfm-rubygem-actioncable-6.0.3.1-1.el7.noarch
tfm-rubygem-hammer_cli-2.1.1-1.el7.noarch
tfm-rubygem-actionmailer-6.0.3.1-1.el7.noarch
tfm-rubygem-oauth-0.5.4-4.el7.noarch
tfm-rubygem-fast_gettext-1.4.1-4.el7.noarch
tfm-rubygem-rack-protection-2.0.3-4.el7.noarch
tfm-rubygem-sequel-5.7.1-3.el7.noarch
tfm-rubygem-ruby2ruby-2.4.2-3.el7.noarch
tfm-rubygem-foreman_hooks-0.3.16-2.fm2_1.el7.noarch
tfm-rubygem-net-scp-1.2.1-4.el7.noarch
tfm-rubygem-statsd-instrument-2.1.4-3.el7.noarch
tfm-rubygem-domain_name-0.5.20160310-4.el7.noarch
vm-tfm-prod.hive.dpdit.de-foreman-proxy-client-1.0-1.noarch
tfm-rubygem-graphql-batch-0.3.10-2.el7.noarch
tfm-rubygem-apipie-dsl-2.2.2-2.el7.noarch
tfm-rubygem-deface-1.5.3-2.el7.noarch
tfm-rubygem-locale-2.0.9-14.el7.noarch
tfm-rubygem-hammer_cli_foreman_bootdisk-0.3.0-1.el7.noarch
tfm-rubygem-amazing_print-1.1.0-1.el7.noarch
tfm-rubygem-deep_cloneable-3.0.0-3.el7.noarch
tfm-rubygem-activejob-6.0.3.1-1.el7.noarch
tfm-rubygem-rsec-0.4.3-4.el7.noarch
tfm-rubygem-sqlite3-1.3.13-6.el7.x86_64
tfm-rubygem-algebrick-0.7.3-7.el7.noarch
tfm-rubygem-pulp_file_client-1.0.1-1.el7.noarch
tfm-rubygem-actionview-6.0.3.1-1.el7.noarch
tfm-rubygem-angular-rails-templates-1.1.0-1.el7.noarch
tfm-rubygem-mime-types-3.2.2-4.el7.noarch
tfm-rubygem-hammer_cli_foreman_docker-0.0.5-1.el7.noarch
tfm-rubygem-apipie-rails-0.5.17-3.el7.noarch
tfm-rubygem-jwt-2.2.1-2.el7.noarch
tfm-rubygem-foreman_ansible-5.1.2-1.fm2_1.el7.noarch
tfm-rubygem-smart_proxy_ansible-3.0.1-6.fm2_1.el7.noarch
tfm-rubygem-typhoeus-1.3.1-1.el7.noarch
tfm-rubygem-concurrent-ruby-1.1.6-2.el7.noarch
tfm-rubygem-polyglot-0.3.5-2.el7.noarch
tfm-rubygem-bundler_ext-0.4.1-5.el7.noarch
tfm-rubygem-gettext_i18n_rails-1.8.0-2.el7.noarch
tfm-rubygem-graphql-1.8.14-2.el7.noarch
tfm-rubygem-clamp-1.1.2-6.el7.noarch
tfm-rubygem-rb-inotify-0.9.7-5.el7.noarch
tfm-rubygem-foreman_remote_execution_core-1.3.0-1.el7.noarch
tfm-rubygem-rack-jsonp-1.3.1-9.el7.noarch
tfm-rubygem-passenger-4.0.18-10.13.el7.x86_64
tfm-rubygem-netrc-0.11.0-5.el7.noarch
tfm-rubygem-mustermann-1.0.2-4.el7.noarch
tfm-rubygem-unf-0.1.3-8.el7.noarch
tfm-rubygem-method_source-0.9.2-1.el7.noarch
vm-tfm-prod.hive.dpdit.de-foreman-client-1.0-1.noarch
tfm-rubygem-promise.rb-0.7.4-2.el7.noarch
vm-tfm-prod.hive.dpdit.de-qpid-router-server-1.0-1.noarch
tfm-rubygem-rubyipmi-0.10.0-6.el7.noarch
tfm-rubygem-nokogiri-1.10.9-1.el7.x86_64
tfm-rubygem-secure_headers-6.3.0-2.el7.noarch
tfm-rubygem-ruby-libvirt-0.7.1-1.el7.x86_64
tfm-rubygem-bcrypt-3.1.12-3.el7.x86_64
tfm-rubygem-marcel-0.3.3-1.el7.noarch
tfm-rubygem-anemone-0.7.2-17.el7.noarch
tfm-rubygem-activesupport-6.0.3.1-1.el7.noarch
tfm-rubygem-rabl-0.14.3-1.el7.noarch
tfm-rubygem-will_paginate-3.1.7-3.el7.noarch
tfm-rubygem-globalid-0.4.2-1.el7.noarch
tfm-rubygem-addressable-2.6.0-2.el7.noarch
tfm-rubygem-unicode-0.4.4.4-3.el7.x86_64
tfm-rubygem-kafo-4.1.0-3.el7.noarch
tfm-rubygem-websocket-driver-0.7.1-1.el7.x86_64
tfm-rubygem-gitlab-sidekiq-fetcher-0.5.2-2.el7.noarch
tfm-rubygem-fog-vsphere-3.3.1-1.el7.noarch
tfm-rubygem-foreman_ansible_core-3.0.3-1.fm2_1.el7.noarch
tfm-rubygem-pulp_certguard_client-0.1.0rc5-1.el7.noarch
tfm-rubygem-pulp_container_client-1.4.1-1.el7.noarch
tfm-rubygem-rails-html-sanitizer-1.3.0-1.el7.noarch
tfm-rubygem-activestorage-6.0.3.1-1.el7.noarch
tfm-rubygem-fx-0.5.0-1.el7.noarch
tfm-rubygem-x-editable-rails-1.5.5-5.el7.noarch
tfm-rubygem-mime-types-data-3.2018.0812-4.el7.noarch
tfm-rubygem-smart_proxy_dynflow-0.2.4-6.fm2_1.el7.noarch
tfm-rubygem-hammer_cli_foreman_tasks-0.0.14-1.fm2_1.el7.noarch
tfm-rubygem-mini_mime-1.0.2-1.el7.noarch
tfm-rubygem-rails-6.0.3.1-1.el7.noarch
tfm-rubygem-foreman-tasks-2.0.2-1.fm2_1.el7.noarch
tfm-rubygem-foreman_default_hostgroup-5.0.0-4.fm2_1.el7.noarch
tfm-rubygem-smart_proxy_remote_execution_ssh-0.3.0-4.fm2_1.el7.noarch
tfm-rubygem-net-ldap-0.16.1-2.el7.noarch
tfm-rubygem-puma-4.3.3-4.el7.x86_64
tfm-rubygem-ethon-0.12.0-1.el7.noarch
tfm-rubygem-rainbow-2.2.1-3.el7.noarch
tfm-rubygem-multi_json-1.14.1-1.el7.noarch
tfm-runtime-6.1-1.el7.x86_64
tfm-rubygem-i18n-1.8.2-1.el7.noarch
tfm-rubygem-sprockets-3.7.2-6.el7.noarch
tfm-rubygem-activerecord-import-1.0.0-2.el7.noarch
tfm-rubygem-sexp_processor-4.10.0-6.el7.noarch
tfm-rubygem-ruby_parser-3.10.1-3.el7.noarch
tfm-rubygem-rake-compiler-1.0.7-3.el7.noarch
tfm-rubygem-gssapi-1.2.0-7.el7.noarch
tfm-rubygem-kafo_wizards-0.0.1-4.el7.noarch
tfm-rubygem-passenger-native-libs-4.0.18-10.13.el7.x86_64
tfm-rubygem-ipaddress-0.8.0-12.el7.noarch
tfm-rubygem-xmlrpc-0.3.0-2.el7.noarch
tfm-rubygem-unf_ext-0.0.7.2-3.el7.x86_64
tfm-rubygem-deacon-1.0.0-4.el7.noarch
tfm-rubygem-stomp-1.4.9-1.el7.noarch
vm-tfm-prod.hive.dpdit.de-qpid-router-client-1.0-1.noarch
tfm-rubygem-logging-2.2.2-5.el7.noarch
tfm-rubygem-mini_portile2-2.4.0-1.el7.noarch
tfm-rubygem-kafo_parsers-1.1.0-3.el7.noarch
tfm-rubygem-rkerberos-0.1.5-18.el7.x86_64
tfm-rubygem-parse-cron-0.1.4-4.fm2_1.el7.noarch
tfm-rubygem-mimemagic-0.3.5-1.el7.noarch
tfm-rubygem-robotex-1.0.0-21.el7.noarch
tfm-rubygem-zeitwerk-2.2.2-1.el7.noarch
tfm-rubygem-activerecord-6.0.3.1-1.el7.noarch
tfm-rubygem-audited-4.9.0-3.el7.noarch
tfm-rubygem-ancestry-3.0.7-1.el7.noarch
tfm-rubygem-public_suffix-3.0.3-2.el7.noarch
tfm-rubygem-apipie-params-0.0.5-4.el7.noarch
tfm-rubygem-ansi-1.5.0-2.el7.noarch
tfm-rubygem-websocket-extensions-0.1.4-1.el7.noarch
tfm-rubygem-sidekiq-5.2.7-3.el7.noarch
tfm-rubygem-rbvmomi-2.2.0-3.el7.noarch
tfm-rubygem-foreman-tasks-core-0.3.4-1.fm2_1.el7.noarch
tfm-rubygem-pulp_rpm_client-3.5.0-1.el7.noarch
tfm-rubygem-pulp_ansible_client-0.2.0b13.dev01588546902-1.el7.noarch
tfm-rubygem-loofah-2.4.0-1.el7.noarch
tfm-rubygem-railties-6.0.3.1-1.el7.noarch
tfm-rubygem-rails-i18n-6.0.0-2.el7.noarch
tfm-rubygem-responders-3.0.0-3.el7.noarch
tfm-rubygem-record_tag_helper-1.0.1-3.el7.noarch
tfm-rubygem-smart_proxy_dynflow_core-0.2.5-1.fm2_1.el7.noarch
tfm-rubygem-hammer_cli_foreman-2.1.2-1.el7.noarch
tfm-rubygem-runcible-2.13.1-1.el7.noarch
tfm-rubygem-actionmailbox-6.0.3.1-1.el7.noarch
katello-ca-consumer-vm-tfm-prod.hive.dpdit.de-1.0-1.noarch
tfm-rubygem-katello-3.16.0-1.el7.noarch
tfm-rubygem-hammer_cli_foreman_ansible-0.3.2-1.fm2_1.el7.noarch
tfm-rubygem-awesome_print-1.8.0-5.el7.noarch
tfm-rubygem-arel-9.0.0-4.el7.noarch
tfm-rubygem-ffi-1.12.2-1.el7.x86_64
tfm-rubygem-safemode-1.3.5-3.el7.noarch
tfm-rubygem-concurrent-ruby-edge-0.6.0-2.fm2_1.el7.noarch
tfm-rubygem-passenger-native-4.0.18-10.13.el7.x86_64
tfm-rubygem-tzinfo-1.2.6-1.el7.noarch
tfm-rubygem-http-cookie-1.0.2-4.el7.noarch
vm-tfm-prod.hive.dpdit.de-foreman-proxy-1.0-1.noarch
tfm-rubygem-little-plugger-1.1.4-2.el7.noarch
tfm-rubygem-thor-1.0.1-2.el7.noarch
tfm-rubygem-daemons-1.2.3-6.el7.noarch
tfm-rubygem-formatador-0.2.1-12.el7.noarch
tfm-rubygem-friendly_id-5.3.0-1.el7.noarch
tfm-rubygem-ldap_fluff-0.4.7-5.el7.noarch
tfm-rubygem-connection_pool-2.2.2-2.el7.noarch
tfm-rubygem-dynflow-1.4.6-1.fm2_1.el7.noarch
tfm-rubygem-pulp_2to3_migration_client-0.2.0-0.1.b6.el7.noarch
tfm-rubygem-actionpack-6.0.3.1-1.el7.noarch
tfm-rubygem-activerecord-session_store-1.1.1-4.el7.noarch
tfm-rubygem-rest-client-2.0.2-3.el7.noarch
tfm-rubygem-hammer_cli_katello-0.22.2-1.el7.noarch
tfm-rubygem-webpack-rails-0.9.8-5.el7.noarch
tfm-rubygem-rack-2.2.2-1.el7.noarch
tfm-rubygem-highline-1.7.8-5.el7.noarch
tfm-rubygem-smart_proxy_pulp-2.1.0-3.fm2_1.el7.noarch
tfm-rubygem-get_process_mem-0.2.1-4.el7.noarch
tfm-rubygem-qpid_messaging-1.38.0-1.el7.x86_64
tfm-rubygem-rack-test-1.1.0-4.el7.noarch
vm-tfm-prod.hive.dpdit.de-qpid-broker-1.0-1.noarch
tfm-rubygem-sinatra-2.0.3-4.el7.noarch
vm-tfm-prod.hive.dpdit.de-apache-1.0-1.noarch
tfm-rubygem-redis-4.1.2-2.el7.noarch
tfm-rubygem-net-ping-2.0.1-4.el7.noarch
tfm-rubygem-nio4r-2.5.2-1.el7.x86_64
tfm-rubygem-hashie-3.6.0-2.el7.noarch
tfm-rubygem-rails-dom-testing-2.0.3-6.el7.noarch
tfm-rubygem-validates_lengths_from_database-0.5.0-7.el7.noarch
tfm-rubygem-css_parser-1.4.7-4.el7.noarch
tfm-rubygem-excon-0.58.0-5.el7.noarch
tfm-rubygem-rack-cors-1.0.2-2.el7.noarch
tfm-rubygem-multipart-post-2.0.0-2.el7.noarch
tfm-rubygem-pulpcore_client-3.4.1-1.el7.noarch
tfm-rubygem-sprockets-rails-3.2.1-6.el7.noarch
tfm-rubygem-roadie-rails-2.1.1-2.el7.noarch
tfm-rubygem-apipie-bindings-0.3.0-2.el7.noarch
tfm-rubygem-mail-2.7.1-1.el7.noarch
tfm-rubygem-foreman_remote_execution-3.3.7-1.fm2_1.el7.noarch
tfm-rubygem-thread_safe-0.3.6-5.el7.noarch
tfm-rubygem-net-ssh-4.2.0-2.el7.noarch
tfm-rubygem-builder-3.2.4-1.el7.noarch
tfm-rubygem-hammer_cli_foreman_remote_execution-0.1.2-1.fm2_1.el7.noarch

Distribution and version:
CentOS Linux release 7.8.2003 (Core)

Other relevant data:

production.log:
    2020-09-30T10:13:25 [I|app|0acd37d1] Started POST "/api/v2/hosts/facts" for 127.0.0.1 at 2020-09-30 10:13:25 +0200
    2020-09-30T10:13:25 [I|app|0acd37d1] Processing by Api::V2::HostsController#facts as JSON
    2020-09-30T10:13:25 [I|app|0acd37d1]   Parameters: {"facts"=>"[FILTERED]", "name"=>"somehost.some.domain", "apiv"=>"v2", "host"=>{"name"=>"somehost.some.domain"}}
    2020-09-30T10:13:25 [I|app|0acd37d1] Import facts for 'somehost.some.domain' completed. Added: 0, Updated: 0, Deleted 245 facts
    2020-09-30T10:13:25 [W|app|0acd37d1] Action failed
    2020-09-30T10:13:25 [I|app|0acd37d1]   Rendering api/v2/errors/custom_error.json.rabl within api/v2/layouts/error_layout
    2020-09-30T10:13:25 [I|app|0acd37d1]   Rendered api/v2/errors/custom_error.json.rabl within api/v2/layouts/error_layout (Duration: 6.1ms | Allocations: 5975)
    2020-09-30T10:13:25 [I|app|0acd37d1] Completed 500 Internal Server Error in 120ms (Views: 9.5ms | ActiveRecord: 58.4ms | Allocations: 29677)

part of ansible -vvv output
<somehost.some.domain> ESTABLISH SSH CONNECTION FOR USER: None
<somehost.some.domain> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/foobar/.ansible/cp/637ebd20f8 -tt somehost.some.domain '/bin/sh -c '"'"'/usr/bin/python /export/home/foobar/.ansible/tmp/ansible-tmp-1601453762.54-42919-62133910765260/AnsiballZ_setup.py && sleep 0'"'"''
<somehost.some.domain> (0, '\r\n{"invocation": {"module_args": {"filter": "ansible_default_ipv4", "gather_subset": ["all"], "fact_path": "/etc/ansible/facts.d", "gather_timeout": 10}}, "ansible_facts": {"ansible_default_ipv4": {"macaddress": "unknown", "network": "10.x.0.0", "mtu": "1500", "broadcast": "10.x.255.255", "interface": "ipmp1", "netmask": "255.255.0.0", "flags": ["UP", "BROADCAST", "RUNNING", "MULTICAST", "IPMP", "PHYSRUNNING"], "address": "10.x.x.x", "device": "ipmp1", "type": "unknown", "gateway": "10.x.x.x"}}}\r\n', 'Shared connection to somehost.some.domainclosed.\r\n')
<somehost.some.domain> ESTABLISH SSH CONNECTION FOR USER: None