Foreman ansible callback failure using method v2_runner_on_ok

Problem:
Foreman callback ansible plugin doesn’t work correctly and it is not passing data to Foreman.

500 Server Error: Internal Server Error
[WARNING]: Failure using method (v2_runner_on_ok) in callback plugin (<ansible.plugins.callback.foreman_callback.CallbackModule object at 0x7efec8825d10>): 'module_name'
Callback Exception:
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_queue_manager.py", line 381, in send_callback
    method(*new_args, **kwargs)
   File "/var/lib/foreman-proxy/ansible/plugins/callback_plugins/foreman_callback.py", line 245, in v2_runner_on_ok
    self.send_facts(host, res)
   File "/var/lib/foreman-proxy/ansible/plugins/callback_plugins/foreman_callback.py", line 101, in send_facts
    if re.search(r'^junos', data['invocation']['module_name']):

Expected outcome:
Make it works.

Foreman and Proxy versions:
3.1.0-rc1

Foreman and Proxy plugin versions:
foreman-tasks 5.1.1
foreman_ansible 6.4.1
foreman_default_hostgroup 6.0.0
foreman_discovery 18.0.4
foreman_hooks 0.3.17
foreman_kubevirt 0.1.9
foreman_openscap 5.1.0
foreman_puppet 2.0.0.alpha.2
foreman_remote_execution 4.8.0
foreman_setup 7.0.0
foreman_snapshot_management 2.0.1
foreman_templates 9.1.0
katello 4.2.1

Distribution and version:

LSB Version: :core-4.1-amd64:core-4.1-noarch
Distributor ID: CentOS
Description: CentOS Linux release 7.9.2009 (Core)
Release: 7.9.2009
Codename: Core

Other relevant data:
Full output from ansible-playbook command:

ansible-playbook 2.9.25
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /bin/ansible-playbook
  python version = 2.7.5 (default, Nov 16 2020, 22:23:17) [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)]
Using /etc/ansible/ansible.cfg as config file
Parsed chd-dev-debrepo01, inventory source with host_list plugin
Skipping callback 'actionable', as we already have a stdout callback.
Skipping callback 'counter_enabled', as we already have a stdout callback.
Skipping callback 'debug', as we already have a stdout callback.
Skipping callback 'dense', as we already have a stdout callback.
Skipping callback 'dense', as we already have a stdout callback.
Skipping callback 'full_skip', as we already have a stdout callback.
Skipping callback 'json', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'null', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.
Skipping callback 'selective', as we already have a stdout callback.
Skipping callback 'skippy', as we already have a stdout callback.
Skipping callback 'stderr', as we already have a stdout callback.
Skipping callback 'unixy', as we already have a stdout callback.
Skipping callback 'yaml', as we already have a stdout callback.

PLAYBOOK: facts.yml **********************************************************************************************************************************************************************************************************************************************************************************************************
1 plays in facts.yml

PLAY [all] *******************************************************************************************************************************************************************************************************************************************************************************************************************

TASK [Gathering Facts] *******************************************************************************************************************************************************************************************************************************************************************************************************
task path: /root/facts.yml:2
<chd-dev-debrepo01> ESTABLISH SSH CONNECTION FOR USER: root
<chd-dev-debrepo01> SSH: EXEC ssh -o ProxyCommand=none -o StrictHostKeyChecking=no -o 'IdentityFile="/usr/share/foreman/.ssh/id_rsa_foreman_proxy"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 chd-dev-debrepo01 '/bin/sh -c '"'"'echo ~root && sleep 0'"'"''
<chd-dev-debrepo01> (0, '/root\n', '')
<chd-dev-debrepo01> ESTABLISH SSH CONNECTION FOR USER: root
<chd-dev-debrepo01> SSH: EXEC ssh -o ProxyCommand=none -o StrictHostKeyChecking=no -o 'IdentityFile="/usr/share/foreman/.ssh/id_rsa_foreman_proxy"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 chd-dev-debrepo01 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /root/.ansible/tmp `"&& mkdir "` echo /root/.ansible/tmp/ansible-tmp-1637232219.59-21032-227331678296735 `" && echo ansible-tmp-1637232219.59-21032-227331678296735="` echo /root/.ansible/tmp/ansible-tmp-1637232219.59-21032-227331678296735 `" ) && sleep 0'"'"''
<chd-dev-debrepo01> (0, 'ansible-tmp-1637232219.59-21032-227331678296735=/root/.ansible/tmp/ansible-tmp-1637232219.59-21032-227331678296735\n', '')
Using module file /usr/lib/python2.7/site-packages/ansible/modules/system/setup.py
<chd-dev-debrepo01> PUT /root/.ansible/tmp/ansible-local-210224QDkit/tmpTf2FS1 TO /root/.ansible/tmp/ansible-tmp-1637232219.59-21032-227331678296735/AnsiballZ_setup.py
<chd-dev-debrepo01> SSH: EXEC sftp -b - -o ProxyCommand=none -o StrictHostKeyChecking=no -o 'IdentityFile="/usr/share/foreman/.ssh/id_rsa_foreman_proxy"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 '[chd-dev-debrepo01]'
<chd-dev-debrepo01> (0, 'sftp> put /root/.ansible/tmp/ansible-local-210224QDkit/tmpTf2FS1 /root/.ansible/tmp/ansible-tmp-1637232219.59-21032-227331678296735/AnsiballZ_setup.py\n', '')
<chd-dev-debrepo01> ESTABLISH SSH CONNECTION FOR USER: root
<chd-dev-debrepo01> SSH: EXEC ssh -o ProxyCommand=none -o StrictHostKeyChecking=no -o 'IdentityFile="/usr/share/foreman/.ssh/id_rsa_foreman_proxy"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 chd-dev-debrepo01 '/bin/sh -c '"'"'chmod u+x /root/.ansible/tmp/ansible-tmp-1637232219.59-21032-227331678296735/ /root/.ansible/tmp/ansible-tmp-1637232219.59-21032-227331678296735/AnsiballZ_setup.py && sleep 0'"'"''
<chd-dev-debrepo01> (0, '', '')
<chd-dev-debrepo01> ESTABLISH SSH CONNECTION FOR USER: root
<chd-dev-debrepo01> SSH: EXEC ssh -o ProxyCommand=none -o StrictHostKeyChecking=no -o 'IdentityFile="/usr/share/foreman/.ssh/id_rsa_foreman_proxy"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -tt chd-dev-debrepo01 '/bin/sh -c '"'"'/usr/bin/python /root/.ansible/tmp/ansible-tmp-1637232219.59-21032-227331678296735/AnsiballZ_setup.py && sleep 0'"'"''
<chd-dev-debrepo01> (0, '\r\n{"invocation": {"module_args": {"filter": "*", "gather_subset": ["all"], "fact_path": "/etc/ansible/facts.d", "gather_timeout": 10}}, "ansible_facts": {"ansible_fibre_channel_wwn": [], "module_setup": true, "ansible_distribution_version": "10", "ansible_distribution_file_variety": "Debian", "ansible_env": {"LANG": "en_US.UTF-8", "TERM": "screen", "SHELL": "/bin/bash", "XDG_RUNTIME_DIR": "/run/user/0", "SHLVL": "0", "SSH_TTY": "/dev/pts/0", "PWD": "/root", "XDG_SESSION_TYPE": "tty", "SSH_CLIENT": "10.9.64.196 56550 22", "LOGNAME": "root", "USER": "root", "PATH": "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "HOME": "/root", "SSH_CONNECTION": "10.9.64.196 56550 10.9.63.204 22", "XDG_SESSION_ID": "142219", "DBUS_SESSION_BUS_ADDRESS": "unix:path=/run/user/0/bus", "XDG_SESSION_CLASS": "user", "_": "/bin/sh"}, "ansible_userspace_bits": "64", "ansible_architecture": "x86_64", "ansible_default_ipv4": {"macaddress": "00:50:56:8b:8e:c9", "network": "10.9.63.0", "mtu": 1500, "broadcast": "10.9.63.255", "alias": "ens160", "netmask": "255.255.255.0", "address": "10.9.63.204", "interface": "ens160", "type": "ether", "gateway": "10.9.63.1"}, "ansible_swapfree_mb": 299, "ansible_default_ipv6": {}, "ansible_cmdline": {"BOOT_IMAGE": "/boot/vmlinuz-4.19.0-16-amd64", "ro": true, "root": "UUID=f20ae82a-c15d-4de4-abe6-e6aaace55580", "quiet": true}, "ansible_machine_id": "d85792fb6f0a4425a4e0bae8109d844c", "ansible_userspace_architecture": "x86_64", "ansible_product_uuid": "420b2bcc-4840-6a0f-1f9b-48159fab3d99", "ansible_pkg_mgr": "apt", "ansible_distribution": "Debian", "ansible_iscsi_iqn": "", "ansible_all_ipv6_addresses": [], "ansible_uptime_seconds": 8466393, "ansible_kernel": "4.19.0-16-amd64", "ansible_system_capabilities_enforced": "True", "ansible_python": {"executable": "/usr/bin/python", "version": {"micro": 16, "major": 2, "releaselevel": "final", "serial": 0, "minor": 7}, "type": "CPython", "has_sslcontext": true, "version_info": [2, 7, 16, "final", 0]}, "ansible_is_chroot": false, "ansible_hostnqn": "", "ansible_user_shell": "/bin/bash", "ansible_product_serial": "VMware-42 0b 2b cc 48 40 6a 0f-1f 9b 48 15 9f ab 3d 99", "ansible_form_factor": "Other", "ansible_distribution_file_parsed": true, "ansible_fips": false, "ansible_user_id": "root", "ansible_selinux_python_present": false, "ansible_kernel_version": "#1 SMP Debian 4.19.181-1 (2021-03-19)", "ansible_local": {}, "ansible_processor_vcpus": 4, "ansible_processor": ["0", "GenuineIntel", "Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz", "1", "GenuineIntel", "Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz", "2", "GenuineIntel", "Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz", "3", "GenuineIntel", "Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz"], "ansible_ssh_host_key_ecdsa_public": "AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPJiW3KZ8KrAxpbPFCg4foForfqF48g7ysb3t2vPH0qbe9SWjcoQcfUjDcg6Q9Ie3rt8s2CmuGGu456yQqcwZT0=", "ansible_user_gid": 0, "ansible_system_vendor": "VMware, Inc.", "ansible_swaptotal_mb": 974, "ansible_distribution_major_version": "10", "ansible_real_group_id": 0, "ansible_lsb": {"release": "10", "major_release": "10", "codename": "buster", "id": "Debian", "description": "Debian GNU/Linux 10 (buster)"}, "ansible_machine": "x86_64", "ansible_ssh_host_key_rsa_public": "AAAAB3NzaC1yc2EAAAADAQABAAABAQCjCfvOMWccgoE9MxaeMkRVdUJE9OMH3VLSOuTvPVJv3+1mskLckishlnqZ6v+vVItz1PTU780QC73Z5C7zpzVhzZdL01EGUCUFn2DW5yiEEErS5zUEfBL19dZ5abUoXJdXzAf99Hqm2LhOGo7rynEsupGaA16C6NZ+y5mJEo9Njr3meS+VlTi3jzUc2tS7lH1PfrHRltukXiHP86rO0hBMREfHFSqgpPTcUoqrJjs90h9XyrQs0Fi7wz/YTHwNP7/vCZ1KG8SzrLgA9yJpKYPU8YzQsFE0UrasMJ3l5Zsxre7WyUMT1H5N6i35U7AJk3ed2AdsNZmvfEvo/YjglhoF", "ansible_user_gecos": "root", "ansible_processor_threads_per_core": 1, "ansible_product_name": "VMware Virtual Platform", "ansible_all_ipv4_addresses": ["10.9.63.204"], "ansible_python_version": "2.7.16", "ansible_product_version": "None", "ansible_service_mgr": "systemd", "ansible_memory_mb": {"real": {"total": 7978, "used": 2521, "free": 5457}, "swap": {"cached": 7, "total": 974, "free": 299, "used": 675}, "nocache": {"used": 1129, "free": 6849}}, "ansible_user_dir": "/root", "gather_subset": ["all"], "ansible_real_user_id": 0, "ansible_virtualization_role": "guest", "ansible_dns": {"nameservers": ["10.9.64.51", "10.10.64.251"], "search": ["companydomain.com"]}, "ansible_effective_group_id": 0, "ansible_lo": {"features": {"tx_checksum_ipv4": "off [fixed]", "generic_receive_offload": "on", "tx_checksum_ipv6": "off [fixed]", "tx_scatter_gather_fraglist": "on [fixed]", "rx_all": "off [fixed]", "highdma": "on [fixed]", "tls_hw_rx_offload": "off [fixed]", "tx_lockless": "on [fixed]", "tx_tcp_ecn_segmentation": "on", "rx_udp_tunnel_port_offload": "off [fixed]", "esp_hw_offload": "off [fixed]", "tx_gso_robust": "off [fixed]", "tx_udp_segmentation": "off [fixed]", "tx_tcp_mangleid_segmentation": "on", "tx_ipxip4_segmentation": "off [fixed]", "tx_checksumming": "on", "vlan_challenged": "on [fixed]", "loopback": "on [fixed]", "tx_ipxip6_segmentation": "off [fixed]", "fcoe_mtu": "off [fixed]", "scatter_gather": "on", "tx_checksum_sctp": "on [fixed]", "tx_vlan_stag_hw_insert": "off [fixed]", "rx_vlan_stag_hw_parse": "off [fixed]", "tx_gso_partial": "off [fixed]", "tx_tcp_segmentation": "on", "rx_vlan_stag_filter": "off [fixed]", "large_receive_offload": "off [fixed]", "tx_scatter_gather": "on [fixed]", "tx_tcp6_segmentation": "on", "rx_checksumming": "on [fixed]", "rx_gro_hw": "off [fixed]", "netns_local": "on [fixed]", "tx_esp_segmentation": "off [fixed]", "generic_segmentation_offload": "on", "rx_fcs": "off [fixed]", "tcp_segmentation_offload": "on", "l2_fwd_offload": "off [fixed]", "rx_vlan_offload": "off [fixed]", "ntuple_filters": "off [fixed]", "tx_gre_csum_segmentation": "off [fixed]", "esp_tx_csum_hw_offload": "off [fixed]", "tx_udp_tnl_segmentation": "off [fixed]", "tx_udp_tnl_csum_segmentation": "off [fixed]", "tx_sctp_segmentation": "on", "udp_fragmentation_offload": "off", "tls_hw_record": "off [fixed]", "tls_hw_tx_offload": "off [fixed]", "tx_checksum_fcoe_crc": "off [fixed]", "hw_tc_offload": "off [fixed]", "tx_checksum_ip_generic": "on [fixed]", "tx_fcoe_segmentation": "off [fixed]", "tx_nocache_copy": "off [fixed]", "rx_vlan_filter": "off [fixed]", "tx_vlan_offload": "off [fixed]", "receive_hashing": "off [fixed]", "tx_gre_segmentation": "off [fixed]"}, "hw_timestamp_filters": [], "mtu": 65536, "device": "lo", "promisc": false, "timestamping": ["tx_software", "rx_software", "software"], "ipv4": {"broadcast": "", "netmask": "255.0.0.0", "network": "127.0.0.0", "address": "127.0.0.1"}, "active": true, "type": "loopback"}, "ansible_memtotal_mb": 7978, "ansible_device_links": {"masters": {}, "labels": {}, "ids": {"sr0": ["ata-VMware_Virtual_SATA_CDRW_Drive_00000000000000000001"]}, "uuids": {"sda5": ["219da1de-11ba-4eb0-9076-c6458326f152"], "sda1": ["f20ae82a-c15d-4de4-abe6-e6aaace55580"]}}, "ansible_apparmor": {"status": "enabled"}, "ansible_proc_cmdline": {"BOOT_IMAGE": "/boot/vmlinuz-4.19.0-16-amd64", "ro": true, "root": "UUID=f20ae82a-c15d-4de4-abe6-e6aaace55580", "quiet": true}, "ansible_memfree_mb": 5457, "ansible_processor_count": 4, "ansible_hostname": "chd-dev-debrepo01", "ansible_interfaces": ["lo", "ens160"], "ansible_selinux": {"status": "Missing selinux Python library"}, "ansible_fqdn": "chd-dev-debrepo01.companydomain.com", "ansible_mounts": [{"block_used": 0, "uuid": "N/A", "size_total": 0, "block_total": 0, "mount": "/debrepo", "block_available": 0, "size_available": 0, "fstype": "autofs", "inode_total": 0, "options": "rw,relatime,fd=6,pgrp=1345,timeout=60,minproto=5,maxproto=5,indirect,pipe_ino=13165092", "device": "/etc/auto.debrepo", "inode_used": 0, "block_size": 4096, "inode_available": 0}, {"block_used": 2788383, "uuid": "f20ae82a-c15d-4de4-abe6-e6aaace55580", "size_total": 145880190976, "block_total": 35615281, "mount": "/", "block_available": 32826898, "size_available": 134458974208, "fstype": "ext4", "inode_total": 9117696, "options": "rw,relatime,errors=remount-ro", "device": "/dev/sda1", "inode_used": 229400, "block_size": 4096, "inode_available": 8888296}], "ansible_nodename": "chd-dev-debrepo01", "ansible_lvm": {"pvs": {}, "lvs": {}, "vgs": {}}, "ansible_domain": "companydomain.com", "ansible_distribution_file_path": "/etc/os-release", "ansible_virtualization_type": "VMware", "ansible_ssh_host_key_ed25519_public": "AAAAC3NzaC1lZDI1NTE5AAAAIDgWPs5UoHo+/cRpv/wVuJc/cxtJ5hGSqLdSzsOGAm94", "ansible_processor_cores": 1, "ansible_bios_version": "6.00", "ansible_date_time": {"weekday_number": "4", "iso8601_basic_short": "20211118T044341", "tz": "CST", "weeknumber": "46", "hour": "04", "year": "2021", "minute": "43", "tz_offset": "-0600", "month": "11", "epoch": "1637232221", "iso8601_micro": "2021-11-18T10:43:41.090629Z", "weekday": "Thursday", "time": "04:43:41", "date": "2021-11-18", "iso8601": "2021-11-18T10:43:41Z", "day": "18", "iso8601_basic": "20211118T044341090629", "second": "41"}, "ansible_distribution_release": "buster", "ansible_os_family": "Debian", "ansible_effective_user_id": 0, "ansible_system": "Linux", "ansible_devices": {"sr0": {"scheduler_mode": "mq-deadline", "rotational": "1", "vendor": "NECVMWar", "sectors": "2097151", "links": {"masters": [], "labels": [], "ids": ["ata-VMware_Virtual_SATA_CDRW_Drive_00000000000000000001"], "uuids": []}, "sas_device_handle": null, "sas_address": null, "virtual": 1, "host": "SATA controller: VMware SATA AHCI controller", "sectorsize": "512", "removable": "1", "support_discard": "0", "model": "VMware SATA CD00", "partitions": {}, "holders": [], "size": "1024.00 MB"}, "sda": {"scheduler_mode": "mq-deadline", "rotational": "1", "vendor": "VMware", "sectors": "293601280", "links": {"masters": [], "labels": [], "ids": [], "uuids": []}, "sas_device_handle": null, "sas_address": null, "virtual": 1, "host": "SCSI storage controller: Broadcom / LSI 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 01)", "sectorsize": "512", "removable": "0", "support_discard": "0", "model": "Virtual disk", "partitions": {"sda5": {"sectorsize": 512, "uuid": "219da1de-11ba-4eb0-9076-c6458326f152", "links": {"masters": [], "labels": [], "ids": [], "uuids": ["219da1de-11ba-4eb0-9076-c6458326f152"]}, "sectors": "1996800", "start": "291602432", "holders": [], "size": "975.00 MB"}, "sda2": {"sectorsize": 512, "uuid": null, "links": {"masters": [], "labels": [], "ids": [], "uuids": []}, "sectors": "2", "start": "291602430", "holders": [], "size": "1.00 KB"}, "sda1": {"sectorsize": 512, "uuid": "f20ae82a-c15d-4de4-abe6-e6aaace55580", "links": {"masters": [], "labels": [], "ids": [], "uuids": ["f20ae82a-c15d-4de4-abe6-e6aaace55580"]}, "sectors": "291598336", "start": "2048", "holders": [], "size": "139.04 GB"}}, "holders": [], "size": "140.00 GB"}}, "ansible_user_uid": 0, "ansible_bios_date": "09/21/2015", "ansible_system_capabilities": ["cap_chown", "cap_dac_override", "cap_dac_read_search", "cap_fowner", "cap_fsetid", "cap_kill", "cap_setgid", "cap_setuid", "cap_setpcap", "cap_linux_immutable", "cap_net_bind_service", "cap_net_broadcast", "cap_net_admin", "cap_net_raw", "cap_ipc_lock", "cap_ipc_owner", "cap_sys_module", "cap_sys_rawio", "cap_sys_chroot", "cap_sys_ptrace", "cap_sys_pacct", "cap_sys_admin", "cap_sys_boot", "cap_sys_nice", "cap_sys_resource", "cap_sys_time", "cap_sys_tty_config", "cap_mknod", "cap_lease", "cap_audit_write", "cap_audit_control", "cap_setfcap", "cap_mac_override", "cap_mac_admin", "cap_syslog", "cap_wake_alarm", "cap_block_suspend", "cap_audit_read+ep"], "ansible_ens160": {"macaddress": "00:50:56:8b:8e:c9", "features": {"tx_checksum_ipv4": "off [fixed]", "generic_receive_offload": "on", "tx_checksum_ipv6": "off [fixed]", "tx_scatter_gather_fraglist": "off [fixed]", "rx_all": "off [fixed]", "highdma": "on", "tls_hw_rx_offload": "off [fixed]", "tx_lockless": "off [fixed]", "tx_tcp_ecn_segmentation": "off [fixed]", "rx_udp_tunnel_port_offload": "off [fixed]", "esp_hw_offload": "off [fixed]", "tx_gso_robust": "off [fixed]", "tx_udp_segmentation": "off [fixed]", "tx_tcp_mangleid_segmentation": "off", "tx_ipxip4_segmentation": "off [fixed]", "tx_checksumming": "on", "vlan_challenged": "off [fixed]", "loopback": "off [fixed]", "tx_ipxip6_segmentation": "off [fixed]", "fcoe_mtu": "off [fixed]", "scatter_gather": "on", "tx_checksum_sctp": "off [fixed]", "tx_vlan_stag_hw_insert": "off [fixed]", "rx_vlan_stag_hw_parse": "off [fixed]", "tx_gso_partial": "off [fixed]", "tx_tcp_segmentation": "on", "rx_vlan_stag_filter": "off [fixed]", "large_receive_offload": "on", "tx_scatter_gather": "on", "tx_tcp6_segmentation": "on", "rx_checksumming": "on", "rx_gro_hw": "off [fixed]", "netns_local": "off [fixed]", "tx_esp_segmentation": "off [fixed]", "generic_segmentation_offload": "on", "rx_fcs": "off [fixed]", "tcp_segmentation_offload": "on", "l2_fwd_offload": "off [fixed]", "rx_vlan_offload": "on", "ntuple_filters": "off [fixed]", "tx_gre_csum_segmentation": "off [fixed]", "esp_tx_csum_hw_offload": "off [fixed]", "tx_udp_tnl_segmentation": "off [fixed]", "tx_udp_tnl_csum_segmentation": "off [fixed]", "tx_sctp_segmentation": "off [fixed]", "udp_fragmentation_offload": "off", "tls_hw_record": "off [fixed]", "tls_hw_tx_offload": "off [fixed]", "tx_checksum_fcoe_crc": "off [fixed]", "hw_tc_offload": "off [fixed]", "tx_checksum_ip_generic": "on", "tx_fcoe_segmentation": "off [fixed]", "tx_nocache_copy": "off", "rx_vlan_filter": "on [fixed]", "tx_vlan_offload": "on", "receive_hashing": "on", "tx_gre_segmentation": "off [fixed]"}, "type": "ether", "pciid": "0000:03:00.0", "module": "vmxnet3", "mtu": 1500, "device": "ens160", "promisc": false, "timestamping": ["rx_software", "software"], "ipv4": {"broadcast": "10.9.63.255", "netmask": "255.255.255.0", "network": "10.9.63.0", "address": "10.9.63.204"}, "active": true, "speed": 10000, "hw_timestamp_filters": []}}}\r\n', 'Connection to chd-dev-debrepo01 closed.\r\n')
<chd-dev-debrepo01> ESTABLISH SSH CONNECTION FOR USER: root
<chd-dev-debrepo01> SSH: EXEC ssh -o ProxyCommand=none -o StrictHostKeyChecking=no -o 'IdentityFile="/usr/share/foreman/.ssh/id_rsa_foreman_proxy"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 chd-dev-debrepo01 '/bin/sh -c '"'"'rm -f -r /root/.ansible/tmp/ansible-tmp-1637232219.59-21032-227331678296735/ > /dev/null 2>&1 && sleep 0'"'"''
<chd-dev-debrepo01> (0, '', '')
ok: [chd-dev-debrepo01]
500 Server Error: Internal Server Error
Callback Exception: 
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_queue_manager.py", line 381, in send_callback
    method(*new_args, **kwargs)
   File "/var/lib/foreman-proxy/ansible/plugins/callback_plugins/foreman_callback.py", line 245, in v2_runner_on_ok
    self.send_facts(host, res)
   File "/var/lib/foreman-proxy/ansible/plugins/callback_plugins/foreman_callback.py", line 101, in send_facts
    if re.search(r'^junos', data['invocation']['module_name']):
META: ran handlers

TASK [Set ansible interpreter] ***********************************************************************************************************************************************************************************************************************************************************************************************
task path: /root/facts.yml:4
ok: [chd-dev-debrepo01] => {
    "ansible_facts": {
        "ansible_python_interpreter": "/usr/bin/python"
    }, 
    "changed": false
}
500 Server Error: Internal Server Error
Callback Exception: 
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_queue_manager.py", line 381, in send_callback
    method(*new_args, **kwargs)
   File "/var/lib/foreman-proxy/ansible/plugins/callback_plugins/foreman_callback.py", line 245, in v2_runner_on_ok
    self.send_facts(host, res)
   File "/var/lib/foreman-proxy/ansible/plugins/callback_plugins/foreman_callback.py", line 101, in send_facts
    if re.search(r'^junos', data['invocation']['module_name']):

TASK [Collect only facts returned by facter] *********************************************************************************************************************************************************************************************************************************************************************************
task path: /root/facts.yml:7
<chd-dev-debrepo01> ESTABLISH SSH CONNECTION FOR USER: root
<chd-dev-debrepo01> SSH: EXEC ssh -o ProxyCommand=none -o StrictHostKeyChecking=no -o 'IdentityFile="/usr/share/foreman/.ssh/id_rsa_foreman_proxy"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 chd-dev-debrepo01 '/bin/sh -c '"'"'echo ~root && sleep 0'"'"''
<chd-dev-debrepo01> (0, '/root\n', '')
<chd-dev-debrepo01> ESTABLISH SSH CONNECTION FOR USER: root
<chd-dev-debrepo01> SSH: EXEC ssh -o ProxyCommand=none -o StrictHostKeyChecking=no -o 'IdentityFile="/usr/share/foreman/.ssh/id_rsa_foreman_proxy"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 chd-dev-debrepo01 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /root/.ansible/tmp `"&& mkdir "` echo /root/.ansible/tmp/ansible-tmp-1637232224.88-21051-133782812663879 `" && echo ansible-tmp-1637232224.88-21051-133782812663879="` echo /root/.ansible/tmp/ansible-tmp-1637232224.88-21051-133782812663879 `" ) && sleep 0'"'"''
<chd-dev-debrepo01> (0, 'ansible-tmp-1637232224.88-21051-133782812663879=/root/.ansible/tmp/ansible-tmp-1637232224.88-21051-133782812663879\n', '')
Using module file /usr/lib/python2.7/site-packages/ansible/modules/system/setup.py
<chd-dev-debrepo01> PUT /root/.ansible/tmp/ansible-local-210224QDkit/tmpaTj0_i TO /root/.ansible/tmp/ansible-tmp-1637232224.88-21051-133782812663879/AnsiballZ_setup.py
<chd-dev-debrepo01> SSH: EXEC sftp -b - -o ProxyCommand=none -o StrictHostKeyChecking=no -o 'IdentityFile="/usr/share/foreman/.ssh/id_rsa_foreman_proxy"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 '[chd-dev-debrepo01]'
<chd-dev-debrepo01> (0, 'sftp> put /root/.ansible/tmp/ansible-local-210224QDkit/tmpaTj0_i /root/.ansible/tmp/ansible-tmp-1637232224.88-21051-133782812663879/AnsiballZ_setup.py\n', '')
<chd-dev-debrepo01> ESTABLISH SSH CONNECTION FOR USER: root
<chd-dev-debrepo01> SSH: EXEC ssh -o ProxyCommand=none -o StrictHostKeyChecking=no -o 'IdentityFile="/usr/share/foreman/.ssh/id_rsa_foreman_proxy"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 chd-dev-debrepo01 '/bin/sh -c '"'"'chmod u+x /root/.ansible/tmp/ansible-tmp-1637232224.88-21051-133782812663879/ /root/.ansible/tmp/ansible-tmp-1637232224.88-21051-133782812663879/AnsiballZ_setup.py && sleep 0'"'"''
<chd-dev-debrepo01> (0, '', '')
<chd-dev-debrepo01> ESTABLISH SSH CONNECTION FOR USER: root
<chd-dev-debrepo01> SSH: EXEC ssh -o ProxyCommand=none -o StrictHostKeyChecking=no -o 'IdentityFile="/usr/share/foreman/.ssh/id_rsa_foreman_proxy"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -tt chd-dev-debrepo01 '/bin/sh -c '"'"'/usr/bin/python /root/.ansible/tmp/ansible-tmp-1637232224.88-21051-133782812663879/AnsiballZ_setup.py && sleep 0'"'"''
<chd-dev-debrepo01> (0, '\r\n{"invocation": {"module_args": {"filter": "*", "gather_subset": ["all"], "fact_path": "/etc/ansible/facts.d", "gather_timeout": 10}}, "ansible_facts": {"ansible_fibre_channel_wwn": [], "module_setup": true, "ansible_distribution_version": "10", "ansible_distribution_file_variety": "Debian", "ansible_env": {"LANG": "C", "LC_NUMERIC": "C", "TERM": "screen", "SHELL": "/bin/bash", "XDG_RUNTIME_DIR": "/run/user/0", "SHLVL": "0", "SSH_TTY": "/dev/pts/0", "HOME": "/root", "PWD": "/root", "XDG_SESSION_TYPE": "tty", "SSH_CLIENT": "10.9.64.196 56566 22", "LOGNAME": "root", "USER": "root", "LC_ALL": "C", "PATH": "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "SSH_CONNECTION": "10.9.64.196 56566 10.9.63.204 22", "XDG_SESSION_ID": "142225", "DBUS_SESSION_BUS_ADDRESS": "unix:path=/run/user/0/bus", "XDG_SESSION_CLASS": "user", "_": "/bin/sh"}, "ansible_userspace_bits": "64", "ansible_architecture": "x86_64", "ansible_default_ipv4": {"macaddress": "00:50:56:8b:8e:c9", "network": "10.9.63.0", "mtu": 1500, "broadcast": "10.9.63.255", "alias": "ens160", "netmask": "255.255.255.0", "address": "10.9.63.204", "interface": "ens160", "type": "ether", "gateway": "10.9.63.1"}, "ansible_swapfree_mb": 299, "ansible_default_ipv6": {}, "ansible_cmdline": {"BOOT_IMAGE": "/boot/vmlinuz-4.19.0-16-amd64", "ro": true, "root": "UUID=f20ae82a-c15d-4de4-abe6-e6aaace55580", "quiet": true}, "ansible_machine_id": "d85792fb6f0a4425a4e0bae8109d844c", "ansible_userspace_architecture": "x86_64", "ansible_product_uuid": "420b2bcc-4840-6a0f-1f9b-48159fab3d99", "ansible_pkg_mgr": "apt", "ansible_distribution": "Debian", "ansible_iscsi_iqn": "", "ansible_all_ipv6_addresses": [], "ansible_uptime_seconds": 8466398, "ansible_kernel": "4.19.0-16-amd64", "ansible_system_capabilities_enforced": "True", "ansible_python": {"executable": "/usr/bin/python", "version": {"micro": 16, "major": 2, "releaselevel": "final", "serial": 0, "minor": 7}, "type": "CPython", "has_sslcontext": true, "version_info": [2, 7, 16, "final", 0]}, "ansible_is_chroot": false, "ansible_hostnqn": "", "ansible_user_shell": "/bin/bash", "ansible_product_serial": "VMware-42 0b 2b cc 48 40 6a 0f-1f 9b 48 15 9f ab 3d 99", "ansible_form_factor": "Other", "ansible_distribution_file_parsed": true, "ansible_fips": false, "ansible_user_id": "root", "ansible_selinux_python_present": false, "ansible_kernel_version": "#1 SMP Debian 4.19.181-1 (2021-03-19)", "ansible_local": {}, "ansible_processor_vcpus": 4, "ansible_processor": ["0", "GenuineIntel", "Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz", "1", "GenuineIntel", "Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz", "2", "GenuineIntel", "Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz", "3", "GenuineIntel", "Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz"], "ansible_ssh_host_key_ecdsa_public": "AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPJiW3KZ8KrAxpbPFCg4foForfqF48g7ysb3t2vPH0qbe9SWjcoQcfUjDcg6Q9Ie3rt8s2CmuGGu456yQqcwZT0=", "ansible_user_gid": 0, "ansible_system_vendor": "VMware, Inc.", "ansible_swaptotal_mb": 974, "ansible_distribution_major_version": "10", "ansible_real_group_id": 0, "ansible_lsb": {"release": "10", "major_release": "10", "codename": "buster", "id": "Debian", "description": "Debian GNU/Linux 10 (buster)"}, "ansible_machine": "x86_64", "ansible_ssh_host_key_rsa_public": "AAAAB3NzaC1yc2EAAAADAQABAAABAQCjCfvOMWccgoE9MxaeMkRVdUJE9OMH3VLSOuTvPVJv3+1mskLckishlnqZ6v+vVItz1PTU780QC73Z5C7zpzVhzZdL01EGUCUFn2DW5yiEEErS5zUEfBL19dZ5abUoXJdXzAf99Hqm2LhOGo7rynEsupGaA16C6NZ+y5mJEo9Njr3meS+VlTi3jzUc2tS7lH1PfrHRltukXiHP86rO0hBMREfHFSqgpPTcUoqrJjs90h9XyrQs0Fi7wz/YTHwNP7/vCZ1KG8SzrLgA9yJpKYPU8YzQsFE0UrasMJ3l5Zsxre7WyUMT1H5N6i35U7AJk3ed2AdsNZmvfEvo/YjglhoF", "ansible_user_gecos": "root", "ansible_processor_threads_per_core": 1, "ansible_product_name": "VMware Virtual Platform", "ansible_all_ipv4_addresses": ["10.9.63.204"], "ansible_python_version": "2.7.16", "ansible_product_version": "None", "ansible_service_mgr": "systemd", "ansible_memory_mb": {"real": {"total": 7978, "used": 2521, "free": 5457}, "swap": {"cached": 7, "total": 974, "free": 299, "used": 675}, "nocache": {"used": 1129, "free": 6849}}, "ansible_user_dir": "/root", "gather_subset": ["all"], "ansible_real_user_id": 0, "ansible_virtualization_role": "guest", "ansible_dns": {"nameservers": ["10.9.64.51", "10.10.64.251"], "search": ["companydomain.com"]}, "ansible_effective_group_id": 0, "ansible_lo": {"features": {"tx_checksum_ipv4": "off [fixed]", "generic_receive_offload": "on", "tx_checksum_ipv6": "off [fixed]", "tx_scatter_gather_fraglist": "on [fixed]", "rx_all": "off [fixed]", "highdma": "on [fixed]", "tls_hw_rx_offload": "off [fixed]", "tx_lockless": "on [fixed]", "tx_tcp_ecn_segmentation": "on", "rx_udp_tunnel_port_offload": "off [fixed]", "esp_hw_offload": "off [fixed]", "tx_gso_robust": "off [fixed]", "tx_udp_segmentation": "off [fixed]", "tx_tcp_mangleid_segmentation": "on", "tx_ipxip4_segmentation": "off [fixed]", "tx_checksumming": "on", "vlan_challenged": "on [fixed]", "loopback": "on [fixed]", "tx_ipxip6_segmentation": "off [fixed]", "fcoe_mtu": "off [fixed]", "scatter_gather": "on", "tx_checksum_sctp": "on [fixed]", "tx_vlan_stag_hw_insert": "off [fixed]", "rx_vlan_stag_hw_parse": "off [fixed]", "tx_gso_partial": "off [fixed]", "tx_tcp_segmentation": "on", "rx_vlan_stag_filter": "off [fixed]", "large_receive_offload": "off [fixed]", "tx_scatter_gather": "on [fixed]", "tx_tcp6_segmentation": "on", "rx_checksumming": "on [fixed]", "rx_gro_hw": "off [fixed]", "netns_local": "on [fixed]", "tx_esp_segmentation": "off [fixed]", "generic_segmentation_offload": "on", "rx_fcs": "off [fixed]", "tcp_segmentation_offload": "on", "l2_fwd_offload": "off [fixed]", "rx_vlan_offload": "off [fixed]", "ntuple_filters": "off [fixed]", "tx_gre_csum_segmentation": "off [fixed]", "esp_tx_csum_hw_offload": "off [fixed]", "tx_udp_tnl_segmentation": "off [fixed]", "tx_udp_tnl_csum_segmentation": "off [fixed]", "tx_sctp_segmentation": "on", "udp_fragmentation_offload": "off", "tls_hw_record": "off [fixed]", "tls_hw_tx_offload": "off [fixed]", "tx_checksum_fcoe_crc": "off [fixed]", "hw_tc_offload": "off [fixed]", "tx_checksum_ip_generic": "on [fixed]", "tx_fcoe_segmentation": "off [fixed]", "tx_nocache_copy": "off [fixed]", "rx_vlan_filter": "off [fixed]", "tx_vlan_offload": "off [fixed]", "receive_hashing": "off [fixed]", "tx_gre_segmentation": "off [fixed]"}, "hw_timestamp_filters": [], "mtu": 65536, "device": "lo", "promisc": false, "timestamping": ["tx_software", "rx_software", "software"], "ipv4": {"broadcast": "", "netmask": "255.0.0.0", "network": "127.0.0.0", "address": "127.0.0.1"}, "active": true, "type": "loopback"}, "ansible_memtotal_mb": 7978, "ansible_device_links": {"masters": {}, "labels": {}, "ids": {"sr0": ["ata-VMware_Virtual_SATA_CDRW_Drive_00000000000000000001"]}, "uuids": {"sda5": ["219da1de-11ba-4eb0-9076-c6458326f152"], "sda1": ["f20ae82a-c15d-4de4-abe6-e6aaace55580"]}}, "ansible_apparmor": {"status": "enabled"}, "ansible_proc_cmdline": {"BOOT_IMAGE": "/boot/vmlinuz-4.19.0-16-amd64", "ro": true, "root": "UUID=f20ae82a-c15d-4de4-abe6-e6aaace55580", "quiet": true}, "ansible_memfree_mb": 5457, "ansible_processor_count": 4, "ansible_hostname": "chd-dev-debrepo01", "ansible_interfaces": ["lo", "ens160"], "ansible_selinux": {"status": "Missing selinux Python library"}, "ansible_fqdn": "chd-dev-debrepo01.companydomain.com", "ansible_mounts": [{"block_used": 0, "uuid": "N/A", "size_total": 0, "block_total": 0, "mount": "/debrepo", "block_available": 0, "size_available": 0, "fstype": "autofs", "inode_total": 0, "options": "rw,relatime,fd=6,pgrp=1345,timeout=60,minproto=5,maxproto=5,indirect,pipe_ino=13165092", "device": "/etc/auto.debrepo", "inode_used": 0, "block_size": 4096, "inode_available": 0}, {"block_used": 2788384, "uuid": "f20ae82a-c15d-4de4-abe6-e6aaace55580", "size_total": 145880190976, "block_total": 35615281, "mount": "/", "block_available": 32826897, "size_available": 134458970112, "fstype": "ext4", "inode_total": 9117696, "options": "rw,relatime,errors=remount-ro", "device": "/dev/sda1", "inode_used": 229400, "block_size": 4096, "inode_available": 8888296}], "ansible_nodename": "chd-dev-debrepo01", "ansible_lvm": {"pvs": {}, "lvs": {}, "vgs": {}}, "ansible_domain": "companydomain.com", "ansible_distribution_file_path": "/etc/os-release", "ansible_virtualization_type": "VMware", "ansible_ssh_host_key_ed25519_public": "AAAAC3NzaC1lZDI1NTE5AAAAIDgWPs5UoHo+/cRpv/wVuJc/cxtJ5hGSqLdSzsOGAm94", "ansible_processor_cores": 1, "ansible_bios_version": "6.00", "ansible_date_time": {"weekday_number": "4", "iso8601_basic_short": "20211118T044346", "tz": "CST", "weeknumber": "46", "hour": "04", "year": "2021", "minute": "43", "tz_offset": "-0600", "month": "11", "epoch": "1637232226", "iso8601_micro": "2021-11-18T10:43:46.070383Z", "weekday": "Thursday", "time": "04:43:46", "date": "2021-11-18", "iso8601": "2021-11-18T10:43:46Z", "day": "18", "iso8601_basic": "20211118T044346070383", "second": "46"}, "ansible_distribution_release": "buster", "ansible_os_family": "Debian", "ansible_effective_user_id": 0, "ansible_system": "Linux", "ansible_devices": {"sr0": {"scheduler_mode": "mq-deadline", "rotational": "1", "vendor": "NECVMWar", "sectors": "2097151", "links": {"masters": [], "labels": [], "ids": ["ata-VMware_Virtual_SATA_CDRW_Drive_00000000000000000001"], "uuids": []}, "sas_device_handle": null, "sas_address": null, "virtual": 1, "host": "SATA controller: VMware SATA AHCI controller", "sectorsize": "512", "removable": "1", "support_discard": "0", "model": "VMware SATA CD00", "partitions": {}, "holders": [], "size": "1024.00 MB"}, "sda": {"scheduler_mode": "mq-deadline", "rotational": "1", "vendor": "VMware", "sectors": "293601280", "links": {"masters": [], "labels": [], "ids": [], "uuids": []}, "sas_device_handle": null, "sas_address": null, "virtual": 1, "host": "SCSI storage controller: Broadcom / LSI 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 01)", "sectorsize": "512", "removable": "0", "support_discard": "0", "model": "Virtual disk", "partitions": {"sda5": {"sectorsize": 512, "uuid": "219da1de-11ba-4eb0-9076-c6458326f152", "links": {"masters": [], "labels": [], "ids": [], "uuids": ["219da1de-11ba-4eb0-9076-c6458326f152"]}, "sectors": "1996800", "start": "291602432", "holders": [], "size": "975.00 MB"}, "sda2": {"sectorsize": 512, "uuid": null, "links": {"masters": [], "labels": [], "ids": [], "uuids": []}, "sectors": "2", "start": "291602430", "holders": [], "size": "1.00 KB"}, "sda1": {"sectorsize": 512, "uuid": "f20ae82a-c15d-4de4-abe6-e6aaace55580", "links": {"masters": [], "labels": [], "ids": [], "uuids": ["f20ae82a-c15d-4de4-abe6-e6aaace55580"]}, "sectors": "291598336", "start": "2048", "holders": [], "size": "139.04 GB"}}, "holders": [], "size": "140.00 GB"}}, "ansible_user_uid": 0, "ansible_bios_date": "09/21/2015", "ansible_system_capabilities": ["cap_chown", "cap_dac_override", "cap_dac_read_search", "cap_fowner", "cap_fsetid", "cap_kill", "cap_setgid", "cap_setuid", "cap_setpcap", "cap_linux_immutable", "cap_net_bind_service", "cap_net_broadcast", "cap_net_admin", "cap_net_raw", "cap_ipc_lock", "cap_ipc_owner", "cap_sys_module", "cap_sys_rawio", "cap_sys_chroot", "cap_sys_ptrace", "cap_sys_pacct", "cap_sys_admin", "cap_sys_boot", "cap_sys_nice", "cap_sys_resource", "cap_sys_time", "cap_sys_tty_config", "cap_mknod", "cap_lease", "cap_audit_write", "cap_audit_control", "cap_setfcap", "cap_mac_override", "cap_mac_admin", "cap_syslog", "cap_wake_alarm", "cap_block_suspend", "cap_audit_read+ep"], "ansible_ens160": {"macaddress": "00:50:56:8b:8e:c9", "features": {"tx_checksum_ipv4": "off [fixed]", "generic_receive_offload": "on", "tx_checksum_ipv6": "off [fixed]", "tx_scatter_gather_fraglist": "off [fixed]", "rx_all": "off [fixed]", "highdma": "on", "tls_hw_rx_offload": "off [fixed]", "tx_lockless": "off [fixed]", "tx_tcp_ecn_segmentation": "off [fixed]", "rx_udp_tunnel_port_offload": "off [fixed]", "esp_hw_offload": "off [fixed]", "tx_gso_robust": "off [fixed]", "tx_udp_segmentation": "off [fixed]", "tx_tcp_mangleid_segmentation": "off", "tx_ipxip4_segmentation": "off [fixed]", "tx_checksumming": "on", "vlan_challenged": "off [fixed]", "loopback": "off [fixed]", "tx_ipxip6_segmentation": "off [fixed]", "fcoe_mtu": "off [fixed]", "scatter_gather": "on", "tx_checksum_sctp": "off [fixed]", "tx_vlan_stag_hw_insert": "off [fixed]", "rx_vlan_stag_hw_parse": "off [fixed]", "tx_gso_partial": "off [fixed]", "tx_tcp_segmentation": "on", "rx_vlan_stag_filter": "off [fixed]", "large_receive_offload": "on", "tx_scatter_gather": "on", "tx_tcp6_segmentation": "on", "rx_checksumming": "on", "rx_gro_hw": "off [fixed]", "netns_local": "off [fixed]", "tx_esp_segmentation": "off [fixed]", "generic_segmentation_offload": "on", "rx_fcs": "off [fixed]", "tcp_segmentation_offload": "on", "l2_fwd_offload": "off [fixed]", "rx_vlan_offload": "on", "ntuple_filters": "off [fixed]", "tx_gre_csum_segmentation": "off [fixed]", "esp_tx_csum_hw_offload": "off [fixed]", "tx_udp_tnl_segmentation": "off [fixed]", "tx_udp_tnl_csum_segmentation": "off [fixed]", "tx_sctp_segmentation": "off [fixed]", "udp_fragmentation_offload": "off", "tls_hw_record": "off [fixed]", "tls_hw_tx_offload": "off [fixed]", "tx_checksum_fcoe_crc": "off [fixed]", "hw_tc_offload": "off [fixed]", "tx_checksum_ip_generic": "on", "tx_fcoe_segmentation": "off [fixed]", "tx_nocache_copy": "off", "rx_vlan_filter": "on [fixed]", "tx_vlan_offload": "on", "receive_hashing": "on", "tx_gre_segmentation": "off [fixed]"}, "type": "ether", "pciid": "0000:03:00.0", "module": "vmxnet3", "mtu": 1500, "device": "ens160", "promisc": false, "timestamping": ["rx_software", "software"], "ipv4": {"broadcast": "10.9.63.255", "netmask": "255.255.255.0", "network": "10.9.63.0", "address": "10.9.63.204"}, "active": true, "speed": 10000, "hw_timestamp_filters": []}}}\r\n', 'Connection to chd-dev-debrepo01 closed.\r\n')
<chd-dev-debrepo01> ESTABLISH SSH CONNECTION FOR USER: root
<chd-dev-debrepo01> SSH: EXEC ssh -o ProxyCommand=none -o StrictHostKeyChecking=no -o 'IdentityFile="/usr/share/foreman/.ssh/id_rsa_foreman_proxy"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 chd-dev-debrepo01 '/bin/sh -c '"'"'rm -f -r /root/.ansible/tmp/ansible-tmp-1637232224.88-21051-133782812663879/ > /dev/null 2>&1 && sleep 0'"'"''
<chd-dev-debrepo01> (0, '', '')
ok: [chd-dev-debrepo01]
500 Server Error: Internal Server Error
Callback Exception: 
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_queue_manager.py", line 381, in send_callback
    method(*new_args, **kwargs)
   File "/var/lib/foreman-proxy/ansible/plugins/callback_plugins/foreman_callback.py", line 245, in v2_runner_on_ok
    self.send_facts(host, res)
   File "/var/lib/foreman-proxy/ansible/plugins/callback_plugins/foreman_callback.py", line 101, in send_facts
    if re.search(r'^junos', data['invocation']['module_name']):

TASK [debug] *****************************************************************************************************************************************************************************************************************************************************************************************************************
task path: /root/facts.yml:15
ok: [chd-dev-debrepo01] => {
    "out": {
        "ansible_facts": {
            "ansible_all_ipv4_addresses": [
                "10.9.63.204"
            ], 
            "ansible_all_ipv6_addresses": [], 
            "ansible_apparmor": {
                "status": "enabled"
            }, 
            "ansible_architecture": "x86_64", 
            "ansible_bios_date": "09/21/2015", 
            "ansible_bios_version": "6.00", 
            "ansible_cmdline": {
                "BOOT_IMAGE": "/boot/vmlinuz-4.19.0-16-amd64", 
                "quiet": true, 
                "ro": true, 
                "root": "UUID=f20ae82a-c15d-4de4-abe6-e6aaace55580"
            }, 
            "ansible_date_time": {
                "date": "2021-11-18", 
                "day": "18", 
                "epoch": "1637232226", 
                "hour": "04", 
                "iso8601": "2021-11-18T10:43:46Z", 
                "iso8601_basic": "20211118T044346070383", 
                "iso8601_basic_short": "20211118T044346", 
                "iso8601_micro": "2021-11-18T10:43:46.070383Z", 
                "minute": "43", 
                "month": "11", 
                "second": "46", 
                "time": "04:43:46", 
                "tz": "CST", 
                "tz_offset": "-0600", 
                "weekday": "Thursday", 
                "weekday_number": "4", 
                "weeknumber": "46", 
                "year": "2021"
            }, 
            "ansible_default_ipv4": {
                "address": "10.9.63.204", 
                "alias": "ens160", 
                "broadcast": "10.9.63.255", 
                "gateway": "10.9.63.1", 
                "interface": "ens160", 
                "macaddress": "00:50:56:8b:8e:c9", 
                "mtu": 1500, 
                "netmask": "255.255.255.0", 
                "network": "10.9.63.0", 
                "type": "ether"
            }, 
            "ansible_system_capabilities_enforced": "True", 
            "ansible_system_vendor": "VMware, Inc.", 
            "ansible_uptime_seconds": 8466398, 
            "ansible_user_dir": "/root", 
            "ansible_user_gecos": "root", 
            "ansible_user_gid": 0, 
            "ansible_user_id": "root", 
            "ansible_user_shell": "/bin/bash", 
            "ansible_user_uid": 0, 
            "ansible_userspace_architecture": "x86_64", 
            "ansible_userspace_bits": "64", 
            "ansible_virtualization_role": "guest", 
            "ansible_virtualization_type": "VMware", 
            "gather_subset": [
                "all"
            ], 
            "module_setup": true
        }, 
        "changed": false, 
        "failed": false
    }
}
META: ran handlers
META: ran handlers

PLAY RECAP *******************************************************************************************************************************************************************************************************************************************************************************************************************
chd-dev-debrepo01          : ok=4    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0      

It has been solved. I found out we were using old foreman_callback.py file https://github.com/theforeman/foreman_ansible/commit/321bd8453ab0ffab6818ea487cf21fba5f3e0be5
instead of a new one from foreman-ansible-modules repository https://github.com/theforeman/foreman-ansible-modules/blob/develop/plugins/callback/foreman.py

Also, 500 Server Error: Internal Server Error was caused by using hostname instead of fqdn.

1 Like