Ansible create host failing to connect

Asumble · April 1, 2021, 12:27pm

Problem:

Ansible fails to connect to create the host

“msg”: "Failed to connect to Foreman server: StopIteration: "

Expected outcome:

Host should get created in Foreman

Foreman and Proxy versions:

1.16.0
1.18.1

Foreman and Proxy plugin versions:

ansible 2.9.17
config file = /etc/ansible/ansible.cfg
configured module search path = [u’/usr/local/ansible/modules/ansible-module-foreman’]
ansible python module location = /usr/lib/python2.7/site-packages/ansible
executable location = /usr/bin/ansible
python version = 2.7.5 (default, Nov 16 2020, 22:23:17) [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)]

Distribution and version:

Other relevant data:

[root@nick ansible]# ansible-playbook -i inventory/AKL/hosts foremancreatehost.yml --limit ‘testnick2.nic.internal’ -vvvvvv
ansible-playbook 2.9.17
config file = /etc/ansible/ansible.cfg
configured module search path = [u’/usr/local/ansible/modules/ansible-module-foreman’]
ansible python module location = /usr/lib/python2.7/site-packages/ansible
executable location = /usr/bin/ansible-playbook
python version = 2.7.5 (default, Nov 16 2020, 22:23:17) [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)]
Using /etc/ansible/ansible.cfg as config file
setting up inventory plugins
host_list declined parsing /etc/ansible/inventory/AKL/hosts as it did not pass its verify_file() method
script declined parsing /etc/ansible/inventory/AKL/hosts as it did not pass its verify_file() method
auto declined parsing /etc/ansible/inventory/AKL/hosts as it did not pass its verify_file() method
Parsed /etc/ansible/inventory/AKL/hosts inventory source with ini plugin
Loading callback plugin default of type stdout, v2.0 from /usr/lib/python2.7/site-packages/ansible/plugins/callback/default.pyc
Attempting to use ‘actionable’ callback.
Skipping callback ‘actionable’, as we already have a stdout callback.
Attempting to use ‘aws_resource_actions’ callback.
Attempting to use ‘cgroup_memory_recap’ callback.
Attempting to use ‘cgroup_perf_recap’ callback.
Attempting to use ‘context_demo’ callback.
Attempting to use ‘counter_enabled’ callback.
Skipping callback ‘counter_enabled’, as we already have a stdout callback.
Attempting to use ‘debug’ callback.
Skipping callback ‘debug’, as we already have a stdout callback.
Attempting to use ‘dense’ callback.
Skipping callback ‘dense’, as we already have a stdout callback.
Attempting to use ‘dense’ callback.
Skipping callback ‘dense’, as we already have a stdout callback.
Attempting to use ‘foreman’ callback.
Attempting to use ‘full_skip’ callback.
Skipping callback ‘full_skip’, as we already have a stdout callback.
Attempting to use ‘grafana_annotations’ callback.
Attempting to use ‘hipchat’ callback.
Attempting to use ‘jabber’ callback.
Attempting to use ‘json’ callback.
Skipping callback ‘json’, as we already have a stdout callback.
Attempting to use ‘junit’ callback.
Attempting to use ‘log_plays’ callback.
Attempting to use ‘logdna’ callback.
Attempting to use ‘logentries’ callback.
Attempting to use ‘logstash’ callback.
Attempting to use ‘mail’ callback.
Attempting to use ‘minimal’ callback.
Skipping callback ‘minimal’, as we already have a stdout callback.
Attempting to use ‘nrdp’ callback.
Attempting to use ‘null’ callback.
Skipping callback ‘null’, as we already have a stdout callback.
Attempting to use ‘oneline’ callback.
Skipping callback ‘oneline’, as we already have a stdout callback.
Attempting to use ‘osx_say’ callback.
Attempting to use ‘profile_roles’ callback.
Attempting to use ‘profile_tasks’ callback.
Attempting to use ‘say’ callback.
Attempting to use ‘selective’ callback.
Skipping callback ‘selective’, as we already have a stdout callback.
Attempting to use ‘skippy’ callback.
Skipping callback ‘skippy’, as we already have a stdout callback.
Attempting to use ‘slack’ callback.
Attempting to use ‘splunk’ callback.
Attempting to use ‘stderr’ callback.
Skipping callback ‘stderr’, as we already have a stdout callback.
Attempting to use ‘sumologic’ callback.
Attempting to use ‘syslog_json’ callback.
Attempting to use ‘timer’ callback.
Attempting to use ‘tree’ callback.
Attempting to use ‘unixy’ callback.
Skipping callback ‘unixy’, as we already have a stdout callback.
Attempting to use ‘yaml’ callback.
Skipping callback ‘yaml’, as we already have a stdout callback.

PLAYBOOK: foremancreatehost.yml ******************************************************************************************************************************************************
Positional arguments: foremancreatehost.yml
subset: testnick2.nic.internal
become_method: sudo
inventory: (u’/etc/ansible/inventory/AKL/hosts’,)
forks: 5
tags: (u’all’,)
verbosity: 6
connection: smart
timeout: 10
1 plays in foremancreatehost.yml

PLAY [all] ***************************************************************************************************************************************************************************
META: ran handlers

TASK [ansible-provision-foreman : Create a host] *************************************************************************************************************************************
task path: /etc/ansible/roles/ansible-provision-foreman/tasks/main.yml:1
ESTABLISH LOCAL CONNECTION FOR USER: root
EXEC /bin/sh -c ‘( umask 77 && mkdir -p “echo /tmp”&& mkdir “echo /tmp/ansible-tmp-1617279362.56-24332-23424257423845” && echo ansible-tmp-1617279362.56-24332-23424257423845="echo /tmp/ansible-tmp-1617279362.56-24332-23424257423845" ) && sleep 0’
Using module_utils file ansible_collections/theforeman/foreman/plugins/module_utils
Using module_utils file ansible_collections/theforeman
Using module_utils file ansible_collections/theforeman/foreman/plugins
Using module_utils file ansible_collections/theforeman/foreman
Using module_utils file ansible_collections
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/basic.py
Using module_utils file ansible_collections/theforeman/foreman/plugins/module_utils/foreman_helper
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/common/_utils.py
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/common/text/formatters.py
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/common/validation.py
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/common/text/converters.py
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/pycompat24.py
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/common/init.py
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/common/text/init.py
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/six/init.py
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/common/_json_compat.py
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/common/_collections_compat.py
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/common/parameters.py
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/parsing/convert_bool.py
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/parsing/init.py
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/compat/selectors.py
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/_text.py
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/common/sys_info.py
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/common/process.py
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/compat/init.py
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/common/file.py
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/common/collections.py
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/compat/_selectors2.py
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/distro/init.py
Using module_utils file /usr/lib/python2.7/site-packages/ansible/module_utils/distro/_distro.py
Using module_utils file ansible_collections/theforeman/foreman/plugins/module_utils/_apypie
Using module file /root/.ansible/collections/ansible_collections/theforeman/foreman/plugins/modules/host.py
PUT /root/.ansible/tmp/ansible-local-24323W8wmQu/tmp8KjSDP TO /tmp/ansible-tmp-1617279362.56-24332-23424257423845/AnsiballZ_host.py
EXEC /bin/sh -c ‘chmod u+x /tmp/ansible-tmp-1617279362.56-24332-23424257423845/ /tmp/ansible-tmp-1617279362.56-24332-23424257423845/AnsiballZ_host.py && sleep 0’
EXEC /bin/sh -c ‘/usr/bin/python2 /tmp/ansible-tmp-1617279362.56-24332-23424257423845/AnsiballZ_host.py && sleep 0’
EXEC /bin/sh -c ‘rm -f -r /tmp/ansible-tmp-1617279362.56-24332-23424257423845/ > /dev/null 2>&1 && sleep 0’
The full traceback is:
WARNING: The below traceback may not be related to the actual failure.
File “/tmp/ansible_theforeman.foreman.host_payload_l2D7TQ/ansible_theforeman.foreman.host_payload.zip/ansible_collections/theforeman/foreman/plugins/module_utils/foreman_helper.py”, line 93, in inner
return f(self, *args, **kwargs)
File “/tmp/ansible_theforeman.foreman.host_payload_l2D7TQ/ansible_theforeman.foreman.host_payload.zip/ansible_collections/theforeman/foreman/plugins/module_utils/foreman_helper.py”, line 617, in connect
self.apply_apidoc_patches()
File “/tmp/ansible_theforeman.foreman.host_payload_l2D7TQ/ansible_theforeman.foreman.host_payload.zip/ansible_collections/theforeman/foreman/plugins/module_utils/foreman_helper.py”, line 627, in apply_apidoc_patches
self._patch_host_update()
File “/tmp/ansible_theforeman.foreman.host_payload_l2D7TQ/ansible_theforeman.foreman.host_payload.zip/ansible_collections/theforeman/foreman/plugins/module_utils/foreman_helper.py”, line 398, in _patch_host_update
_host_update_taxonomy_param = next(x for x in _host_update[‘params’] if x[‘name’] == param)
fatal: [testnick2.nic.internal]: FAILED! => {
“changed”: false,
“invocation”: {
“module_args”: {
“activation_keys”: null,
“architecture”: “x86_64”,
“build”: true,
“comment”: null,
“compute_attributes”: {
“cpus”: 2,
“memory_mb”: 4096
},
“compute_profile”: “vmware”,
“compute_resource”: “vcenter.nic.internal”,
“config_groups”: null,
“content_source”: null,
“content_view”: null,
“domain”: null,
“enabled”: null,
“environment”: “tdr”,
“hostgroup”: “Base Sever/Centos 7.8.2003”,
“image”: null,
“interfaces_attributes”: [
{
“attached_devices”: null,
“attached_to”: null,
“bond_options”: null,
“compute_attributes”: {
“domain”: “tdr.internal”,
“interface”: “VMXNET3”,
“ip”: “192.168.10.11”,
“name”: “nic1”,
“network”: “home”,
“subnet”: “192.168.10.0”
},
“domain”: null,
“identifier”: null,
“ip”: null,
“ip6”: null,
“mac”: null,
“managed”: null,
“mode”: null,
“mtu”: null,
“name”: null,
“password”: null,
“primary”: true,
“provider”: null,
“provision”: true,
“subnet”: null,
“subnet6”: null,
“tag”: null,
“type”: “interface”,
“username”: null,
“virtual”: null
}
],
“ip”: null,
“kickstart_repository”: null,
“lifecycle_environment”: null,
“location”: null,
“mac”: null,
“managed”: true,
“medium”: null,
“name”: “testnick2.nic.internal”,
“openscap_proxy”: null,
“operatingsystem”: “Centos 7.8.2003”,
“organization”: null,
“owner”: “unix”,
“owner_group”: null,
“parameters”: null,
“password”: “VALUE_SPECIFIED_IN_NO_LOG_PARAMETER”,
“provision_method”: null,
“ptable”: “Centos - LVM - / , swap”,
“puppet_ca_proxy”: “puppet-2.nic.internal”,
“puppet_proxy”: “puppet-2.nic.internal”,
“puppetclasses”: null,
“pxe_loader”: “PXELinux BIOS”,
“realm”: null,
“root_pass”: “VALUE_SPECIFIED_IN_NO_LOG_PARAMETER”,
“server_url”: “http://foreman-1.test”,
“state”: “present”,
“subnet”: null,
“subnet6”: null,
“username”: “admin”,
“validate_certs”: false
}
},
“msg”: "Failed to connect to Foreman server: StopIteration: "
}

PLAY RECAP ***************************************************************************************************************************************************************************
testnick2.tdr.internal : ok=0 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0

Role Task.

name: “Create a host”
theforeman.foreman.host:
username: user
password: password
server_url: “{{ server_url }}”
name: testnick2.home.test
hostgroup: Base Sever/Centos 7.8.2003
managed: true
build: yes
compute_profile: “{{ compute_profile }}”
compute_resource: “{{ computer_resource }}”
compute_attributes:
cpus: 2
memory_mb: 4096
interfaces_attributes:
- type: “interface”
  primary: true
  compute_attributes:
  name: nic1
  network: home
  interface: “VMXNET3”
  subnet: 192.168.10.1
  ip: 192.168.10.12
  domain: “{{ domain }}”
  provision: yes
  operatingsystem: Centos 7.8.2003
  architecture: x86_64
  pxe_loader: PXELinux BIOS
  puppet_ca_proxy: {{ puppet_ca_proxy }}
  puppet_proxy: {{ puppet_proxy }}
  root_pass: whatev3er
  environment: test
  ptable: Centos - LVM - / , swap
  owner: god
  state: present
  validate_certs: false
  delegate_to: localhost

Anyone know why its failing to connect?

Nick Tailor
nicktailor.com

evgeni · April 1, 2021, 1:33pm

I think I know what is failing here, and could produce a fix/patch for that.

But at the same time: Foreman 1.16 is really old and not supported – and not tested with our Ansible Modules, so there might be more dragons and those might be harder to fix.

evgeni · April 1, 2021, 1:52pm

Is your setup running with Organizations/Locations disabled?

evgeni · April 1, 2021, 1:57pm

You can try the following patch, but other things might still be broken:

diff --git plugins/module_utils/foreman_helper.py plugins/module_utils/foreman_helper.py
index 432c76df..c9a3abda 100644
--- plugins/module_utils/foreman_helper.py
+++ plugins/module_utils/foreman_helper.py
@@ -396,8 +396,9 @@ class ForemanAnsibleModule(AnsibleModule):
 
         _host_update = next(x for x in _host_methods if x['name'] == 'update')
         for param in ['location_id', 'organization_id']:
-            _host_update_taxonomy_param = next(x for x in _host_update['params'] if x['name'] == param)
-            _host_update['params'].remove(_host_update_taxonomy_param)
+            _host_update_taxonomy_param = next((x for x in _host_update['params'] if x['name'] == param), None)
+            if _host_update_taxonomy_param is not None:
+                _host_update['params'].remove(_host_update_taxonomy_param)
 
     @_check_patch_needed(fixed_version='2.0.0')
     def _patch_templates_resource_name(self):

Asumble · April 1, 2021, 2:00pm

Yes I believe it is.

The error i get is

403 Client Error: Forbidden for url: http://foreman-1.nic.corp-apps.com/api/v2/reports

How ever my curl command with credentials works.

I did not install the foreman proxy plugin on the foreman side.

Do i need this?

evgeni · April 1, 2021, 2:02pm

the host module doesn’t hit the reports endpoint, only the callback does, which is unrelated to the error you post above.

and the above error should go away with enabling org/loc (those are mandatory in later foreman releases anyways) or trying my untested patch

Asumble · April 1, 2021, 2:03pm

Sorry my foreman version is 1.19 i copied the smart proxy stuff.

my bad. If you could help that would be awesome.

Cheers

Asumble · April 1, 2021, 2:05pm

Im happy to try the patch? Is the patch on the ansible module side or the foreman side?

Cheers

evgeni · April 1, 2021, 2:06pm

Ansible module

Asumble · April 1, 2021, 2:07pm

Ahh i see it okay.

Let me see if i can figure out how to add that patch…

Give me a few minutes.

Thanks for your help

Asumble · April 1, 2021, 2:16pm

Okay patch added and got a new error

        "username": "admin", 
        "validate_certs": false
    }
}, 
"msg": "Found no results while searching for hostgroups with title=\"Base Sever/Centos 7.8.2003\""

}

PLAY RECAP ***************************************************************************************************************************************************************************
testnick2.nic.internal : ok=0 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0

403 Client Error: Forbidden for url: http://foreman-1.nic.corp-apps.com/api/v2/reports

Asumble · April 1, 2021, 2:24pm

Sweet so that means its connecting now right?

You rock. What did we exactly fix there?

Thank you so much. Now i gotta figure out why the hostgroups are not showing up.

Cheers

Nick Tailor

evgeni · April 1, 2021, 2:30pm

Yeah, it connects now and doesn’t find your hostgroup, probably because it looks for a title, but you have spaces in there (which titles usually don’t).

The fix was to avoid trying to touch a specific resource that is only available when you have Org/Loc enabled.

Asumble · April 1, 2021, 2:33pm

Sweet! one step closer. Thank you so much man!!! I’v been racking my head with this for a few nights now. I will see i can get it find the hostgroup title and!..hopefully create a host in the foreman using ansible. Then my life will be complete…hhaha. I hope i can buy you a beer some day.

Cheers

Nick Tailor.

Asumble · April 1, 2021, 2:35pm

I still get that call back error. But im guessing that isnt a show stopper?
Cool blog btw.

Cheers

evgeni · April 1, 2021, 2:37pm

Yeah, the reports endpoint doesn’t exist in your version, and if the callback is failing the only thing you are missing is the nice report inside of foreman.

The actual work still has been done.

Asumble · April 2, 2021, 12:19pm

Hey, just wanted to say thank you. I was able to get the host creation to work. I will be writing a how to on my blog nicktailor.com for everyone since there is no real good documentation online.

Theres are just 2 issues…im trying to find a way to solve

Is there anyway to define the compute profile resources via ansible yaml?

disksize & datastore to use?

        "compute_attributes": {
            "cpus": 2, 
            "datastore": "esx_04", 
            "memory_mb": 4096, 
            "size": "50GB"
        },

Doesnt seem like i can. If it could do that I would be in heaven.

Everything else seems to work great.

Cheers

Nick Tailor

evgeni · April 6, 2021, 9:16am

Sure, there is a compute_attributes parameter to the host module, so yours above should be something like:

- name: deploy test01.example.com
  theforeman.foreman.host:
    name: test01.example.com
    …
    compute_attributes:
      cpus: 2 
      datastore: "esx_04"
      memory_mb: 4096
      size: 50GB

But I think the correct syntax would be using volumes_attributes subkey, resulting in:

- name: deploy test01.example.com
  theforeman.foreman.host:
    name: test01.example.com
    …
    compute_attributes:
      cpus: 2 
      memory_mb: 4096
      volumes_attributes:
        0:
          size_gb: 50
          datastore: "esx_04"

angry_yak_shaver · March 6, 2023, 3:59am

Hey @evgeni any plans to upstream that check? I have it committed here if you want to pull it:

angry_yak_shaver · March 6, 2023, 4:02am

(tried to file a PR but I couldn’t find my way in the project)

Show me so I can help with other things - we use Foreman a lot at work and I believe we can help.