Katello/Foreman Ansible "Run Ansible roles" no longer executing playbooks

Support
, , ,
1

Problem: After upgrading from 1.24.0 to 1.24.2 Katello/Foreman Ansible no longer executes the playbook as expected.

Expected outcome:. Get hosts facts and execute playbook against the host

Foreman and Proxy versions: 1.24.2

Foreman and Proxy plugin versions: 1.24 Katello: 3.14.1

Distribution and version:

image
image1232×986 147 KB

Other relevant data:

CentOS 7.7.1908

Foreman prod logs

2020-03-16T10:42:35 [I|app|5bd6e225] Completed 200 OK in 26ms (Views: 0.2ms | ActiveRecord: 3.0ms)
2020-03-16T10:42:35 [I|app|4a8f1444] Started POST "/foreman_tasks/api/tasks/callback" for 10.9.64.196 at 2020-03-16 10:42:35 -0500
2020-03-16T10:42:35 [I|app|4a8f1444] Processing by ForemanTasks::Api::TasksController#callback as */*
2020-03-16T10:42:35 [I|app|4a8f1444]   Parameters: {"callbacks"=>[{"callback"=>{"task_id"=>"c57b9a2d-86d2-4561-ba25-18d52418946b", "step_id"=>3}, "data"=>{"result"=>[{"output_type"=>"stdout", "output"=>"[WARNING]:  * Failed to parse /tmp/d20200316-24411-1rj3tcl/inventory/hosts with\n", "timestamp"=>1584373346.8121903}, {"output_type"=>"stdout", "output"=>"script plugin: failed to parse executable inventory script results from\n", "timestamp"=>1584373346.812233}, {"output_type"=>"stdout", "output"=>"/tmp/d20200316-24411-1rj3tcl/inventory/hosts: Syntax Error while loading YAML.\n", "timestamp"=>1584373346.812282}, {"output_type"=>"stdout", "output"=>"found unknown escape character  The error appears to be in '<string>': line 1,\n", "timestamp"=>1584373346.8123298}, {"output_type"=>"stdout", "output"=>"column 2839, but may be elsewhere in the file depending on the exact syntax\n", "timestamp"=>1584373346.8123689}, {"output_type"=>"stdout", "output"=>"problem.\n", "timestamp"=>1584373346.812404}, {"output_type"=>"stdout", "output"=>"[WARNING]:  * Failed to parse /tmp/d20200316-24411-1rj3tcl/inventory/hosts with\n", "timestamp"=>1584373346.8124557}, {"output_type"=>"stdout", "output"=>"yaml plugin: YAML inventory has invalid structure, it should be a dictionary,\n", "timestamp"=>1584373346.8124921}, {"output_type"=>"stdout", "output"=>"got: <class 'ansible.parsing.yaml.objects.AnsibleUnicode'>\n", "timestamp"=>1584373346.8125274}, {"output_type"=>"stdout", "output"=>"[WARNING]:  * Failed to parse /tmp/d20200316-24411-1rj3tcl/inventory/hosts with\n", "timestamp"=>1584373346.8125622}, {"output_type"=>"stdout", "output"=>"ini plugin: /tmp/d20200316-24411-1rj3tcl/inventory/hosts:2: Expected key=value\n", "timestamp"=>1584373346.8125975}, {"output_type"=>"stdout", "output"=>"host variable assignment, got: <<-EOS\n", "timestamp"=>1584373346.8126442}, {"output_type"=>"stdout", "output"=>"[WARNING]: Unable to parse /tmp/d20200316-24411-1rj3tcl/inventory/hosts as an\n", "timestamp"=>1584373346.8126807}, {"output_type"=>"stdout", "output"=>"inventory source\n", "timestamp"=>1584373346.8127146}, {"output_type"=>"stdout", "output"=>"[WARNING]: Unable to parse /tmp/d20200316-24411-1rj3tcl/inventory as an\n", "timestamp"=>1584373346.8127491}, {"output_type"=>"stdout", "output"=>"inventory source\n", "timestamp"=>1584373346.812805}, {"output_type"=>"stdout", "output"=>"[WARNING]: No inventory was parsed, only implicit localhost is available\n", "timestamp"=>1584373346.8128407}, {"output_type"=>"stdout", "output"=>"[WARNING]: provided hosts list is empty, only localhost is available. Note that\n", "timestamp"=>1584373346.8128757}, {"output_type"=>"stdout", "output"=>"the implicit localhost does not match 'all'\n", "timestamp"=>1584373346.81291}, {"output_type"=>"stdout", "output"=>"\n", "timestamp"=>1584373347.815868}, {"output_type"=>"stdout", "output"=>"\r\nPLAY [all] *********************************************************************\n", "timestamp"=>1584373347.8159304}, {"output_type"=>"stdout", "output"=>"skipping: no hosts matched\n", "timestamp"=>1584373347.8159819}, {"output_type"=>"stdout", "output"=>"PLAY RECAP *********************************************************************\n", "timestamp"=>1584373347.8160594}], "exit_status"=>0}}], "task"=>{}}
2

Hi,
does the playbook by any chance contain some special characters (something like \s) or so?

3

Hi,
None at all. Here is an example of one of the many job template playbooks

---
- hosts: all
  tasks:
    - name: Collect only facts returned by facter
      setup:
        gather_subset:
          - '!all'
          - '!any'
          - facter
      register: out
    - debug: var=out

Output from Job Invocations:

[WARNING]: * Failed to parse /tmp/d20200317-3347-xq7oxf/inventory/hosts with
script plugin: failed to parse executable inventory script results from
/tmp/d20200317-3347-xq7oxf/inventory/hosts: Syntax Error while loading YAML.
found unknown escape character The error appears to be in ‘’: line 1,
column 2839, but may be elsewhere in the file depending on the exact syntax
problem.
[WARNING]: * Failed to parse /tmp/d20200317-3347-xq7oxf/inventory/hosts with
yaml plugin: YAML inventory has invalid structure, it should be a dictionary,
got: <class ‘ansible.parsing.yaml.objects.AnsibleUnicode’>
[WARNING]: * Failed to parse /tmp/d20200317-3347-xq7oxf/inventory/hosts with
ini plugin: /tmp/d20200317-3347-xq7oxf/inventory/hosts:2: Expected key=value
host variable assignment, got: <<-EOS
[WARNING]: Unable to parse /tmp/d20200317-3347-xq7oxf/inventory/hosts as an
inventory source
[WARNING]: Unable to parse /tmp/d20200317-3347-xq7oxf/inventory as an inventory
source
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that
the implicit localhost does not match ‘all’
PLAY [all] *********************************************************************
skipping: no hosts matched
PLAY RECAP *********************************************************************
Exit status: 0

Here is the ansible.log example error from running anisble-inventory against any of generate foreman /tmp files

ansible-inventory -i ‘/tmp/d20200316-24411-1pfypxs/inventory/hosts’ --list --yaml

2020-03-16 18:45:11,814 p=12569 u=root n=ansible | [WARNING]: * Failed to parse /tmp/d20200316-24411-1pfypxs/inventory/hosts with script plugin: failed to parse executable inventory script results from /tmp/d20200316-24411-1pfypxs/inventory/hosts: Syntax Error while loading YAML. found unknown escape character The error appears to be in ‘’: line 1,
column 2839, but may be elsewhere in the file depending on the exact syntax problem.

2020-03-16 18:45:11,814 p=12569 u=root n=ansible | [WARNING]: * Failed to parse /tmp/d20200316-24411-1pfypxs/inventory/hosts with yaml plugin: YAML inventory has invalid structure, it should be a dictionary, got: <class ‘ansible.parsing.yaml.objects.AnsibleUnicode’>

2020-03-16 18:45:11,815 p=12569 u=root n=ansible | [WARNING]: * Failed to parse /tmp/d20200316-24411-1pfypxs/inventory/hosts with constructed plugin: ‘AnsibleUnicode’ object has no attribute ‘get’

2020-03-16 18:45:11,815 p=12569 u=root n=ansible | [WARNING]: Unable to parse /tmp/d20200316-24411-1pfypxs/inventory/hosts as an inventory source

2020-03-16 18:45:11,815 p=12569 u=root n=ansible | [WARNING]: No inventory was parsed, only implicit localhost is available

2020-03-16 18:45:11,816 p=12569 u=root n=ansible | all:
children:
ungrouped: {}

4

Also using ansible-inventory to debug before the upgrade foreman generated a script which is parsed correctly

ansible-inventory -i '/tmp/d20200301-3348-r5tza5/inventory/hosts' --list --yaml -vvvv
ansible-inventory 2.9.3
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /bin/ansible-inventory
  python version = 2.7.5 (default, Aug  7 2019, 00:51:29) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]
Using /etc/ansible/ansible.cfg as config file
setting up inventory plugins
Parsed /tmp/d20200301-3348-r5tza5/inventory/hosts inventory source with script plugin
all:
  children:
    ungrouped:
      hosts:

And now the script is not:

ansible-inventory -i '/tmp/d20200317-3347-xq7oxf/inventory/host'  --list --yaml -vvvv
ansible-inventory 2.9.3
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /bin/ansible-inventory
  python version = 2.7.5 (default, Aug  7 2019, 00:51:29) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]
Using /etc/ansible/ansible.cfg as config file
setting up inventory plugins
Skipping due to inventory source not existing or not being readable by the current user
script declined parsing /tmp/d20200317-3347-xq7oxf/inventory/host as it did not pass its verify_file() method
host_list declined parsing /tmp/d20200317-3347-xq7oxf/inventory/host as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
virtualbox declined parsing /tmp/d20200317-3347-xq7oxf/inventory/host as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
yaml declined parsing /tmp/d20200317-3347-xq7oxf/inventory/host as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
constructed declined parsing /tmp/d20200317-3347-xq7oxf/inventory/host as it did not pass its verify_file() method
[WARNING]: Unable to parse /tmp/d20200317-3347-xq7oxf/inventory/host as an inventory source

[WARNING]: No inventory was parsed, only implicit localhost is available
5

What version of tfm-rubygem-foreman_ansible_core do you have? If you’re hitting what I think you are, then it should be fixed in 3.0.3 which should be already available in 1.24 repos.

6

tfm-rubygem-foreman_ansible_core 3.0.2 fm24

7

What issue am I encountering in 1.24.2?

8

Try updating to 3.0.3 and restarting the smart_proxy_dynflow_core service afterwards.

What issue am I encountering in 1.24.2?

It feels like this Bug #29336: When variable (or template input) contains \s or other non-string formatting characters, execution fails - Ansible - Foreman . The error is almost exactly the same as what I was seeing when fixing that issue, so it may be a good start.