Problem:
When executing Build Ansible Roles the execution just stops with an unreadable error message:
1645: TASK [Apply roles] *************************************************************
1646:
1647: TA
1648: Exit status: 0
The report shows that something failed, but no error message or detailing WHAT failed is shown.
Expected outcome:
Report should contain all messages printed by ansible. Output in console window of execution should show unscrambled output
Foreman and Proxy versions:
3.1.0
Foreman and Proxy plugin versions:
foreman-tasks 5.2.0
foreman_ansible 7.0.1
foreman_discovery 19.0.1
foreman_puppet 2.0.0
foreman_remote_execution 5.0.1
Distribution and version:
Rocky Linux 8.5
Other relevant data:
ansible [core 2.12.1]
config file = /etc/foreman-proxy/ansible.cfg
configured module search path = [’/root/.ansible/plugins/modules’, ‘/usr/share/ansible/plugins/modules’]
ansible python module location = /usr/local/lib/python3.8/site-packages/ansible
ansible collection location = /etc/ansible/collections:/usr/share/ansible/collections
executable location = /usr/local/bin/ansible
python version = 3.8.8 (default, Nov 9 2021, 13:31:34) [GCC 8.5.0 20210514 (Red Hat 8.5.0-3)]
jinja version = 3.0.3
libyaml = True
When debugging foreman-proxy there’s the same scrambled output in the broadcast:
2022-01-25T10:04:58 [D] [foreman_ansible] - handling event broadcast: {
"uuid": "00155d00-1443-5282-c8f4-00000000002b",
"counter": 17,
"stdout": "\r\nTA",
"start_line": 1645,
"end_line": 1646,
"runner_ident": "f48f5a15-2999-48ec-8059-248f40d301dd",
"event": "playbook_on_task_start",
"pid": 19003,
"created": "2022-01-25T10:04:36.330516",
"parent_uuid": "00155d00-1443-5282-c8f4-000000000006",
"event_data": {
"playbook": "playbook.yml",
"playbook_uuid": "a747230f-d235-4019-84d5-249b8a69fbb1",
"play": "all",
"play_uuid": "00155d00-1443-5282-c8f4-000000000006",
"play_pattern": "all",
"task": "Setup iptables/interfaces files",
"task_uuid": "00155d00-1443-5282-c8f4-00000000002b",
"task_action": "set_fact",
"task_args": "",
"task_path": "/data/ansible/roles/kubernetes/tasks/main.yml:6",
"role": "kubernetes",
"name": "kubernetes : Setup iptables/interfaces files",
"is_conditional": false,
"uuid": "00155d00-1443-5282-c8f4-00000000002b"
}
}
Could you please set the setting Setting > Remote Execution > Cleanup working directories to No and run the job again?
Then you should have the working directory of ansible run on the smart-proxy and under artifacts/<uuid>/job_events you can see an raw output of the job. It might give us little bit more info if we’ve just read the output wrong, or if there is something wrong with the ansible run itself 
Hi,
during my test run the scrambled output didn’t happen, but I found the missing error output in a -partial.json file.
The partial file is a fully valid json, and has the event “event”: “runner_on_failed”
Not sure where to go from here … should the error message also be display in the Task Runner output? (I would definitly prefer that)
I might got a hint why it’s happening.
I removed the default foreman output task and now the error message is actually displayed!
So maybe there’s too many lines, or the large json (output was around 1500 lines before) output breaks a logic somewhere?
Ansible generates the artifacts files as -partial and then atomically moves them to become the standard, non-partial, files that are guaranteed to be valid JSONs. If ansible dies before it finalizes an artifact we can’t really do much about it.
What do you mean by removing the default foreman output task?
That feels unlikely, 1500 lines isn’t that much. I recall having much larger outputs.
Can you reproduce this reliably? Could you summarise the steps for us so we could reproduce it locally?
I seem to not be able to reproduce it. It consistently happens with my kubernetes role, but if I pull out the parts that actually fail it shows the message correctly.
What I have also noticed is, that the output of large amounts of text (i.e . debug: var: foreman) take super long time, 2 - 3 minutes. You can actually follow the stdout file and see it’s super slow printing the json.
