Is there a way to see the output from each host, in one place, from a job?
So instead of having to click on each one in the list, we want to be able to see the output from all of them, in one place. I’ve tried generating a report but that doesnt show much either.
Report:
We’ll be running jobs against 90,000 hosts in some cases…
As far as I know not in the ui. We’re currently working on a redesign that would, among other things, allow you to view outputs of multiple hosts at once, but I don’t think it will be feasible to watch 90k hosts at once even when the redesign is done.
However, there is the /api/job_invocations/:id/outputs api, which allows you to get outputs of multiple (even all) hosts within a job. So either this or the report is probably your best bet. You ca build your own on top of the api or change the template yourself, depending on what you exactly want to see.
Ok thanks. Sounds awesome.
Its the number 1 thing all our users ask for. Since rarely do we run on one host at a time. Even when you run a job on 5 hosts, the combined output would be nice.
Ya, for 90k hosts it might be a bit much, but lots of times we are simply trying to get a single salt grain back from them, and could easily jq/grep through a unified output to get what we want.
Thanks for the info about the endpoint. Ill see what I can do with it.
Side note, with Broadcom buying out vmware, and increasing the costs of SaltStack UI (now VMware Aria) have gone up 10x. I think you’ll be seeing an increase in people looking for better/other options. Our cost savings where I am, moving from SaltStack to Foreman is over $7M. So, thank you for all you do.
@aruzicka Are you saying that you think the job invocation report could be changed to show the output from each job? The documentation is a bit lacking, and Im trying to figure out where to start. Would the job invocation template actually be able to get the template invocation id for each jid? Then spit out the standard error/out, etc?
I see in the help section: sub_task_for_host template_invocations_hosts for JobInvocation. So it seems like its possible, but Im having a heck of a time figuring it out.
Here is what I have so far:
name: Job - Detailed Invocation Report
snippet: false
template_inputs:
- name: job_id
required: true
input_type: user
description: Id of job invocation to report
advanced: false
value_type: plain
resource_type: JobInvocation
hidden_value: false
- name: hosts
required: false
input_type: user
description: Field for filter hosts of job invocation. Leave blank for all hosts.
advanced: false
value_type: search
resource_type: Host
hidden_value: false
model: ReportTemplate
require:
- plugin: foreman_remote_execution
version: 4.4.0
-%>
<%- report_headers 'Host', 'Task ID', 'Action', 'stdout', 'stderr', 'debug', 'Result', 'Finished' -%>
<%- invocation = find_job_invocation_by_id(input('job_id')) -%>
<%- parts = ["job_invocation.id = #{input('job_id')}"] -%>
<%- parts << input('hosts') unless input('hosts').blank? -%>
<%- search = parts.map { |part| '(' + part + ')' }.join(' AND ') -%>
<%- load_hosts(search: search).each do |batch| -%>
<%- batch.each do |host| -%>
<%- task = invocation.sub_task_for_host(host) -%>
<%- if task.present? -%>
<%- outputs = { 'stdout' => [], 'stderr' => [], 'debug' => [] } -%>
<%- task.action_continuous_output.each do |output| -%>
<%- outputs[output['output_type']] << output['output'] -%>
<%- end -%>
<%- report_row(
'Host': host.name,
'Task ID': task.id,
'Action': task.action,
'stdout': join_with_line_break(outputs['stdout']),
'stderr': join_with_line_break(outputs['stderr']),
'debug': join_with_line_break(outputs['debug']),
'Result': task.result,
'Finished': task.ended_at.blank? ? nil : format_time(task.ended_at),
) -%>
<%- else -%>
<%- report_row(
'Host': host.name,
'Task ID': 'N/A',
'Action': 'N/A',
'stdout': 'No Task Found',
'stderr': 'No Task Found',
'debug': 'No Task Found',
'Result': 'N/A',
'Finished': 'N/A',
) -%>
<%- end -%>
<%- end -%>
<%- end -%>
<%= report_render -%>
Its printing out the task_id now, but that doesnt seem to provide information on the output of each task/job.
Any help would be awesome.Aha! Its working. Now I just need to make the output look a little nicer:
name: Job - Detailed Invocation Report with Debugging
snippet: false
template_inputs:
- name: job_id
required: true
input_type: user
description: Id of job invocation to report
advanced: false
value_type: plain
resource_type: JobInvocation
hidden_value: false
- name: hosts
required: false
input_type: user
description: Field for filter hosts of job invocation. Leave blank for all hosts.
advanced: false
value_type: search
resource_type: Host
hidden_value: false
model: ReportTemplate
require:
- plugin: foreman_remote_execution
version: 4.4.0
-%>
<%- report_headers 'Host', 'Task ID', 'Action', 'stdout', 'stderr', 'debug', 'Result', 'Finished', 'SubTask Inspection' -%>
<%- invocation = find_job_invocation_by_id(input('job_id')) -%>
<%- parts = ["job_invocation.id = #{input('job_id')}"] -%>
<%- parts << input('hosts') unless input('hosts').blank? -%>
<%- search = parts.map { |part| '(' + part + ')' }.join(' AND ') -%>
<%- load_hosts(search: search).each do |batch| -%>
<%- batch.each do |host| -%>
<%- # Use sub_task_for_host to retrieve the sub-task -%>
<%- sub_task = invocation.sub_task_for_host(host) -%>
<%- if sub_task.present? -%>
<%- # Initialize outputs hash -%>
<%- outputs = { 'stdout' => [], 'stderr' => [], 'debug' => [] } -%>
<%- # Attempt to access outputs -%>
<%- if sub_task.respond_to?(:action_continuous_output) && sub_task.action_continuous_output.present? -%>
<%- sub_task.action_continuous_output.each do |output| -%>
<%- outputs[output['output_type']] << output['output'] -%>
<%- end -%>
<%- else -%>
<%- # If action_continuous_output is not available, attempt alternative methods or note absence -%>
<%- outputs['stdout'] << 'No stdout output available' -%>
<%- outputs['stderr'] << 'No stderr output available' -%>
<%- outputs['debug'] << 'No debug output available' -%>
<%- end -%>
<%- # Report row with an additional inspection column -%>
<%- report_row(
'Host': host.name,
'Task ID': sub_task.id,
'Action': sub_task.action,
'stdout': join_with_line_break(outputs['stdout']),
'stderr': join_with_line_break(outputs['stderr']),
'debug': join_with_line_break(outputs['debug']),
'Result': sub_task.result,
'Finished': sub_task.ended_at.blank? ? nil : format_time(sub_task.ended_at),
'SubTask Inspection': sub_task.inspect
) -%>
<%- else -%>
<%- report_row(
'Host': host.name,
'Task ID': 'N/A',
'Action': 'N/A',
'stdout': 'No Task Found',
'stderr': 'No Task Found',
'debug': 'No Task Found',
'Result': 'N/A',
'Finished': 'N/A',
'SubTask Inspection': 'No SubTask Available'
) -%>
<%- end -%>
<%- end -%>
<%- end -%>
<%= report_render -%>
Output:
e[0;32m10-173-26-210.ssnc-corp.cloud:e[0;0m
e[0;36m----------e[0;0m
e[0;36m ID: run_commande[0;0m
e[0;36mFunction: cmd.rune[0;0m
e[0;36m Name: cat /etc/salt/minion.d/99-metadata-grains.conf|grep -i patching_groupe[0;0m
e[0;36m Result: Truee[0;0m
e[0;36m Comment: Command "cat /etc/salt/minion.d/99-metadata-grains.conf|grep -i patching_group" rune[0;0m
e[0;36m Started: 14:27:39.229273e[0;0m
e[0;36mDuration: 16.323 mse[0;0m
e[0;36m Changes:
e[0;36m----------e[0;0m
e[0;36mpide[0;0m:
e[0;1;33m1051484e[0;0m
e[0;36mretcodee[0;0m:
e[0;1;33m0e[0;0m
e[0;36mstderre[0;0m:
e[0;36mstdoute[0;0m:
e[0;32m ssnc_patching_group: 3rd-sun-2am-5ame[0;0me[0;0m
e[0;36m
Summary for 10-173-26-210.ssnc-corp.cloud
------------e[0;0m
e[0;32mSucceeded: 1e[0;0m (e[0;32mchanged=1e[0;0m)
e[0;36mFailed: 0e[0;0m
e[0;36m------------
Total states run: 1e[0;0m
e[0;36mTotal run time: 16.323 mse[0;0m
Exit status: 0```WooHoo. Just gotta strip the ANSI characters. I hope others find this helpful. Thanks again for the reply Adam. This will work perfectly for our needs.
name: Job - Detailed Invocation Report without ANSI Codes
snippet: false
template_inputs:
- name: job_id
required: true
input_type: user
description: Id of job invocation to report
advanced: false
value_type: plain
resource_type: JobInvocation
hidden_value: false
- name: hosts
required: false
input_type: user
description: Field for filter hosts of job invocation. Leave blank for all hosts.
advanced: false
value_type: search
resource_type: Host
hidden_value: false
model: ReportTemplate
require:
- plugin: foreman_remote_execution
version: 4.4.0
-%>
<%# Define a helper method to strip ANSI codes %>
<% def strip_ansi_codes(text)
text.to_s.gsub(/\e\[[0-9;]*m/, '')
end %>
<%- report_headers 'Host', 'Task ID', 'Action', 'stdout', 'stderr', 'debug', 'Result', 'Finished' -%>
<%- invocation = find_job_invocation_by_id(input('job_id')) -%>
<%- parts = ["job_invocation.id = #{input('job_id')}"] -%>
<%- parts << input('hosts') unless input('hosts').blank? -%>
<%- search = parts.map { |part| '(' + part + ')' }.join(' AND ') -%>
<%- load_hosts(search: search).each do |batch| -%>
<%- batch.each do |host| -%>
<%- sub_task = invocation.sub_task_for_host(host) -%>
<%- if sub_task.present? -%>
<%- outputs = { 'stdout' => [], 'stderr' => [], 'debug' => [] } -%>
<%- if sub_task.respond_to?(:action_continuous_output) && sub_task.action_continuous_output.present? -%>
<%- sub_task.action_continuous_output.each do |output| -%>
<%- outputs[output['output_type']] << output['output'] -%>
<%- end -%>
<%- else -%>
<%- outputs['stdout'] << 'No stdout output available' -%>
<%- outputs['stderr'] << 'No stderr output available' -%>
<%- outputs['debug'] << 'No debug output available' -%>
<%- end -%>
<%- report_row(
'Host': host.name,
'Task ID': sub_task.id,
'Action': sub_task.action,
'stdout': strip_ansi_codes(join_with_line_break(outputs['stdout'])),
'stderr': strip_ansi_codes(join_with_line_break(outputs['stderr'])),
'debug': strip_ansi_codes(join_with_line_break(outputs['debug'])),
'Result': sub_task.result,
'Finished': sub_task.ended_at.blank? ? nil : format_time(sub_task.ended_at),
) -%>
<%- else -%>
<%- report_row(
'Host': host.name,
'Task ID': 'N/A',
'Action': 'N/A',
'stdout': 'No Task Found',
'stderr': 'No Task Found',
'debug': 'No Task Found',
'Result': 'N/A',
'Finished': 'N/A',
) -%>
<%- end -%>
<%- end -%>
<%- end -%>
<%= report_render -%>