Two Issues with Foreman Callback - Creating new hosts and Message of the text reports is ill formatted

Problem:
Running a job in awx the callback creates 3 new hosts. And the message of the text looks to be output in json.

Expected outcome:
Doesn’t create new hosts and updates existing ones. The job details for the reporting functionality looks closer to the input from puppet.

Foreman and Proxy versions:
1.17
Foreman and Proxy plugin versions:
not using a proxy in the awx container. just using callback script from
/usr/lib/python2.7/site-packages/awx/plugins/callback/foreman.py
/usr/lib/python2.7/site-packages/ansible/plugins/callback/foreman.py

setup the foreman ssl cert and foreman.ini in /etc/ansible/foreman.ini
provides cert, key, url, user and password.

Other relevant data:
I do notice it just to pull container info everytime a job runs from awx.
2018-05-24 12:44:30 38819b54 [app] [I] Started GET “/node/sa1x-foreman-p1.harlandclarke.local?format=yml” for 10.105.115.42 at 2018-05-24 12:44:30 -0500
2018-05-24 12:44:30 38819b54 [app] [I] Processing by HostsController#externalNodes as YML
2018-05-24 12:44:30 38819b54 [app] [I] Parameters: {“name”=>“sa1x-foreman-p1.harlandclarke.local”}
2018-05-24 12:44:30 38819b54 [app] [I] Current user: foreman_api_admin (administrator)
2018-05-24 12:44:30 38819b54 [app] [I] Rendering text template
2018-05-24 12:44:30 38819b54 [app] [I] Rendered text template (0.0ms)
2018-05-24 12:44:30 38819b54 [app] [I] Completed 200 OK in 265ms (Views: 0.7ms | ActiveRecord: 28.2ms)
2018-05-24 12:44:32 7747e2a8 [app] [I] Started POST “/api/hosts/facts” for 10.105.115.42 at 2018-05-24 12:44:32 -0500
2018-05-24 12:44:32 7747e2a8 [app] [I] Processing by Api::V2::HostsController#facts as JSON
2018-05-24 12:44:32 7747e2a8 [app] [I] Parameters: {“facts”=>“[FILTERED]”, “name”=>“sa1x-foreman-p1.harlandclarke.local”, “certname”=>“sa1x-foreman-p1.harlandclarke.local”, “apiv”=>“v2”, “host”=>{“certname”=>“sa1x-foreman-p1.harlandclarke.local”, “name”=>“sa1x-foreman-p1.harlandclarke.local”}}
2018-05-24 12:44:32 7747e2a8 [app] [I] Current user: foreman_api_admin (administrator)
2018-05-24 12:44:32 7747e2a8 [audit] [I] [sa1x-foreman-p1.harlandclarke.local] deleted 0 (10.6ms)
2018-05-24 12:44:33 7747e2a8 [audit] [I] [sa1x-foreman-p1.harlandclarke.local] updated 22 (904.4ms)
2018-05-24 12:44:33 7747e2a8 [audit] [I] [sa1x-foreman-p1.harlandclarke.local] added 0 (14.4ms)
2018-05-24 12:44:33 7747e2a8 [app] [I] Import facts for ‘sa1x-foreman-p1.harlandclarke.local’ completed. Added: 0, Updated: 22, Deleted 0 facts
2018-05-24 12:44:33 7747e2a8 [app] [I] Completed 201 Created in 1099ms (Views: 6.0ms | ActiveRecord: 244.1ms)
2018-05-24 12:44:33 1c604664 [app] [I] Started GET “/node/sa1x-foreman-p1.harlandclarke.local?format=yml” for 10.105.115.42 at 2018-05-24 12:44:33 -0500
2018-05-24 12:44:33 1c604664 [app] [I] Processing by HostsController#externalNodes as YML
2018-05-24 12:44:33 1c604664 [app] [I] Parameters: {“name”=>“sa1x-foreman-p1.harlandclarke.local”}
2018-05-24 12:44:33 1c604664 [app] [I] Current user: foreman_api_admin (administrator)
2018-05-24 12:44:34 1c604664 [app] [I] Rendering text template
2018-05-24 12:44:34 1c604664 [app] [I] Rendered text template (0.0ms)
2018-05-24 12:44:34 1c604664 [app] [I] Completed 200 OK in 238ms (Views: 0.6ms | ActiveRecord: 22.4ms)
2018-05-24 12:44:34 9d7dac13 [app] [I] Started POST “/api/config_reports” for 10.105.115.42 at 2018-05-24 12:44:34 -0500
2018-05-24 12:44:34 9d7dac13 [app] [I] Processing by Api::V2::ConfigReportsController#create as JSON
2018-05-24 12:44:34 9d7dac13 [app] [I] Parameters: {“config_report”=>“[FILTERED]”, “apiv”=>“v2”}
2018-05-24 12:44:34 9d7dac13 [app] [I] Current user: foreman_api_admin (administrator)
2018-05-24 12:44:34 9d7dac13 [app] [I] Imported report for sa1x-foreman-p1.harlandclarke.local in 0.04 seconds, status refreshed in 0.02 seconds
2018-05-24 12:44:34 9d7dac13 [app] [I] Rendering api/v2/config_reports/create.json.rabl
2018-05-24 12:44:34 9d7dac13 [app] [I] Rendered api/v2/config_reports/create.json.rabl (12.4ms)
2018-05-24 12:44:34 9d7dac13 [app] [I] Completed 201 Created in 91ms (Views: 12.2ms | ActiveRecord: 26.0ms)

This is just the rest of the previous log.

   2018-05-24 12:42:37 e1a816ab [app] [I] Started POST "/api/v2/reports" for 10.105.115.43 at 2018-05-24 12:42:37 -0500
    2018-05-24 12:42:37 e1a816ab [app] [I] Processing by Api::V2::ReportsController#create as JSON
    2018-05-24 12:42:37 e1a816ab [app] [I]   Parameters: {"report"=>"[FILTERED]", "apiv"=>"v2"}
    2018-05-24 12:42:37 e1a816ab [app] [I] Current user: foreman_api_admin (administrator)
    2018-05-24 12:42:37 e1a816ab [app] [E] Unprocessable entity ConfigReport (id: new):
     |   Reported at has already been taken
     |
    2018-05-24 12:42:37 e1a816ab [app] [I]   Rendering api/v2/errors/unprocessable_entity.json.rabl within api/v2/layouts/error_layout
    2018-05-24 12:42:37 e1a816ab [app] [I]   Rendered api/v2/errors/unprocessable_entity.json.rabl within api/v2/layouts/error_layout (2.0ms)
    2018-05-24 12:42:37 e1a816ab [app] [I] Completed 422 Unprocessable Entity in 43ms (Views: 7.6ms | ActiveRecord: 8.6ms)
    2018-05-24 12:42:42 39dd16ea [app] [I] Started POST "/api/v2/reports" for 10.105.115.43 at 2018-05-24 12:42:42 -0500
    2018-05-24 12:42:42 39dd16ea [app] [I] Processing by Api::V2::ReportsController#create as JSON
    2018-05-24 12:42:42 39dd16ea [app] [I]   Parameters: {"report"=>"[FILTERED]", "apiv"=>"v2"}
    2018-05-24 12:42:42 39dd16ea [app] [I] Current user: foreman_api_admin (administrator)
    2018-05-24 12:42:42 39dd16ea [app] [I] Imported report for localhost in 0.05 seconds, status refreshed in 0.02 seconds
    2018-05-24 12:42:42 39dd16ea [app] [I]   Rendering api/v2/reports/create.json.rabl
    2018-05-24 12:42:42 39dd16ea [app] [I]   Rendered api/v2/reports/create.json.rabl (11.4ms)
    2018-05-24 12:42:42 39dd16ea [app] [I] Completed 201 Created in 91ms (Views: 10.4ms | ActiveRecord: 25.2ms)
    2018-05-24 12:42:42 cdc6ea5c [app] [I] Started POST "/api/v2/reports" for 10.105.115.43 at 2018-05-24 12:42:42 -0500
    2018-05-24 12:42:42 cdc6ea5c [app] [I] Processing by Api::V2::ReportsController#create as JSON
    2018-05-24 12:42:42 cdc6ea5c [app] [I]   Parameters: {"report"=>"[FILTERED]", "apiv"=>"v2"}
    2018-05-24 12:42:42 cdc6ea5c [app] [I] Current user: foreman_api_admin (administrator)
    2018-05-24 12:42:42 cdc6ea5c [app] [E] Unprocessable entity ConfigReport (id: new):
     |   Reported at has already been taken
     |
    2018-05-24 12:42:42 cdc6ea5c [app] [I]   Rendering api/v2/errors/unprocessable_entity.json.rabl within api/v2/layouts/error_layout
    2018-05-24 12:42:42 cdc6ea5c [app] [I]   Rendered api/v2/errors/unprocessable_entity.json.rabl within api/v2/layouts/error_layout (0.8ms)
    2018-05-24 12:42:42 cdc6ea5c [app] [I] Completed 422 Unprocessable Entity in 30ms (Views: 1.5ms | ActiveRecord: 8.5ms)
    2018-05-24 12:42:48 2d37c4af [app] [I] Started POST "/api/v2/hosts/facts" for 10.105.115.43 at 2018-05-24 12:42:48 -0500
    2018-05-24 12:42:48 2d37c4af [app] [I] Processing by Api::V2::HostsController#facts as JSON
    2018-05-24 12:42:48 2d37c4af [app] [I]   Parameters: {"facts"=>"[FILTERED]", "name"=>"sa1x-spacewalk-p1", "apiv"=>"v2", "host"=>{"name"=>"sa1x-spacewalk-p1"}}
    2018-05-24 12:42:48 2d37c4af [app] [I] Current user: foreman_api_admin (administrator)
    2018-05-24 12:42:49 2d37c4af [audit] [I] [sa1x-spacewalk-p1.harlandclarke.local] deleted 86 (42.8ms)
    2018-05-24 12:42:52 2d37c4af [audit] [I] [sa1x-spacewalk-p1.harlandclarke.local] updated 36 (3101.4ms)
    2018-05-24 12:42:52 2d37c4af [audit] [I] [sa1x-spacewalk-p1.harlandclarke.local] added 0 (42.4ms)
    2018-05-24 12:42:52 2d37c4af [app] [I] Import facts for 'sa1x-spacewalk-p1.harlandclarke.local' completed. Added: 0, Updated: 36, Deleted 86 facts
    2018-05-24 12:42:52 2d37c4af [app] [I] Completed 201 Created in 3916ms (Views: 3.6ms | ActiveRecord: 874.4ms)
    2018-05-24 12:42:52 ecce2b14 [app] [I] Started POST "/api/v2/hosts/facts" for 10.105.115.43 at 2018-05-24 12:42:52 -0500
    2018-05-24 12:42:52 ecce2b14 [app] [I] Processing by Api::V2::HostsController#facts as JSON
    2018-05-24 12:42:52 ecce2b14 [app] [I]   Parameters: {"facts"=>"[FILTERED]", "name"=>"sa1x-spacewalk-p1", "apiv"=>"v2", "host"=>{"name"=>"sa1x-spacewalk-p1"}}
    2018-05-24 12:42:52 ecce2b14 [app] [I] Current user: foreman_api_admin (administrator)
    2018-05-24 12:42:53 ecce2b14 [audit] [I] [sa1x-spacewalk-p1.harlandclarke.local] deleted 0 (44.8ms)
    2018-05-24 12:42:56 ecce2b14 [audit] [I] [sa1x-spacewalk-p1.harlandclarke.local] updated 0 (3108.3ms)
    2018-05-24 12:42:56 ecce2b14 [audit] [I] [sa1x-spacewalk-p1.harlandclarke.local] added 0 (40.0ms)
    2018-05-24 12:42:56 ecce2b14 [app] [I] Import facts for 'sa1x-spacewalk-p1.harlandclarke.local' completed. Added: 0, Updated: 0, Deleted 0 facts
    2018-05-24 12:42:56 ecce2b14 [app] [I] Completed 201 Created in 3941ms (Views: 3.9ms | ActiveRecord: 841.6ms)
    2018-05-24 12:43:03 9d863c15 [app] [I] Started POST "/api/v2/reports" for 10.105.115.43 at 2018-05-24 12:43:03 -0500
    2018-05-24 12:43:03 9d863c15 [app] [I] Processing by Api::V2::ReportsController#create as JSON
    2018-05-24 12:43:03 9d863c15 [app] [I]   Parameters: {"report"=>"[FILTERED]", "apiv"=>"v2"}
    2018-05-24 12:43:03 9d863c15 [app] [I] Current user: foreman_api_admin (administrator)
    2018-05-24 12:43:03 9d863c15 [app] [I] Imported report for sa1x-spacewalk-p1 in 0.11 seconds, status refreshed in 0.02 seconds
    2018-05-24 12:43:03 9d863c15 [app] [I]   Rendering api/v2/reports/create.json.rabl
    2018-05-24 12:43:03 9d863c15 [app] [I]   Rendered api/v2/reports/create.json.rabl (41.1ms)
    2018-05-24 12:43:03 9d863c15 [app] [I] Completed 201 Created in 193ms (Views: 39.5ms | ActiveRecord: 42.3ms)
    2018-05-24 12:43:03 18cbfda5 [app] [I] Started POST "/api/v2/reports" for 10.105.115.43 at 2018-05-24 12:43:03 -0500
    2018-05-24 12:43:03 18cbfda5 [app] [I] Processing by Api::V2::ReportsController#create as JSON
    2018-05-24 12:43:03 18cbfda5 [app] [I]   Parameters: {"report"=>"[FILTERED]", "apiv"=>"v2"}
    2018-05-24 12:43:03 18cbfda5 [app] [I] Current user: foreman_api_admin (administrator)
    2018-05-24 12:43:03 18cbfda5 [app] [I] Imported report for sa1x-spacewalk-p1 in 0.08 seconds, status refreshed in 0.02 seconds
    2018-05-24 12:43:03 18cbfda5 [app] [I]   Rendering api/v2/reports/create.json.rabl
    2018-05-24 12:43:03 18cbfda5 [app] [I]   Rendered api/v2/reports/create.json.rabl (16.5ms)
    2018-05-24 12:43:03 18cbfda5 [app] [I] Completed 201 Created in 128ms (Views: 14.8ms | ActiveRecord: 41.4ms)

Hey,

Foreman attempts to set operating system major and minor version as well as OS name etc from your facts. It looks like your input is not correct and major/minor are not integers, thus this fails to save in the database. Please send us facts from the host which fails, in json format (facter --json). I assume you are using Puppet.

Workaround: Set ignore_facts_for_operatingsystem setting in Administer menu to not set OS from facts.

That looks like Ansible to me. @ansible folks, any thoughts on this one?

@lzap @Gwmngilfen - Yes this is specifically about trying to get reports from AWX (open source tower). We are using puppet as well for some different tasks and I am trying to get the reports into foreman, as overall location for reporting on hosts. the AWX container labeled as localhost isn’t really an issue (it runs in a docker container and is updating a host entry called localhost but I am not worried about grabbing it’s os major and minor. I am more specifically looking at the sa1x-spacewalk-p1 instance creating new hosts when a job is run in awx. it doesn’t combine with the existing host even. I am looking for any help with the integration between awx and foreman. (or if the callbacks can be used in that type of environment.)

Hmm @dLobatog ?

You changed the title but that doesn’t reflect my issue I don’t care about the container os. Like if that is for development that is fine with me. Specifically I am wondering why the callback isn’t updating an existing host record and why when I set the stdout to a callback of foreman it is being sent as Json and doesn’t look clean inside of foreman.

I have restored the original title for you.

1 Like

Sorry, do not hesitate to create separate threads for separate issues for better clarity.

Admittedly I thought they both revolve around an issue with the foreman ansible callback module. But if it would be easier I can split them. I just provided the tail of a log after kicking off an awx job.

@ansible - Hi all, I was wondering if you any of you had some information about this? The callback is creating new hosts in ansible each time a job is run in awx. And the formatting seems to comeback in a json format that doesn’t display nicely in foreman when run for those hosts. I’d like to help more and can try to provided better feedback.

@ansible, @Gwmngilfen, @lzap - I am wondering if anyone knows specifically can I use the foreman callback features from inside of the awx 1.0.3 container? Even if it is something you are just looking into and you need sometime - any feedback into this thread would be very much appreciated.

Thanks,
Jeff

I haven’t read the whole thread, but if the question is whether callback would work on awx container, I’d say yes. awx container probably contains recent version of ansible engine which has the foreman callback builtin. You only need to enable it in /etc/ansible/ansible.cfg so as long as the container allows you doing that, it should be fine.