Foreman 3.0 - Problem with Ansible Callback

Hi,

We are trying to get Foreman up and running by using Ansible, moving away completely from Puppet as we merely use it for Content Management and a few Ansible Playbooks to get a Base OS up and running.

Running an Ansible Playbook directly from the CLI on the Foreman Proxy, i.e a simple playbook with “setup” we can see the hosts being registered in Foreman and Facts/Reports being sent over, indicating that the Callback is working, but the timestamp on the reports are 2hours wrong which always marks the host as “Out of sync” due to the default 30min timeout. While we can work around this by increasing the Timeout value it would be good to try and solve the actual issue.

What’s worse is that when running any ansible-playbooks by utilizing a job the callback no longer works and is just throwing an “Callback disabled by environment. Disabling the Foreman callback plugin.”

We are a bit stumbled on what is causing the callbacks to be disabled when using jobs but works perfectly fine when using “ansible-playbook”.

Any help/hints are greatly appreciated.

Welcome Aanuka,

While I’m not in a position to immediately assist as I’m running an older version Foreman version and not yet using Katello content management - An obvious question will probably be asked by those with knowledge to help.

How does your Foreman Server / Smart-Proxy Time Zone compare to that of your managed hosts; and probably just as importantly to GMT?

Regards,
Peter

PS: In raising this issue in the Support forum, you seemed to have deleted to the Support template when asking your question. So readers are missing OS name and version, and other details. You mention Content Management but not the Katello version, you mention Ansible callback, but not the Ansible Version or the Ansible Plugin Version. :slightly_smiling_face:

PPS: Another thing I ran into on older versions of Foreman, which I believe is still current, is that on EL8 Foreman systems, you should not install the EPEL repository, and ansible should come from the centos-release-ansible-29 repository - Please validate this in the current documentation before taking it as still being the case.

PPPS: I also initially started by running ansible on the foreman server, but was advised that the developer/community expectation is that ansible being run on a separate system, such as a dedicated Ansible Tower (AWX) system.

Thanks for all the tips, and sorry about the template.
As I can’t seem to edit the first post I will write it down below:

Problem:
Ansible CallBack not working through Jobs, no facts/reports being sent to Foreman.
“Callback disabled by environment. Disabling the Foreman callback plugin”
Callbacks being made when running “ansible-playbook” on the Foreman host but being populated with the wrong time.

Expected outcome:
Facts/Reports populating in Foreman with correct time.

Foreman and Proxy versions:
Foreman 3.0.0

Foreman and Proxy plugin versions:
Foreman 3.0.0
katello 4.2.0.1
foreman_ansible 6.4.1
ansible 2.9.27

Distribution and version:
Rocky Linux 8.4 on Foreman/Katello
Rocky Linux 8.4 on Content Hosts

Regarding your questions:

  • All hosts are placed in the same Timezone, this being Europe/Stockholm and using the same NTP Server.

  • Ansible has been installed using the SIG - ansible-29 Repository. EPEL is disabled.

  • I will look into using another server, but as we are merely sitting at 10 hosts at the moment we didn’t see the need for that setup.

So according to Google, Europe/Stockholm is GMT +2 hours. That was what I was expecting.

First try echo $TZ ; and then run timedatectl and review the output.

Maybe try running Ansible with a TZ=GMT prefix (or TZ=Europe/Stockholm) and see if that helps.

Hopefully with this additional info, other community members can assist you further.

Yes, that’s correct.
I did a try with putting the Foreman server in UTC and that did “solve” the issue with the wrong Report Time and fixing the Out of sync issue.

Now I just need to figure out why Callback are disabled running the Job inside foreman compared to running ansible-playbook

IIRC when running ansible from Foreman the callback is only enabled if you’re applying roles as those should form the baseline and be used to decide whether a host is in sync or not. Having one-off jobs send reports would mess with this.

You seem to be correct, I tried running the “Run Ansible roles” and that did in fact report back to Foreman.

The wording in the documentation (Foreman :: Plugin Manuals)
“Every time you run a playbook or an Ansible module, Foreman will receive facts and a report for the host you have executed it on. If you ran Ansible from Foreman itself, the output of your Ansible run will be saved in our database.”

This seemed to indicate that running jobs inside Foreman would also trigger a callback, but I take it that this only applies when running this through ansible-playbook on the Ansible server.

Good day,

I am having the same issue with UTC timezone while integrating AWX and Foreman. All the reports are off and are giving me sync errors. I have found this recommendation from redhat - How Do I Adjust Playbook and Log Timezone in Ansible Tower? - Red Hat Customer Portal which sets the appropriate time zone for the AWX API and it helps. My problem is that my environment has multiple timezones, so it does not really solve my issue. Is using UTC the only work around?

I understood it this way all the time as well, and I was curious why I don’t see any reports when I’m running jobs and roles from Foreman. I see reports when I run them directly on Ansible server.