Ansible callback not working. No information logged in foreman/production.log

Hi all,

I’ve been struggling to get this to work for two days and I’m not getting any further and I’m out of ideas and read everything I could find as far as similar issues other folks had. None seem to help in my case. Help is much appreciated.

I am getting no errors when I run ansible “hostname” -m setup. I see the facts fine on the ansible server and I can run ansible playbooks but the host never gets added to Foreman and no errors or information about the callback getting logged in the production.log.

Problem:
Ansible callback not working. No information logged in foreman/production.log

Expected outcome:

Hostname and facts imported to Foreman.

Foreman and Proxy versions:
root@ansible01 callback]# rpm -qa | grep -i foreman
foreman-postgresql-1.17.0-1.el7.noarch
tfm-rubygem-foreman_remote_execution_core-1.1.0-1.fm1_17.el7.noarch
foreman-release-1.17.0-1.el7.noarch
foreman-selinux-1.17.0-1.el7.noarch
foreman-installer-1.17.0-1.el7.noarch
foreman-release-scl-4-1.el7.noarch
foreman-1.17.0-1.el7.noarch
foreman-proxy-1.17.0-1.el7.noarch
foreman-cli-1.17.0-1.el7.noarch
tfm-rubygem-foreman-tasks-core-0.2.4-1.fm1_17.el7.noarch
tfm-rubygem-foreman-tasks-0.11.2-1.fm1_17.el7.noarch
tfm-rubygem-foreman_ansible_core-2.0.2-1.fm1_17.el7.noarch
tfm-rubygem-foreman_remote_execution-1.4.5-2.fm1_17.el7.noarch
foreman-debug-1.17.0-1.el7.noarch
tfm-rubygem-hammer_cli_foreman-0.12.0-2.el7.noarch
tfm-rubygem-foreman_ansible-2.0.1-1.fm1_17.el7.noarch

Other relevant data:
added to ansible.cfg
[defaults]
bin_ansible_callbacks = True
callback_whitelist = foreman foreman-proxy

[callback_foreman]
verify_certs = True # this is a boolean
ssl_key = “/etc/puppetlabs/puppet/ssl/private_keys/ansible01.localhost.local.pem”
ssl_cert = “/etc/puppetlabs/puppet/ssl/certs/ansible01.localhost.local.pem”
url = “https://ansible01.localhost.local

using /etc/hosts file instead of DNS,

ssh remote execution tasks to the ansible server works fine, ssh keys to the server i’m trying to import works also.

This is the command I used to install foreman and plugin.
foreman-installer --enable-foreman-plugin-remote-execution --enable-foreman-proxy-plugin-remote-execution-ssh --enable-foreman-plugin-ansible

[e.g. logs from Foreman and/or the Proxy, modified templates, commands issued, etc]

Not sure if this is helpful but when I remove foreman-proxy from callback_whitelist = foreman foreman-proxy

I get the errors below hence why I added it.

ansible ubuntu16 -m setup -vvv
ansible 2.5.1
config file = /etc/ansible/ansible.cfg
configured module search path = [u’/home/a-flaporta/.ansible/plugins/modules’, u’/usr/share/ansible/plugins/modules’]
ansible python module location = /usr/lib/python2.7/site-packages/ansible
executable location = /usr/bin/ansible
python version = 2.7.5 (default, Aug 4 2017, 00:39:18) [GCC 4.8.5 20150623 (Red Hat 4.8.5-16)]
Using /etc/ansible/ansible.cfg as config file
Parsed /etc/ansible/hosts inventory source with ini plugin
ERROR! Unexpected Exception, this is probably a bug: ‘https://ansible01.localhost.local
the full traceback was:

Traceback (most recent call last):
File “/usr/bin/ansible”, line 118, in
exit_code = cli.run()
File “/usr/lib/python2.7/site-packages/ansible/cli/adhoc.py”, line 181, in run
result = self._tqm.run(play)
File “/usr/lib/python2.7/site-packages/ansible/executor/task_queue_manager.py”, line 229, in run
self.load_callbacks()
File “/usr/lib/python2.7/site-packages/ansible/executor/task_queue_manager.py”, line 210, in load_callbacks
callback_obj.set_options()
File “/usr/share/ansible/plugins/callback/foreman.py”, line 97, in set_options
self.FOREMAN_URL = self.get_option(‘https://ansible01.localhost.local’)
File “/usr/lib/python2.7/site-packages/ansible/plugins/callback/init.py”, line 89, in get_option
return self._plugin_options[k]
KeyError: ‘https://ansible01.localhost.local

The one in Ansible 2.5 and 2.5.1 is broken. There’s a fix in develop and 2.5-stable but unreleased.

https://github.com/ansible/ansible/commit/082b03c829fe568e179c7917fd72274ad1db7980#diff-8c9fb0d3068dcb750eb2c59165eed061

I also added some more changes which are only in develop, but not sure they are needed to get basic functionality working:

https://github.com/ansible/ansible/commit/d303c0706c69831f4e5233a5c9d43b8f2db6e5ac#diff-8c9fb0d3068dcb750eb2c59165eed061

Hi ekohl,

Thank you for the information. Is there any work around in the meantime or I need to wait for the 2.5-stable to be released.

Thank you,

The only workaround I know is downloading foreman.py and replace it in the callback directory.

Hi ekohl,

I appreciate your help and patience… Where can I download the latest working foreman.py.

Thank you,
Fab

https://github.com/ansible/ansible/commit/d303c0706c69831f4e5233a5c9d43b8f2db6e5ac#diff-8c9fb0d3068dcb750eb2c59165eed061

Looking a bit deeper it appears 2.5.1 should already include the fix. Since it doesn’t work for you, I wonder what’s wrong. It looks like it’s resolving the value for URL and then using it as a key. I can’t really explain why that would happen.

Thanks for the update. Do you know why it only happens when I remove foreman-proxy from the callback_whitelist line in /etc/ansible/ansible.cfg? I’m not sure how that line actually effects the callback plugin. Does the callback plugin not run when I add foreman-proxy to the whitelist?

Thank you

Ignore last reply… just noticed the callback_whitelist was not set correctly. adding foreman-proxy makes no difference.

Thank you all for your help.

I updated Ansible to version 2.5.2 and hosts are now being added successfully to Foreman.

I did have to fix permissions to be able to read the SSL certificates for my user account that runs ansible and also modified the /etc/environment file as shown below.

FOREMAN_URL=https://ansible01.localhost.local
FOREMAN_SSL_CERT=/etc/puppetlabs/puppet/ssl/certs/ansible01.localhost.local.pem
FOREMAN_SSL_KEY=/etc/puppetlabs/puppet/ssl/private_keys/ansible01.localhost.local.pem
FOREMAN_SSL_VERIFY=/etc/puppetlabs/puppet/ssl/certs/ca.pem

I’m happy this working and can stop obsessing over fixing it. :slight_smile: have a great weekend.

AFAIK since Ansible 2.5 environment variables are no longer used and changing /etc/environment should no longer be needed.