Foreman-Ansible Callback Connection Refused (Errno 111)

Support
1

Hi, ladies and gentlemen. I'm trying to setup Foreman with Ansible.
Looks like I'm missing something. After the test command on the Ansible
server "ansible -m setup localhost" Ansible writes many green strings with
configuration info, and in the end where the Callback should be used it
says:

[WARNING]: Failure using method (v2_runner_on_ok) in callback plugin
(</usr/share/ansible/plugins/callback/foreman_callback.CallbackModule
object at 0x16995d0>):
HTTPSConnectionPool(host='myAWShost.myRegion.compute.internal',
port=3000): Max retries exceeded with url: /api/v2/hosts/facts (Caused by
<class
'socket.error'>: [Errno 111] Connection refused)

I did everything as in the manual, may be I have ony changed the default
directory of a plugin (/usr/share/ansible/plugins/), but i made
appropriate changes to Ansible config, as plugin itself works.
In the Foreman UI I see some options appeared related to Ansible, like
"Configure -> Ansible / Roles". Also I added my Ansible server to the
trusted_puppetmaster_hosts and create_new_host_when_facts_are_uploaded
are ON.
The problem is, as I see, in connection to Foreman. In my Foreman Callback
.py file *FOREMAN_URL *is the same, as it is in the results of the
following command
openssl s_client -showcerts
-connect myAWShost.myRegion.compute.internal:443
This same address was by default. Also I copied Foreman's cert .pem and ssl
.pem to the Ansible server and added this paths in the foreman_callback.py
file.

When I try to turn off FOREMAN_SSL_VERIFY, I'm getting another error:

ERROR! Unexpected Exception: 'module' object has no attribute 'packages'
the full traceback was:

Traceback (most recent call last):

  • File "/usr/bin/ansible", line 92, in <module>*
  • exit_code = cli.run()*
  • File "/usr/lib/python2.6/site-packages/ansible/cli/adhoc.py", line 193,
    in run*
  • result = self._tqm.run(play)*
  • File
    "/usr/lib/python2.6/site-packages/ansible/executor/task_queue_manager.py",
    line 202, in run*
  • self.load_callbacks()*
  • File
    "/usr/lib/python2.6/site-packages/ansible/executor/task_queue_manager.py",
    line 188, in load_callbacks*
  • self._callback_plugins.append(callback_plugin())*
  • File "/usr/share/ansible/plugins/callback/foreman_callback.py", line 69,
    in init*
  • self.ssl_verify = self._ssl_verify()*
  • File "/usr/share/ansible/plugins/callback/foreman_callback.py", line 83,
    in _ssl_verify*
  • requests.packages.urllib3.disable_warnings()*
    AttributeError: 'module' object has no attribute 'packages'

Need help, what have I done wrong? May be something wrong with the
installation, don't know…

2

> Hi, ladies and gentlemen. I'm trying to setup Foreman with Ansible.
> Looks like I'm missing something. After the test command on the Ansible
> server "ansible -m setup localhost" Ansible writes many green strings with
> configuration info, and in the end where the Callback should be used it
> says:
>
> [WARNING]: Failure using method (v2_runner_on_ok) in callback plugin
> (</usr/share/ansible/plugins/callback/foreman_callback.CallbackModule
> object at 0x16995d0>):
> HTTPSConnectionPool(host='myAWShost.myRegion.compute.internal',
> port=3000): Max retries exceeded with url: /api/v2/hosts/facts (Caused by
> <class
> 'socket.error'>: [Errno 111] Connection refused)

The installation seems fine from the rest of your email, don't worry too
much about that. The problem seems to be that your Foreman host is not
accepting connections from this host on that port (notice that 'connection
refused' message).

I don't think the problem is in Foreman itself but rather in how the
host is configured. Can you run that same command, but from the Foreman
host itself? You might see different results if the firewall was
blocking the connection.

Have you checked /var/log/foreman/production.log on the Foreman host?
From the looks of it I don't think the requests have even got to the
host. I'd check firewall, SELinux, Apache logs…

You should set FOREMAN_SSL_CERT and FOREMAN_SSL_KEY if you intend the
callback to work through HTTPS (or enable http for that Foreman route)

https://theforeman.org/plugins/foreman_ansible/1.x/index.html#2.1Ansiblecallback
has more info, if you don't feel like configuring stuff and you have
Ansible 2.2, the callback is shipped by default with it. You should only
have to enable it and set the appropriate variables.

··· On 11/21, vladimir@gochange.co wrote:

I did everything as in the manual, may be I have ony changed the default
directory of a plugin (/usr/share/ansible/plugins/), but i made
appropriate changes to Ansible config, as plugin itself works.
In the Foreman UI I see some options appeared related to Ansible, like
"Configure -> Ansible / Roles". Also I added my Ansible server to the
trusted_puppetmaster_hosts and create_new_host_when_facts_are_uploaded
are ON.
The problem is, as I see, in connection to Foreman. In my Foreman Callback
.py file *FOREMAN_URL *is the same, as it is in the results of the
following command
openssl s_client -showcerts
-connect myAWShost.myRegion.compute.internal:443
This same address was by default. Also I copied Foreman’s cert .pem and ssl
.pem to the Ansible server and added this paths in the foreman_callback.py
file.

When I try to turn off FOREMAN_SSL_VERIFY, I’m getting another error:

ERROR! Unexpected Exception: ‘module’ object has no attribute 'packages’
the full traceback was:

Traceback (most recent call last):

  • File “/usr/bin/ansible”, line 92, in *
  • exit_code = cli.run()*
  • File “/usr/lib/python2.6/site-packages/ansible/cli/adhoc.py”, line 193,
    in run*
  • result = self._tqm.run(play)*
  • File
    "/usr/lib/python2.6/site-packages/ansible/executor/task_queue_manager.py",
    line 202, in run*
  • self.load_callbacks()*
  • File
    "/usr/lib/python2.6/site-packages/ansible/executor/task_queue_manager.py",
    line 188, in load_callbacks*
  • self._callback_plugins.append(callback_plugin())*
  • File “/usr/share/ansible/plugins/callback/foreman_callback.py”, line 69,
    in init*
  • self.ssl_verify = self._ssl_verify()*
  • File “/usr/share/ansible/plugins/callback/foreman_callback.py”, line 83,
    in _ssl_verify*
  • requests.packages.urllib3.disable_warnings()*
    AttributeError: ‘module’ object has no attribute ‘packages’

Need help, what have I done wrong? May be something wrong with the
installation, don’t know…

–
You received this message because you are subscribed to the Google Groups “Foreman users” group.
To unsubscribe from this group and stop receiving emails from it, send an email to foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at https://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.

–
Daniel Lobato Garcia

@dLobatog
blog.daniellobato.me
daniellobato.me

GPG: http://keys.gnupg.net/pks/lookup?op=get&search=0x7A92D6DD38D6DE30
Keybase: https://keybase.io/elobato

3

Hello!

I actually have the same problem while running ansible on the same host
than the foreman.
TCP port 3000 is simply closed. And if I use the env variables to use port
80, I get an auth error.
What's port 3000 supposed to be the entry point of? The Foreman Web UI or
something else?
I'm confused about how the callback is supposed to authenticate itself. Any
help appreciated :slight_smile:

Thanks!

··· On Thursday, November 24, 2016 at 10:26:49 AM UTC+1, Daniel Lobato wrote: > > On 11/21, vlad...@gochange.co wrote: > > Hi, ladies and gentlemen. I'm trying to setup Foreman with Ansible. > > Looks like I'm missing something. After the test command on the Ansible > > server "ansible -m setup localhost" Ansible writes many green strings > with > > configuration info, and in the end where the Callback should be used it > > says: > > > > *[WARNING]: Failure using method (v2_runner_on_ok) in callback plugin > > ( > object at 0x16995d0>):* > > *HTTPSConnectionPool(host='myAWShost.myRegion.compute.internal', > > port=3000): Max retries exceeded with url: /api/v2/hosts/facts (Caused > by > > > *'socket.error'>: [Errno 111] Connection refused)* > > The installation seems fine from the rest of your email, don't worry too > much about that. The problem seems to be that your Foreman host is not > accepting connections from this host on that port (notice that 'connection > refused' message). > > I don't think the problem is in Foreman itself but rather in how the > host is configured. Can you run that same command, but from the Foreman > host itself? You might see different results if the firewall was > blocking the connection. > > Have you checked /var/log/foreman/production.log on the Foreman host? > From the looks of it I don't think the requests have even got to the > host. I'd check firewall, SELinux, Apache logs... > > You should set FOREMAN_SSL_CERT and FOREMAN_SSL_KEY if you intend the > callback to work through HTTPS (or enable http for that Foreman route) > > > https://theforeman.org/plugins/foreman_ansible/1.x/index.html#2.1Ansiblecallback > has more info, if you don't feel like configuring stuff and you have > Ansible 2.2, the callback is shipped by default with it. You should only > have to enable it and set the appropriate variables. > > > I did everything as in the manual, may be I have ony changed the default > > directory of a plugin (*/usr/share/ansible/plugins/*), but i made > > appropriate changes to Ansible config, as plugin itself works. > > In the Foreman UI I see some options appeared related to Ansible, like > > "Configure -> Ansible / Roles". Also I added my Ansible server to the > > *trusted_puppetmaster_hosts* and > *create_new_host_when_facts_are_uploaded* > > are ON. > > The problem is, as I see, in connection to Foreman. In my Foreman > Callback > > .py file *FOREMAN_URL *is the same, as it is in the results of the > > following command > > *openssl s_client -showcerts > > -connect myAWShost.myRegion.compute.internal:443* > > This same address was by default. Also I copied Foreman's cert .pem and > ssl > > .pem to the Ansible server and added this paths in the > *foreman_callback.py* > > file. > > > > When I try to turn off *FOREMAN_SSL_VERIFY*, I'm getting another error: > > > > *ERROR! Unexpected Exception: 'module' object has no attribute > 'packages'* > > *the full traceback was:* > > > > *Traceback (most recent call last):* > > * File "/usr/bin/ansible", line 92, in * > > * exit_code = cli.run()* > > * File "/usr/lib/python2.6/site-packages/ansible/cli/adhoc.py", line > 193, > > in run* > > * result = self._tqm.run(play)* > > * File > > > "/usr/lib/python2.6/site-packages/ansible/executor/task_queue_manager.py", > > line 202, in run* > > * self.load_callbacks()* > > * File > > > "/usr/lib/python2.6/site-packages/ansible/executor/task_queue_manager.py", > > line 188, in load_callbacks* > > * self._callback_plugins.append(callback_plugin())* > > * File "/usr/share/ansible/plugins/callback/foreman_callback.py", line > 69, > > in __init__* > > * self.ssl_verify = self._ssl_verify()* > > * File "/usr/share/ansible/plugins/callback/foreman_callback.py", line > 83, > > in _ssl_verify* > > * requests.packages.urllib3.disable_warnings()* > > *AttributeError: 'module' object has no attribute 'packages'* > > > > Need help, what have I done wrong? May be something wrong with the > > installation, don't know... > > > > -- > > You received this message because you are subscribed to the Google > Groups "Foreman users" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to foreman-user...@googlegroups.com . > > To post to this group, send email to forema...@googlegroups.com > . > > Visit this group at https://groups.google.com/group/foreman-users. > > For more options, visit https://groups.google.com/d/optout. > > > -- > Daniel Lobato Garcia > > @dLobatog > blog.daniellobato.me > daniellobato.me > > GPG: http://keys.gnupg.net/pks/lookup?op=get&search=0x7A92D6DD38D6DE30 > Keybase: https://keybase.io/elobato >
4

I'm seeing a similar errors, but mine seem to be related to ssh
connections. However, I only see the error from foreman UI and not from the
cli. Is this task executed as a particular user? My firewall is disabled,
and i have Selinux in permissive in order to troubleshoot this.

## From the foreman UI.
Enter PLAY [all]

··· *********************************************************************

TASK [setup]

fatal: [myserver.home.local]: UNREACHABLE! => {“changed”: false, “msg”:
“Failed to connect to the host via ssh: Permission denied
(publickey,gssapi-keyex,gssapi-with-mic,password).\r\n”, “unreachable”:
true}
to retry, use: --limit
@/tmp/d20161204-2426-e3aev2/foreman-playbook-3d1080b7-deca-46fe-b4f4-8bd75e00a612.retry

PLAY RECAP

myserver.home.local : ok=0 changed=0 unreachable=1 failed=0

[WARNING]: Failure using method (v2_playbook_on_stats) in callback plugin
(</usr/lib/python2.7/site-
packages/ansible/plugins/callback/foreman.CallbackModule object at
0x1492a10>):
(‘Connection aborted.’, error(111, ‘Connection refused’))

# As the foreman user
-bash-4.2$ ansible-playbook --limitmyserver.home.local
/etc/ansible/roles/redhat-register.yml

PLAY [all]

TASK [setup]

ok: [myserver.home.local]
[WARNING]: Failure using method (v2_runner_on_ok) in callback plugin
(</usr/lib/python2.7/site-packages/ansible/plugins/callback/foreman.CallbackModule
object at 0x2fe60d0>): (‘Connection aborted.’, error(111,
‘Connection refused’))

TASK [debug]

ok: [myserver.home.local] => {
“msg”: “Hello World!”
}

PLAY RECAP

myserver.home.local : ok=2 changed=0 unreachable=0 failed=0

[WARNING]: Failure using method (v2_playbook_on_stats) in callback plugin
(</usr/lib/python2.7/site-packages/ansible/plugins/callback/foreman.CallbackModule
object at 0x2fe60d0>): (‘Connection aborted.’,
error(111, ‘Connection refused’))

# As root

[root@foreman roles]# ansible-playbook --limit myserver.home.local
/etc/ansible/roles/redhat-register.yml

PLAY [all]

TASK [setup]

ok: [myserver.home.local]

TASK [debug]

ok: [myserver.home.local] => {
“msg”: “Hello World!”
}

PLAY RECAP

myserver.home.local : ok=2 changed=0 unreachable=0 failed=0

On Sunday, December 4, 2016 at 1:47:21 PM UTC-5, Bernard Landon wrote:

Hello!

I actually have the same problem while running ansible on the same host
than the foreman.
TCP port 3000 is simply closed. And if I use the env variables to use port
80, I get an auth error.
What’s port 3000 supposed to be the entry point of? The Foreman Web UI or
something else?
I’m confused about how the callback is supposed to authenticate itself.
Any help appreciated :slight_smile:

Thanks!

On Thursday, November 24, 2016 at 10:26:49 AM UTC+1, Daniel Lobato wrote:

On 11/21, vlad...@gochange.co wrote:

Hi, ladies and gentlemen. I’m trying to setup Foreman with Ansible.
Looks like I’m missing something. After the test command on the Ansible
server “ansible -m setup localhost” Ansible writes many green strings
with
configuration info, and in the end where the Callback should be used it
says:

[WARNING]: Failure using method (v2_runner_on_ok) in callback plugin
(</usr/share/ansible/plugins/callback/foreman_callback.CallbackModule
object at 0x16995d0>):
HTTPSConnectionPool(host=‘myAWShost.myRegion.compute.internal’,
port=3000): Max retries exceeded with url: /api/v2/hosts/facts (Caused
by
<class
’socket.error’>: [Errno 111] Connection refused)

The installation seems fine from the rest of your email, don’t worry too
much about that. The problem seems to be that your Foreman host is not
accepting connections from this host on that port (notice that
’connection
refused’ message).

I don’t think the problem is in Foreman itself but rather in how the
host is configured. Can you run that same command, but from the Foreman
host itself? You might see different results if the firewall was
blocking the connection.

Have you checked /var/log/foreman/production.log on the Foreman host?
From the looks of it I don’t think the requests have even got to the
host. I’d check firewall, SELinux, Apache logs…

You should set FOREMAN_SSL_CERT and FOREMAN_SSL_KEY if you intend the
callback to work through HTTPS (or enable http for that Foreman route)

Foreman :: Plugin Manuals
has more info, if you don’t feel like configuring stuff and you have
Ansible 2.2, the callback is shipped by default with it. You should only
have to enable it and set the appropriate variables.

I did everything as in the manual, may be I have ony changed the
default
directory of a plugin (/usr/share/ansible/plugins/), but i made
appropriate changes to Ansible config, as plugin itself works.
In the Foreman UI I see some options appeared related to Ansible, like
"Configure -> Ansible / Roles". Also I added my Ansible server to the
trusted_puppetmaster_hosts and
create_new_host_when_facts_are_uploaded
are ON.
The problem is, as I see, in connection to Foreman. In my Foreman
Callback
.py file *FOREMAN_URL *is the same, as it is in the results of the
following command
openssl s_client -showcerts
-connect myAWShost.myRegion.compute.internal:443
This same address was by default. Also I copied Foreman’s cert .pem and
ssl
.pem to the Ansible server and added this paths in the
foreman_callback.py
file.

When I try to turn off FOREMAN_SSL_VERIFY, I’m getting another error:

ERROR! Unexpected Exception: ‘module’ object has no attribute
’packages’
the full traceback was:

Traceback (most recent call last):

  • File “/usr/bin/ansible”, line 92, in *
  • exit_code = cli.run()*
  • File “/usr/lib/python2.6/site-packages/ansible/cli/adhoc.py”, line
    193,
    in run*
  • result = self._tqm.run(play)*
  • File

“/usr/lib/python2.6/site-packages/ansible/executor/task_queue_manager.py”,

line 202, in run*

  • self.load_callbacks()*
  • File

“/usr/lib/python2.6/site-packages/ansible/executor/task_queue_manager.py”,

line 188, in load_callbacks*

  • self._callback_plugins.append(callback_plugin())*
  • File “/usr/share/ansible/plugins/callback/foreman_callback.py”, line
    69,
    in init*
  • self.ssl_verify = self._ssl_verify()*
  • File “/usr/share/ansible/plugins/callback/foreman_callback.py”, line
    83,
    in _ssl_verify*
  • requests.packages.urllib3.disable_warnings()*
    AttributeError: ‘module’ object has no attribute ‘packages’

Need help, what have I done wrong? May be something wrong with the
installation, don’t know…

–
You received this message because you are subscribed to the Google
Groups “Foreman users” group.
To unsubscribe from this group and stop receiving emails from it, send
an email to foreman-user...@googlegroups.com.
To post to this group, send email to forema...@googlegroups.com.
Visit this group at https://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.

–
Daniel Lobato Garcia

@dLobatog
blog.daniellobato.me
daniellobato.me

GPG: http://keys.gnupg.net/pks/lookup?op=get&search=0x7A92D6DD38D6DE30
Keybase: https://keybase.io/elobato

5

So you do have a service running on port 3000 or you export the url variable to use another one?

6

No. It's been a while, but I think when you install Foreman from source or
without passenger then it listens on port 3000. I ran the foreman
installer, so i exported the required variables like so:

export FOREMAN_URL=https://foreman.home.local
export
FOREMAN_SSL_CERT=/etc/puppetlabs/puppet/ssl/certs/foreman.home.local.pem
export
FOREMAN_SSL_KEY=/etc/puppetlabs/puppet/ssl/private_keys/foreman.home.local.pem
export FOREMAN_SSL_VERIFY=/etc/puppetlabs/puppet/ssl/certs/ca.pem

@dLobatog I wonder why not just have the callback script read
/etc/ansible/foreman.ini? Just like the foreman inventory script, seems
pretty straight forward.

··· On Sun, Dec 4, 2016 at 6:44 PM, Bernard Landon wrote:

So you do have a service running on port 3000 or you export the url
variable to use another one?

–
You received this message because you are subscribed to the Google Groups
"Foreman users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at https://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.

–
swygue neron —>>

7

This error means that the client cannot connect to the port on the computer running server script. This can be caused by few things, like lack of routing to the destination or you have a firewall somewhere between your client and the server - it could be on server itself or on the client etc. Note that a server must perform the sequence socket(), bind(), listen(), accept() (possibly repeating the accept() to service more than one client), while a client only needs the sequence socket(), connect(). Also note that the server does not sendall()/recv() on the socket it is listening on but on the new socket returned by accept(). Try the following:

  • Check if you really have that port listening on the server (this should tell you if your code does what you think it should): based on you OS, but on linux you could do something like netstat -ntulp
  • Check from the server, if you’re accepting the connections to the server: again based on your OS, but telnet LISTENING_IP LISTENING_PORT should do the job
  • Check if you can access the port of the server from the client , but not using the code: just us the telnet (or appropriate command for your OS) from the client