Foreman remote execution fails

Support
1

i have just installed the latest version of foreman (1.13.0) on Centos 7.2.
everything works well except when i am trying to run remote execution on a
machine (installed via foreman-installer) it fails with :

1:
Error initializing command: Net::SSH::ConnectionTimeout -
Net::SSH::ConnectionTimeout
2:
Exit status: EXCEPTION

basically it says "pending" on the job status and after about 2 minutes it
throws the above message on the host details.

it seems the remote host doesn't get the request for some reason.

all features are enabled on the smart proxy section.
FW & Selinux are disabled.

according to foreman-proxy logs there is nothing suspicious:

accept: 192.168.249.222:49686
D, [2016-10-19T11:35:35.979099 #5730] DEBUG – : Rack::Handler::WEBrick is
invoked.
D, [2016-10-19T11:35:35.988823 #5730] DEBUG – : verifying remote client
192.168.220.222 against trusted_hosts ["foreman.example.com"]
D, [2016-10-19T11:35:35.991867 #5730] DEBUG – : Proxy request from
foreman.example.com:8443/dynflow/tasks/count to https://foreman.example.com
:8008/tasks/
count
D, [2016-10-19T11:35:36.081274 #5730] DEBUG – : Proxy request status 200 -
#<Net::HTTPOK:0x000000025f36b8>
I, [2016-10-19T11:35:36.082396 #5730] INFO – : 192.168.220.222 - -
[19/Oct/2016:11:35:36 +0000] "GET /dynflow/tasks/count?state=running
HTTP/1.1" 200 29 0.0939

D, [2016-10-19T11:35:36.123514 #5730] DEBUG – : close:
192.168.220.222:49686
D, [2016-10-19T11:35:36.343377 #5730] DEBUG – : accept:
192.168.220.222:49688
D, [2016-10-19T11:35:36.349244 #5730] DEBUG – : Rack::Handler::WEBrick is
invoked.
D, [2016-10-19T11:35:36.350386 #5730] DEBUG – : verifying remote client
192.168.220.222 against trusted_hosts ["foreman.example.com"]
D, [2016-10-19T11:35:36.350751 #5730] DEBUG – : Proxy request from
foreman.example.com:8443/dynflow/tasks/ to https://foreman.example.com:8008/tasks/

D, [2016-10-19T11:35:36.852074 #5730] DEBUG – : Proxy request status 200 -
#<Net::HTTPOK:0x00000002a812e8>
I, [2016-10-19T11:35:36.852681 #5730] INFO – : 192.168.220.222 - -
[19/Oct/2016:11:35:36 +0000] "POST /dynflow/tasks/ HTTP/1.1" 200 50 0.5026

D, [2016-10-19T11:35:36.893861 #5730] DEBUG – : close:
192.168.220.222:49688
D, [2016-10-19T11:36:36.346125 #5730] DEBUG – : accept:
192.168.220.222:49697
D, [2016-10-19T11:36:36.352459 #5730] DEBUG – : Rack::Handler::WEBrick is
invoked.
D, [2016-10-19T11:36:36.353314 #5730] DEBUG – : verifying remote client
192.168.220.222 against trusted_hosts ["foreman.example.com"]
D, [2016-10-19T11:36:36.353691 #5730] DEBUG – : Proxy request from
foreman.example.com:8443/dynflow/tasks/1063de40-3671-4a2d-9370-22068b642009/status
to http
s://foreman.example.com:8008/tasks/1063de40-3671-4a2d-9370-22068b642009/status

D, [2016-10-19T11:36:36.406618 #5730] DEBUG – : Proxy request status 200 -
#<Net::HTTPOK:0x00000002afc178>
I, [2016-10-19T11:36:36.407055 #5730] INFO – : 192.168.220.222 - -
[19/Oct/2016:11:36:36 +0000] "GET
/dynflow/tasks/1063de40-3671-4a2d-9370-22068b642009/status
HTTP/1.1" 200 1882 0.0539

D, [2016-10-19T11:36:36.447722 #5730] DEBUG – : close:
192.168.220.222:49697

any ideas what could be the reason ? is it possibly a bug ?

2

Is the target host configured to accept the SSH connection from the proxy?

https://theforeman.org/plugins/foreman_remote_execution/0.3/index.html#2.2RemoteHostsConfiguration

··· On Wednesday, 19 October 2016 13:00:17 UTC+1, Dor Azoulay wrote: > > > i have just installed the latest version of foreman (1.13.0) on Centos 7.2. > everything works well except when i am trying to run remote execution on a > machine (installed via foreman-installer) it fails with : > > 1: > Error initializing command: Net::SSH::ConnectionTimeout - > Net::SSH::ConnectionTimeout > 2: > Exit status: EXCEPTION > > basically it says "pending" on the job status and after about 2 minutes it > throws the above message on the host details. > > it seems the remote host doesn't get the request for some reason. > > all features are enabled on the smart proxy section. > FW & Selinux are disabled. > > according to foreman-proxy logs there is nothing suspicious: > > accept: 192.168.249.222:49686 > D, [2016-10-19T11:35:35.979099 #5730] DEBUG -- : Rack::Handler::WEBrick is > invoked. > D, [2016-10-19T11:35:35.988823 #5730] DEBUG -- : verifying remote client > 192.168.220.222 against trusted_hosts ["foreman.example.com"] > D, [2016-10-19T11:35:35.991867 #5730] DEBUG -- : Proxy request from > foreman.example.com:8443/dynflow/tasks/count to https:// > foreman.example.com:8008/tasks/ > count > D, [2016-10-19T11:35:36.081274 #5730] DEBUG -- : Proxy request status 200 > - # > I, [2016-10-19T11:35:36.082396 #5730] INFO -- : 192.168.220.222 - - > [19/Oct/2016:11:35:36 +0000] "GET /dynflow/tasks/count?state=running > HTTP/1.1" 200 29 0.0939 > > D, [2016-10-19T11:35:36.123514 #5730] DEBUG -- : close: > 192.168.220.222:49686 > D, [2016-10-19T11:35:36.343377 #5730] DEBUG -- : accept: > 192.168.220.222:49688 > D, [2016-10-19T11:35:36.349244 #5730] DEBUG -- : Rack::Handler::WEBrick is > invoked. > D, [2016-10-19T11:35:36.350386 #5730] DEBUG -- : verifying remote client > 192.168.220.222 against trusted_hosts ["foreman.example.com"] > D, [2016-10-19T11:35:36.350751 #5730] DEBUG -- : Proxy request from > foreman.example.com:8443/dynflow/tasks/ to https://foreman.example.com:8008/tasks/ > > D, [2016-10-19T11:35:36.852074 #5730] DEBUG -- : Proxy request status 200 > - # > I, [2016-10-19T11:35:36.852681 #5730] INFO -- : 192.168.220.222 - - > [19/Oct/2016:11:35:36 +0000] "POST /dynflow/tasks/ HTTP/1.1" 200 50 0.5026 > > D, [2016-10-19T11:35:36.893861 #5730] DEBUG -- : close: > 192.168.220.222:49688 > D, [2016-10-19T11:36:36.346125 #5730] DEBUG -- : accept: > 192.168.220.222:49697 > D, [2016-10-19T11:36:36.352459 #5730] DEBUG -- : Rack::Handler::WEBrick is > invoked. > D, [2016-10-19T11:36:36.353314 #5730] DEBUG -- : verifying remote client > 192.168.220.222 against trusted_hosts ["foreman.example.com"] > D, [2016-10-19T11:36:36.353691 #5730] DEBUG -- : Proxy request from > foreman.example.com:8443/dynflow/tasks/1063de40-3671-4a2d-9370-22068b642009/status > to http > s://foreman.example.com:8008/tasks/1063de40-3671-4a2d-9370-22068b642009/status > > D, [2016-10-19T11:36:36.406618 #5730] DEBUG -- : Proxy request status 200 > - # > I, [2016-10-19T11:36:36.407055 #5730] INFO -- : 192.168.220.222 - - > [19/Oct/2016:11:36:36 +0000] "GET > /dynflow/tasks/1063de40-3671-4a2d-9370-22068b642009/status > HTTP/1.1" 200 1882 0.0539 > > D, [2016-10-19T11:36:36.447722 #5730] DEBUG -- : close: > 192.168.220.222:49697 > > > any ideas what could be the reason ? is it possibly a bug ? >
3

Hello,
from what you posted it seems that there is a problem with opening the
SSH connection from proxy to the target host. Could you please check
/var/log/foreman-proxy/smart_proxy_dynflow_core.log? There could be
something helpful there (it is a log file of a smart proxy helper
process which handles the ssh connections). It might help even more if
you could set the log level for this process (in
/etc/smart_proxy_dynflow_core/settings.yml) to DEBUG, rerun the job
invocation and check the logs afterwards.

Hope this helps shed some light into what might be wrong

– Adam

Dor Azoulay <dorazoulay@gmail.com> writes:

··· > i have just installed the latest version of foreman (1.13.0) on Centos 7.2. > everything works well except when i am trying to run remote execution on a > machine (installed via foreman-installer) it fails with : > > 1: > Error initializing command: Net::SSH::ConnectionTimeout - > Net::SSH::ConnectionTimeout > 2: > Exit status: EXCEPTION > > basically it says "pending" on the job status and after about 2 minutes it > throws the above message on the host details. > > it seems the remote host doesn't get the request for some reason. > > all features are enabled on the smart proxy section. > FW & Selinux are disabled. > > according to foreman-proxy logs there is nothing suspicious: > > accept: 192.168.249.222:49686 > D, [2016-10-19T11:35:35.979099 #5730] DEBUG -- : Rack::Handler::WEBrick is > invoked. > D, [2016-10-19T11:35:35.988823 #5730] DEBUG -- : verifying remote client > 192.168.220.222 against trusted_hosts ["foreman.example.com"] > D, [2016-10-19T11:35:35.991867 #5730] DEBUG -- : Proxy request from > foreman.example.com:8443/dynflow/tasks/count to https://foreman.example.com > :8008/tasks/ > count > D, [2016-10-19T11:35:36.081274 #5730] DEBUG -- : Proxy request status 200 - > # > I, [2016-10-19T11:35:36.082396 #5730] INFO -- : 192.168.220.222 - - > [19/Oct/2016:11:35:36 +0000] "GET /dynflow/tasks/count?state=running > HTTP/1.1" 200 29 0.0939 > > D, [2016-10-19T11:35:36.123514 #5730] DEBUG -- : close: > 192.168.220.222:49686 > D, [2016-10-19T11:35:36.343377 #5730] DEBUG -- : accept: > 192.168.220.222:49688 > D, [2016-10-19T11:35:36.349244 #5730] DEBUG -- : Rack::Handler::WEBrick is > invoked. > D, [2016-10-19T11:35:36.350386 #5730] DEBUG -- : verifying remote client > 192.168.220.222 against trusted_hosts ["foreman.example.com"] > D, [2016-10-19T11:35:36.350751 #5730] DEBUG -- : Proxy request from > foreman.example.com:8443/dynflow/tasks/ to https://foreman.example.com:8008/tasks/ > > D, [2016-10-19T11:35:36.852074 #5730] DEBUG -- : Proxy request status 200 - > # > I, [2016-10-19T11:35:36.852681 #5730] INFO -- : 192.168.220.222 - - > [19/Oct/2016:11:35:36 +0000] "POST /dynflow/tasks/ HTTP/1.1" 200 50 0.5026 > > D, [2016-10-19T11:35:36.893861 #5730] DEBUG -- : close: > 192.168.220.222:49688 > D, [2016-10-19T11:36:36.346125 #5730] DEBUG -- : accept: > 192.168.220.222:49697 > D, [2016-10-19T11:36:36.352459 #5730] DEBUG -- : Rack::Handler::WEBrick is > invoked. > D, [2016-10-19T11:36:36.353314 #5730] DEBUG -- : verifying remote client > 192.168.220.222 against trusted_hosts ["foreman.example.com"] > D, [2016-10-19T11:36:36.353691 #5730] DEBUG -- : Proxy request from > foreman.example.com:8443/dynflow/tasks/1063de40-3671-4a2d-9370-22068b642009/status > to http > s://foreman.example.com:8008/tasks/1063de40-3671-4a2d-9370-22068b642009/status > > D, [2016-10-19T11:36:36.406618 #5730] DEBUG -- : Proxy request status 200 - > # > I, [2016-10-19T11:36:36.407055 #5730] INFO -- : 192.168.220.222 - - > [19/Oct/2016:11:36:36 +0000] "GET > /dynflow/tasks/1063de40-3671-4a2d-9370-22068b642009/status > HTTP/1.1" 200 1882 0.0539 > > D, [2016-10-19T11:36:36.447722 #5730] DEBUG -- : close: > 192.168.220.222:49697 > > > any ideas what could be the reason ? is it possibly a bug ? > > -- > You received this message because you are subscribed to the Google Groups "Foreman users" group. > To unsubscribe from this group and stop receiving emails from it, send an email to foreman-users+unsubscribe@googlegroups.com. > To post to this group, send email to foreman-users@googlegroups.com. > Visit this group at https://groups.google.com/group/foreman-users. > For more options, visit https://groups.google.com/d/optout.
4

Yes, the root user on the remote host has foreman-proxy ssh public key,
anything else i might have missed ?

··· On Wednesday, October 19, 2016 at 3:54:06 PM UTC+3, Duncan Innes wrote: > > Is the target host configured to accept the SSH connection from the proxy? > > > https://theforeman.org/plugins/foreman_remote_execution/0.3/index.html#2.2RemoteHostsConfiguration > > On Wednesday, 19 October 2016 13:00:17 UTC+1, Dor Azoulay wrote: >> >> >> i have just installed the latest version of foreman (1.13.0) on Centos >> 7.2. >> everything works well except when i am trying to run remote execution on >> a machine (installed via foreman-installer) it fails with : >> >> 1: >> Error initializing command: Net::SSH::ConnectionTimeout - >> Net::SSH::ConnectionTimeout >> 2: >> Exit status: EXCEPTION >> >> basically it says "pending" on the job status and after about 2 minutes >> it throws the above message on the host details. >> >> it seems the remote host doesn't get the request for some reason. >> >> all features are enabled on the smart proxy section. >> FW & Selinux are disabled. >> >> according to foreman-proxy logs there is nothing suspicious: >> >> accept: 192.168.249.222:49686 >> D, [2016-10-19T11:35:35.979099 #5730] DEBUG -- : Rack::Handler::WEBrick >> is invoked. >> D, [2016-10-19T11:35:35.988823 #5730] DEBUG -- : verifying remote client >> 192.168.220.222 against trusted_hosts ["foreman.example.com"] >> D, [2016-10-19T11:35:35.991867 #5730] DEBUG -- : Proxy request from >> foreman.example.com:8443/dynflow/tasks/count to https:// >> foreman.example.com:8008/tasks/ >> count >> D, [2016-10-19T11:35:36.081274 #5730] DEBUG -- : Proxy request status 200 >> - # >> I, [2016-10-19T11:35:36.082396 #5730] INFO -- : 192.168.220.222 - - >> [19/Oct/2016:11:35:36 +0000] "GET /dynflow/tasks/count?state=running >> HTTP/1.1" 200 29 0.0939 >> >> D, [2016-10-19T11:35:36.123514 #5730] DEBUG -- : close: >> 192.168.220.222:49686 >> D, [2016-10-19T11:35:36.343377 #5730] DEBUG -- : accept: >> 192.168.220.222:49688 >> D, [2016-10-19T11:35:36.349244 #5730] DEBUG -- : Rack::Handler::WEBrick >> is invoked. >> D, [2016-10-19T11:35:36.350386 #5730] DEBUG -- : verifying remote client >> 192.168.220.222 against trusted_hosts ["foreman.example.com"] >> D, [2016-10-19T11:35:36.350751 #5730] DEBUG -- : Proxy request from >> foreman.example.com:8443/dynflow/tasks/ to https://foreman.example.com:8008/tasks/ >> >> D, [2016-10-19T11:35:36.852074 #5730] DEBUG -- : Proxy request status 200 >> - # >> I, [2016-10-19T11:35:36.852681 #5730] INFO -- : 192.168.220.222 - - >> [19/Oct/2016:11:35:36 +0000] "POST /dynflow/tasks/ HTTP/1.1" 200 50 0.5026 >> >> D, [2016-10-19T11:35:36.893861 #5730] DEBUG -- : close: >> 192.168.220.222:49688 >> D, [2016-10-19T11:36:36.346125 #5730] DEBUG -- : accept: >> 192.168.220.222:49697 >> D, [2016-10-19T11:36:36.352459 #5730] DEBUG -- : Rack::Handler::WEBrick >> is invoked. >> D, [2016-10-19T11:36:36.353314 #5730] DEBUG -- : verifying remote client >> 192.168.220.222 against trusted_hosts ["foreman.example.com"] >> D, [2016-10-19T11:36:36.353691 #5730] DEBUG -- : Proxy request from >> foreman.example.com:8443/dynflow/tasks/1063de40-3671-4a2d-9370-22068b642009/status >> to http >> s://foreman.example.com:8008/tasks/1063de40-3671-4a2d-9370-22068b642009/status >> >> D, [2016-10-19T11:36:36.406618 #5730] DEBUG -- : Proxy request status 200 >> - # >> I, [2016-10-19T11:36:36.407055 #5730] INFO -- : 192.168.220.222 - - >> [19/Oct/2016:11:36:36 +0000] "GET >> /dynflow/tasks/1063de40-3671-4a2d-9370-22068b642009/status >> HTTP/1.1" 200 1882 0.0539 >> >> D, [2016-10-19T11:36:36.447722 #5730] DEBUG -- : close: >> 192.168.220.222:49697 >> >> >> any ideas what could be the reason ? is it possibly a bug ? >> >
5

Thanks for the insight, i didn't knew it has separate logs.

i have enabled debug log, and it seems like foreman-proxy is trying to
access the remote host with different IP ( it resolves to the host
secondary IP ) which results in connection error.
i have tried it with several hosts which have the same issue.

the good news are that there are other hosts now which works OK, so i guess
the problem is within resolving the hosts ips.

Thank you very much, your input regarding the debug logs helped a lot.

··· On Wednesday, October 19, 2016 at 3:10:54 PM UTC+3, aruz...@redhat.com wrote: > > Hello, > from what you posted it seems that there is a problem with opening the > SSH connection from proxy to the target host. Could you please check > /var/log/foreman-proxy/smart_proxy_dynflow_core.log? There could be > something helpful there (it is a log file of a smart proxy helper > process which handles the ssh connections). It might help even more if > you could set the log level for this process (in > /etc/smart_proxy_dynflow_core/settings.yml) to DEBUG, rerun the job > invocation and check the logs afterwards. > > Hope this helps shed some light into what might be wrong > > -- Adam > > Dor Azoulay <doraz...@gmail.com > writes: > > > i have just installed the latest version of foreman (1.13.0) on Centos > 7.2. > > everything works well except when i am trying to run remote execution on > a > > machine (installed via foreman-installer) it fails with : > > > > 1: > > Error initializing command: Net::SSH::ConnectionTimeout - > > Net::SSH::ConnectionTimeout > > 2: > > Exit status: EXCEPTION > > > > basically it says "pending" on the job status and after about 2 minutes > it > > throws the above message on the host details. > > > > it seems the remote host doesn't get the request for some reason. > > > > all features are enabled on the smart proxy section. > > FW & Selinux are disabled. > > > > according to foreman-proxy logs there is nothing suspicious: > > > > accept: 192.168.249.222:49686 > > D, [2016-10-19T11:35:35.979099 #5730] DEBUG -- : Rack::Handler::WEBrick > is > > invoked. > > D, [2016-10-19T11:35:35.988823 #5730] DEBUG -- : verifying remote client > > 192.168.220.222 against trusted_hosts ["foreman.example.com"] > > D, [2016-10-19T11:35:35.991867 #5730] DEBUG -- : Proxy request from > > foreman.example.com:8443/dynflow/tasks/count to > https://foreman.example.com > > :8008/tasks/ > > count > > D, [2016-10-19T11:35:36.081274 #5730] DEBUG -- : Proxy request status > 200 - > > # > > I, [2016-10-19T11:35:36.082396 #5730] INFO -- : 192.168.220.222 - - > > [19/Oct/2016:11:35:36 +0000] "GET /dynflow/tasks/count?state=running > > HTTP/1.1" 200 29 0.0939 > > > > D, [2016-10-19T11:35:36.123514 #5730] DEBUG -- : close: > > 192.168.220.222:49686 > > D, [2016-10-19T11:35:36.343377 #5730] DEBUG -- : accept: > > 192.168.220.222:49688 > > D, [2016-10-19T11:35:36.349244 #5730] DEBUG -- : Rack::Handler::WEBrick > is > > invoked. > > D, [2016-10-19T11:35:36.350386 #5730] DEBUG -- : verifying remote client > > 192.168.220.222 against trusted_hosts ["foreman.example.com"] > > D, [2016-10-19T11:35:36.350751 #5730] DEBUG -- : Proxy request from > > foreman.example.com:8443/dynflow/tasks/ to > https://foreman.example.com:8008/tasks/ > > > > D, [2016-10-19T11:35:36.852074 #5730] DEBUG -- : Proxy request status > 200 - > > # > > I, [2016-10-19T11:35:36.852681 #5730] INFO -- : 192.168.220.222 - - > > [19/Oct/2016:11:35:36 +0000] "POST /dynflow/tasks/ HTTP/1.1" 200 50 > 0.5026 > > > > D, [2016-10-19T11:35:36.893861 #5730] DEBUG -- : close: > > 192.168.220.222:49688 > > D, [2016-10-19T11:36:36.346125 #5730] DEBUG -- : accept: > > 192.168.220.222:49697 > > D, [2016-10-19T11:36:36.352459 #5730] DEBUG -- : Rack::Handler::WEBrick > is > > invoked. > > D, [2016-10-19T11:36:36.353314 #5730] DEBUG -- : verifying remote client > > 192.168.220.222 against trusted_hosts ["foreman.example.com"] > > D, [2016-10-19T11:36:36.353691 #5730] DEBUG -- : Proxy request from > > > foreman.example.com:8443/dynflow/tasks/1063de40-3671-4a2d-9370-22068b642009/status > > to http > > s:// > foreman.example.com:8008/tasks/1063de40-3671-4a2d-9370-22068b642009/status > > > > D, [2016-10-19T11:36:36.406618 #5730] DEBUG -- : Proxy request status > 200 - > > # > > I, [2016-10-19T11:36:36.407055 #5730] INFO -- : 192.168.220.222 - - > > [19/Oct/2016:11:36:36 +0000] "GET > > /dynflow/tasks/1063de40-3671-4a2d-9370-22068b642009/status > > HTTP/1.1" 200 1882 0.0539 > > > > D, [2016-10-19T11:36:36.447722 #5730] DEBUG -- : close: > > 192.168.220.222:49697 > > > > > > any ideas what could be the reason ? is it possibly a bug ? > > > > -- > > You received this message because you are subscribed to the Google > Groups "Foreman users" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to foreman-user...@googlegroups.com . > > To post to this group, send email to forema...@googlegroups.com > . > > Visit this group at https://groups.google.com/group/foreman-users. > > For more options, visit https://groups.google.com/d/optout. >
6

2 mins sounds like a timeout. I only got those delays when the key wasn't
properly in place so the target was waiting for a password. Can you test
manually SSHing from Satellite to the target host? Is root allowed to SSH
into the target?

Other issue I faced was /var/tmp being mounted noexec, but this comes back
pretty quickly with a fail.

··· On Wednesday, 19 October 2016 14:38:46 UTC+1, Dor Azoulay wrote: > > Yes, the root user on the remote host has foreman-proxy ssh public key, > anything else i might have missed ? > > On Wednesday, October 19, 2016 at 3:54:06 PM UTC+3, Duncan Innes wrote: >> >> Is the target host configured to accept the SSH connection from the proxy? >> >> >> https://theforeman.org/plugins/foreman_remote_execution/0.3/index.html#2.2RemoteHostsConfiguration >> >> On Wednesday, 19 October 2016 13:00:17 UTC+1, Dor Azoulay wrote: >>> >>> >>> i have just installed the latest version of foreman (1.13.0) on Centos >>> 7.2. >>> everything works well except when i am trying to run remote execution on >>> a machine (installed via foreman-installer) it fails with : >>> >>> 1: >>> Error initializing command: Net::SSH::ConnectionTimeout - >>> Net::SSH::ConnectionTimeout >>> 2: >>> Exit status: EXCEPTION >>> >>> basically it says "pending" on the job status and after about 2 minutes >>> it throws the above message on the host details. >>> >>> it seems the remote host doesn't get the request for some reason. >>> >>> all features are enabled on the smart proxy section. >>> FW & Selinux are disabled. >>> >>> according to foreman-proxy logs there is nothing suspicious: >>> >>> accept: 192.168.249.222:49686 >>> D, [2016-10-19T11:35:35.979099 #5730] DEBUG -- : Rack::Handler::WEBrick >>> is invoked. >>> D, [2016-10-19T11:35:35.988823 #5730] DEBUG -- : verifying remote client >>> 192.168.220.222 against trusted_hosts ["foreman.example.com"] >>> D, [2016-10-19T11:35:35.991867 #5730] DEBUG -- : Proxy request from >>> foreman.example.com:8443/dynflow/tasks/count to https:// >>> foreman.example.com:8008/tasks/ >>> count >>> D, [2016-10-19T11:35:36.081274 #5730] DEBUG -- : Proxy request status >>> 200 - # >>> I, [2016-10-19T11:35:36.082396 #5730] INFO -- : 192.168.220.222 - - >>> [19/Oct/2016:11:35:36 +0000] "GET /dynflow/tasks/count?state=running >>> HTTP/1.1" 200 29 0.0939 >>> >>> D, [2016-10-19T11:35:36.123514 #5730] DEBUG -- : close: >>> 192.168.220.222:49686 >>> D, [2016-10-19T11:35:36.343377 #5730] DEBUG -- : accept: >>> 192.168.220.222:49688 >>> D, [2016-10-19T11:35:36.349244 #5730] DEBUG -- : Rack::Handler::WEBrick >>> is invoked. >>> D, [2016-10-19T11:35:36.350386 #5730] DEBUG -- : verifying remote client >>> 192.168.220.222 against trusted_hosts ["foreman.example.com"] >>> D, [2016-10-19T11:35:36.350751 #5730] DEBUG -- : Proxy request from >>> foreman.example.com:8443/dynflow/tasks/ to https://foreman.example.com:8008/tasks/ >>> >>> D, [2016-10-19T11:35:36.852074 #5730] DEBUG -- : Proxy request status >>> 200 - # >>> I, [2016-10-19T11:35:36.852681 #5730] INFO -- : 192.168.220.222 - - >>> [19/Oct/2016:11:35:36 +0000] "POST /dynflow/tasks/ HTTP/1.1" 200 50 0.5026 >>> >>> D, [2016-10-19T11:35:36.893861 #5730] DEBUG -- : close: >>> 192.168.220.222:49688 >>> D, [2016-10-19T11:36:36.346125 #5730] DEBUG -- : accept: >>> 192.168.220.222:49697 >>> D, [2016-10-19T11:36:36.352459 #5730] DEBUG -- : Rack::Handler::WEBrick >>> is invoked. >>> D, [2016-10-19T11:36:36.353314 #5730] DEBUG -- : verifying remote client >>> 192.168.220.222 against trusted_hosts ["foreman.example.com"] >>> D, [2016-10-19T11:36:36.353691 #5730] DEBUG -- : Proxy request from >>> foreman.example.com:8443/dynflow/tasks/1063de40-3671-4a2d-9370-22068b642009/status >>> to http >>> s://foreman.example.com:8008/tasks/1063de40-3671-4a2d-9370-22068b642009/status >>> >>> D, [2016-10-19T11:36:36.406618 #5730] DEBUG -- : Proxy request status >>> 200 - # >>> I, [2016-10-19T11:36:36.407055 #5730] INFO -- : 192.168.220.222 - - >>> [19/Oct/2016:11:36:36 +0000] "GET >>> /dynflow/tasks/1063de40-3671-4a2d-9370-22068b642009/status >>> HTTP/1.1" 200 1882 0.0539 >>> >>> D, [2016-10-19T11:36:36.447722 #5730] DEBUG -- : close: >>> 192.168.220.222:49697 >>> >>> >>> any ideas what could be the reason ? is it possibly a bug ? >>> >>