Yo,
I've been woken up a lot by foreman-proxy/pagerduty over the last few
nights, due to simple 'GET /version' checks timing out (e.g. 10s default).
To remedy the situation I bumped it up to 300s but am still getting the odd
alert (seeing about ~30s average response time)… looking deeper:
Top
···
----
28840 foreman+ 20 0 1236184 451452 5720 S 99.5 2.7 13:17.18
ruby
Basically maxing out a core, memory gets as high as 4G virtual and 2G
resident. Bit excessive, n’est pas?
Log
I, [2015-09-29T14:13:47.611369 #28840] INFO – : Enumerated hosts on
10.X.A.0
I, [2015-09-29T14:13:56.711172 #28840] INFO – : Enumerated hosts on
10.X.B.0
10.10.192.137 - - [29/Sep/2015 14:13:59] “GET /version HTTP/1.1” 200 19
0.0005
I, [2015-09-29T14:14:04.767038 #28840] INFO – : Enumerated hosts on
10.X.C.0
I, [2015-09-29T14:14:06.043815 #28840] INFO – : Enumerated hosts on
10.X.A.0
10.10.192.137 - - [29/Sep/2015 14:14:16] “GET /version HTTP/1.1” 200 19
0.0007
I, [2015-09-29T14:14:17.202636 #28840] INFO – : Enumerated hosts on
10.X.D.0
10.10.192.137 - - [29/Sep/2015 14:15:01] “GET /version HTTP/1.1” 200 19
0.0006
I, [2015-09-29T14:15:12.143120 #28840] INFO – : Enumerated hosts on
10.10.C.0
I, [2015-09-29T14:15:18.870285 #28840] INFO – : Enumerated hosts on
10.10.E.0
I don’t know if this is indicative of a problem but it is constantly
enumerating, which again seems excessive as nothing is being provisioned,
so I’ve no idea why this needs to be repeated all the time.
Environment
root@ns0:~# dpkg -l | grep foreman
ii foreman-debug
1.9.2-1 all provides support utility
foreman-debug.
ii foreman-proxy
1.9.2-1 all RESTful proxies for DNS,
DHCP, TFTP, and Puppet
This particular domain has about 150 hosts in it.
So basically a call out (to save my sanity) has anyone noticed this
behaviour, or offer any insight as to why this may be happening?
Regards
Si
–
DataCentred Limited registered in England and Wales no. 05611763
I'm not seeing anything in Redmine about it, and I agree that it's
very odd to be constantly enumerating records. There's definitely no
incoming API requests to the proxy?
It could well be a bug, but I can't see anything on Redmine that is
obviously related. If you have a chance to try to replicate it on a
test VM, that would be awesome. If not, log it with the info you have
and we'll try to take a look.
Cheers,
Greg
···
On 29 September 2015 at 14:22, Simon Murray wrote:
> Yo,
>
> I've been woken up a lot by foreman-proxy/pagerduty over the last few
> nights, due to simple 'GET /version' checks timing out (e.g. 10s default).
> To remedy the situation I bumped it up to 300s but am still getting the odd
> alert (seeing about ~30s average response time)... looking deeper:
>
> Top
> ----
>
> 28840 foreman+ 20 0 1236184 451452 5720 S 99.5 2.7 13:17.18 ruby
>
> Basically maxing out a core, memory gets as high as 4G virtual and 2G
> resident. Bit excessive, n'est pas?
>
> Log
> ---
>
> I, [2015-09-29T14:13:47.611369 #28840] INFO -- : Enumerated hosts on
> 10.X.A.0
> I, [2015-09-29T14:13:56.711172 #28840] INFO -- : Enumerated hosts on
> 10.X.B.0
> 10.10.192.137 - - [29/Sep/2015 14:13:59] "GET /version HTTP/1.1" 200 19
> 0.0005
> I, [2015-09-29T14:14:04.767038 #28840] INFO -- : Enumerated hosts on
> 10.X.C.0
> I, [2015-09-29T14:14:06.043815 #28840] INFO -- : Enumerated hosts on
> 10.X.A.0
> 10.10.192.137 - - [29/Sep/2015 14:14:16] "GET /version HTTP/1.1" 200 19
> 0.0007
> I, [2015-09-29T14:14:17.202636 #28840] INFO -- : Enumerated hosts on
> 10.X.D.0
> 10.10.192.137 - - [29/Sep/2015 14:15:01] "GET /version HTTP/1.1" 200 19
> 0.0006
> I, [2015-09-29T14:15:12.143120 #28840] INFO -- : Enumerated hosts on
> 10.10.C.0
> I, [2015-09-29T14:15:18.870285 #28840] INFO -- : Enumerated hosts on
> 10.10.E.0
>
> I don't know if this is indicative of a problem but it is constantly
> enumerating, which again seems excessive as nothing is being provisioned, so
> I've no idea why this needs to be repeated all the time.
>
> Environment
> -----------
>
> root@ns0:~# dpkg -l | grep foreman
> ii foreman-debug 1.9.2-1
> all provides support utility foreman-debug.
> ii foreman-proxy 1.9.2-1
> all RESTful proxies for DNS, DHCP, TFTP, and Puppet
>
> This particular domain has about 150 hosts in it.
>
> So basically a call out (to save my sanity) has anyone noticed this
> behaviour, or offer any insight as to why this may be happening?
>
> Regards
> Si
