Has anyone else noticed increased memory usage by the foreman ruby
processes after upgrading? Our foreman servers used to be able to manage 20
odd nodes easily with 1GB memory, after the ugprade they are now maxing out
the memory with foreman:ruby being the main culprit.
Yeah, since upgrading to 1.8.1 we've had to double the amount of RAM on our
Foreman server.
···
On Friday, May 29, 2015 at 1:14:42 AM UTC-7, Andrew Lau wrote:
>
> Hi,
>
> Has anyone else noticed increased memory usage by the foreman ruby
> processes after upgrading? Our foreman servers used to be able to manage 20
> odd nodes easily with 1GB memory, after the ugprade they are now maxing out
> the memory with foreman:ruby being the main culprit.
>
> Regards,
> Andrew.
>
Are you able to tell if the number of processes usually running has
increased, or if individual processes are taking up more memory?
Running "passenger-status" is an easy way to check the current number of
processes and the amount of memory they're roughly taking up.
Also check for any stray rake processes, especially when using trends.
···
On 29/05/15 09:14, Andrew Lau wrote:
> Hi,
>
> Has anyone else noticed increased memory usage by the foreman ruby
> processes after upgrading? Our foreman servers used to be able to manage
> 20 odd nodes easily with 1GB memory, after the ugprade they are now
> maxing out the memory with foreman:ruby being the main culprit.
> > Hi,
> >
> > Has anyone else noticed increased memory usage by the foreman ruby
> > processes after upgrading? Our foreman servers used to be able to manage
> > 20 odd nodes easily with 1GB memory, after the ugprade they are now
> > maxing out the memory with foreman:ruby being the main culprit.
>
> Are you able to tell if the number of processes usually running has
> increased, or if individual processes are taking up more memory?
>
> Running "passenger-status" is an easy way to check the current number of
> processes and the amount of memory they're roughly taking up.
>
> Also check for any stray rake processes, especially when using trends.
>
Appears to be UI related, perhaps a stray ajax call that leads to a memory
leak. I haven't noticed much now that everything's configured, if someone
logs in and makes about 10+ changes to puppet classes I noticed there was 3
ruby processes (2x foreman, 1x puppet). Right now there's only 1xforeman
and 1x puppet. The additional memory usage didn't die until a reboot or
restart of httpd.
···
On 2 June 2015 at 20:36, Dominic Cleal wrote:
> On 29/05/15 09:14, Andrew Lau wrote:
Ah okay, so additional processes would make sense as there is more AJAX
in 1.8 versus 1.7.
You may want to try setting and reducing PassengerPoolIdleTime from its
default of 300 seconds
(https://www.phusionpassenger.com/documentation/Users%20guide%20Apache.html)
to ensure that second instance is stopped sooner, but I don't think you
can get away from the need for enough RAM for both processes at once.
The other thing worth mentioning is to ensure your installation is
minimal, as it has a large effect on the memory footprint. If you're
not using some plugins or compute resource support, remove the packages.
···
On 03/06/15 12:25, Andrew Lau wrote:
>
> On 2 June 2015 at 20:36, Dominic Cleal > wrote:
>
> On 29/05/15 09:14, Andrew Lau wrote:
> > Hi,
> >
> > Has anyone else noticed increased memory usage by the foreman ruby
> > processes after upgrading? Our foreman servers used to be able to manage
> > 20 odd nodes easily with 1GB memory, after the ugprade they are now
> > maxing out the memory with foreman:ruby being the main culprit.
>
> Are you able to tell if the number of processes usually running has
> increased, or if individual processes are taking up more memory?
>
> Running "passenger-status" is an easy way to check the current number of
> processes and the amount of memory they're roughly taking up.
>
> Also check for any stray rake processes, especially when using trends.
>
>
> Appears to be UI related, perhaps a stray ajax call that leads to a
> memory leak. I haven't noticed much now that everything's configured, if
> someone logs in and makes about 10+ changes to puppet classes I noticed
> there was 3 ruby processes (2x foreman, 1x puppet). Right now there's
> only 1xforeman and 1x puppet. The additional memory usage didn't die
> until a reboot or restart of httpd.
>
> > I upgraded 3 servers from 1.8.0, 1.8.0 RC3? and 1.7.x
>
> Can you confirm it's a regression between 1.8 and 1.8.1 ?
>
I wish I could, otherwise I would've opened a bug report instead. I didn't
notice it while on 1.8RC3 or 1.8.0 but I was only on those versions for
about a week.
···
On Tuesday, 2 June 2015 18:34:43 UTC+10, Lukas Zapletal wrote:
Are you able to identify type of workload that is causing high memory usage? Is it lots of UI requests? Lots of API requests? Lots of incoming reports? Or ENC workload?