Hi,
We have been using katello/foreman for quite a while, and our node grow
from something like 10 to around 500 now, the katello server was running on
a VM that we extended CPU cores and memory couple of times and now it was
set 16 cores and 32G memories, but sometimes we still saw some of our
puppet node run into error due to "server busy", and puppet master service
seems to be busy, is there any way that we can tune the performance or
scale out by adding more katello servers?
Thanks
-Sinux
> Hi,
>
> We have been using katello/foreman for quite a while, and our node grow
> from something like 10 to around 500 now, the katello server was running on
> a VM that we extended CPU cores and memory couple of times and now it was
> set 16 cores and 32G memories, but sometimes we still saw some of our
> puppet node run into error due to "server busy", and puppet master service
> seems to be busy, is there any way that we can tune the performance or
> scale out by adding more katello servers?
>
You should start by identifing what consume your resources, for example, if
its the puppet master services, then its pretty easy to move away the
puppet master service to another vms.
once you identify further, we can probably help with better scale / LB, you
can also learn a bit from a recent blog about HA/LB [1]
Ohad
[1] Foreman :: Journey to High Availability
···
On Thu, Mar 17, 2016 at 9:33 AM, sinux shen wrote:
Thanks
-Sinux
–
You received this message because you are subscribed to the Google Groups
"Foreman users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at https://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.
Hi, Ohad,
Yes, this is exactly the puppet master that caused our CPU usage very high,
I actually tried to offload the puppet master by adding capsule server that
only run puppet master, but I have some problem while I run a puppet agent
with puppet CA pointing to katello server while puppet master pointing to a
new master, here is some error, it seems to be some certificate verify
issue:
goldloki ODW [~] 226# puppet agent -t
Notice: Ignoring --listen on onetime run
Warning: Unable to fetch my node definition, but the agent run will
continue:
Warning: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate
B: certificate verify failed: [self signed certificate in certificate chain
for /CN=Puppet CA: smartpxyfm.anim.odw.com.cn]
Info: Retrieving pluginfacts
Error: /File[/var/lib/puppet/facts.d]: Failed to generate additional
resources using 'eval_generate': SSL_connect returned=1 errno=0 state=SSLv3
read server certificate B: certificate verify failed: [self signed
certificate in certificate chain for /CN=Puppet CA:
smartpxyfm.anim.odw.com.cn]
Error: /File[/var/lib/puppet/facts.d]: Could not evaluate: Could not
retrieve file metadata for puppet://smartpxyfm.anim.odw.com.cn/pluginfacts:
SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B:
certificate verify failed: [self signed certificate in certificate chain
for /CN=Puppet CA: smartpxyfm.anim.odw.com.cn]
Info: Retrieving plugin
Error: /File[/var/lib/puppet/lib]: Failed to generate additional resources
using 'eval_generate': SSL_connect returned=1 errno=0 state=SSLv3 read
server certificate B: certificate verify failed: [self signed certificate
in certificate chain for /CN=Puppet CA: smartpxyfm.anim.odw.com.cn]
Error: /File[/var/lib/puppet/lib]: Could not evaluate: Could not retrieve
file metadata for puppet://smartpxyfm.anim.odw.com.cn/plugins: SSL_connect
returned=1 errno=0 state=SSLv3 read server certificate B: certificate
verify failed: [self signed certificate in certificate chain for /CN=Puppet
CA: smartpxyfm.anim.odw.com.cn]
Info: Loading facts
Error: Could not retrieve catalog from remote server: SSL_connect
returned=1 errno=0 state=SSLv3 read server certificate B: certificate
verify failed: [self signed certificate in certificate chain for /CN=Puppet
CA: smartpxyfm.anim.odw.com.cn]
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run
Error: Could not send report: SSL_connect returned=1 errno=0 state=SSLv3
read server certificate B: certificate verify failed: [self signed
certificate in certificate chain for /CN=Puppet CA:
smartpxyfm.anim.odw.com.cn]
Please give me some hint about how we can offload puppet from
katello/foreman server,
Thanks
···
On Thursday, March 17, 2016 at 3:44:30 PM UTC+8, ohad wrote:
>
>
>
> On Thu, Mar 17, 2016 at 9:33 AM, sinux shen > wrote:
>
>> Hi,
>>
>> We have been using katello/foreman for quite a while, and our node grow
>> from something like 10 to around 500 now, the katello server was running on
>> a VM that we extended CPU cores and memory couple of times and now it was
>> set 16 cores and 32G memories, but sometimes we still saw some of our
>> puppet node run into error due to "server busy", and puppet master service
>> seems to be busy, is there any way that we can tune the performance or
>> scale out by adding more katello servers?
>>
>
> You should start by identifing what consume your resources, for example,
> if its the puppet master services, then its pretty easy to move away the
> puppet master service to another vms.
>
> once you identify further, we can probably help with better scale / LB,
> you can also learn a bit from a recent blog about HA/LB [1]
>
> Ohad
>
> [1] http://theforeman.org/2015/12/journey_to_high_availability.html
>
>
>
>>
>> Thanks
>> -Sinux
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Foreman users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to foreman-user...@googlegroups.com .
>> To post to this group, send email to forema...@googlegroups.com
>> .
>> Visit this group at https://groups.google.com/group/foreman-users.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
Hi Ohad,
I am a little confused about how to split puppet ca and puppet master in
foreman, what I did was to setup a smart proxy that only run puppet, I
manually changed one of my test host and point puppet server to the new
smart proxy, but it gave me the error that I pasted in my previous email.
Thanks
-Sinux
···
On Friday, March 18, 2016 at 7:48:50 PM UTC+8, sinux shen wrote:
>
> Hi, Ohad,
>
> Yes, this is exactly the puppet master that caused our CPU usage very
> high, I actually tried to offload the puppet master by adding capsule
> server that only run puppet master, but I have some problem while I run a
> puppet agent with puppet CA pointing to katello server while puppet master
> pointing to a new master, here is some error, it seems to be some
> certificate verify issue:
>
> goldloki ODW [~] 226# puppet agent -t
> Notice: Ignoring --listen on onetime run
> Warning: Unable to fetch my node definition, but the agent run will
> continue:
> Warning: SSL_connect returned=1 errno=0 state=SSLv3 read server
> certificate B: certificate verify failed: [self signed certificate in
> certificate chain for /CN=Puppet CA: smartpxyfm.anim.odw.com.cn]
> Info: Retrieving pluginfacts
> Error: /File[/var/lib/puppet/facts.d]: Failed to generate additional
> resources using 'eval_generate': SSL_connect returned=1 errno=0 state=SSLv3
> read server certificate B: certificate verify failed: [self signed
> certificate in certificate chain for /CN=Puppet CA:
> smartpxyfm.anim.odw.com.cn]
> Error: /File[/var/lib/puppet/facts.d]: Could not evaluate: Could not
> retrieve file metadata for puppet://smartpxyfm.anim.odw.com.cn/pluginfacts:
> SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B:
> certificate verify failed: [self signed certificate in certificate chain
> for /CN=Puppet CA: smartpxyfm.anim.odw.com.cn]
> Info: Retrieving plugin
> Error: /File[/var/lib/puppet/lib]: Failed to generate additional resources
> using 'eval_generate': SSL_connect returned=1 errno=0 state=SSLv3 read
> server certificate B: certificate verify failed: [self signed certificate
> in certificate chain for /CN=Puppet CA: smartpxyfm.anim.odw.com.cn]
> Error: /File[/var/lib/puppet/lib]: Could not evaluate: Could not retrieve
> file metadata for puppet://smartpxyfm.anim.odw.com.cn/plugins:
> SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B:
> certificate verify failed: [self signed certificate in certificate chain
> for /CN=Puppet CA: smartpxyfm.anim.odw.com.cn]
> Info: Loading facts
> Error: Could not retrieve catalog from remote server: SSL_connect
> returned=1 errno=0 state=SSLv3 read server certificate B: certificate
> verify failed: [self signed certificate in certificate chain for /CN=Puppet
> CA: smartpxyfm.anim.odw.com.cn]
> Warning: Not using cache on failed catalog
> Error: Could not retrieve catalog; skipping run
> Error: Could not send report: SSL_connect returned=1 errno=0 state=SSLv3
> read server certificate B: certificate verify failed: [self signed
> certificate in certificate chain for /CN=Puppet CA:
> smartpxyfm.anim.odw.com.cn]
>
> Please give me some hint about how we can offload puppet from
> katello/foreman server,
>
> Thanks
>
> On Thursday, March 17, 2016 at 3:44:30 PM UTC+8, ohad wrote:
>>
>>
>>
>> On Thu, Mar 17, 2016 at 9:33 AM, sinux shen wrote:
>>
>>> Hi,
>>>
>>> We have been using katello/foreman for quite a while, and our node grow
>>> from something like 10 to around 500 now, the katello server was running on
>>> a VM that we extended CPU cores and memory couple of times and now it was
>>> set 16 cores and 32G memories, but sometimes we still saw some of our
>>> puppet node run into error due to "server busy", and puppet master service
>>> seems to be busy, is there any way that we can tune the performance or
>>> scale out by adding more katello servers?
>>>
>>
>> You should start by identifing what consume your resources, for example,
>> if its the puppet master services, then its pretty easy to move away the
>> puppet master service to another vms.
>>
>> once you identify further, we can probably help with better scale / LB,
>> you can also learn a bit from a recent blog about HA/LB [1]
>>
>> Ohad
>>
>> [1] http://theforeman.org/2015/12/journey_to_high_availability.html
>>
>>
>>
>>>
>>> Thanks
>>> -Sinux
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Foreman users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to foreman-user...@googlegroups.com.
>>> To post to this group, send email to forema...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/foreman-users.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
Another thing I am confused now is how to make puppet master certificated
signed by puppet CA, because it seems that when I run puppet agent on a
client, it shows me the certificated chain verified failed
···
On Monday, March 21, 2016 at 9:42:16 AM UTC+8, sinux shen wrote:
>
> Hi Ohad,
>
> I am a little confused about how to split puppet ca and puppet master in
> foreman, what I did was to setup a smart proxy that only run puppet, I
> manually changed one of my test host and point puppet server to the new
> smart proxy, but it gave me the error that I pasted in my previous email.
>
> Thanks
> -Sinux
>
> On Friday, March 18, 2016 at 7:48:50 PM UTC+8, sinux shen wrote:
>>
>> Hi, Ohad,
>>
>> Yes, this is exactly the puppet master that caused our CPU usage very
>> high, I actually tried to offload the puppet master by adding capsule
>> server that only run puppet master, but I have some problem while I run a
>> puppet agent with puppet CA pointing to katello server while puppet master
>> pointing to a new master, here is some error, it seems to be some
>> certificate verify issue:
>>
>> goldloki ODW [~] 226# puppet agent -t
>> Notice: Ignoring --listen on onetime run
>> Warning: Unable to fetch my node definition, but the agent run will
>> continue:
>> Warning: SSL_connect returned=1 errno=0 state=SSLv3 read server
>> certificate B: certificate verify failed: [self signed certificate in
>> certificate chain for /CN=Puppet CA: smartpxyfm.anim.odw.com.cn]
>> Info: Retrieving pluginfacts
>> Error: /File[/var/lib/puppet/facts.d]: Failed to generate additional
>> resources using 'eval_generate': SSL_connect returned=1 errno=0 state=SSLv3
>> read server certificate B: certificate verify failed: [self signed
>> certificate in certificate chain for /CN=Puppet CA:
>> smartpxyfm.anim.odw.com.cn]
>> Error: /File[/var/lib/puppet/facts.d]: Could not evaluate: Could not
>> retrieve file metadata for puppet://
>> smartpxyfm.anim.odw.com.cn/pluginfacts: SSL_connect returned=1 errno=0
>> state=SSLv3 read server certificate B: certificate verify failed: [self
>> signed certificate in certificate chain for /CN=Puppet CA:
>> smartpxyfm.anim.odw.com.cn]
>> Info: Retrieving plugin
>> Error: /File[/var/lib/puppet/lib]: Failed to generate additional
>> resources using 'eval_generate': SSL_connect returned=1 errno=0 state=SSLv3
>> read server certificate B: certificate verify failed: [self signed
>> certificate in certificate chain for /CN=Puppet CA:
>> smartpxyfm.anim.odw.com.cn]
>> Error: /File[/var/lib/puppet/lib]: Could not evaluate: Could not retrieve
>> file metadata for puppet://smartpxyfm.anim.odw.com.cn/plugins:
>> SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B:
>> certificate verify failed: [self signed certificate in certificate chain
>> for /CN=Puppet CA: smartpxyfm.anim.odw.com.cn]
>> Info: Loading facts
>> Error: Could not retrieve catalog from remote server: SSL_connect
>> returned=1 errno=0 state=SSLv3 read server certificate B: certificate
>> verify failed: [self signed certificate in certificate chain for /CN=Puppet
>> CA: smartpxyfm.anim.odw.com.cn]
>> Warning: Not using cache on failed catalog
>> Error: Could not retrieve catalog; skipping run
>> Error: Could not send report: SSL_connect returned=1 errno=0 state=SSLv3
>> read server certificate B: certificate verify failed: [self signed
>> certificate in certificate chain for /CN=Puppet CA:
>> smartpxyfm.anim.odw.com.cn]
>>
>> Please give me some hint about how we can offload puppet from
>> katello/foreman server,
>>
>> Thanks
>>
>> On Thursday, March 17, 2016 at 3:44:30 PM UTC+8, ohad wrote:
>>>
>>>
>>>
>>> On Thu, Mar 17, 2016 at 9:33 AM, sinux shen wrote:
>>>
>>>> Hi,
>>>>
>>>> We have been using katello/foreman for quite a while, and our node grow
>>>> from something like 10 to around 500 now, the katello server was running on
>>>> a VM that we extended CPU cores and memory couple of times and now it was
>>>> set 16 cores and 32G memories, but sometimes we still saw some of our
>>>> puppet node run into error due to "server busy", and puppet master service
>>>> seems to be busy, is there any way that we can tune the performance or
>>>> scale out by adding more katello servers?
>>>>
>>>
>>> You should start by identifing what consume your resources, for example,
>>> if its the puppet master services, then its pretty easy to move away the
>>> puppet master service to another vms.
>>>
>>> once you identify further, we can probably help with better scale / LB,
>>> you can also learn a bit from a recent blog about HA/LB [1]
>>>
>>> Ohad
>>>
>>> [1] http://theforeman.org/2015/12/journey_to_high_availability.html
>>>
>>>
>>>
>>>>
>>>> Thanks
>>>> -Sinux
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Foreman users" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to foreman-user...@googlegroups.com.
>>>> To post to this group, send email to forema...@googlegroups.com.
>>>> Visit this group at https://groups.google.com/group/foreman-users.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>>
