I recently had a thread up here regarding server builds from an isolated
network via a smart proxy. I have now been able to complete a build via
provisioning from the smart proxy back to the master foreman server however
during the post install script puppet is run however it complains that
waitforcert is disabled etc. as it cannot retreieve the CA I suspect. The
problem is with the CA for the host that is set to be the internal Master
Foreman server and not the proxy, for some reason it has to be set to the
master foreman server else the provisioning template doesnt get passed to
the proxy (??) even though the smart proxy has puppetCA enabled.
I'm unsure what needs to be changed to get this working - I would like the
Smart Proxy to be able to act as a puppet master and a PuppetCA as the
isolated hosts have no direct access to the Foreman Master server however
everytime i try and set my proxy to work as a CA it breaks something… :-/
The Template feature has no interaction with the Puppet and PuppetCa
features, last I checked, so I think there's multiple issues here that are
being conflated.
Waitforcert errors usually refer to the appropriate line for the host
missing from the autosign file on it's puppet master. During the
provisioning process (that is, when the host has requested the provision
template and is running kickstart/preseed) check the contents of the
autosign file on the host's master (I guess thats your puppet proxy). It
should contain the name of the host. Check also that the host is
contacting that master in it's post-build run of puppet.
Greg
···
On 14 April 2016 at 13:49, James Denton wrote:
Hi
I recently had a thread up here regarding server builds from an isolated
network via a smart proxy. I have now been able to complete a build via
provisioning from the smart proxy back to the master foreman server however
during the post install script puppet is run however it complains that
waitforcert is disabled etc. as it cannot retreieve the CA I suspect. The
problem is with the CA for the host that is set to be the internal Master
Foreman server and not the proxy, for some reason it has to be set to the
master foreman server else the provisioning template doesnt get passed to
the proxy (??) even though the smart proxy has puppetCA enabled.
I’m unsure what needs to be changed to get this working - I would like the
Smart Proxy to be able to act as a puppet master and a PuppetCA as the
isolated hosts have no direct access to the Foreman Master server however
everytime i try and set my proxy to work as a CA it breaks something… :-/
I wouldn't have thought that the template feature has any interaction with
puppetCA however it does, in this instance, appear to be the case.
The issue(s) i had previously that you responded to (the token not being
obtained from the master via proxy) was solved from setting the Puppet CA
from Proxy to Master.
On the proxy i have set autosign to be *.domain.com as a test as i read
this in a simliar post.
My problems are as follows:
There is no direct comms from the isolated network to the Foreman
Master, only via the smart proxy - Although the PuppetCA feature is enabled
on the Proxy if i run a client against it, it fails (at build time and
performing a manual puppet run).
If i generate a CA on the proxy (seperate from the master) clients
obtain a cert although i get the error "node not found" which i assume to
be that there is an issue between the proxy and master foreman instance.
If the proxy is using its own CA then the master has problems connecting
to it over 8443 so i had to set the proxy (under smart proxies tab) to be
http/8000 - this however doesnt resolve the issue above with "node not
found". If the proxy doesnt have its own CA, Apache/Passenger fails to
start.
I would like the Master to be the only CA and I assumed that the
PuppetCA feature would proxy requests through.
I'm happy to send any log snippets or config files
Thanks again for your support.
···
On Thursday, April 14, 2016 at 7:57:08 PM UTC+1, Greg Sutcliffe wrote:
>
> On 14 April 2016 at 13:49, James Denton <jrden...@gmail.com > > wrote:
>
>> Hi
>>
>> I recently had a thread up here regarding server builds from an isolated
>> network via a smart proxy. I have now been able to complete a build via
>> provisioning from the smart proxy back to the master foreman server however
>> during the post install script puppet is run however it complains that
>> waitforcert is disabled etc. as it cannot retreieve the CA I suspect. The
>> problem is with the CA for the host that is set to be the internal Master
>> Foreman server and not the proxy, for some reason it has to be set to the
>> master foreman server else the provisioning template doesnt get passed to
>> the proxy (??) even though the smart proxy has puppetCA enabled.
>>
>> I'm unsure what needs to be changed to get this working - I would like
>> the Smart Proxy to be able to act as a puppet master and a PuppetCA as the
>> isolated hosts have no direct access to the Foreman Master server however
>> everytime i try and set my proxy to work as a CA it breaks something.. :-/
>>
>
> The Template feature has no interaction with the Puppet and PuppetCa
> features, last I checked, so I think there's multiple issues here that are
> being conflated.
>
> Waitforcert errors usually refer to the appropriate line for the host
> missing from the autosign file on it's puppet master. During the
> provisioning process (that is, when the host has requested the provision
> template and is running kickstart/preseed) check the contents of the
> autosign file on the host's master (I guess thats your puppet proxy). It
> should contain the name of the host. Check also that the host *is*
> contacting *that* master in it's post-build run of puppet.
>
> Greg
>
Here's the confusion, I think. The PuppetCA feature doesn't imply any
proxying, it just represents a puppetmaster+ca box. If the box isn't
capable of signing puppet certs, the smart-proxy shouldn't have that
feature enabled.
In general, what you want should work, but you need to ensure your SSL
traffic is set up correctly. There's several sets of certificate traffic
happening here:
The agent<->puppetmaster traffic
The puppetmaster->ENC (Foreman UI) traffic
The smart-proxy->Foreman traffic
The Foreman->smart-proxy traffic
All of these can be configured separately, although by default (when
everything is on one box) it's all done from a single CA - and that's the
key here. You're mixing CAs across this setup, so that's leading to
communication errors.
If you only plan to have a single puppetmaster (on the smart-proxy) then
it's probably simplest to generate a certificate for the Foreman box from
there, and replace Foreman's current certs. Alternatively, if you must keep
two separate CAs, you'll need to work through each of those traffic types
and ensure communication is right (e.g. for proxy -> Foreman traffic, you
can configure the proxy to use certs from the Foreman CA in
/etc/foreman-proxy/settings.yml; likewise the ENC comms is configured in
/etc/puppet/foreman.yml)
Hope that helps!
Greg
···
On 15 April 2016 at 09:54, James Denton wrote:
Hi Greg
I wouldn’t have thought that the template feature has any interaction with
puppetCA however it does, in this instance, appear to be the case.
The issue(s) i had previously that you responded to (the token not being
obtained from the master via proxy) was solved from setting the Puppet CA
from Proxy to Master.
On the proxy i have set autosign to be *.domain.com as a test as i read
this in a simliar post.
My problems are as follows:
There is no direct comms from the isolated network to the Foreman
Master, only via the smart proxy - Although the PuppetCA feature is enabled
on the Proxy if i run a client against it, it fails (at build time and
performing a manual puppet run).
If i generate a CA on the proxy (seperate from the master) clients
obtain a cert although i get the error “node not found” which i assume to
be that there is an issue between the proxy and master foreman instance.
If the proxy is using its own CA then the master has problems
connecting to it over 8443 so i had to set the proxy (under smart proxies
tab) to be http/8000 - this however doesnt resolve the issue above with
"node not found". If the proxy doesnt have its own CA, Apache/Passenger
fails to start.
I would like the Master to be the only CA and I assumed that the
PuppetCA feature would proxy requests through.
One adjustment that might solve this is to setup Apache on the puppet
master to proxy_pass the certificate requests through to the puppet CA.
PuppetLabs suggests this as a possible CA configuration in the Scaling
Guide: https://docs.puppet.com/guides/scaling_multiple_masters.html#option-2-proxy-certificate-traffic.
As I've been investigating deploying multiple puppet-masters with Foreman,
I've found that it's not easy to setup remote puppet-masters if the entire
network behind them doesn't have access to the ONE puppetCA. I haven't
tested this with Foreman, but I'm working on building a suitable lab
environment. For us, doing the CA proxy method in apache would solve
adding nodes by installing puppet and running puppet agent --test
–server=<remote master> --waitforcert=30, whereas with out the proxy
feature, the remote admin has to be aware and specify the --ca_server flag
and point it to the puppetca server.
Greg, do you think the CA proxy config in apache on the puppet master work
resolve this issue?
···
On Friday, April 15, 2016 at 9:00:12 AM UTC-4, Greg Sutcliffe wrote:
>
> On 15 April 2016 at 09:54, James Denton <jrden...@gmail.com > > wrote:
>
>> Hi Greg
>>
>> I wouldn't have thought that the template feature has any interaction
>> with puppetCA however it does, in this instance, appear to be the case.
>>
>> The issue(s) i had previously that you responded to (the token not being
>> obtained from the master via proxy) was solved from setting the Puppet CA
>> from Proxy to Master.
>>
>> On the proxy i have set autosign to be *.domain.com as a test as i read
>> this in a simliar post.
>>
>> My problems are as follows:
>> 1) There is no direct comms from the isolated network to the Foreman
>> Master, only via the smart proxy - Although the PuppetCA feature is enabled
>> on the Proxy if i run a client against it, it fails (at build time and
>> performing a manual puppet run).
>> 2) If i generate a CA on the proxy (seperate from the master) clients
>> obtain a cert although i get the error "node not found" which i assume to
>> be that there is an issue between the proxy and master foreman instance.
>> 3) If the proxy is using its own CA then the master has problems
>> connecting to it over 8443 so i had to set the proxy (under smart proxies
>> tab) to be http/8000 - this however doesnt resolve the issue above with
>> "node not found". If the proxy doesnt have its own CA, Apache/Passenger
>> fails to start.
>> 4) I would like the Master to be the only CA and I assumed that the
>> PuppetCA feature would proxy requests through.
>>
>
> Here's the confusion, I think. The PuppetCA feature doesn't imply any
> proxying, it just represents a puppetmaster+ca box. If the box isn't
> capable of signing puppet certs, the smart-proxy shouldn't have that
> feature enabled.
>
> In general, what you want should work, but you need to ensure your SSL
> traffic is set up correctly. There's several sets of certificate traffic
> happening here:
>
> * The agent<->puppetmaster traffic
> * The puppetmaster->ENC (Foreman UI) traffic
> * The smart-proxy->Foreman traffic
> * The Foreman->smart-proxy traffic
>
> All of these *can* be configured separately, although by default (when
> everything is on one box) it's all done from a single CA - and that's the
> key here. You're mixing CAs across this setup, so that's leading to
> communication errors.
>
> If you only plan to have a single puppetmaster (on the smart-proxy) then
> it's probably simplest to generate a certificate for the Foreman box from
> there, and replace Foreman's current certs. Alternatively, if you must keep
> two separate CAs, you'll need to work through each of those traffic types
> and ensure communication is right (e.g. for proxy -> Foreman traffic, you
> can configure the proxy to use certs from the Foreman CA in
> /etc/foreman-proxy/settings.yml; likewise the ENC comms is configured in
> /etc/puppet/foreman.yml)
>
> Hope that helps!
> Greg
>
Thank Sean - I now have the CA part working with proxy_pass solution you
provided.
However…
It appears though that I have an issue with the puppet client server
retreiving its node definition from the Foreman-Proxy/Puppetmaster server.
I would of thought that the smart proxy obtains the node definition from
the Foreman Master but when i run puppet manually it times out and cannot
find it.
The following log file extract is taken from the Smart Proxy:
Apr 19 11:26:37 foreman-proxy puppet-master[78937]: Starting Puppet master
version 3.8.6Apr 19 11:27:37 foreman-proxy puppet-master[78952]: Failed to
find testclient.test.dmz via exec: Execution of '/etc/puppet/node.rb
testclient.test.dmz' returned 1:Apr 19 11:28:48 foreman-proxy
puppet-master[78952]: Failed when searching for node testclient.test.dmz:
Failed to find testclient.test.dmz via exec: Execution of
'/etc/puppet/node.rb testclient.test.dmz' returned 1:Apr 19 11:28:48
foreman-proxy puppet-master[78952]: Failed when searching for node
testclient.test.dmz: Failed to find testclient.test.dmz via exec: Execution
of '/etc/puppet/node.rb testclient.test.dmz' returned 1:Apr 19 11:30:01
foreman-proxy puppet-master[78952]: Report processor failed: Could not send
report to Foreman at https://foreman-master.test.com/api/reports:
Connection timed out - connect(2)Apr 19 11:30:01 foreman-proxy
puppet-master[78952]: /usr/lib/ruby/1.8/net/http.rb:560:in initialize'/usr/lib/ruby/1.8/net/http.rb:560:inopen'/usr/lib/ruby/1.8/net/http.rb:560:in connect'/usr/lib/ruby/1.8/timeout.rb:53:intimeout'/usr/lib/ruby/1.8/timeout.rb:101:in timeout'/usr/lib/ruby/1.8/net/http.rb:560:inconnect'/usr/lib/ruby/1.8/net/http.rb:553:in do_start'/usr/lib/ruby/1.8/net/http.rb:542:instart'/usr/lib/ruby/1.8/net/http.rb:1035:in request'/usr/lib/ruby/site_ruby/1.8/puppet/reports/foreman.rb:63:inprocess'/usr/lib/ruby/site_ruby/1.8/puppet/indirector/report/processor.rb:37:in process'/usr/lib/ruby/site_ruby/1.8/puppet/indirector/report/processor.rb:53:inprocessors'/usr/lib/ruby/site_ruby/1.8/puppet/indirector/report/processor.rb:51:in each'/usr/lib/ruby/site_ruby/1.8/puppet/indirector/report/processor.rb:51:inprocessors'/usr/lib/ruby/site_ruby/1.8/puppet/indirector/report/processor.rb:30:in process'/usr/lib/ruby/site_ruby/1.8/puppet/indirector/report/processor.rb:14:insave'/usr/lib/ruby/site_ruby/1.8/puppet/indirector/indirection.rb:283:in save'/usr/lib/ruby/site_ruby/1.8/puppet/network/http/api/v1.rb:160:indo_save'/usr/lib/ruby/site_ruby/1.8/puppet/network/http/api/v1.rb:50:in send'/usr/lib/ruby/site_ruby/1.8/puppet/network/http/api/v1.rb:50:incall'/usr/lib/ruby/site_ruby/1.8/puppet/context.rb:64:in override'/usr/lib/ruby/site_ruby/1.8/puppet.rb:246:inoverride'/usr/lib/ruby/site_ruby/1.8/puppet/network/http/api/v1.rb:49:in call'/usr/lib/ruby/site_ruby/1.8/puppet/network/http/route.rb:82:inprocess'/usr/lib/ruby/site_ruby/1.8/puppet/network/http/route.rb:81:in each'/usr/lib/ruby/site_ruby/1.8/puppet/network/http/route.rb:81:inprocess'/usr/lib/ruby/site_ruby/1.8/puppet/network/http/handler.rb:63:in process'/usr/lib/ruby/site_ruby/1.8/puppet/util/profiler/around_profiler.rb:58:inprofile'/usr/lib/ruby/site_ruby/1.8/puppet/util/profiler.rb:51:in profile'/usr/lib/ruby/site_ruby/1.8/puppet/network/http/handler.rb:61:inprocess'/usr/lib/ruby/site_ruby/1.8/puppet/network/http/rack.r
I then tried to manually perform a test to the URL mentioned using curl:
On the puppet client when I run puppet agent -t I get the following:
[root@testclient ~]# puppet agent -tWarning: Unable to fetch my node
definition, but the agent run will continue:Warning: Error 400 on SERVER:
Failed to find testclient.test.dmz via exec: Execution of
'/etc/puppet/node.rb testclient.test.dmz' returned 1:Info: Retrieving
pluginfactsInfo: Retrieving pluginInfo: Loading factsError: Could not
retrieve catalog from remote server: Error 400 on SERVER: Failed when
searching for node testclient.test.dmz: Failed to find testclient.test.dmz
via exec: Execution of '/etc/puppet/node.rb testclient.test.dmz' returned
1:Warning: Not using cache on failed catalogError: Could not retrieve
catalog; skipping runI dont see anything logged on the master foreman
server in the production.log.
These are my current URL settings on the proxy foreman server:
···
On Friday, April 15, 2016 at 4:47:57 PM UTC+1, Sean A wrote:
>
> One adjustment that might solve this is to setup Apache on the puppet
> master to proxy_pass the certificate requests through to the puppet CA.
> PuppetLabs suggests this as a possible CA configuration in the Scaling
> Guide:
> https://docs.puppet.com/guides/scaling_multiple_masters.html#option-2-proxy-certificate-traffic.
> As I've been investigating deploying multiple puppet-masters with Foreman,
> I've found that it's not easy to setup remote puppet-masters if the entire
> network behind them doesn't have access to the ONE puppetCA. I haven't
> tested this with Foreman, but I'm working on building a suitable lab
> environment. For us, doing the CA proxy method in apache would solve
> adding nodes by installing puppet and running puppet agent --test
> --server= --waitforcert=30, whereas with out the proxy
> feature, the remote admin has to be aware and specify the --ca_server flag
> and point it to the puppetca server.
>
> Greg, do you think the CA proxy config in apache on the puppet master work
> resolve this issue?
>
> On Friday, April 15, 2016 at 9:00:12 AM UTC-4, Greg Sutcliffe wrote:
>>
>> On 15 April 2016 at 09:54, James Denton wrote:
>>
>>> Hi Greg
>>>
>>> I wouldn't have thought that the template feature has any interaction
>>> with puppetCA however it does, in this instance, appear to be the case.
>>>
>>> The issue(s) i had previously that you responded to (the token not being
>>> obtained from the master via proxy) was solved from setting the Puppet CA
>>> from Proxy to Master.
>>>
>>> On the proxy i have set autosign to be *.domain.com as a test as i read
>>> this in a simliar post.
>>>
>>> My problems are as follows:
>>> 1) There is no direct comms from the isolated network to the Foreman
>>> Master, only via the smart proxy - Although the PuppetCA feature is enabled
>>> on the Proxy if i run a client against it, it fails (at build time and
>>> performing a manual puppet run).
>>> 2) If i generate a CA on the proxy (seperate from the master) clients
>>> obtain a cert although i get the error "node not found" which i assume to
>>> be that there is an issue between the proxy and master foreman instance.
>>> 3) If the proxy is using its own CA then the master has problems
>>> connecting to it over 8443 so i had to set the proxy (under smart proxies
>>> tab) to be http/8000 - this however doesnt resolve the issue above with
>>> "node not found". If the proxy doesnt have its own CA, Apache/Passenger
>>> fails to start.
>>> 4) I would like the Master to be the only CA and I assumed that the
>>> PuppetCA feature would proxy requests through.
>>>
>>
>> Here's the confusion, I think. The PuppetCA feature doesn't imply any
>> proxying, it just represents a puppetmaster+ca box. If the box isn't
>> capable of signing puppet certs, the smart-proxy shouldn't have that
>> feature enabled.
>>
>> In general, what you want should work, but you need to ensure your SSL
>> traffic is set up correctly. There's several sets of certificate traffic
>> happening here:
>>
>> * The agent<->puppetmaster traffic
>> * The puppetmaster->ENC (Foreman UI) traffic
>> * The smart-proxy->Foreman traffic
>> * The Foreman->smart-proxy traffic
>>
>> All of these *can* be configured separately, although by default (when
>> everything is on one box) it's all done from a single CA - and that's the
>> key here. You're mixing CAs across this setup, so that's leading to
>> communication errors.
>>
>> If you only plan to have a single puppetmaster (on the smart-proxy) then
>> it's probably simplest to generate a certificate for the Foreman box from
>> there, and replace Foreman's current certs. Alternatively, if you must keep
>> two separate CAs, you'll need to work through each of those traffic types
>> and ensure communication is right (e.g. for proxy -> Foreman traffic, you
>> can configure the proxy to use certs from the Foreman CA in
>> /etc/foreman-proxy/settings.yml; likewise the ENC comms is configured in
>> /etc/puppet/foreman.yml)
>>
>> Hope that helps!
>> Greg
>>
>
