Currently we have a Foreman server within our internal network with a Smart
Proxy in an isolated network with the correct ports opened between them. In
order for us to build hosts within the isolated network they will need to
use the Smart Proxy for everything from TFTP to Puppet as new hosts will
not be able to communicate directly with the Foreman master. My question is
for initial Provisioning and installation - Does the Smart proxy need to
have installation media locally or does it proxy off requests for the
install media to the Master Foreman server? If not is there a plugin or
tool available to allow this?
The installation media URLs configured in Foreman must be accessible
directly to the hosts being provisioned and to the smart proxy itself.
The hosts use them for the whole installation, so create a new
installation medium for each location.
The smart proxy itself doesn't provide any access to the installation
media, it only uses it (to download the OS PXE boot files). You could
look at tools like aptly or mrepo to easily mirror media.
···
On 31/03/16 08:48, James Denton wrote:
> Hi all,
>
> I wonder if someone could please answer this.
>
> Currently we have a Foreman server within our internal network with a
> Smart Proxy in an isolated network with the correct ports opened between
> them. In order for us to build hosts within the isolated network they
> will need to use the Smart Proxy for everything from TFTP to Puppet as
> new hosts will not be able to communicate directly with the Foreman
> master. My question is for initial Provisioning and installation - Does
> the Smart proxy need to have installation media locally or does it proxy
> off requests for the install media to the Master Foreman server? If not
> is there a plugin or tool available to allow this?
I have one last (I hope) issue with provisioning a new server via the
smart-proxy. I have set the Templates feature up on the Smart Proxy and set
a new server to build. Once build on the new server has been set i see that
the pxelinux.cfg dir on the proxy is populated (as expected) with the
correct file for the new server:
So the client will boot, obtain a DHCP IP and grab the init/kernel files as
expected, however when it tries to download the ks file (as above) it fails
to download. Port 80 is definatly open on the smart proxy server and when i
try a curl or wget on the above file it also fails with "file not found".
So my question is, does the Foreman master transfer the ks file to the
smart-proxy when the client is set to build (i.e. should it exist somewhere
on the proxy) or does the smart-proxy obtain the file from the master
foreman when required by the install client?
Your help is appreciated
···
On Thursday, March 31, 2016 at 8:48:48 AM UTC+1, James Denton wrote:
>
> Hi all,
>
> I wonder if someone could please answer this.
>
> Currently we have a Foreman server within our internal network with a
> Smart Proxy in an isolated network with the correct ports opened between
> them. In order for us to build hosts within the isolated network they will
> need to use the Smart Proxy for everything from TFTP to Puppet as new hosts
> will not be able to communicate directly with the Foreman master. My
> question is for initial Provisioning and installation - Does the Smart
> proxy need to have installation media locally or does it proxy off requests
> for the install media to the Master Foreman server? If not is there a
> plugin or tool available to allow this?
>
> Thanks!
>
I setup a simple tcpdump running on both the foreman and proxy, and it
shows no calls are made between them at the time of the template request.
···
On Thursday, March 31, 2016 at 8:48:48 AM UTC+1, James Denton wrote:
>
> Hi all,
>
> I wonder if someone could please answer this.
>
> Currently we have a Foreman server within our internal network with a
> Smart Proxy in an isolated network with the correct ports opened between
> them. In order for us to build hosts within the isolated network they will
> need to use the Smart Proxy for everything from TFTP to Puppet as new hosts
> will not be able to communicate directly with the Foreman master. My
> question is for initial Provisioning and installation - Does the Smart
> proxy need to have installation media locally or does it proxy off requests
> for the install media to the Master Foreman server? If not is there a
> plugin or tool available to allow this?
>
> Thanks!
>
···
On Thursday, March 31, 2016 at 9:02:09 AM UTC+1, Dominic Cleal wrote:
>
> On 31/03/16 08:48, James Denton wrote:
> > Hi all,
> >
> > I wonder if someone could please answer this.
> >
> > Currently we have a Foreman server within our internal network with a
> > Smart Proxy in an isolated network with the correct ports opened between
> > them. In order for us to build hosts within the isolated network they
> > will need to use the Smart Proxy for everything from TFTP to Puppet as
> > new hosts will not be able to communicate directly with the Foreman
> > master. My question is for initial Provisioning and installation - Does
> > the Smart proxy need to have installation media locally or does it proxy
> > off requests for the install media to the Master Foreman server? If not
> > is there a plugin or tool available to allow this?
>
> The installation media URLs configured in Foreman must be accessible
> directly to the hosts being provisioned and to the smart proxy itself.
> The hosts use them for the whole installation, so create a new
> installation medium for each location.
>
> The smart proxy itself doesn't provide any access to the installation
> media, it only uses it (to download the OS PXE boot files). You could
> look at tools like aptly or mrepo to easily mirror media.
>
> --
> Dominic Cleal
> dom...@cleal.org
>
The proxy requests it from Foreman when the call is made to the proxy -
it's not stored on the proxy (othewise it could get out of date if, for
example, your kickstart makes use of ERB variables which have changed in
the time between enabling build mode and booting the client)
Check the Foreman logs to see if the proxy is requesting the kickstart for
the host, and if there's any associated error.
···
On 6 April 2016 at 08:55, James Denton wrote:
So my question is, does the Foreman master transfer the ks file to the
smart-proxy when the client is set to build (i.e. should it exist somewhere
on the proxy) or does the smart-proxy obtain the file from the master
foreman when required by the install client?
—# Enable this if the Proxy should handle template requests on behalf of
Foreman# Can be true, false, or http/https to enable just one of the
protocols:enabled: true# This plugin also requires that :foreman_url: be
set in the main settings.yml# This lets the plugin know how to obtain the
templates from foreman.# This allows the proxy to define how hosts that are
being provisioned where to# obtain the templates from. Most installers
don't support https, so it's recommended# to enable an http port listener
in the main config file too, and use it in# the url below## :template_url
is the URL the host should use to contact the proxy for a template.# The
default protocol is http on port 80 unless otherwise specified in the url.#
Examples:# https://1.2.3.4:8443 <https://1.2.3.4:8443> # default
proxy https port# http://1.2.3.4:8000 <http://1.2.3.4:8000> #
default proxy http port# https://smart-proxy.example.com
<https://smart-proxy.example.com> # assumes port 443# smart-proxy.example.com:8080 <http://smart-proxy.example.com:8080> #
assumes http:template_url: http://smartproxy.test.com
<http://smartproxy.test.com>
···
On Thursday, April 7, 2016 at 2:46:15 PM UTC+1, James Denton wrote:
>
> I setup a simple tcpdump running on both the foreman and proxy, and it
> shows no calls are made between them at the time of the template request.
>
> On Thursday, March 31, 2016 at 8:48:48 AM UTC+1, James Denton wrote:
>>
>> Hi all,
>>
>> I wonder if someone could please answer this.
>>
>> Currently we have a Foreman server within our internal network with a
>> Smart Proxy in an isolated network with the correct ports opened between
>> them. In order for us to build hosts within the isolated network they will
>> need to use the Smart Proxy for everything from TFTP to Puppet as new hosts
>> will not be able to communicate directly with the Foreman master. My
>> question is for initial Provisioning and installation - Does the Smart
>> proxy need to have installation media locally or does it proxy off requests
>> for the install media to the Master Foreman server? If not is there a
>> plugin or tool available to allow this?
>>
>> Thanks!
>>
>
Thanks, I have looked at Katello previously but will look again.
I also have another issue in regards to TFTP and Smart Proxy. As mentioned
in my original post, the Smart Proxy i am using resides on a differant
subnet from the master. The TFTP feature is enabled on the Smart Proxy
(config is picked up by the master) and set in correctly in the "subnet"
configuration. However when i press "build" for a new client the correct
pxelinux.0 and pxelinux.cfg/ files are not present or popualted on the
Smart Proxy, either for that particular client or generically. I have seen
similar issues in other threads but no definitive answer. The log files on
both the master and proxy are not showing any errors. Can someone help?
···
On Thursday, March 31, 2016 at 2:34:51 PM UTC+1, Lukas Zapletal wrote:
>
> > Thanks for the reply Dominic
>
> Take a look on Katello plugin.
>
> http://www.katello.org/
>
> --
> Later,
> Lukas #lzap Zapletal
>
Thanks Greg, that answers my question although I have tried 2 differant
proxies (one on the same subnet as the master) and neither appear to
transfer the KS/Template file from the Foreman master server to the build
client, is there any settings outside of the template.yml file that need to
be set and which port would it connect to the master on?
Thanks
···
On Wednesday, April 6, 2016 at 12:48:19 PM UTC+1, Greg Sutcliffe wrote:
>
>
>
> On 6 April 2016 at 08:55, James Denton <jrden...@gmail.com > > wrote:
>
>> So my question is, does the Foreman master transfer the ks file to the
>> smart-proxy when the client is set to build (i.e. should it exist somewhere
>> on the proxy) or does the smart-proxy obtain the file from the master
>> foreman when required by the install client?
>>
>
> The proxy requests it from Foreman when the call is made to the proxy -
> it's not stored on the proxy (othewise it could get out of date if, for
> example, your kickstart makes use of ERB variables which have changed in
> the time between enabling build mode and booting the client)
>
> Check the Foreman logs to see if the proxy is requesting the kickstart for
> the host, and if there's any associated error.
>
On the Foreman master in /var/log/httpd/foreman_access.log I dont see much
other than when i tried to manually retreive the provision template via
wget:
···
On Wednesday, April 6, 2016 at 12:48:19 PM UTC+1, Greg Sutcliffe wrote:
>
>
>
> On 6 April 2016 at 08:55, James Denton <jrden...@gmail.com > > wrote:
>
>> So my question is, does the Foreman master transfer the ks file to the
>> smart-proxy when the client is set to build (i.e. should it exist somewhere
>> on the proxy) or does the smart-proxy obtain the file from the master
>> foreman when required by the install client?
>>
>
> The proxy requests it from Foreman when the call is made to the proxy -
> it's not stored on the proxy (othewise it could get out of date if, for
> example, your kickstart makes use of ERB variables which have changed in
> the time between enabling build mode and booting the client)
>
> Check the Foreman logs to see if the proxy is requesting the kickstart for
> the host, and if there's any associated error.
>
Seems like the request is being correctly proxied, but Foreman isn't
finding the host. You might need to enable debug logs on Foreman, but since
it's a 404 rather than an error, I'd guess the host either isn't in build
mode, or the token has expired. What's your token_duration setting and has
the host been in build mode longer than that?
You're right that the TFTP Proxy must be set on the subnet - also ensure
the subnet is set on the provisioning interface when creating the host,
and that this interface has the Managed tickbox enabled.
···
On 01/04/16 08:15, James Denton wrote:
> Thanks, I have looked at Katello previously but will look again.
>
> I also have another issue in regards to TFTP and Smart Proxy. As
> mentioned in my original post, the Smart Proxy i am using resides on a
> differant subnet from the master. The TFTP feature is enabled on the
> Smart Proxy (config is picked up by the master) and set in correctly in
> the "subnet" configuration. However when i press "build" for a new
> client the correct pxelinux.0 and pxelinux.cfg/ files are not present or
> popualted on the Smart Proxy, either for that particular client or
> generically. I have seen similar issues in other threads but no
> definitive answer. The log files on both the master and proxy are not
> showing any errors. Can someone help?
I have/had debug enabled already on the logs but its not providing much
further information im afraid.
The build mode set on the client is no more than a min before its powered
on for the build and i have cancelled and set build multiple times. You
mention Foreman finding the host, how do you mean exactly? The client for
installation is in an isolated network and so the Foreman master will not
have any direct access to it and vice versa from the build client - hence
the need for the Smart Proxy which has access to both the Foreman master
and the build client to handle TFTP, Templates etc.
Thanks
···
On Wednesday, April 6, 2016 at 4:54:04 PM UTC+1, Greg Sutcliffe wrote:
>
> Seems like the request is being correctly proxied, but Foreman isn't
> finding the host. You might need to enable debug logs on Foreman, but since
> it's a 404 rather than an error, I'd guess the host either isn't in build
> mode, or the token has expired. What's your token_duration setting and has
> the host been in build mode longer than that?
>
>
> Greg
>
I just mean finding it in the DB, thats all. However what you posted
earlier look like Apache logs - I was asking for the Foreman logs (usually
/var/log/foreman/production.log) - I'd be interesting so seen a tail of
that while a build is in progress, especially if debug is enabled.
Greg
···
On 6 April 2016 at 18:43, James Denton wrote:
Hi Greg,
I have/had debug enabled already on the logs but its not providing much
further information im afraid.
The build mode set on the client is no more than a min before its powered
on for the build and i have cancelled and set build multiple times. You
mention Foreman finding the host, how do you mean exactly? The client for
installation is in an isolated network and so the Foreman master will not
have any direct access to it and vice versa from the build client - hence
the need for the Smart Proxy which has access to both the Foreman master
and the build client to handle TFTP, Templates etc.
That appears to have resolved it for the internal Smart Proxy, setting the
template URL to be 8000 and IP based.
I appreciate the help! I will look now at trying the same on the server
within an isolated network.
···
On Friday, April 8, 2016 at 1:34:31 AM UTC+1, Greg Sutcliffe wrote:
>
> On 7 April 2016 at 17:35, James Denton <jrden...@gmail.com > > wrote:
>
>> *:template_url: http://smartproxy.test.com *
>>
>
> Is that verbatim? The proxy is normally on 8000 or 8433 (your config above
> says 8443) rather than 80/443 - is the port correctly assigned?
>
> On 7 April 2016 at 14:46, James Denton <jrden...@gmail.com > > wrote:
>
>> I setup a simple tcpdump running on both the foreman and proxy, and it
>> shows no calls are made between them at the time of the template request.
>>
>
> That would imply template_url is wrong, doublecheck it and potentially try
> http.
>
> Greg
>
>
Thanks again Dominic - Its this bit that was missing:
interface has the Managed tickbox enabled
I will try an install now, thanks again!
···
On Friday, April 1, 2016 at 8:18:41 AM UTC+1, Dominic Cleal wrote:
>
> On 01/04/16 08:15, James Denton wrote:
> > Thanks, I have looked at Katello previously but will look again.
> >
> > I also have another issue in regards to TFTP and Smart Proxy. As
> > mentioned in my original post, the Smart Proxy i am using resides on a
> > differant subnet from the master. The TFTP feature is enabled on the
> > Smart Proxy (config is picked up by the master) and set in correctly in
> > the "subnet" configuration. However when i press "build" for a new
> > client the correct pxelinux.0 and pxelinux.cfg/ files are not present or
> > popualted on the Smart Proxy, either for that particular client or
> > generically. I have seen similar issues in other threads but no
> > definitive answer. The log files on both the master and proxy are not
> > showing any errors. Can someone help?
>
> There are quite a few conditions that must be true for these to be
> created. Have a work through the list here:
>
> http://projects.theforeman.org/projects/foreman/wiki/Troubleshooting#No-TFTP-menus-or-files-are-created-for-new-hosts
>
> You're right that the TFTP Proxy must be set on the subnet - also ensure
> the subnet is set on the provisioning interface when creating the host,
> and that this interface has the Managed tickbox enabled.
>
> --
> Dominic Cleal
> dom...@cleal.org
>
I have re-ran the kickstart and i see no output on the master
production.log at the time the request for the template is made:-/
Here is the current log level settings in /etc/foreman/settings.yml:
# Log settings for the current environment can be adjusted by adding them#
here. For example, if you want to increase the log level.:logging: :level:
debug# Individual logging types can be toggled on/off here:loggers:
This has always been set to debug as have the smart proxies log level.
Thanks
···
On Wednesday, April 6, 2016 at 7:45:52 PM UTC+1, Greg Sutcliffe wrote:
>
> On 6 April 2016 at 18:43, James Denton <jrden...@gmail.com > > wrote:
>
>> Hi Greg,
>>
>> I have/had debug enabled already on the logs but its not providing much
>> further information im afraid.
>>
>> The build mode set on the client is no more than a min before its powered
>> on for the build and i have cancelled and set build multiple times. You
>> mention Foreman finding the host, how do you mean exactly? The client for
>> installation is in an isolated network and so the Foreman master will not
>> have any direct access to it and vice versa from the build client - hence
>> the need for the Smart Proxy which has access to both the Foreman master
>> and the build client to handle TFTP, Templates etc.
>>
>
> I just mean finding it in the DB, thats all. However what you posted
> earlier look like Apache logs - I was asking for the Foreman logs (usually
> /var/log/foreman/production.log) - I'd be interesting so seen a tail of
> that while a build is in progress, especially if debug is enabled.
>
> Greg
>
Today i tried again to build from the other smart proxy which provisions
hosts in the isolated network now that the correct FW rules are in place
and i have the same issue with obtaining the provision token for the host
however i do now see actions being logged to the production.log on the
foreman master server:
>
> Started GET "/unattended/provision?url=
http%3A%2F%2F10.148.6.34%3A8000&token=4bee34fe-ba86-4866-a776-c9a282d435a9"
for 10.148.6.34 at 2016-04-12 15:12:01 +0100
2016-04-12 15:12:01 [app] [I] Processing by UnattendedController#provision
as HTML
2016-04-12 15:12:01 [app] [I] Parameters: {"url"=>"http://10.148.6.34:8000",
"token"=>"4bee34fe-ba86-4866-a776-c9a282d435a9", "unattended"=>{}}
2016-04-12 15:12:01 [app] [I] Found extest1.test.dmz
2016-04-12 15:12:02 [app] [I] Redirected to
2016-04-12 15:12:02 [app] [I] Completed 500 Internal Server Error in 885ms
2016-04-12 15:12:02 [app] [F]
> ActionController::RedirectBackError (No HTTP_REFERER was set in the
request to this action, so redirect_to :back could not be called
successfully. If this is a test, make sure to specify
request.env["HTTP_REFERER"].):
> app/controllers/application_controller.rb:275:in process_error' > app/controllers/application_controller.rb:107:insmart_proxy_exception'
> lib/middleware/catch_json_parse_errors.rb:9:in `call'
>
>
Can someone explain from this what the error is?
Thanks
···
On Friday, April 8, 2016 at 11:10:54 AM UTC+1, James Denton wrote:
>
> Hi Greg
>
> That appears to have resolved it for the internal Smart Proxy, setting the
> template URL to be 8000 and IP based.
>
> I appreciate the help! I will look now at trying the same on the server
> within an isolated network.
>
> On Friday, April 8, 2016 at 1:34:31 AM UTC+1, Greg Sutcliffe wrote:
>>
>> On 7 April 2016 at 17:35, James Denton wrote:
>>
>>> *:template_url: http://smartproxy.test.com *
>>>
>>
>> Is that verbatim? The proxy is normally on 8000 or 8433 (your config
>> above says 8443) rather than 80/443 - is the port correctly assigned?
>>
>> On 7 April 2016 at 14:46, James Denton wrote:
>>
>>> I setup a simple tcpdump running on both the foreman and proxy, and it
>>> shows no calls are made between them at the time of the template request.
>>>
>>
>> That would imply template_url is wrong, doublecheck it and potentially
>> try http.
>>
>> Greg
>>
>>
