New Foreman User. Not clear on what to do next

Hello,

I've installed foreman/puppet on a host, using the isntructions for Foreman
1.3

I've got a few issues though, and the manual isn't very clear on what my
next steps should be.

I asumed that the next steps would be: Get my hosts to properly check in
with the puppet master. Then have foreman import the hosts from the puppet
master? Am I here correct?

I proceeded with an attempt to get one host to get to talk to the puppet
master. I ran on the host the following command:

puppet agent --test --server dc1-pup1.cloudlynx.local --waitforcert 120

And then on the puppet master I signed the request:

puppet cert sign -all.

And now however when I run:

[root@dc1-ipa1 puppet]# puppet agent --test --server puppet
Warning: Unable to fetch my node definition, but the agent run will
continue:
Warning: Error 400 on SERVER: Failed to find dc1-ipa1.cloudlynx.local via
exec: Execution of '/etc/puppet/node.rb dc1-ipa1 returned 1: — false

Info: Retrieving plugin
Error: Could not retrieve catalog from remote server: Error 400 on SERVER:
Failed when searching for node dc1-ipa1: Failed to find dc1-ipa1 via exec:
Execution of '/etc/puppet/node.rb dc1-ipa1' returned 1: — false

Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

I tested this by running node.rb on the puppetmaster:

[root@dc1-pup1 conf.d]# /etc/puppet/node.rb dc1-ipa1
— false
Error retrieving node dc1-ipa1: Net::HTTPNotFound

I asume that this error message is because the host isn't know in foreman
yet. But getting the host in to foreman is what I want. This looks a bit
like a chicken and egg problem.
What next? I'm a bit stuck here. How do I proceed?

What are the steps to get existing host known to foreman in an efficient
way?

> Hello,
>
> I've installed foreman/puppet on a host, using the isntructions for
> Foreman 1.3
>

Welcome to the community :slight_smile:

> I tested this by running node.rb on the puppetmaster:
>
> [root@dc1-pup1 conf.d]# /etc/puppet/node.rb dc1-ipa1
> — false
> Error retrieving node dc1-ipa1: Net::HTTPNotFound
>

This is saying that node.rb cannot find your foreman server - check the URL
in the settings at the top of the file is correct. Once this script can
contact Foreman, it will create a Host as part of the puppet run, if the
Host does not already exist.

HTH,
Greg

··· On 17 October 2013 15:45, Krist van Besien wrote:

The URL in the file is correct. I can actually see the requests in the http
logs. What I did is modify the foreman.conf and puppet.conf in
/etc/httpd/conf.d so that I get separate logfiles. In the foreman httpd log
I see that all requests return with an error code 403…

So something is not right.

··· On Thursday, October 17, 2013 5:02:30 PM UTC+2, Greg Sutcliffe wrote: > > I tested this by running node.rb on the puppetmaster: >> >> [root@dc1-pup1 conf.d]# /etc/puppet/node.rb dc1-ipa1 >> --- false >> Error retrieving node dc1-ipa1: Net::HTTPNotFound >> > > This is saying that node.rb cannot find your foreman server - check the > URL in the settings at the top of the file is correct. Once this script can > contact Foreman, it will create a Host as part of the puppet run, if the > Host does not already exist. >

Could you check Foreman's log file? /var/log/foreman/production.log

If it's Foreman itself denying access (there are some auth mechanisms)
then it'll log the reason why.

··· On 18/10/13 07:12, Krist van Besien wrote: > > > On Thursday, October 17, 2013 5:02:30 PM UTC+2, Greg Sutcliffe wrote: > > I tested this by running node.rb on the puppetmaster: > > [root@dc1-pup1 conf.d]# /etc/puppet/node.rb dc1-ipa1 > --- false > Error retrieving node dc1-ipa1: Net::HTTPNotFound > > > This is saying that node.rb cannot find your foreman server - check > the URL in the settings at the top of the file is correct. Once this > script can contact Foreman, it will create a Host as part of the > puppet run, if the Host does not already exist. > > > The URL in the file is correct. I can actually see the requests in the > http logs. What I did is modify the foreman.conf and puppet.conf in > /etc/httpd/conf.d so that I get separate logfiles. In the foreman httpd > log I see that all requests return with an error code 403... > > So something is not right.


Dominic Cleal
Red Hat Engineering

That appears to be the case:
Started GET "/node/dc1-sat1.domain.local?format=yml" for 172.16.1.103 at
2013-10-18 09:29:57 +0200
Processing by HostsController#externalNodes as YML
Parameters: {"name"=>"dc1-sat1.domain.local"}
No smart proxy server found on ["dc1-pup1.domain.local"] and is not in
trusted_puppetmaster_hosts
Redirected to https://dc1-pup1.domain.local/users/login
Filter chain halted as :require_puppetmaster_or_login rendered or redirected
Completed 403 Forbidden in 3ms (ActiveRecord: 0.3ms)
Connecting to database specified by database.yml

But how to fix this?

The smart proxy does appear to be running on this host:
[root@dc1-pup1 log]# ps -fp cat /var/run/foreman-proxy/foreman-proxy.pid
UID PID PPID C STIME TTY TIME CMD
497 11099 1 0 Oct17 ? 00:00:02 /usr/bin/ruby
/usr/share/foreman-proxy/bin/smart-proxy

But the foreman-proxy log does not show any access attempts.

So what do I check next?

(As an aside: Foreman logs everything to production.log. Does "production"
and "development" in foreman map to the Puppet environments? I have all my
hosts configured to use the "development" environment. So why is everything
logged in production.log?)

··· On Friday, October 18, 2013 9:24:01 AM UTC+2, Dominic Cleal wrote: > > On 18/10/13 07:12, Krist van Besien wrote: > > > > > > On Thursday, October 17, 2013 5:02:30 PM UTC+2, Greg Sutcliffe wrote: > > > > I tested this by running node.rb on the puppetmaster: > > > > [root@dc1-pup1 conf.d]# /etc/puppet/node.rb dc1-ipa1 > > --- false > > Error retrieving node dc1-ipa1: Net::HTTPNotFound > > > > > > This is saying that node.rb cannot find your foreman server - check > > the URL in the settings at the top of the file is correct. Once this > > script can contact Foreman, it will create a Host as part of the > > puppet run, if the Host does not already exist. > > > > > > The URL in the file is correct. I can actually see the requests in the > > http logs. What I did is modify the foreman.conf and puppet.conf in > > /etc/httpd/conf.d so that I get separate logfiles. In the foreman httpd > > log I see that all requests return with an error code 403... > > > > So something is not right. > > Could you check Foreman's log file? /var/log/foreman/production.log > > If it's Foreman itself denying access (there are some auth mechanisms) > then it'll log the reason why. >

>
>
>
>
> If it's Foreman itself denying access (there are some auth mechanisms)
> then it'll log the reason why.
>
>
> That's indeed what it looks like:
>
> Started GET "/node/dc1-sat1.cloudlynx.local?format=yml" for 172.16.1.103
> at 2013-10-18 09:29:57 +0200
> Processing by HostsController#externalNodes as YML
> Parameters: {"name"=>"dc1-sat1.domain.local"}
> No smart proxy server found on ["dc1-pup1.cloudlynx.local"] and is not
> in trusted_puppetmaster_hosts
> Redirected to https://dc1-pup1.domain.local/users/login
> Filter chain halted as :require_puppetmaster_or_login rendered or redirected
> Completed 403 Forbidden in 3ms (ActiveRecord: 0.3ms)
> Connecting to database specified by database.yml
>
> So how do I fix this?
>
> I did a "default" install, that did install the puppet master. I asume
> that the settings would be correct, but don't know where to start looking…

Yes, it should have been. The install should have registered a smart
proxy (look under More>Config>Smart proxies) with the FQDN of your host.

If it's not registered, then I guess there was an error during install.
Try adding it with the URL "https://dc1-pup1.cloudlynx.local:8443".

If it's there but under a different hostname to
"dc1-pup1.cloudlynx.local", try editing it to use that hostname and it
should then permit the puppetmaster to query Foreman.

> (As an aside: there is a "production" and a "development" log. What is
> the difference? Do they somehow relate to the puppet "development" and
> "production" logs? If so, how come everything gets logged to production,
> even though my hosts are all part of the evelopment environment…)

It's no relation to Puppet environments. It's referring to the Rails
environments that Foreman itself runs in - we usually only use production.

··· On 18/10/13 08:39, Krist van Besien wrote: > On Friday, October 18, 2013 9:24:01 AM UTC+2, Dominic Cleal wrote:


Dominic Cleal
Red Hat Engineering

The install at the moment is meant to register the smart-proxy by default,
but unfortunately it does not because apache is not started before trying
to register.

The simple way is to run the install twice on setup.

··· On Friday, October 18, 2013 8:52:22 AM UTC+1, Dominic Cleal wrote: > > On 18/10/13 08:39, Krist van Besien wrote: > > > > > > On Friday, October 18, 2013 9:24:01 AM UTC+2, Dominic Cleal wrote: > > > > > > If it's Foreman itself denying access (there are some auth > mechanisms) > > then it'll log the reason why. > > > > > > That's indeed what it looks like: > > > > Started GET "/node/dc1-sat1.cloudlynx.local?format=yml" for 172.16.1.103 > > at 2013-10-18 09:29:57 +0200 > > Processing by HostsController#externalNodes as YML > > Parameters: {"name"=>"dc1-sat1.domain.local"} > > No smart proxy server found on ["dc1-pup1.cloudlynx.local"] and is not > > in trusted_puppetmaster_hosts > > Redirected to https://dc1-pup1.domain.local/users/login > > Filter chain halted as :require_puppetmaster_or_login rendered or > redirected > > Completed 403 Forbidden in 3ms (ActiveRecord: 0.3ms) > > Connecting to database specified by database.yml > > > > So how do I fix this? > > > > I did a "default" install, that did install the puppet master. I asume > > that the settings would be correct, but don't know where to start > looking... > > Yes, it should have been. The install should have registered a smart > proxy (look under More>Config>Smart proxies) with the FQDN of your host. > > If it's not registered, then I guess there was an error during install. > Try adding it with the URL "https://dc1-pup1.cloudlynx.local:8443". > > If it's there but under a different hostname to > "dc1-pup1.cloudlynx.local", try editing it to use that hostname and it > should then permit the puppetmaster to query Foreman. > > > (As an aside: there is a "production" and a "development" log. What is > > the difference? Do they somehow relate to the puppet "development" and > > "production" logs? If so, how come everything gets logged to production, > > even though my hosts are all part of the evelopment environment...) > > It's no relation to Puppet environments. It's referring to the Rails > environments that Foreman itself runs in - we usually only use production. > > -- > Dominic Cleal > Red Hat Engineering >

Thanks for the tip. I registered the proxy. Getting there…
(And learning a lot in the process… )

One other issue however:
foreman-rake puppet:import:hosts_and_facts
didn't work at first.

Only after applying this patch did it work:
http://projects.theforeman.org/projects/foreman/repository/revisions/956e2ed4b5f45a89786d4702d33680b66474f159/diff/

Will this end up in the rpms soon, so I can just do a yum update?

··· On Friday, October 18, 2013 9:52:22 AM UTC+2, Dominic Cleal wrote: > > > > Yes, it should have been. The install should have registered a smart > proxy (look under More>Config>Smart proxies) with the FQDN of your host. > >

Sorry about this, I felt it was too complex to try and fix just before
1.3 was released. I'll try and revisit it soon:

http://projects.theforeman.org/issues/3083

··· On 18/10/13 09:29, Alan Sergeant wrote: > The install at the moment is meant to register the smart-proxy by > default, but unfortunately it does not because apache is not started > before trying to register. > > The simple way is to run the install twice on setup.


Dominic Cleal
Red Hat Engineering

Yep, that'll be released in 1.3.1 which I'd say we could do within a
couple of weeks, once all the .0 issues have stabilised.

··· On 18/10/13 09:32, Krist van Besien wrote: > > > On Friday, October 18, 2013 9:52:22 AM UTC+2, Dominic Cleal wrote: > > > > Yes, it should have been. The install should have registered a smart > proxy (look under More>Config>Smart proxies) with the FQDN of your > host. > > > Thanks for the tip. I registered the proxy. Getting there... > (And learning a lot in the process... ) > > One other issue however: > foreman-rake puppet:import:hosts_and_facts > didn't work at first. > > Only after applying this patch did it work: > http://projects.theforeman.org/projects/foreman/repository/revisions/956e2ed4b5f45a89786d4702d33680b66474f159/diff/ > > Will this end up in the rpms soon, so I can just do a yum update?


Dominic Cleal
Red Hat Engineering

I'm still having a few issues here.

I'm now configuring my puppet agents, and on the first run it gives me this
error:

puppet agent --test
Warning: Local environment: "development" doesn't match server specified
node environment "production", switching agent to "production".

And all the hosts get registered in the "production" environment.
I did set "environment = developmen" in the puppet.conf on the client.
Something is still overriding it. What could that be?

Krist

That's Puppet. In Puppet 3.X the ENC sets the environment authoritatively.
You have two options - 1) change the environment to development in Foreman,
or 2) set the enc_environment Setting to false in the Foreman UI. The
latter will prevent Foreman from sending an environment to Puppet, and thus
the Puppetmaster will allow the agent to decide on an environment.

Greg

··· On 18 October 2013 10:32, Krist van Besien wrote:

I’m still having a few issues here.

I’m now configuring my puppet agents, and on the first run it gives me
this error:

puppet agent --test
Warning: Local environment: “development” doesn’t match server specified
node environment “production”, switching agent to “production”.

And all the hosts get registered in the “production” environment.
I did set “environment = developmen” in the puppet.conf on the client.
Something is still overriding it. What could that be?

If it is registering for the first time an existing box it doesn't know
about it so by default it assigns it to the production environment. Once it
is registered use the gui, cli or api to assign the environment for the
boxes, next Puppet run will get it right.

Jim :slight_smile:

··· On 18 October 2013 10:32, Krist van Besien wrote:

I’m still having a few issues here.

I’m now configuring my puppet agents, and on the first run it gives me
this error:

puppet agent --test
Warning: Local environment: “development” doesn’t match server specified
node environment “production”, switching agent to “production”.

And all the hosts get registered in the “production” environment.
I did set “environment = developmen” in the puppet.conf on the client.
Something is still overriding it. What could that be?

Krist


You received this message because you are subscribed to the Google Groups
"Foreman users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at http://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/groups/opt_out.

Yes. Found that setting in Foreman.

Is there an overview of all the available settings in Foreman somewhere,
and what they do?

Krist

··· On Friday, October 18, 2013 11:40:11 AM UTC+2, Greg Sutcliffe wrote: > > > That's Puppet. In Puppet 3.X the ENC sets the environment authoritatively. > You have two options - 1) change the environment to development in Foreman, > or 2) set the enc_environment Setting to false in the Foreman UI. The > latter will prevent Foreman from sending an environment to Puppet, and thus > the Puppetmaster will allow the agent to decide on an environment. > >

Quite a few are documented[1] but we would love contributions to the docs
if you find ones that aren't :slight_smile:

Greg
[1]Foreman :: Manual

··· On 18 October 2013 10:45, Krist van Besien wrote:

Yes. Found that setting in Foreman.

Is there an overview of all the available settings in Foreman somewhere,
and what they do?

This behaviour is actually configurable :). You can set
default_puppet_environment the the value new hosts should go in. This can
be pretty handy if you want to default to something less mission-critical
than "production" :slight_smile:

··· On 18 October 2013 10:43, James Bailey wrote:

If it is registering for the first time an existing box it doesn’t know
about it so by default it assigns it to the production environment. Once it
is registered use the gui, cli or api to assign the environment for the
boxes, next Puppet run will get it right.

Jim :slight_smile:

Things are looking good now. Thanks to everyone who replied.

Of to the weekend now, and the mountains :slight_smile: