Foreman-proxy: dns does not initialize if dns_provider option set

Hi,

if my dns.yml only has (in addition to — of course)

:enabled: https

then proxy.log says

INFO – : 'dns' settings were initialized with default values:
:dns_provider: nsupdate

if I change dns.yml to

:dns_provider: nsupdate

(or as I really want to do:
:enabled: https
:dns_provider: nsupdate_gss
:dns_server: 192.168.180.104
:dns_tsig_keytab: /etc/foreman/foremanproxy.keytab
:dns_tsig_principal: foremanproxy@BORG.TREK
)

dns settings does not initialize, there is no entry in proxy.log

Im using foreman 1.9.0 and foreman-proxy 1.9.0 on Ubuntu 14.04

any help greatly appreciated,

/Erik

> Hi,
>
> if my dns.yml only has (in addition to — of course)
>
> :enabled: https
>
> then proxy.log says
>
> INFO – : 'dns' settings were initialized with default values:
> :dns_provider: nsupdate

Could you provide a bit more of the log file please?

> if I change dns.yml to
>
> :dns_provider: nsupdate
>
> (or as I really want to do:
> :enabled: https
> :dns_provider: nsupdate_gss
> :dns_server: 192.168.180.104
> :dns_tsig_keytab: /etc/foreman/foremanproxy.keytab
> :dns_tsig_principal: foremanproxy@BORG.TREK
> )
>
> dns settings does not initialize, there is no entry in proxy.log

Check that you have ":settings_directory: /etc/foreman-proxy/settings.d"
in /etc/foreman-proxy/settings.yml for it to find these additional files.

··· On 27/08/15 09:34, Erik Hjelmås wrote:


Dominic Cleal
dominic@cleal.org

thank you for your reply, sorry for not paying attention to my post,

>
> > Hi,
> >
> > if my dns.yml only has (in addition to — of course)
> >
> > :enabled: https
> >
> > then proxy.log says
> >
> > INFO – : 'dns' settings were initialized with default values:
> > :dns_provider: nsupdate
>
> Could you provide a bit more of the log file please?
>

If i don't set provider (notice dns is initialized with default provider):

$ grep dns_provider: /etc/foreman-proxy/settings.d/dns.yml
#:dns_provider: nsupdate_gss
$ service foreman-proxy restart

  • Restarting foreman-proxy foreman-proxy
    …done.
    $ tail -n 12 /var/log/foreman-proxy/proxy.log
    I, [2015-09-14T09:38:42.902127 #28143] INFO – : 'bmc' module is disabled.
    I, [2015-09-14T09:38:42.902657 #28143] INFO – : 'realm' module is
    disabled.
    W, [2015-09-14T09:41:08.299193 #28263] WARN – : Couldn't find settings
    file /etc/foreman-proxy/settings.d/foreman_proxy.yml. Using default
    settings.
    I, [2015-09-14T09:41:08.299607 #28263] INFO – : 'foreman_proxy' settings
    were initialized with default values: :enabled: true
    I, [2015-09-14T09:41:08.307588 #28263] INFO – : 'facts' module is
    disabled.
    I, [2015-09-14T09:41:08.308671 #28263] INFO – : 'dns' settings were
    initialized with default values: :dns_provider: nsupdate
    I, [2015-09-14T09:41:08.316116 #28263] INFO – : 'templates' module is
    disabled.
    I, [2015-09-14T09:41:08.316864 #28263] INFO – : 'tftp' module is disabled.
    I, [2015-09-14T09:41:08.317531 #28263] INFO – : 'dhcp' module is disabled.
    I, [2015-09-14T09:41:08.963583 #28263] INFO – : 'puppet' settings were
    initialized with default values: :puppet_provider: puppetrun, :puppetdir:
    /etc/puppet, :salt_puppetrun_cmd: puppet.run, :use_cache: true
    I, [2015-09-14T09:41:08.968332 #28263] INFO – : 'bmc' module is disabled.
    I, [2015-09-14T09:41:08.968958 #28263] INFO – : 'realm' module is
    disabled.

If i do set provider (notice dns does not appear at all in the log):

$ grep dns_provider: /etc/foreman-proxy/settings.d/dns.yml
:dns_provider: nsupdate_gss
$ service foreman-proxy restart

  • Restarting foreman-proxy foreman-proxy
    …done.
    $ tail -n 10 /var/log/foreman-proxy/proxy.log
    I, [2015-09-14T09:45:32.926533 #28407] INFO – : 'realm' module is
    disabled.
    W, [2015-09-14T09:46:16.872296 #28453] WARN – : Couldn't find settings
    file /etc/foreman-proxy/settings.d/foreman_proxy.yml. Using default
    settings.
    I, [2015-09-14T09:46:16.872841 #28453] INFO – : 'foreman_proxy' settings
    were initialized with default values: :enabled: true
    I, [2015-09-14T09:46:16.881412 #28453] INFO – : 'facts' module is
    disabled.
    I, [2015-09-14T09:46:16.890708 #28453] INFO – : 'templates' module is
    disabled.
    I, [2015-09-14T09:46:16.891413 #28453] INFO – : 'tftp' module is disabled.
    I, [2015-09-14T09:46:16.892185 #28453] INFO – : 'dhcp' module is disabled.
    I, [2015-09-14T09:46:17.455115 #28453] INFO – : 'puppet' settings were
    initialized with default values: :puppet_provider: puppetrun, :puppetdir:
    /etc/puppet, :salt_puppetrun_cmd: puppet.run, :use_cache: true
    I, [2015-09-14T09:46:17.459676 #28453] INFO – : 'bmc' module is disabled.
    I, [2015-09-14T09:46:17.460182 #28453] INFO – : 'realm' module is
    disabled.

> > if I change dns.yml to
> >
> > :dns_provider: nsupdate
> >
> > (or as I really want to do:
> > :enabled: https
> > :dns_provider: nsupdate_gss
> > :dns_server: 192.168.180.104
> > :dns_tsig_keytab: /etc/foreman/foremanproxy.keytab
> > :dns_tsig_principal: foremanproxy@BORG.TREK
> > )
> >
> > dns settings does not initialize, there is no entry in proxy.log
>
> Check that you have ":settings_directory: /etc/foreman-proxy/settings.d"
> in /etc/foreman-proxy/settings.yml for it to find these additional files.
>
>
$ grep settings_directory /etc/foreman-proxy/settings.yml
:settings_directory: /etc/foreman-proxy/settings.d

thanks for looking into this,
/Erik

··· On Wednesday, September 2, 2015 at 1:42:51 PM UTC+2, Dominic Cleal wrote: > On 27/08/15 09:34, Erik Hjelmås wrote:


Dominic Cleal
dom...@cleal.org <javascript:>

I think this is working correctly, I saw something very similar last week.

I don't believe there's any log message when a module is initialised
successfully - the only message is the "initialised with default values"
which happens only when the module uses some default values.

In 1.9, the only default value is the dns_provider, so when you specify
it there aren't any more default values used and so it doesn't log.

This is confusing, we could add an additional log message simply to say
it's initialised - I'd suggest filing a bug at
Foreman.

··· On 14/09/15 08:49, Erik Hjelmås wrote: > thank you for your reply, sorry for not paying attention to my post, > > On Wednesday, September 2, 2015 at 1:42:51 PM UTC+2, Dominic Cleal wrote: > > On 27/08/15 09:34, Erik Hjelmås wrote: > > Hi, > > > > if my dns.yml only has (in addition to --- of course) > > > > :enabled: https > > > > then proxy.log says > > > > INFO -- : 'dns' settings were initialized with default values: > > :dns_provider: nsupdate > > Could you provide a bit more of the log file please? > > > If i don't set provider (notice dns is initialized with default provider): > > $ grep dns_provider: /etc/foreman-proxy/settings.d/dns.yml > #:dns_provider: nsupdate_gss > $ service foreman-proxy restart > * Restarting foreman-proxy foreman-proxy > ...done. > $ tail -n 12 /var/log/foreman-proxy/proxy.log > I, [2015-09-14T09:38:42.902127 #28143] INFO -- : 'bmc' module is disabled. > I, [2015-09-14T09:38:42.902657 #28143] INFO -- : 'realm' module is > disabled. > W, [2015-09-14T09:41:08.299193 #28263] WARN -- : Couldn't find settings > file /etc/foreman-proxy/settings.d/foreman_proxy.yml. Using default > settings. > I, [2015-09-14T09:41:08.299607 #28263] INFO -- : 'foreman_proxy' > settings were initialized with default values: :enabled: true > I, [2015-09-14T09:41:08.307588 #28263] INFO -- : 'facts' module is > disabled. > I, [2015-09-14T09:41:08.308671 #28263] INFO -- : 'dns' settings were > initialized with default values: :dns_provider: nsupdate > I, [2015-09-14T09:41:08.316116 #28263] INFO -- : 'templates' module is > disabled. > I, [2015-09-14T09:41:08.316864 #28263] INFO -- : 'tftp' module is disabled. > I, [2015-09-14T09:41:08.317531 #28263] INFO -- : 'dhcp' module is disabled. > I, [2015-09-14T09:41:08.963583 #28263] INFO -- : 'puppet' settings were > initialized with default values: :puppet_provider: puppetrun, > :puppetdir: /etc/puppet, :salt_puppetrun_cmd: puppet.run, :use_cache: true > I, [2015-09-14T09:41:08.968332 #28263] INFO -- : 'bmc' module is disabled. > I, [2015-09-14T09:41:08.968958 #28263] INFO -- : 'realm' module is > disabled. > > If i do set provider (notice dns does not appear at all in the log): > > $ grep dns_provider: /etc/foreman-proxy/settings.d/dns.yml > :dns_provider: nsupdate_gss > $ service foreman-proxy restart > * Restarting foreman-proxy foreman-proxy > ...done. > $ tail -n 10 /var/log/foreman-proxy/proxy.log > I, [2015-09-14T09:45:32.926533 #28407] INFO -- : 'realm' module is > disabled. > W, [2015-09-14T09:46:16.872296 #28453] WARN -- : Couldn't find settings > file /etc/foreman-proxy/settings.d/foreman_proxy.yml. Using default > settings. > I, [2015-09-14T09:46:16.872841 #28453] INFO -- : 'foreman_proxy' > settings were initialized with default values: :enabled: true > I, [2015-09-14T09:46:16.881412 #28453] INFO -- : 'facts' module is > disabled. > I, [2015-09-14T09:46:16.890708 #28453] INFO -- : 'templates' module is > disabled. > I, [2015-09-14T09:46:16.891413 #28453] INFO -- : 'tftp' module is disabled. > I, [2015-09-14T09:46:16.892185 #28453] INFO -- : 'dhcp' module is disabled. > I, [2015-09-14T09:46:17.455115 #28453] INFO -- : 'puppet' settings were > initialized with default values: :puppet_provider: puppetrun, > :puppetdir: /etc/puppet, :salt_puppetrun_cmd: puppet.run, :use_cache: true > I, [2015-09-14T09:46:17.459676 #28453] INFO -- : 'bmc' module is disabled. > I, [2015-09-14T09:46:17.460182 #28453] INFO -- : 'realm' module is > disabled.


Dominic Cleal
dominic@cleal.org

sorry but this is still quite strange:

dns is enabled with https and nsupdate_gss as mentioned in previous posts:

$ grep -Ev '^#' /etc/foreman-proxy/settings.d/dns.yml

··· --- :enabled: https :dns_provider: nsupdate_gss :dns_server: dir01.borg.trek :dns_ttl: 86400 :dns_tsig_keytab: /etc/foreman/foremanproxy.keytab :dns_tsig_principal: foremanproxy/dir01.borg.trek@BORG.TREK

$ ping dir01.borg.trek
PING dir01.borg.trek (192.168.180.104) 56(84) bytes of data.
64 bytes from dir01.borg.trek (192.168.180.104): icmp_seq=1 ttl=128
time=0.918 ms
64 bytes from dir01.borg.trek (192.168.180.104): icmp_seq=2 ttl=128
time=1.19 ms
^C
— dir01.borg.trek ping statistics —
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.918/1.055/1.193/0.141 ms

$ kinit foremanproxy@BORG.TREK -k -t /etc/foreman/foremanproxy.keytab
$

Ive enabled debug logging according to
http://theforeman.org/manuals/1.9/index.html#7.2Debugging
(also for foreman-proxy and restarted both of course)

dns should then use nsupdate_gss (with keytab etc which Ive verified
authenticates without errors) to add the hostname to dns if a new host is
created or if it changes its name right?

so I tcpdump -n dst host 192.168.180.104 (traffic to the dns server) and
detect nothing from foreman (tried first changing name of a host, then
tried adding a new host)

if I add a new host (by running puppet agent on it), the host appears in
foreman and I sign the certificate, but no sign of any dns updates

(the new hosts name is ‘test’ with ip address ending in 148)
$ grep -E ‘(test|148)’ foreman-proxy/proxy.log | grep 06:28
D, [2015-09-17T06:28:15.223216 #26106] DEBUG – : Executing /usr/bin/sudo
-S /usr/bin/puppet cert --ssldir /var/lib/puppet/ssl --sign test.borg.trek
I, [2015-09-17T06:28:16.974627 #26106] INFO – : signed puppet certificate
for test.borg.trek
192.168.180.100 - - [17/Sep/2015 06:28:16] “POST /puppet/ca/test.borg.trek
HTTP/1.1” 200 - 1.7542

In other words, the puppet part get logged by foreman-proxy, but no sign of
any dns/nsupdate

in the foreman log there’s no sign of dns either:

$ grep -E ‘(dns|DNS|nsupdate|gss|proxy)’ foreman-proxy/proxy.log
W, [2015-09-17T06:04:32.849175 #26101] WARN – : Couldn’t find settings
file /etc/foreman-proxy/settings.d/foreman_proxy.yml. Using default
settings.
I, [2015-09-17T06:04:32.849553 #26101] INFO – : ‘foreman_proxy’ settings
were initialized with default values: :enabled: true

and the warning about ‘Couldn’t find settings file
/etc/foreman-proxy/settings.d/foreman_proxy.yml’ is not important according
to
https://groups.google.com/forum/#!topic/foreman-users/K8M5t16ITlA

sorry Im stuck, of course I can manually update my DNS server, but I really
want to get this to work, but with no logs and no network traffic (and
maybe me misunderstanding how this is supposed to work?) it is hard

any help still greatly appreciated

/Erik

Yes, if the domain has the DNS Proxy set. In Foreman, go to Infra >
Domains > your domain and ensure DNS Proxy is set at the bottom of the page.

··· On 17/09/15 05:54, Erik Hjelmås wrote: > dns should then use nsupdate_gss (with keytab etc which Ive verified > authenticates without errors) to add the hostname to dns if a new host > is created or if it changes its name right?


Dominic Cleal
dominic@cleal.org

yes, this was already set

I tried creating a New host in foreman now, and that works, that is the
config (in case useful for others):

:enabled: https
:dns_provider: nsupdate_gss
:dns_server: dir01.borg.trek
:dns_ttl: 86400
:dns_tsig_keytab: /etc/foreman/foremanproxy.keytab
:dns_tsig_principal: foremanproxy@BORG.TREK

and with a chmod 644 /etc/foreman/foremanproxy.keytab
did the trick (the keytab shouldnt be 644 of course, but chown
foreman:foreman did not help, I'll figure this out later)

But DNS does not update if I join a new host using puppet, in other words
what I normally do with a new host is just run
puppet agent -t --server=foreman
on the client and the sign the key in foreman, shouldnt this create a DNS
record for the host as well? And what if I change the hostname in foreman
shouldnt that update DNS?

thanks so much for all help so far!

/Erik

··· On Thursday, September 17, 2015 at 8:57:02 AM UTC+2, Dominic Cleal wrote: > > On 17/09/15 05:54, Erik Hjelmås wrote: > > dns should then use nsupdate_gss (with keytab etc which Ive verified > > authenticates without errors) to add the hostname to dns if a new host > > is created or if it changes its name right? > > Yes, if the domain has the DNS Proxy set. In Foreman, go to Infra > > Domains > your domain and ensure DNS Proxy is set at the bottom of the > page. > >

>
> > dns should then use nsupdate_gss (with keytab etc which Ive verified
> > authenticates without errors) to add the hostname to dns if a new
> host
> > is created or if it changes its name right?
>
> Yes, if the domain has the DNS Proxy set. In Foreman, go to Infra >
> Domains > your domain and ensure DNS Proxy is set at the bottom of
> the page.
>
>
> yes, this was already set
>
> I tried creating a New host in foreman now, and that works, that is the
> config (in case useful for others):
>
> :enabled: https
> :dns_provider: nsupdate_gss
> :dns_server: dir01.borg.trek
> :dns_ttl: 86400
> :dns_tsig_keytab: /etc/foreman/foremanproxy.keytab
> :dns_tsig_principal: foremanproxy@BORG.TREK
>
> and with a chmod 644 /etc/foreman/foremanproxy.keytab
> did the trick (the keytab shouldnt be 644 of course, but chown
> foreman:foreman did not help, I'll figure this out later)

The keytab should be accessible to the foreman-proxy user, not the
foreman user.

> But DNS does not update if I join a new host using puppet, in other
> words what I normally do with a new host is just run
> puppet agent -t --server=foreman
> on the client and the sign the key in foreman, shouldnt this create a
> DNS record for the host as well?

No, hosts created via Puppet runs are unmanaged, while hosts created via
the UI (when :unattended: true in settings.yaml) are managed. Managed
means the full provisioning cycle, including DNS, DHCP etc records will
be created.

You can edit the host and click the Manage Host button to change its
state. This should create records when you save it, but you'll probably
need to set extra attributes.

> And what if I change the hostname in
> foreman shouldnt that update DNS?

It should do, if the host is managed.

··· On 17/09/15 13:55, Erik Hjelmås wrote: > On Thursday, September 17, 2015 at 8:57:02 AM UTC+2, Dominic Cleal wrote: > On 17/09/15 05:54, Erik Hjelmås wrote:


Dominic Cleal
dominic@cleal.org

Fantastic! Setting the host to managed solved the problem, this means that
when I register a new host from puppet, in addition to assigning a puppet
role to it, I have to mark it as managed and rename it (and then rename it
back to its original name again :), I can live with that, the important
thing is that now I can do it all from Foreman,

thank you so much Dominic!!

/Erik

PS! and chown foreman-proxy:foreman-proxy on the keytab file fixed the
access issues as well of course

No problem. In 1.10, you'll be able to click "Rebuild Configs" from the
host list to regenerate DHCP/DNS records after making it managed, which
should avoid the need for renaming the host. You could probably file a
feature request to do this automatically when a host is made managed, it
would often be useful.

··· On 17/09/15 15:03, Erik Hjelmås wrote: > Fantastic! Setting the host to managed solved the problem, this means > that when I register a new host from puppet, in addition to assigning a > puppet role to it, I have to mark it as managed and rename it (and then > rename it back to its original name again :), I can live with that, the > important thing is that now I can do it all from Foreman, > > thank you so much Dominic!!


Dominic Cleal
dominic@cleal.org

> No problem. In 1.10, you'll be able to click "Rebuild Configs" from the
> host list to regenerate DHCP/DNS records after making it managed, which
> should avoid the need for renaming the host. You could probably file a
> feature request to do this automatically when a host is made managed, it
> would often be useful.
>
>
excellent! looking forward to 1.10

"Rebuild Configs" will make the workflow much nicer, and I guess it would
be useful to have the option of running it automatically so I have filed a
feature request for it now at Feature #11879: Automatic "Rebuild configs" when making a host Managed in 1.10 - Foreman

/Erik