Build notifying results in '409 Conflict' and building loop

Support
1

Hi,

Notifying foreman about successful host build doesn't work for me.

This shows up in foreman production log:

Started GET "/unattended/built?token=a8b7240a-506e-42a6-b8c9-ee2699b56fe9"
for 10.11.63.2 at 2014-09-14 21:22:35 +0000
Processing by UnattendedController#built as /
Parameters: {"token"=>"a8b7240a-506e-42a6-b8c9-ee2699b56fe9"}
Found dnstest.mydomain.net
unattended: dnstest.mydomain.net is Built!
DHCP records dnstest.mydomain.net-52:54:00:48:e3:59/10.0.210.254 already
exists
DNS A Records dnstest.mydomain.net/10.0.210.254 already exists
DHCP records dnstest.mydomain.net-52:54:00:48:e3:59/10.0.210.254 already
exists
DNS A Records dnstest.mydomain.net/10.0.210.254 already exists
Completed 409 Conflict in 1669ms (ActiveRecord: 18.4ms)

And when I try to notify manually I get the same error on the client:

wget --no-check-certificate
http://10.11.10.210:80/unattended/built?token=a8b7240a-506e-42a6-b8c9-ee2699b56fe9
–2014-09-14 21:16:41–
http://10.11.10.210/unattended/built?token=a8b7240a-506e-42a6-b8c9-ee2699b56fe9
Connecting to 10.11.10.210:80… connected.
HTTP request sent, awaiting response… 409 Conflict
2014-09-14 21:16:42 ERROR 409: Conflict.

I thought it is this bug Bug #2279: ERROR 409: Conflict - Foreman or it's
related to Bug #4375: Kickstart provisioning is on a loop when using compute profiles with ovirt Compute Resource. - Foreman as mentioned in the
last comment, but I'm using KVM not oVirt (occurs on bare metal too), and I
have the same problem on ubuntu clients as well, while the reported bug
showed up only on CentOS'es, so I think there's something wrong with my
config.

If I remove my host's DHCP/DNS entries by hand, wget finishes properly
without any errors, but my host is still being left in the building stage.

I have 2 proxys in two networks, one with TFTP, DNS, DHCP, Puppet, and
Puppet CA and another only with TFTP, DHCP - the problem occurs when using
either of them.

I'm using:
rubygem-hammer_cli_foreman-0.1.3-1.el6.noarch
ruby193-rubygem-foreman_bootdisk-3.1.2-1.el6.noarch
foreman-selinux-1.6.0-1.el6.noarch
foreman-postgresql-1.6.0-1.el6.noarch
foreman-1.6.0-1.el6.noarch
foreman-installer-1.6.0-1.el6.noarch
foreman-libvirt-1.6.0-1.el6.noarch
foreman-cli-1.6.0-1.el6.noarch
foreman-proxy-1.6.0-1.el6.noarch
ruby193-rubygem-foreman_setup-2.0.4-1.el6.noarch
foreman-compute-1.6.0-1.el6.noarch
foreman-console-1.6.0-1.el6.noarch
foreman-release-1.6.0-1.el6.noarch
on my foreman host and proxy (CentOS 6.5)

Has anyone stumbled upon such problem?

2

I managed to narrow the problem a bit.
Both my foreman-proxies are behind NAT and they complain in their logs with:

10.11.10.210 - - [17/Sep/2014 05:33:59] "GET /serverName HTTP/1.1" 200 24
0.0009
10.11.10.210 - - [17/Sep/2014 05:33:59] "GET /10.0.210.0/52:54:00:37:fa:03
HTTP/1.1" 200 176 0.0059
E, [2014-09-17T05:33:59.749992 #18284] ERROR – : Record
10.0.210.0/10.11.63.2 not found
10.11.10.210 - - [17/Sep/2014 05:33:59] "GET /10.0.210.0/10.11.63.2
HTTP/1.1" 404 38 0.0058
10.11.10.210 - - [17/Sep/2014 05:33:59] "GET /10.0.210.0/52:54:00:37:fa:03
HTTP/1.1" 200 176 0.0053
E, [2014-09-17T05:33:59.973169 #18284] ERROR – : Record
10.0.210.0/10.11.63.2 not found
10.11.10.210 - - [17/Sep/2014 05:33:59] "GET /10.0.210.0/10.11.63.2
HTTP/1.1" 404 38 0.0057

10.11.63.2 - is doing NAT of course, no wonder there is no record for that
somewhere.
I set up a 3rd proxy and a new host in the same network as foreman and
tested it - notifying foreman about successful build works.

The question now is how to get that working behind NAT. Shouldn't it work
right out of the box? I thought that what (now default) host tokens were
introduced for ( Feature #1069: Unattended install behind firewall and built status - Foreman )

3

> The question now is how to get that working behind NAT. Shouldn't it work
> right out of the box? I thought that what (now default) host tokens were
> introduced for ( Feature #1069: Unattended install behind firewall and built status - Foreman )

That's right, is this upgraded instance? It is possible than you have
a version that ships with tokens by default, but your settings and
templates are not "tokenized" yet.

Check your templates and global settings.

··· -- Later, Lukas #lzap Zapletal
4

> Check your templates and global settings.
>
Thanks for the tip, I had update_ip_from_built_request set to true in
provisioning settings.

Notifying now works from behind NAT.