IP autosuggestion

We've been struggling a bit to get DHCP and Foreman to play nicely
together, and I'm a bit confused about the expected behaviour around IP
autosuggestion. Could well be that we're doing something wrong so hoping
someone can point us in the right direction.

We initially had a DHCP pool configuration which covered pretty much the
entire subnet, including the range we'd defined in Foreman for IP
auto-suggestion.

subnet 10.10.192.0 netmask 255.255.255.0 {
pool {
failover peer "dhcp-failover";
range 10.10.192.8 10.10.192.249;
}
filename "/pxelinux.0";
next-server 10.10.192.250;
option subnet-mask 255.255.255.0;
option routers 10.10.192.254;
option domain-search "sal01.datacentred.co.uk";
default-lease-time 86400;
max-lease-time 172800;
}

This caused some issues as we'd get dynamic and static leases present for
the same IP - machines booted initially onto a dynamic lease, and then were
assigned a static lease from Foreman during provisioning so would have two
leases present.

In order to try and solve that problem, we changed the DHCP range so that
we had a dynamic pool for initial discovery, and defined the range for
Foreman auto-suggestion as outside of that pool ( but obviously still
inside the subnet )

subnet 10.10.192.0 netmask 255.255.255.0 {
pool {
failover peer "dhcp-failover";
range 10.10.192.16 10.10.192.100;
}
filename "/pxelinux.0";
next-server 10.10.192.250;
option subnet-mask 255.255.255.0;
option routers 10.10.192.254;
option domain-search "sal01.datacentred.co.uk";
default-lease-time 86400;
max-lease-time 172800;
}

We've got Foreman configured to suggest IP's from 101 to 249 but we still
don't get the expected behaviour. Foreman fills in the suggested IP address
with the IP from the dynamic pool which the machine has got during
discovery and not from the range we've configured for auto-suggestion.

Are we doing something wrong here ? We're using Foreman 1.4.2

Thanks

Matt

··· -- DataCentred Limited registered in England and Wales no. 05611763

Did you fill in the same range in the Subnet UI in Foreman? Currently
Foreman doesn't directly use what's in the dhcpd.conf when finding a
free IP, but it does respect the range configured in Foreman's own UI.

Greg

Yes, the auto-suggestion range is filled in from the Subnet UI after we
pull the subnet in from the proxy.

Start of IP range
Optional: Starting IP Address for IP auto suggestion
End of IP range
Optional: Ending IP Address for IP auto suggestion

··· On Thursday, April 3, 2014 5:19:19 PM UTC+1, Greg Sutcliffe wrote: > > Did you fill in the same range in the Subnet UI in Foreman? Currently > Foreman doesn't directly use what's in the dhcpd.conf when finding a > free IP, but it does respect the range configured in Foreman's own UI. > > Greg >


DataCentred Limited registered in England and Wales no. 05611763

Interesting, so I'd expect the autosuggestion to fill in an IP from
101 to 249. This is a brand new host in Foreman right? You're not
editing an existing Host with an IP set? Assuming so, could you tail
the proxy logs while filling out the form and see if you get any
requests to /dhcp/10.10.192.0/unused_ip ?

Greg

> Yes, the auto-suggestion range is filled in from the Subnet UI after we
> pull the subnet in from the proxy.
>
>
> Start of IP range
> Optional: Starting IP Address for IP auto suggestion
> End of IP range
> Optional: Ending IP Address for IP auto suggestion

Foreman also tries to actually ping those addresses first. Any chance
those you are expecting to be assigned are occupied already. I mean does
respond to pings.

··· -- Later,

Lukas “lzap” Zapletal
irc: lzap #theforeman

So after a bit more digging it looks if the machine already has a lease (
which it does because it's been through the discovery plugin process ),
then that lease is suggested as opposed to a new lease from the IP auto
suggestion range, even if the current lease is outside of the range for IP
auto suggestion. Is that the correct behaviour ? In our case we have a
dynamic range we use for discovery but we then want to re-assign
provisioned machines into a different range once they have a static IP set
in Foreman.

··· On Friday, April 4, 2014 3:38:38 PM UTC+1, Lukas Zapletal wrote: > > > Yes, the auto-suggestion range is filled in from the Subnet UI after we > > pull the subnet in from the proxy. > > > > > > Start of IP range > > Optional: Starting IP Address for IP auto suggestion > > End of IP range > > Optional: Ending IP Address for IP auto suggestion > > Foreman also tries to actually ping those addresses first. Any chance > those you are expecting to be assigned are occupied already. I mean does > respond to pings. > > -- > Later, > > Lukas "lzap" Zapletal > irc: lzap #theforeman >


DataCentred Limited registered in England and Wales no. 05611763

> So after a bit more digging it looks if the machine already has a lease (
> which it does because it's been through the discovery plugin process ),
> then that lease is suggested as opposed to a new lease from the IP auto
> suggestion range, even if the current lease is outside of the range for IP
> auto suggestion. Is that the correct behaviour ? In our case we have a
> dynamic range we use for discovery but we then want to re-assign
> provisioned machines into a different range once they have a static IP set
> in Foreman.

The way it works (from lib/proxy/dhcp/subnet.rb):

  free_ips = valid_range(args) - records.collect{|r| r.ip}

Speaking about discovery, you can determine that the request comes from
a discovery host using it's hostname which is being sent in a DHCP
request I suppose. I think I can change this from "localhost" to let's
say "discovery". Would that help for you? You might use different pool
for all hosts with name "discovery" maybe.

··· -- Later,

Lukas “lzap” Zapletal
irc: lzap #theforeman

I think the confusion comes about because the expected behaviour - having
filled out the autosuggestion range range in the Foreman UI - is that it'll
only return a suggested address from within that range. The function
within subnet.rb re-uses the existing lease if found for the autosuggestion
when it comes to provisioning, which may well be outside of this range when
you're using the auto discovery feature like we're doing.

··· On Wednesday, 9 April 2014 10:21:03 UTC+1, Lukas Zapletal wrote: > > > So after a bit more digging it looks if the machine already has a lease > ( > > which it does because it's been through the discovery plugin process ), > > then that lease is suggested as opposed to a new lease from the IP auto > > suggestion range, even if the current lease is outside of the range for > IP > > auto suggestion. Is that the correct behaviour ? In our case we have a > > dynamic range we use for discovery but we then want to re-assign > > provisioned machines into a different range once they have a static IP > set > > in Foreman. > > The way it works (from lib/proxy/dhcp/subnet.rb): > > free_ips = valid_range(args) - records.collect{|r| r.ip} > > Speaking about discovery, you can determine that the request comes from > a discovery host using it's hostname which is being sent in a DHCP > request I suppose. I think I can change this from "localhost" to let's > say "discovery". Would that help for you? You might use different pool > for all hosts with name "discovery" maybe. >

-Nick


DataCentred Limited registered in England and Wales no. 05611763

The bit of code we're looking at from lib/proxy/dhcp/subnet.rb is :

  # first check if we already have a record for this host
  # if we do, we can simply reuse the same ip address.
  if args[:mac] and r=has_mac?(args[:mac])
    logger.debug "Found an existing dhcp record #{r}, reusing..."
    return r.ip
  end

I think the behaviour we were expecting in pseudo code would be something
like :

if host has been provisioned before in foreman ( ie. it is a registered
host and has a static DHCP lease defined )

re-use existing dhcp record

elif host is a discovered host ( ie. it has a dynamic lease )

check autosuggestion range and suggest free IP

Matt

··· On Wednesday, April 9, 2014 11:55:31 AM UTC+1, Nick Jones wrote: > > > > On Wednesday, 9 April 2014 10:21:03 UTC+1, Lukas Zapletal wrote: >> >> > So after a bit more digging it looks if the machine already has a lease >> ( >> > which it does because it's been through the discovery plugin process ), >> > then that lease is suggested as opposed to a new lease from the IP auto >> > suggestion range, even if the current lease is outside of the range for >> IP >> > auto suggestion. Is that the correct behaviour ? In our case we have a >> > dynamic range we use for discovery but we then want to re-assign >> > provisioned machines into a different range once they have a static IP >> set >> > in Foreman. >> >> The way it works (from lib/proxy/dhcp/subnet.rb): >> >> free_ips = valid_range(args) - records.collect{|r| r.ip} >> >> Speaking about discovery, you can determine that the request comes from >> a discovery host using it's hostname which is being sent in a DHCP >> request I suppose. I think I can change this from "localhost" to let's >> say "discovery". Would that help for you? You might use different pool >> for all hosts with name "discovery" maybe. >> > > I think the confusion comes about because the expected behaviour - having > filled out the autosuggestion range range in the Foreman UI - is that it'll > *only* return a suggested address from within that range. The function > within subnet.rb re-uses the existing lease if found for the autosuggestion > when it comes to provisioning, which may well be outside of this range when > you're using the auto discovery feature like we're doing. > > -- > > -Nick > > DataCentred Limited registered in England and Wales no. 05611763


DataCentred Limited registered in England and Wales no. 05611763

Guys,

I created bug to track this feature.

http://projects.theforeman.org/issues/5123

Feel free to send us a patch. I am not sure when we can get to this. If
you do it quick, you can have this in Foreman 1.5 :slight_smile:

LZ

··· On Wed, Apr 09, 2014 at 04:03:16AM -0700, Matt Jarvis wrote: > > > On Wednesday, April 9, 2014 11:55:31 AM UTC+1, Nick Jones wrote: > > > > > > > > On Wednesday, 9 April 2014 10:21:03 UTC+1, Lukas Zapletal wrote: > >> > >> > So after a bit more digging it looks if the machine already has a lease > >> ( > >> > which it does because it's been through the discovery plugin process ), > >> > then that lease is suggested as opposed to a new lease from the IP auto > >> > suggestion range, even if the current lease is outside of the range for > >> IP > >> > auto suggestion. Is that the correct behaviour ? In our case we have a > >> > dynamic range we use for discovery but we then want to re-assign > >> > provisioned machines into a different range once they have a static IP > >> set > >> > in Foreman. > >> > >> The way it works (from lib/proxy/dhcp/subnet.rb): > >> > >> free_ips = valid_range(args) - records.collect{|r| r.ip} > >> > >> Speaking about discovery, you can determine that the request comes from > >> a discovery host using it's hostname which is being sent in a DHCP > >> request I suppose. I think I can change this from "localhost" to let's > >> say "discovery". Would that help for you? You might use different pool > >> for all hosts with name "discovery" maybe. > >> > > > > I think the confusion comes about because the expected behaviour - having > > filled out the autosuggestion range range in the Foreman UI - is that it'll > > *only* return a suggested address from within that range. The function > > within subnet.rb re-uses the existing lease if found for the autosuggestion > > when it comes to provisioning, which may well be outside of this range when > > you're using the auto discovery feature like we're doing. > > > > -- > > > > -Nick > > > > DataCentred Limited registered in England and Wales no. 05611763 > > > > The bit of code we're looking at from lib/proxy/dhcp/subnet.rb is : > > # first check if we already have a record for this host > # if we do, we can simply reuse the same ip address. > if args[:mac] and r=has_mac?(args[:mac]) > logger.debug "Found an existing dhcp record #{r}, reusing..." > return r.ip > end > > I think the behaviour we were expecting in pseudo code would be something > like : > > if host has been provisioned before in foreman ( ie. it is a registered > host and has a static DHCP lease defined ) > > re-use existing dhcp record > > elif host is a discovered host ( ie. it has a dynamic lease ) > > check autosuggestion range and suggest free IP > > > Matt > > -- > DataCentred Limited registered in England and Wales no. 05611763


Later,

Lukas “lzap” Zapletal
irc: lzap #theforeman

With reference to https://github.com/theforeman/smart-proxy/pull/148, this
issue is partially solved by our change to the proxy in that the API call
will now return an IP from the auto-suggestion range if the current IP is
not in that range.

However, when you provision a discovered host, the initial view in Foreman
is not populated by the API call and just adds in the current IP. By
switching to a different subnet and back again, you can force the API call
to happen, at which point the auto-suggested IP is filled in to the form.

For discovered hosts we would ideally like to present the auto-suggested IP
rather than the current IP when switching through to the provisioning view,
ie. to have the API call happen when first switching to the edit view, but
realise this may not be the ideal behaviour under all circumstances. Would
a change in Foreman to do this be acceptable ? What's the view from other
users of the discovery features ?

··· On Wednesday, April 9, 2014 12:11:10 PM UTC+1, Lukas Zapletal wrote: > > Guys, > > I created bug to track this feature. > > http://projects.theforeman.org/issues/5123 > > Feel free to send us a patch. I am not sure when we can get to this. If > you do it quick, you can have this in Foreman 1.5 :-) > > LZ > > On Wed, Apr 09, 2014 at 04:03:16AM -0700, Matt Jarvis wrote: > > > > > > On Wednesday, April 9, 2014 11:55:31 AM UTC+1, Nick Jones wrote: > > > > > > > > > > > > On Wednesday, 9 April 2014 10:21:03 UTC+1, Lukas Zapletal wrote: > > >> > > >> > So after a bit more digging it looks if the machine already has a > lease > > >> ( > > >> > which it does because it's been through the discovery plugin > process ), > > >> > then that lease is suggested as opposed to a new lease from the IP > auto > > >> > suggestion range, even if the current lease is outside of the range > for > > >> IP > > >> > auto suggestion. Is that the correct behaviour ? In our case we > have a > > >> > dynamic range we use for discovery but we then want to re-assign > > >> > provisioned machines into a different range once they have a static > IP > > >> set > > >> > in Foreman. > > >> > > >> The way it works (from lib/proxy/dhcp/subnet.rb): > > >> > > >> free_ips = valid_range(args) - records.collect{|r| r.ip} > > >> > > >> Speaking about discovery, you can determine that the request comes > from > > >> a discovery host using it's hostname which is being sent in a DHCP > > >> request I suppose. I think I can change this from "localhost" to > let's > > >> say "discovery". Would that help for you? You might use different > pool > > >> for all hosts with name "discovery" maybe. > > >> > > > > > > I think the confusion comes about because the expected behaviour - > having > > > filled out the autosuggestion range range in the Foreman UI - is that > it'll > > > *only* return a suggested address from within that range. The > function > > > within subnet.rb re-uses the existing lease if found for the > autosuggestion > > > when it comes to provisioning, which may well be outside of this range > when > > > you're using the auto discovery feature like we're doing. > > > > > > -- > > > > > > -Nick > > > > > > DataCentred Limited registered in England and Wales no. 05611763 > > > > > > > > The bit of code we're looking at from lib/proxy/dhcp/subnet.rb is : > > > > # first check if we already have a record for this host > > # if we do, we can simply reuse the same ip address. > > if args[:mac] and r=has_mac?(args[:mac]) > > logger.debug "Found an existing dhcp record #{r}, reusing..." > > return r.ip > > end > > > > I think the behaviour we were expecting in pseudo code would be > something > > like : > > > > if host has been provisioned before in foreman ( ie. it is a registered > > host and has a static DHCP lease defined ) > > > > re-use existing dhcp record > > > > elif host is a discovered host ( ie. it has a dynamic lease ) > > > > check autosuggestion range and suggest free IP > > > > > > Matt > > > > -- > > DataCentred Limited registered in England and Wales no. 05611763 > > -- > Later, > > Lukas "lzap" Zapletal > irc: lzap #theforeman >


DataCentred Limited registered in England and Wales no. 05611763

I think easy help would be to delete IP address from the host in the
discovery plugin when provisioning it. This should cause request.

But as I noted in the PR comment, I'd rather prefer new button next to
the IP address field that would cause request for another IP address
which could be also helpful in other scenarios.

LZ

··· On Thu, Apr 10, 2014 at 03:42:56AM -0700, Matt Jarvis wrote: > With reference to https://github.com/theforeman/smart-proxy/pull/148, this > issue is partially solved by our change to the proxy in that the API call > will now return an IP from the auto-suggestion range if the current IP is > not in that range. > > However, when you provision a discovered host, the initial view in Foreman > is not populated by the API call and just adds in the current IP. By > switching to a different subnet and back again, you can force the API call > to happen, at which point the auto-suggested IP is filled in to the form. > > For discovered hosts we would ideally like to present the auto-suggested IP > rather than the current IP when switching through to the provisioning view, > ie. to have the API call happen when first switching to the edit view, but > realise this may not be the ideal behaviour under all circumstances. Would > a change in Foreman to do this be acceptable ? What's the view from other > users of the discovery features ? > > On Wednesday, April 9, 2014 12:11:10 PM UTC+1, Lukas Zapletal wrote: > > > > Guys, > > > > I created bug to track this feature. > > > > http://projects.theforeman.org/issues/5123 > > > > Feel free to send us a patch. I am not sure when we can get to this. If > > you do it quick, you can have this in Foreman 1.5 :-) > > > > LZ > > > > On Wed, Apr 09, 2014 at 04:03:16AM -0700, Matt Jarvis wrote: > > > > > > > > > On Wednesday, April 9, 2014 11:55:31 AM UTC+1, Nick Jones wrote: > > > > > > > > > > > > > > > > On Wednesday, 9 April 2014 10:21:03 UTC+1, Lukas Zapletal wrote: > > > >> > > > >> > So after a bit more digging it looks if the machine already has a > > lease > > > >> ( > > > >> > which it does because it's been through the discovery plugin > > process ), > > > >> > then that lease is suggested as opposed to a new lease from the IP > > auto > > > >> > suggestion range, even if the current lease is outside of the range > > for > > > >> IP > > > >> > auto suggestion. Is that the correct behaviour ? In our case we > > have a > > > >> > dynamic range we use for discovery but we then want to re-assign > > > >> > provisioned machines into a different range once they have a static > > IP > > > >> set > > > >> > in Foreman. > > > >> > > > >> The way it works (from lib/proxy/dhcp/subnet.rb): > > > >> > > > >> free_ips = valid_range(args) - records.collect{|r| r.ip} > > > >> > > > >> Speaking about discovery, you can determine that the request comes > > from > > > >> a discovery host using it's hostname which is being sent in a DHCP > > > >> request I suppose. I think I can change this from "localhost" to > > let's > > > >> say "discovery". Would that help for you? You might use different > > pool > > > >> for all hosts with name "discovery" maybe. > > > >> > > > > > > > > I think the confusion comes about because the expected behaviour - > > having > > > > filled out the autosuggestion range range in the Foreman UI - is that > > it'll > > > > *only* return a suggested address from within that range. The > > function > > > > within subnet.rb re-uses the existing lease if found for the > > autosuggestion > > > > when it comes to provisioning, which may well be outside of this range > > when > > > > you're using the auto discovery feature like we're doing. > > > > > > > > -- > > > > > > > > -Nick > > > > > > > > DataCentred Limited registered in England and Wales no. 05611763 > > > > > > > > > > > > The bit of code we're looking at from lib/proxy/dhcp/subnet.rb is : > > > > > > # first check if we already have a record for this host > > > # if we do, we can simply reuse the same ip address. > > > if args[:mac] and r=has_mac?(args[:mac]) > > > logger.debug "Found an existing dhcp record #{r}, reusing..." > > > return r.ip > > > end > > > > > > I think the behaviour we were expecting in pseudo code would be > > something > > > like : > > > > > > if host has been provisioned before in foreman ( ie. it is a registered > > > host and has a static DHCP lease defined ) > > > > > > re-use existing dhcp record > > > > > > elif host is a discovered host ( ie. it has a dynamic lease ) > > > > > > check autosuggestion range and suggest free IP > > > > > > > > > Matt > > > > > > -- > > > DataCentred Limited registered in England and Wales no. 05611763 > > > > -- > > Later, > > > > Lukas "lzap" Zapletal > > irc: lzap #theforeman > > > > -- > DataCentred Limited registered in England and Wales no. 05611763


Later,

Lukas “lzap” Zapletal
irc: lzap #theforeman

Lol, I just posted exactly the same conclusion in the PR request thread.
We'll look into both and submit some code :wink:

··· On Thursday, April 10, 2014 11:52:11 AM UTC+1, Lukas Zapletal wrote: > > I think easy help would be to delete IP address from the host in the > discovery plugin when provisioning it. This should cause request. > > But as I noted in the PR comment, I'd rather prefer new button next to > the IP address field that would cause request for another IP address > which could be also helpful in other scenarios. > > LZ > > On Thu, Apr 10, 2014 at 03:42:56AM -0700, Matt Jarvis wrote: > > With reference to https://github.com/theforeman/smart-proxy/pull/148, > this > > issue is partially solved by our change to the proxy in that the API > call > > will now return an IP from the auto-suggestion range if the current IP > is > > not in that range. > > > > However, when you provision a discovered host, the initial view in > Foreman > > is not populated by the API call and just adds in the current IP. By > > switching to a different subnet and back again, you can force the API > call > > to happen, at which point the auto-suggested IP is filled in to the > form. > > > > For discovered hosts we would ideally like to present the auto-suggested > IP > > rather than the current IP when switching through to the provisioning > view, > > ie. to have the API call happen when first switching to the edit view, > but > > realise this may not be the ideal behaviour under all circumstances. > Would > > a change in Foreman to do this be acceptable ? What's the view from > other > > users of the discovery features ? > > > > On Wednesday, April 9, 2014 12:11:10 PM UTC+1, Lukas Zapletal wrote: > > > > > > Guys, > > > > > > I created bug to track this feature. > > > > > > http://projects.theforeman.org/issues/5123 > > > > > > Feel free to send us a patch. I am not sure when we can get to this. > If > > > you do it quick, you can have this in Foreman 1.5 :-) > > > > > > LZ > > > > > > On Wed, Apr 09, 2014 at 04:03:16AM -0700, Matt Jarvis wrote: > > > > > > > > > > > > On Wednesday, April 9, 2014 11:55:31 AM UTC+1, Nick Jones wrote: > > > > > > > > > > > > > > > > > > > > On Wednesday, 9 April 2014 10:21:03 UTC+1, Lukas Zapletal wrote: > > > > >> > > > > >> > So after a bit more digging it looks if the machine already has > a > > > lease > > > > >> ( > > > > >> > which it does because it's been through the discovery plugin > > > process ), > > > > >> > then that lease is suggested as opposed to a new lease from the > IP > > > auto > > > > >> > suggestion range, even if the current lease is outside of the > range > > > for > > > > >> IP > > > > >> > auto suggestion. Is that the correct behaviour ? In our case > we > > > have a > > > > >> > dynamic range we use for discovery but we then want to > re-assign > > > > >> > provisioned machines into a different range once they have a > static > > > IP > > > > >> set > > > > >> > in Foreman. > > > > >> > > > > >> The way it works (from lib/proxy/dhcp/subnet.rb): > > > > >> > > > > >> free_ips = valid_range(args) - records.collect{|r| r.ip} > > > > >> > > > > >> Speaking about discovery, you can determine that the request > comes > > > from > > > > >> a discovery host using it's hostname which is being sent in a > DHCP > > > > >> request I suppose. I think I can change this from "localhost" to > > > let's > > > > >> say "discovery". Would that help for you? You might use different > > > pool > > > > >> for all hosts with name "discovery" maybe. > > > > >> > > > > > > > > > > I think the confusion comes about because the expected behaviour - > > > having > > > > > filled out the autosuggestion range range in the Foreman UI - is > that > > > it'll > > > > > *only* return a suggested address from within that range. The > > > function > > > > > within subnet.rb re-uses the existing lease if found for the > > > autosuggestion > > > > > when it comes to provisioning, which may well be outside of this > range > > > when > > > > > you're using the auto discovery feature like we're doing. > > > > > > > > > > -- > > > > > > > > > > -Nick > > > > > > > > > > DataCentred Limited registered in England and Wales no. 05611763 > > > > > > > > > > > > > > > > The bit of code we're looking at from lib/proxy/dhcp/subnet.rb is : > > > > > > > > # first check if we already have a record for this host > > > > # if we do, we can simply reuse the same ip address. > > > > if args[:mac] and r=has_mac?(args[:mac]) > > > > logger.debug "Found an existing dhcp record #{r}, > reusing..." > > > > return r.ip > > > > end > > > > > > > > I think the behaviour we were expecting in pseudo code would be > > > something > > > > like : > > > > > > > > if host has been provisioned before in foreman ( ie. it is a > registered > > > > host and has a static DHCP lease defined ) > > > > > > > > re-use existing dhcp record > > > > > > > > elif host is a discovered host ( ie. it has a dynamic lease ) > > > > > > > > check autosuggestion range and suggest free IP > > > > > > > > > > > > Matt > > > > > > > > -- > > > > DataCentred Limited registered in England and Wales no. 05611763 > > > > > > -- > > > Later, > > > > > > Lukas "lzap" Zapletal > > > irc: lzap #theforeman > > > > > > > -- > > DataCentred Limited registered in England and Wales no. 05611763 > > -- > Later, > > Lukas "lzap" Zapletal > irc: lzap #theforeman >


DataCentred Limited registered in England and Wales no. 05611763