10GBs NIC PXE Boot not working?

Hi,

I just can't get our servers (IBM X3650 M5) to PXE boot. It works fine with
the 1GBs nics but not with the 10Gbs. I enabled debugging on the tftp
server but it looks like the server is not even reaching it. During boot I
can see that both NICS register with DHCP and that's it.

I wonder if it is a setting on the 10GBs switch like DHCP helper adress?

Cheers
Oli

'Oliver Weinmann' via Foreman users writes:
> I just can't get our servers (IBM X3650 M5) to PXE boot.

I don't have experience with that particular server type, but we also
have (Quanta and Dalco/Intel) servers with GigE and 10GE interfaces. We
never use the GigE (except the separate BMC/IPMI port) and always
PXEboot from the 10GEs. So it should be possible (unless IBM did
something nasty).

> It works fine with the 1GBs nics but not with the 10Gbs. I enabled
> debugging on the tftp server but it looks like the server is not even
> reaching it. During boot I can see that both NICS register with DHCP
> and that's it.

> I wonder if it is a setting on the 10GBs switch like DHCP helper adress?

If your 10GE ports are on the same logical subnet (VLAN) as the GigE
ports, then that shouldn't be necessary.

What we do is that we put the (untagged/default VLAN) 10GE ports in a
special "provisioning" network that the DHCP server controlled by
Foreman (Smart Proxy) has access to. So we don't need DHCP helper
addresses.

(Once the systems are installed, they will not use that VLAN anymore.
They use separate VLANs that are used for production traffic.)

Another possible reason for your problem may be that the system doesn't
try to PXEboot from the 10GE ports. That's something that you can see
by staring at the console while it's trying to boot, and/or by running
tcpdump looking for packets from the 10GE interfaces (you can filter by
their MAC addresses; of course you must run the tcpdump on a port that
"sees" all traffic from those servers' 10GE ports). If the server
doesn't want to boot from the 10GEs, then maybe you can fix it by
changing BIOS settings. But depending on the age of the server/BIOS, it
may even turn out that PXE isn't even supported on the 10GE ports.

Hope this helps,

··· -- Simon.

Not sure if this helps but I had a similar problem.

The problem ended up being the start up time for a port on the switch.

We disabled STP and the startup delay on the switch.

··· On 11/06/2016 04:45 AM, Simon Leinen wrote: > 'Oliver Weinmann' via Foreman users writes: >> I just can't get our servers (IBM X3650 M5) to PXE boot. > I don't have experience with that particular server type, but we also > have (Quanta and Dalco/Intel) servers with GigE and 10GE interfaces. We > never use the GigE (except the separate BMC/IPMI port) and always > PXEboot from the 10GEs. So it should be possible (unless IBM did > something nasty). > >> It works fine with the 1GBs nics but not with the 10Gbs. I enabled >> debugging on the tftp server but it looks like the server is not even >> reaching it. During boot I can see that both NICS register with DHCP >> and that's it. >> I wonder if it is a setting on the 10GBs switch like DHCP helper adress? > If your 10GE ports are on the same logical subnet (VLAN) as the GigE > ports, then that shouldn't be necessary. > > What we do is that we put the (untagged/default VLAN) 10GE ports in a > special "provisioning" network that the DHCP server controlled by > Foreman (Smart Proxy) has access to. So we don't need DHCP helper > addresses. > > (Once the systems are installed, they will not use that VLAN anymore. > They use separate VLANs that are used for production traffic.) > > Another possible reason for your problem may be that the system doesn't > try to PXEboot from the 10GE ports. That's something that you can see > by staring at the console while it's trying to boot, and/or by running > tcpdump looking for packets from the 10GE interfaces (you can filter by > their MAC addresses; of course you must run the tcpdump on a port that > "sees" all traffic from those servers' 10GE ports). If the server > doesn't want to boot from the 10GEs, then *maybe* you can fix it by > changing BIOS settings. But depending on the age of the server/BIOS, it > may even turn out that PXE isn't even supported on the 10GE ports. > > Hope this helps,


Alvin Starr || voice: (905)513-7688
Netvel Inc. || Cell: (416)806-0133
alvin@netvel.net ||

Thanks for all your hints. I think the problem
Is the dhcp helper address. Our dhcp sits on a different vlan. I noticed that it doesn't work on the 1gb nics too so it is not a problem with the 10gb nics. It seems that some of our switches are not configured with a dhcp helper address. We mainly use Cisco where it can be configured but I have not found a configuration parameter on the Lenovo g8124 10gb switches yet. I will have a look once I'm back in the office on Tuesday.

··· Von meinem iPhone gesendet

Am 06.11.2016 um 14:54 schrieb Alvin Starr alvin@netvel.net:

Not sure if this helps but I had a similar problem.

The problem ended up being the start up time for a port on the switch.

We disabled STP and the startup delay on the switch.

On 11/06/2016 04:45 AM, Simon Leinen wrote:
‘Oliver Weinmann’ via Foreman users writes:

I just can’t get our servers (IBM X3650 M5) to PXE boot.
I don’t have experience with that particular server type, but we also
have (Quanta and Dalco/Intel) servers with GigE and 10GE interfaces. We
never use the GigE (except the separate BMC/IPMI port) and always
PXEboot from the 10GEs. So it should be possible (unless IBM did
something nasty).

It works fine with the 1GBs nics but not with the 10Gbs. I enabled
debugging on the tftp server but it looks like the server is not even
reaching it. During boot I can see that both NICS register with DHCP
and that’s it.
I wonder if it is a setting on the 10GBs switch like DHCP helper adress?
If your 10GE ports are on the same logical subnet (VLAN) as the GigE
ports, then that shouldn’t be necessary.

What we do is that we put the (untagged/default VLAN) 10GE ports in a
special “provisioning” network that the DHCP server controlled by
Foreman (Smart Proxy) has access to. So we don’t need DHCP helper
addresses.

(Once the systems are installed, they will not use that VLAN anymore.
They use separate VLANs that are used for production traffic.)

Another possible reason for your problem may be that the system doesn’t
try to PXEboot from the 10GE ports. That’s something that you can see
by staring at the console while it’s trying to boot, and/or by running
tcpdump looking for packets from the 10GE interfaces (you can filter by
their MAC addresses; of course you must run the tcpdump on a port that
"sees" all traffic from those servers’ 10GE ports). If the server
doesn’t want to boot from the 10GEs, then maybe you can fix it by
changing BIOS settings. But depending on the age of the server/BIOS, it
may even turn out that PXE isn’t even supported on the 10GE ports.

Hope this helps,


Alvin Starr || voice: (905)513-7688
Netvel Inc. || Cell: (416)806-0133
alvin@netvel.net ||

You received this message because you are subscribed to a topic in the Google Groups “Foreman users” group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/foreman-users/R7mOV1ZtV9E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at https://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.