PXE Issues + Provisioning Templates Broken

Okay I'm new to Foreman and oVirt but have been trying to set something up
in my lab for our developers.

Unfortunately i'm slightly outside of the IT realm at my company so I don't
have direct access to all the core facilities such as DHCP/DNS Servers.
Also it seems we have an existing PXE server on our network.

I managed to get Foreman Installed and configured most of my provisioning
tasks as well as linked Foreman to my oVirt instance in both directions
Foreman -> oVirt Compute and oVirt -> Foreman External Provider - not sure
I needed both but I configured it anyway.

During my initial testing of deploying a VM to oVirt everything seems to
work except when my VM PXE boots I'm grabbing an IP outside of my
infrastructure and it's trying to load windows so likely it's a domain PXE
server that seems to be taking precedence.

While troubleshooting this I found some iPXE to PXE chain booting thread
<Fetch boot files via http instead of TFTP - Foreman>
which I thought I might try to see if I could get it to grab my TFTP server
details in a different manner. I ran through these settings and now I
can't make any changes to my Provisioning Templates whatsoever.

TFTP setup

Copy the iPXE firmware to the TFTP root directory:

cp /usr/share/ipxe/ipxe.lkrn /var/lib/tftpboot/

The source directory can be different on linux distributions, this is for
Red Hats. The file is shipped in ipxe-bootimgs package.

Do not use symbolic links as TFTP runs in chroot. When using SELinux,
remember to correct file contexts:

restorecon -RvF /var/lib/tftpboot/

When I try and Submit any changes I just get an error screen. I'm not sure
which log to look in as the main /var/log/foreman/production.log doesn't
update when I try and make these changes to try and track down the issue.

I have removed the ipxe.lkrn from /var/lib/tftpboot/ to try and back out
these changes, but so far no luck. I have restarted foreman.service as
well as rebooted the node entirely and every time I get the same error.

Internal Server Error

The server encountered an internal error or misconfiguration and was unable
to complete your request.

Please contact the server administrator at [no address given] to inform
them of the time this error occurred, and the actions you performed just
before this error.

More information about this error may be available in the server error log.

So here I have two questions feedback on either would be most appreciated.

  1. How to point my PXE boot to my foreman server rather than the corporate
    PXE server?

Is this on the oVirt side or Foreman Side to isolate which PXE server to
point to?

  1. Which logs do I need to check to fix my Provisioning Template debacle?

> Okay I'm new to Foreman and oVirt but have been trying to set something up
> in my lab for our developers.
>

Welcome :slight_smile:

> During my initial testing of deploying a VM to oVirt everything seems to
> work except when my VM PXE boots I'm grabbing an IP outside of my
> infrastructure and it's trying to load windows so likely it's a domain PXE
> server that seems to be taking precedence.
>

So one option would be to use premade images that can be configured when
they spin up (I think oVirt calls them "templates", just to overload that
term further :P). That would be one way to bypass the need for a TFTP
server.

However, Foreman will expect to know the IP so it can SSH in and configure
the image once booted. You may have some issues there - but rather than
derail further now, we can discuss it if that's an interesting direction.

> I have restarted foreman.service

You need to restart Apache, by default. The foreman service is provided by
the packages for convenience but we use Apache/Passenger in the installer.

> 1. How to point my PXE boot to my foreman server rather than the corporate
> PXE server?
>
> Is this on the oVirt side or Foreman Side to isolate which PXE server to
> point to?
>
> Assuming you dont want to use images, as per above, then there's three
steps:

  1. Where does the VM get DHCP from. If it's a private network, you can
    control it. If it's bridged, it's time to talk nicely to the DHCP guys :slight_smile:
  2. The DHCP server hands out bootp and next-server records that direct the
    VM to the TFTP server - these need to be right
  3. Or, the existing PXE server needs a class file (something for your
    network segment probably) that chain loads your tftp server.

> 2. Which logs do I need to check to fix my Provisioning Template debacle?
>
I'm not clear on which page you're hitting Submit on, but in general you
probably want to enable debug logs. Take a look at
Troubleshooting - Foreman for
some general hints.

Greg

··· On 26 October 2016 at 11:08, Travis Ross wrote:

Thanks for the quick response Greg - I'm traveling in India so my timezone
is a bit off.

Images - These were my initial thought of use with oVirt until I started
thinking about post deployment. I want there to be as little work
necessary on the VM's as they will be for SQC and Engineering Developers to
quickly test features and dispose of the VM. As you mentioned Foreman
could still work with the Template - but how would it connect. I could
technically have a static IP that gets changed after build - but what
happens when 2 users try and create a VM at the same time. That's likely
not the best solution.

It sounds like I need to get with IT and work on some integration. I'm
sure they will help me out as I had our AD/Linux Team assist in getting
oVirt integrated with our Domain which required a bit more than a simple
login. Unfortunately that work may have been in vain as it looks like now
my User Frontend will be Foreman rather than oVirt for VM Creation.

I'll have to contact my IT Network team and see what would be easiest with
the least amount of questions… my subnet is only for my lab and my
servers so likely just creating a rule on my DHCP server would be easiest
to forward to my Foreman TFTP server. Making changes to the Corporate PXE
server might be a bit more of a pain and stir up more questions than i'm
interested in dealing with.

I wonder outside of PXE is it possible to have Foreman mount the ISO to the
system and have it boot off local media? I have the ISO's stored in an ISO
Repository in oVirt so they are available, I'm just not sure if this is
something that Foreman supports. I guess the VM needs to be on the network
for Foreman to contact it to go through kickstart so simply booting to the
media isn't sufficient. (probably answered my own question there just
thinking allowed).

Regarding my Logs. I'll go through enabling debugging to see if I can
spawn an error. The "Submit" i'm referring to is when I make any changes
to a Provisioning Template, whether it be association or script editing.
When you "Submit" the changes, I am redirected to the error page and
nothing takes place. It seems like the actions I took corrupted something
within the application. It's not like I have much done on the system to if
it's too much trouble I can just start from scratch considering it'll
probably take me awhile to get the DHCP settings updated I can't do much
provisioning anyway.

Thanks,
-Travis

··· On Wednesday, October 26, 2016 at 6:51:04 AM UTC-5, Greg Sutcliffe wrote: > > On 26 October 2016 at 11:08, Travis Ross <tross...@gmail.com > > wrote: > >> Okay I'm new to Foreman and oVirt but have been trying to set something >> up in my lab for our developers. >> > > Welcome :) > > >> During my initial testing of deploying a VM to oVirt everything seems to >> work except when my VM PXE boots I'm grabbing an IP outside of my >> infrastructure and it's trying to load windows so likely it's a domain PXE >> server that seems to be taking precedence. >> > > So one option would be to use premade images that can be configured when > they spin up (I think oVirt calls them "templates", just to overload that > term further :P). That would be one way to bypass the need for a TFTP > server. > > However, Foreman will expect to know the IP so it can SSH in and configure > the image once booted. You may have some issues there - but rather than > derail further now, we can discuss it if that's an interesting direction. > > > I have restarted foreman.service > > You need to restart Apache, by default. The foreman service is provided by > the packages for convenience but we use Apache/Passenger in the installer. > >> 1. How to point my PXE boot to my foreman server rather than the >> corporate PXE server? >> >> Is this on the oVirt side or Foreman Side to isolate which PXE server to >> point to? >> >> Assuming you dont want to use images, as per above, then there's three > steps: > > 1) Where does the VM get DHCP from. If it's a private network, you can > control it. If it's bridged, it's time to talk nicely to the DHCP guys :) > 2) The DHCP server hands out bootp and next-server records that direct the > VM to the TFTP server - these need to be right > 3) Or, the existing PXE server needs a class file (something for your > network segment probably) that chain loads your tftp server. > >> 2. Which logs do I need to check to fix my Provisioning Template debacle? >> > I'm not clear on which page you're hitting Submit on, but in general you > probably want to enable debug logs. Take a look at > http://projects.theforeman.org/projects/foreman/wiki/Troubleshooting for > some general hints. > > Greg >

In addition to Lukas' reply for iPXE, another option for
image/template-based booting is this - Foreman 1.13 supports using
user-data with oVirt images. In this scenario, Foreman doesn't need to SSH
to the box, so it works even if Foreman can't know the IP. In return
though, the VMs need to be able to reach Foreman (they'll make a call to
/unattended/built when they're done). Might be worth investigating :slight_smile:

Greg

··· On 27 October 2016 at 05:00, Travis Ross wrote:

Thanks for the quick response Greg - I’m traveling in India so my timezone
is a bit off.

Images - These were my initial thought of use with oVirt until I started
thinking about post deployment. I want there to be as little work
necessary on the VM’s as they will be for SQC and Engineering Developers to
quickly test features and dispose of the VM. As you mentioned Foreman
could still work with the Template - but how would it connect. I could
technically have a static IP that gets changed after build - but what
happens when 2 users try and create a VM at the same time. That’s likely
not the best solution.

Hello,

it looks like you are using oVirt which has iPXE firmware built in.
You don't need to use PXE at all! Configure your infrastructure as
described in

http://projects.theforeman.org/projects/foreman/wiki/Fetch_boot_files_via_http_instead_of_TFTP#C-Chainbooting-virtual-machines

The only change you need to do is on your DHCP server that does this:
"if iPXE is booting, then hand over iPXE template, otherwise hand over
PXELinux". The ISC DHCP bit is on the wiki page. Other DHCP servers
work similar way I think.

··· On Thu, Oct 27, 2016 at 6:00 AM, Travis Ross wrote: > Thanks for the quick response Greg - I'm traveling in India so my timezone > is a bit off. > > Images - These were my initial thought of use with oVirt until I started > thinking about post deployment. I want there to be as little work necessary > on the VM's as they will be for SQC and Engineering Developers to quickly > test features and dispose of the VM. As you mentioned Foreman could still > work with the Template - but how would it connect. I could technically have > a static IP that gets changed after build - but what happens when 2 users > try and create a VM at the same time. That's likely not the best solution. > > It sounds like I need to get with IT and work on some integration. I'm sure > they will help me out as I had our AD/Linux Team assist in getting oVirt > integrated with our Domain which required a bit more than a simple login. > Unfortunately that work may have been in vain as it looks like now my User > Frontend will be Foreman rather than oVirt for VM Creation. > > I'll have to contact my IT Network team and see what would be easiest with > the least amount of questions... my subnet is only for my lab and my servers > so likely just creating a rule on my DHCP server would be easiest to forward > to my Foreman TFTP server. Making changes to the Corporate PXE server might > be a bit more of a pain and stir up more questions than i'm interested in > dealing with. > > I wonder outside of PXE is it possible to have Foreman mount the ISO to the > system and have it boot off local media? I have the ISO's stored in an ISO > Repository in oVirt so they are available, I'm just not sure if this is > something that Foreman supports. I guess the VM needs to be on the network > for Foreman to contact it to go through kickstart so simply booting to the > media isn't sufficient. (probably answered my own question there just > thinking allowed). > > > > Regarding my Logs. I'll go through enabling debugging to see if I can spawn > an error. The "Submit" i'm referring to is when I make any changes to a > Provisioning Template, whether it be association or script editing. When > you "Submit" the changes, I am redirected to the error page and nothing > takes place. It seems like the actions I took corrupted something within > the application. It's not like I have much done on the system to if it's > too much trouble I can just start from scratch considering it'll probably > take me awhile to get the DHCP settings updated I can't do much provisioning > anyway. > > > Thanks, > -Travis > > On Wednesday, October 26, 2016 at 6:51:04 AM UTC-5, Greg Sutcliffe wrote: >> >> On 26 October 2016 at 11:08, Travis Ross wrote: >>> >>> Okay I'm new to Foreman and oVirt but have been trying to set something >>> up in my lab for our developers. >> >> >> Welcome :) >> >>> >>> During my initial testing of deploying a VM to oVirt everything seems to >>> work except when my VM PXE boots I'm grabbing an IP outside of my >>> infrastructure and it's trying to load windows so likely it's a domain PXE >>> server that seems to be taking precedence. >> >> >> So one option would be to use premade images that can be configured when >> they spin up (I think oVirt calls them "templates", just to overload that >> term further :P). That would be one way to bypass the need for a TFTP >> server. >> >> However, Foreman will expect to know the IP so it can SSH in and configure >> the image once booted. You may have some issues there - but rather than >> derail further now, we can discuss it if that's an interesting direction. >> >> > I have restarted foreman.service >> >> You need to restart Apache, by default. The foreman service is provided by >> the packages for convenience but we use Apache/Passenger in the installer. >>> >>> 1. How to point my PXE boot to my foreman server rather than the >>> corporate PXE server? >>> >>> Is this on the oVirt side or Foreman Side to isolate which PXE server to >>> point to? >> >> Assuming you dont want to use images, as per above, then there's three >> steps: >> >> 1) Where does the VM get DHCP from. If it's a private network, you can >> control it. If it's bridged, it's time to talk nicely to the DHCP guys :) >> 2) The DHCP server hands out bootp and next-server records that direct the >> VM to the TFTP server - these need to be right >> 3) Or, the existing PXE server needs a class file (something for your >> network segment probably) that chain loads your tftp server. >>> >>> 2. Which logs do I need to check to fix my Provisioning Template debacle? >> >> I'm not clear on which page you're hitting Submit on, but in general you >> probably want to enable debug logs. Take a look at >> http://projects.theforeman.org/projects/foreman/wiki/Troubleshooting for >> some general hints. >> >> Greg > > -- > You received this message because you are subscribed to the Google Groups > "Foreman users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to foreman-users+unsubscribe@googlegroups.com. > To post to this group, send email to foreman-users@googlegroups.com. > Visit this group at https://groups.google.com/group/foreman-users. > For more options, visit https://groups.google.com/d/optout.


Later,
Lukas @lzap Zapletal

Thanks guys. I'm on my way to the airport and will be traveling for the
next 30 hours so I won't get a chance to work on this till Monday.

I did look at the guide Lukas provided, but I was looking at topic A so
looks like I was working with the wrong configuration. Still looks like
I'll need the network admin assistance with some setup so I'll need to
identify the best contact for that.

Appreciate the assistance!

Travis

··· On Oct 27, 2016 10:02 PM, "Greg Sutcliffe" wrote:

On 27 October 2016 at 05:00, Travis Ross trossmobile@gmail.com wrote:

Thanks for the quick response Greg - I’m traveling in India so my
timezone is a bit off.

Images - These were my initial thought of use with oVirt until I started
thinking about post deployment. I want there to be as little work
necessary on the VM’s as they will be for SQC and Engineering Developers to
quickly test features and dispose of the VM. As you mentioned Foreman
could still work with the Template - but how would it connect. I
could technically have a static IP that gets changed after build - but what
happens when 2 users try and create a VM at the same time. That’s likely
not the best solution.

In addition to Lukas’ reply for iPXE, another option for
image/template-based booting is this - Foreman 1.13 supports using
user-data with oVirt images. In this scenario, Foreman doesn’t need to SSH
to the box, so it works even if Foreman can’t know the IP. In return
though, the VMs need to be able to reach Foreman (they’ll make a call to
/unattended/built when they’re done). Might be worth investigating :slight_smile:

Greg


You received this message because you are subscribed to a topic in the
Google Groups “Foreman users” group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/foreman-users/Q0xaeKxKQZk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at https://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.