Discovery Image foreman-proxy.service entered failed state

I just downloaded fdi v 3.1.2 and I'm hitting a snag when trying to reboot
the new system at provisioning time. The first symptom I noticed was that
the new system seemed to be taking a long time to register with my foreman
server which is on a small VLAN (currently only 5 hosts). The foreman
server is receiving and responding to DHCP requests quickly and via both
ssh and the console I have verified that the new client has the correct
network information.
At first I thought I was hitting the same issue as discussed in this
thread: https://groups.google.com/forum/#!topic/foreman-users/QYhdjL7ay3Q but
my fdi is not getting the error from the generate-proxy-cert process.
After some fiddling I tried restarting foreman-proxy manually (systemctl
start foreman-proxy.service) and it would just hang.
Next I tried running /usr/share/foreman-proxy/bin/smart-proxy manually
(ruby --debug /usr/share/foreman-proxy/bin/smart-proxy) and that provided
some interesting results. Please note I have to retype the errors below as
I can't transfer them directly to the internet:

Exception LoadError' at /usr/share/rubygems/rubygems.rb:1096 - cannot load such file -- rubygems/defaults/ruby ExceptionLoadError' at
/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55 - cannot load
such file --abrt
Exception `LoadError' at
/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:141 - cannot load
such file --abrt
/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55: warning:
loading in progress, circular require considered harmful -
/usr/share/foreman-proxy/lib/proxy/log_buffer/decorator.rb

That's the beginning of the errors…if you need more just let me know but,
like I said, I'd have to type it in so I'm trying to avoid that if possible.

Any ideas?

Thanks!

>
> Next I tried running /usr/share/foreman-proxy/bin/smart-proxy manually
> (ruby --debug /usr/share/foreman-proxy/bin/smart-proxy) and that provided
> some interesting results. Please note I have to retype the errors below as
> I can't transfer them directly to the internet:
>

As a side not:
You can enable sshd on the fdi
run discovery-debug > somefile
and then replace the info you don't wanna leak (using sed magic or
something)

So I've done some more digging and I think I've found the main symptom.
When foreman-proxy.service is started it takes, on average, 36 seconds to
start. Systemd steps in at 30 seconds, based on DefaultTimeoutStartSec from
system.conf and kills the start process. I'm not sure where the start
process is hanging up.
I've eliminated the hardware as an issue because I'm seeing the same
behavior on a virtual machine and bare metal.

Let me know where to look next and/or what info I can provide to help.

Thanks!

··· On Thursday, May 26, 2016 at 8:54:38 AM UTC-4, AML wrote:

I just downloaded fdi v 3.1.2 and I’m hitting a snag when trying to reboot
the new system at provisioning time. The first symptom I noticed was that
the new system seemed to be taking a long time to register with my foreman
server which is on a small VLAN (currently only 5 hosts). The foreman
server is receiving and responding to DHCP requests quickly and via both
ssh and the console I have verified that the new client has the correct
network information.
At first I thought I was hitting the same issue as discussed in this
thread: https://groups.google.com/forum/#!topic/foreman-users/QYhdjL7ay3Q but
my fdi is not getting the error from the generate-proxy-cert process.
After some fiddling I tried restarting foreman-proxy manually (systemctl
start foreman-proxy.service) and it would just hang.
Next I tried running /usr/share/foreman-proxy/bin/smart-proxy manually
(ruby --debug /usr/share/foreman-proxy/bin/smart-proxy) and that provided
some interesting results. Please note I have to retype the errors below as
I can’t transfer them directly to the internet:

Exception LoadError' at /usr/share/rubygems/rubygems.rb:1096 - cannot load such file -- rubygems/defaults/ruby ExceptionLoadError’ at
/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55 - cannot load
such file --abrt
Exception `LoadError’ at
/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:141 - cannot load
such file --abrt
/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55: warning:
loading in progress, circular require considered harmful -
/usr/share/foreman-proxy/lib/proxy/log_buffer/decorator.rb

That’s the beginning of the errors…if you need more just let me know
but, like I said, I’d have to type it in so I’m trying to avoid that if
possible.

Any ideas?

Thanks!

Is anyone else experiencing the foreman-proxy service taking longer than 30
seconds to start in the discovery image?

Thanks!

··· On Thursday, May 26, 2016 at 12:19:54 PM UTC-4, AML wrote:

So I’ve done some more digging and I think I’ve found the main symptom.
When foreman-proxy.service is started it takes, on average, 36 seconds to
start. Systemd steps in at 30 seconds, based on DefaultTimeoutStartSec from
system.conf and kills the start process. I’m not sure where the start
process is hanging up.
I’ve eliminated the hardware as an issue because I’m seeing the same
behavior on a virtual machine and bare metal.

Let me know where to look next and/or what info I can provide to help.

Thanks!

On Thursday, May 26, 2016 at 8:54:38 AM UTC-4, AML wrote:

I just downloaded fdi v 3.1.2 and I’m hitting a snag when trying to
reboot the new system at provisioning time. The first symptom I noticed was
that the new system seemed to be taking a long time to register with my
foreman server which is on a small VLAN (currently only 5 hosts). The
foreman server is receiving and responding to DHCP requests quickly and via
both ssh and the console I have verified that the new client has the
correct network information.
At first I thought I was hitting the same issue as discussed in this
thread: https://groups.google.com/forum/#!topic/foreman-users/QYhdjL7ay3Q but
my fdi is not getting the error from the generate-proxy-cert process.
After some fiddling I tried restarting foreman-proxy manually (systemctl
start foreman-proxy.service) and it would just hang.
Next I tried running /usr/share/foreman-proxy/bin/smart-proxy manually
(ruby --debug /usr/share/foreman-proxy/bin/smart-proxy) and that provided
some interesting results. Please note I have to retype the errors below as
I can’t transfer them directly to the internet:

Exception LoadError' at /usr/share/rubygems/rubygems.rb:1096 - cannot load such file -- rubygems/defaults/ruby ExceptionLoadError’ at
/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55 - cannot load
such file --abrt
Exception `LoadError’ at
/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:141 - cannot load
such file --abrt
/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55: warning:
loading in progress, circular require considered harmful -
/usr/share/foreman-proxy/lib/proxy/log_buffer/decorator.rb

That’s the beginning of the errors…if you need more just let me know
but, like I said, I’d have to type it in so I’m trying to avoid that if
possible.

Any ideas?

Thanks!