Unable to perform power BMC operation

Jorick_Astrego · September 18, 2014, 9:50am

Hi,

Yesterday I did some testing with ovirt/the foreman integration. I have
several hosts discovered and deployed some yesterday.

Today I cannot deploy a discovered host anymore (didn't change
anything), I get the following errors:

ForemanDiscovery: Rebooting node2.test.nu as its being discovered
and assigned
Failed to reboot: ERF12-1772 [ProxyAPI::ProxyException]: Unable to
perform power BMC operation ([Errno::ECONNREFUSED]: Connection
refused - connect(2)) for proxy http://xx.xxx.xxx.72:8443/bmc
Rolling back due to a problem: [Rebooting node2.test.nu 10000     failed

As I said, this is weird as it worked for 2 months now and I upgraded to
The Foreman 1.6 a couple of weeks ago.

I then decided to disable BMC in /etc/foreman-proxy/settings.d/bmc.yml ,
restarted the services. But this doesn't help.

Where does it get it's BMC info from and how to disable???

Kind regards,

Jorick Astrego
Netbulae B.V.

ohadlevy · September 18, 2014, 10:04am

> Hi,
>
> Yesterday I did some testing with ovirt/the foreman integration. I have
> several hosts discovered and deployed some yesterday.
>
> Today I cannot deploy a discovered host anymore (didn't change anything),
> I get the following errors:
>
> ForemanDiscovery: Rebooting node2.test.nu as its being discovered and
> assigned
> Failed to reboot: ERF12-1772 [ProxyAPI::ProxyException]: Unable to perform
> power BMC operation ([Errno::ECONNREFUSED]: Connection refused -
> connect(2)) for proxy http://xx.xxx.xxx.72:8443/bmc
> Rolling back due to a problem: [Rebooting node2.test.nu 10000
> failed
>
> this implies the proxy service is not running or fw is rejecting the
connection.

Ohad

···

On Thu, Sep 18, 2014 at 12:50 PM, Jorick Astrego wrote:

As I said, this is weird as it worked for 2 months now and I upgraded to
The Foreman 1.6 a couple of weeks ago.

I then decided to disable BMC in /etc/foreman-proxy/settings.d/bmc.yml ,
restarted the services. But this doesn’t help.

Where does it get it’s BMC info from and how to disable???

Kind regards,

Jorick Astrego
Netbulae B.V.

–
You received this message because you are subscribed to the Google Groups
“Foreman users” group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at http://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.

Jorick_Astrego · September 18, 2014, 10:23am

>
>
>
> Hi,
>
> Yesterday I did some testing with ovirt/the foreman integration. I
> have several hosts discovered and deployed some yesterday.
>
> Today I cannot deploy a discovered host anymore (didn't change
> anything), I get the following errors:
>
> ForemanDiscovery: Rebooting node2.test.nu
> <http://node2.test.nu> as its being discovered and assigned
> Failed to reboot: ERF12-1772 [ProxyAPI::ProxyException]:
> Unable to perform power BMC operation ([Errno::ECONNREFUSED]:
> Connection refused - connect(2)) for proxy
> http://xx.xxx.xxx.72:8443/bmc
> Rolling back due to a problem: [Rebooting node2.test.nu
> <http://node2.test.nu> 10000 failed
>
> this implies the proxy service is not running or fw is rejecting the
> connection.
>
> Ohad

The proxy service is running fine and I can deploy a libvirt vm using
network install without problems.

The ip "proxy http://xx.xxx.xxx.72:8443/bmc"
<http://xx.xxx.xxx.72:8443/bmc>isn't the proxy address but the ip of the
discovered host I'm trying to deploy. So it tries to use the dhcp ip of
the host itself as proxy???

Jorick

<http://xx.xxx.xxx.72:8443/bmc>

···

On 09/18/2014 12:04 PM, Ohad Levy wrote: > On Thu, Sep 18, 2014 at 12:50 PM, Jorick Astrego > <j.astrego@netbulae.eu > wrote: > > > As I said, this is weird as it worked for 2 months now and I > upgraded to The Foreman 1.6 a couple of weeks ago. > > I then decided to disable BMC in > /etc/foreman-proxy/settings.d/bmc.yml , restarted the services. > But this doesn't help. > > Where does it get it's BMC info from and how to disable??? > > Kind regards, > > Jorick Astrego > Netbulae B.V. >

ohadlevy · September 18, 2014, 10:34am

>
>
>
>
>
>> Hi,
>>
>> Yesterday I did some testing with ovirt/the foreman integration. I have
>> several hosts discovered and deployed some yesterday.
>>
>> Today I cannot deploy a discovered host anymore (didn't change anything),
>> I get the following errors:
>>
>> ForemanDiscovery: Rebooting node2.test.nu as its being discovered and
>> assigned
>> Failed to reboot: ERF12-1772 [ProxyAPI::ProxyException]: Unable to
>> perform power BMC operation ([Errno::ECONNREFUSED]: Connection refused -
>> connect(2)) for proxy http://xx.xxx.xxx.72:8443/bmc
>> Rolling back due to a problem: [Rebooting node2.test.nu 10000
>> failed
>>
>> this implies the proxy service is not running or fw is rejecting the
> connection.
>
> Ohad
>
>
> The proxy service is running fine and I can deploy a libvirt vm using
> network install without problems.
>
> The ip "proxy http://xx.xxx.xxx.72:8443/bmc"
> <http://xx.xxx.xxx.72:8443/bmc>isn't the proxy address but the ip of the
> discovered host I'm trying to deploy. So it tries to use the dhcp ip of the
> host itself as proxy???
>

I see, The proxy code is actually running inside the discovered host (it
saved us from rewriting an agent) with minimal functionality (such as power
management and fact reporting).

in order to trigger a reboot, the proxy needs to be reachable on the
discovered host from foreman (until we use the smart proxy as a real proxy
for this purpose). is that the case?

Ohad

···

On Thu, Sep 18, 2014 at 1:23 PM, Jorick Astrego wrote: > On 09/18/2014 12:04 PM, Ohad Levy wrote: > On Thu, Sep 18, 2014 at 12:50 PM, Jorick Astrego > wrote:

Jorick

http://xx.xxx.xxx.72:8443/bmc

As I said, this is weird as it worked for 2 months now and I upgraded to
The Foreman 1.6 a couple of weeks ago.

I then decided to disable BMC in /etc/foreman-proxy/settings.d/bmc.yml ,
restarted the services. But this doesn’t help.

Where does it get it’s BMC info from and how to disable???

Kind regards,

Jorick Astrego
Netbulae B.V.

–
You received this message because you are subscribed to the Google Groups
“Foreman users” group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at http://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.

Jorick_Astrego · September 18, 2014, 10:49am

Ah I see, the host is discovered and it ip address is pingable. The
http://xx.xxx.xxx.72 <http://xx.xxx.xxx.72:8443/bmc>:8443/bmc url isn't
working though. I still use the same discovery image as I used for the
last 2 months.

Jorick

···

On 09/18/2014 12:34 PM, Ohad Levy wrote: > > > On Thu, Sep 18, 2014 at 1:23 PM, Jorick Astrego > wrote: > > > On 09/18/2014 12:04 PM, Ohad Levy wrote: >> >> >> On Thu, Sep 18, 2014 at 12:50 PM, Jorick Astrego >> <j.astrego@netbulae.eu > wrote: >> >> Hi, >> >> Yesterday I did some testing with ovirt/the foreman >> integration. I have several hosts discovered and deployed >> some yesterday. >> >> Today I cannot deploy a discovered host anymore (didn't >> change anything), I get the following errors: >> >> ForemanDiscovery: Rebooting node2.test.nu >> as its being discovered and assigned >> Failed to reboot: ERF12-1772 [ProxyAPI::ProxyException]: >> Unable to perform power BMC operation >> ([Errno::ECONNREFUSED]: Connection refused - connect(2)) >> for proxy http://xx.xxx.xxx.72:8443/bmc >> Rolling back due to a problem: [Rebooting node2.test.nu >> 10000 failed >> >> this implies the proxy service is not running or fw is rejecting >> the connection. >> >> Ohad > > The proxy service is running fine and I can deploy a libvirt vm > using network install without problems. > > The ip "proxy http://xx.xxx.xxx.72:8443/bmc" > isn't the proxy address but the ip > of the discovered host I'm trying to deploy. So it tries to use > the dhcp ip of the host itself as proxy??? > > > I see, The proxy code is actually running inside the discovered host > (it saved us from rewriting an agent) with minimal functionality (such > as power management and fact reporting). > > in order to trigger a reboot, the proxy needs to be reachable on the > discovered host from foreman (until we use the smart proxy as a real > proxy for this purpose). is that the case? > > Ohad > >

Jorick_Astrego · September 19, 2014, 11:16am

Found it! And it's a nice one.

I was looking at the facts after discovery and noticed eth0 had no ip
and there was an ovirtmgmt interface:

interfaces eth0,eth1,eth2,eth3,eth4,eth5,lo,ovirtmgmt
ipaddress 10.xxx.xxx.88
ipaddress_eth1 10.xxx.xxx.88
ipaddress_eth2 10.xxx.xxx.46
ipaddress_eth3 10.xxx.xxx.47
ipaddress_lo 127.0.0.1

So my thinking was that as there was ovirt-node installed on the host
and the discovery image also uses ovirt-node. What if the discovery
image reads the network config from disk somehow. I reinitialized the
disk and rebooted in discovery mode.

Tada, no more ovirtmgmt interface:

interfaces eth0,eth1,eth2,eth3,eth4,eth5,lo
ipaddress 10.xxx.xxx.85
ipaddress_eth0 10.xxx.xxx.85
ipaddress_eth1 10.xxx.xxx.89
ipaddress_eth2 10.xxx.xxx.46
ipaddress_eth3 10.xxx.xxx.47

The discovery image shouldn't read anything from the disk I think. I'll
file a bug for it.

Kind regards,

Jorick Astrego
Netbulae B.V.

···

On 09/18/2014 12:49 PM, Jorick Astrego wrote: > > On 09/18/2014 12:34 PM, Ohad Levy wrote: >> >> >> On Thu, Sep 18, 2014 at 1:23 PM, Jorick Astrego >> <j.astrego@netbulae.eu > wrote: >> >> >> On 09/18/2014 12:04 PM, Ohad Levy wrote: >>> >>> >>> On Thu, Sep 18, 2014 at 12:50 PM, Jorick Astrego >>> <j.astrego@netbulae.eu > wrote: >>> >>> Hi, >>> >>> Yesterday I did some testing with ovirt/the foreman >>> integration. I have several hosts discovered and deployed >>> some yesterday. >>> >>> Today I cannot deploy a discovered host anymore (didn't >>> change anything), I get the following errors: >>> >>> ForemanDiscovery: Rebooting node2.test.nu >>> as its being discovered and assigned >>> Failed to reboot: ERF12-1772 [ProxyAPI::ProxyException]: >>> Unable to perform power BMC operation >>> ([Errno::ECONNREFUSED]: Connection refused - connect(2)) >>> for proxy http://xx.xxx.xxx.72:8443/bmc >>> Rolling back due to a problem: [Rebooting node2.test.nu >>> 10000 failed >>> >>> this implies the proxy service is not running or fw is rejecting >>> the connection. >>> >>> Ohad >> >> The proxy service is running fine and I can deploy a libvirt vm >> using network install without problems. >> >> The ip "proxy http://xx.xxx.xxx.72:8443/bmc" >> isn't the proxy address but the ip >> of the discovered host I'm trying to deploy. So it tries to use >> the dhcp ip of the host itself as proxy??? >> >> >> I see, The proxy code is actually running inside the discovered host >> (it saved us from rewriting an agent) with minimal functionality >> (such as power management and fact reporting). >> >> in order to trigger a reboot, the proxy needs to be reachable on the >> discovered host from foreman (until we use the smart proxy as a real >> proxy for this purpose). is that the case? >> >> Ohad >> >> > Ah I see, the host is discovered and it ip address is pingable. The > http://xx.xxx.xxx.72 :8443/bmc url > isn't working though. I still use the same discovery image as I used > for the last 2 months. > > Jorick >