Discovery Image 3.0.4 updated

Hello,

we have updated the discovery image to version 3.0.4. Few rather
cosmetic bugs were fixed. This is the very first version signed by our
Foreman Discovery GPG key, which will be used to sign ISO/tar balls.

To verify individual ISO/tar files, import our Discovery Plugin key 7E81E7B0
and verify individual files:

wget 'http://keys.fedoraproject.org:11371/pks/lookup?search=0x7E81E7B0&op=get' -O- | gpg --import

gpg --verify --multifile *.asc

gpg: Signature made Fri 06 Nov 2015 09:42:55 AM UTC using RSA key ID 7E81E7B0
gpg: Good signature from "Foreman Discovery <foreman-xxx@googlegroups.com>"

Changelog:

Fixes #12340 - resend button shows countdown now (23 hours ago)
Fixes #12323 - def number of register calls is zero (2 days ago)
Fixes #12275 - fixed error messages and NIC sort (2 days ago)
Fixes #12322 - disabled resend button for PXE-less (4 days ago)

Download here:

http://downloads.theforeman.org/discovery/releases/3.0/

(This image needs newest Foreman Discovery plugin, which is currently only
available for 1.10 RC version. We plan to release it also for 1.9 soon.)

··· -- Later, Lukas #lzap Zapletal

Lukas, thanks!

One question - what about releasing a version of smart-proxy running on
this image as a package (both rpm and deb)? I see that image runs:

GET /version (on machine running FDI 3.0.4)

{version: "1.11.0-develop"}

while the latest smart-proxy I can find as a package is 1.10-rc2:

GET /version (on machine running our internal discovery OS with Foreman
smart-proxy installed):

{

version: "1.10.0-RC2"

}

I'm testing Foreman 1.10-rc2 with Discovery plugin from the same 1.10 repo (tfm-rubygem-foreman_discovery-4.1.2-1.fm1_10.el7.noarch)
on CentOS7. The problem I'm having with 1.10-rc2 smart-proxy is that even
though I register/discover the hosts just fine, I can't refresh facts from
them and getting this error:

Oops, we're sorry but something went wrong

Warning!

Could not get facts from proxy https://<IP>:8443: 404 Resource Not Found:
<!DOCTYPE html> <html> <head> <style type="text/css"> body {
text-align:center;font-family:helvetica,arial;font-size:22px;
color:#888;margin:20px} #c {margin:0 auto;width:500px;text-align:left}
</style> </head> <body> <h2>Sinatra doesn&rsquo;t know this ditty.</h2>
<img src='https://<IP>:8443/sinatra/404.png'> <div id="c"> Try this:
<pre># in usr/share/foreman-proxy/modules/root/root_api.rb class
Proxy::RootApi get '/inventory/facter' do "Hello World" end end </pre>
</div> </body> </html>

I tried FDI 3.0.4 just now to see if this works and it does, so I checked
the versions and see the discrepancy hence my question.

Please let me know.

Thanks!

Hey,

please avoid double-posting next time :slight_smile:

> GET /version (on machine running FDI 3.0.4)
>
> {version: "1.11.0-develop"}

Interesting, this build was supposed to be build against 1.10:

http://ci.theforeman.org/view/Packaging/job/packaging_discovery_image/61/parameters/

And it looks like a bug in our build script which I think ignores what
we provide in the CI parameters:

http://ci.theforeman.org/view/Packaging/job/packaging_discovery_image/61/consoleFull

Will take a look on that. Thanks for report!

> I tried FDI 3.0.4 just now to see if this works and it does, so I checked
> the versions and see the discrepancy hence my question.

You can't use 2.X image with Plugin 4.1. That won't work (registering
will tho).

··· -- Later, Lukas #lzap Zapletal

> And it looks like a bug in our build script which I think ignores what
> we provide in the CI parameters:

I've released version 3.0.5 which is build correctly against 1.10 proxy.
Also it contains additional fixes:

Fixes #12420 - primary detected correctly for PXE-less
Fixes #12430 - unattended mode sends custom facts now

http://downloads.theforeman.org/discovery/releases/3.0/

LZ

··· -- Later, Lukas #lzap Zapletal

Thanks, Lukas! I'll try 3.0.5 as well, but it does not solve my problem I
described above.

I don't use older Foreman image(s) at this point (eventually I'm hoping to
migrate to it though, however, it is not a fast thing to pull off as we
have lots of other dependencies on the image we run for discovery), I was
always installing a smart-proxy on our internal netbooted discovery image
(ubuntu14.04-based).

But installing a 1.10-rc2 smart-proxy does not work now it seems for facts
refresh. Is that by design and the only supported discovery image now is
FDI? If so, I was wondering what made you to make that decision?
If I'm missing something though (I'm hoping I do), please let me know what
you think it may be. I'll provide more details about installation if
needed, just let me know what details you need.

Thanks!

BTW, there seems to be a new issue with 3.0.5 - discovery_bootif is not
detected properly any more:

[root@fdi ~]# FACTERLIB=/usr/share/fdi/facts:/opt/extension/facts facter |
egrep "mac|ipmi|disc"
discovery_bootif => 00:00:00:00:00:00
discovery_bootip => <correct-ip>
discovery_release => 20151113.1
discovery_version => 3.0.5
ipmi_1_gateway => <gateway>
ipmi_1_ipaddress => <ip>
ipmi_1_ipaddress_source => DHCP Address
ipmi_1_macaddress => 2c:60:0c:bc:8e:d9
ipmi_1_subnet_mask => 255.255.252.0
ipmi_8_gateway => 0.0.0.0
ipmi_8_ipaddress => 0.0.0.0
ipmi_8_ipaddress_source => Unspecified
ipmi_8_macaddress => 00:00:00:00:00:00
ipmi_8_subnet_mask => 0.0.0.0
ipmi_enabled => true
ipmi_gateway => <gateway>
ipmi_ipaddress => <ip>
ipmi_ipaddress_source => DHCP Address
ipmi_macaddress => 2c:60:0c:bc:8e:d9
ipmi_subnet_mask => 255.255.252.0
macaddress => 2c:60:0c:95:8d:dd
macaddress_ens20f0 => 2c:60:0c:95:8d:dd
macaddress_ens20f1 => 2c:60:0c:95:8d:de
[root@fdi ~]#

That results in an error on Foreman side during fact refresh:

Could not get facts from proxy https://<ip>:8443: ERF42-0481
[Foreman::Exception]: Unable to detect primary interface using MAC
'00:00:00:00:00:00' specified by discovery_fact 'discovery_bootif'

This is the same machine on which 3.0.4 worked just fine.

Any advice/suggestions on packages to install smart-proxy w/ discovery on
other than FDI images, Lukas?

Hello,

> I don't use older Foreman image(s) at this point (eventually I'm hoping to
> migrate to it though, however, it is not a fast thing to pull off as we
> have lots of other dependencies on the image we run for discovery), I was
> always installing a smart-proxy on our internal netbooted discovery image
> (ubuntu14.04-based).
>
> But installing a 1.10-rc2 smart-proxy does not work now it seems for facts
> refresh. Is that by design and the only supported discovery image now is
> FDI? If so, I was wondering what made you to make that decision?
> If I'm missing something though (I'm hoping I do), please let me know what
> you think it may be. I'll provide more details about installation if
> needed, just let me know what details you need.

we have changed the FDI API, there is a new smart-proxy component to run
on the image:

Refresh Facts call now uses this new API:

Its /inventory/facter path. So change your image according to this.

Also not that port has changed, FDI now listens on:

8448 - HTTP
8443 - HTTPS

LZ

··· -- Later, Lukas #lzap Zapletal

> always installing a smart-proxy on our internal netbooted discovery image
> (ubuntu14.04-based).

Are you able to publish your work as open source? Others might be
interested in alternative DI…

··· -- Later, Lukas #lzap Zapletal

> Any advice/suggestions on packages to install smart-proxy w/ discovery on
> other than FDI images, Lukas?
>

If there's some other way to bring that capability into other images
(GEM-based or whatever), I'd like to give a try too.

Thanks!

> Are you able to publish your work as open source? Others might be
> interested in alternative DI…
>
>
This is an open-source-based image. For those who's really interested, here
are a couple of links to start from:
https://wiki.ubuntu.com/LiveCDNetboot
https://help.ubuntu.com/community/LiveCDCustomization

There's a huge drawback though - the squashed FS is mounted over NFS, which
may not suitable for all environments. We have been OK with that for a
while now, however, it is time for us to get rid of that for different
reasons and that's why I'm looking at FDI as a possible candidate - I need
HTTP and I need to be able to customize the image during the boot (adding
some scripts from a git repo and execute them, etc.).

> discovery_bootif => 00:00:00:00:00:00

I am unable to reproduce this, show me your kernel command line. Use
discovery-debug and pastebin the output please.

··· -- Later, Lukas #lzap Zapletal

>
> we have changed the FDI API, there is a new smart-proxy component to run
> on the image:
>
> https://github.com/theforeman/smart_proxy_discovery_image
>
> Refresh Facts call now uses this new API:
>
>
> https://github.com/theforeman/smart_proxy_discovery_image/blob/master/lib/smart_proxy_discovery_image/inventory_api.rb
>
> Its /inventory/facter path. So change your image according to this.
>
> Also not that port has changed, FDI now listens on:
>
> 8448 - HTTP
> 8443 - HTTPS
>
>
Yes, I understand that and here's what I tried to do so far:

  • install foreman-proxy 1.10-rc3 (as DEB package)
  • install smart_proxy_discovery_image-1.0.5.gem (since there's no DEB
    package for this component)
  • settings.yml is configured with the ports you specified above

Yet I still get this error:

Oops, we're sorry but something went wrong

Warning!

Could not get facts from proxy https://<IP>:8443: 404 Resource Not Found:
<!DOCTYPE html> <html> <head> <style type="text/css"> body {
text-align:center;font-family:helvetica,arial;font-size:22px;
color:#888;margin:20px} #c {margin:0 auto;width:500px;text-align:left}
</style> </head> <body> <h2>Sinatra doesn&rsquo;t know this ditty.</h2>
<img src='https://<IP>:8443/sinatra/404.png'> <div id="c"> Try this:
<pre># in usr/share/foreman-proxy/modules/root/root_api.rb class
Proxy::RootApi get '/inventory/facter' do "Hello World" end end </pre>
</div> </body> </html>

I've compared /usr/share/foreman-proxy/modules/root/root_api.rb on a
machine running FDI 3.0.5 to the one I run and they are exactly identical,
so I'm missing something else. If smart_proxy_discovery_image was a DEB
package, maybe that dependency would have been satisfied.

Suggestions?

> I am unable to reproduce this, show me your kernel command line. Use
> discovery-debug and pastebin the output please.
>
>
Here's the output from a same machine running 3.0.4 (bootif mac is proper)
and 3.0.5 (bootif is set to 0 by rescue):

[root@fdi ~]# facter | grep discovery
discovery_bootif => 2c:60:0c:95:8d:de
discovery_bootip => <IP>
discovery_release => 20151106.1
discovery_version => 3.0.4

[root@fdi ~]# cat /proc/cmdline
rootflags=loop root=live:/fdi.iso rootfstype=auto ro rd.live.image
acpi=force rd.luks=0 rd.md=0 rd.dm=0 rd.lvm=0 rd.bootif=0 rd.neednet=0
rd.debug=1 nomodeset proxy.url=https://<foreman> proxy.type=foreman
fdi.ssh=1 fdi.rootpw=<pw> fdi.initnet=all fdi.cachefacts=5
fdi.uploadsleep=30 console=ttyS1,115200n8 BOOTIF=01-2c-60-0c-95-8d-de
net.ifnames=0
[root@fdi ~]#

[root@fdi ~]# facter | grep discovery
discovery_bootif => 00:00:00:00:00:00
discovery_bootip => <IP>
discovery_release => 20151113.1
discovery_version => 3.0.5

[root@fdi ~]# cat /proc/cmdline
rootflags=loop root=live:/fdi.iso rootfstype=auto ro rd.live.image
acpi=force rd.luks=0 rd.md=0 rd.dm=0 rd.lvm=0 rd.bootif=0 rd.neednet=0
rd.debug=1 nomodeset proxy.url=https://<foreman> proxy.type=foreman
fdi.ssh=1 fdi.rootpw=<pw> fdi.initnet=all fdi.cachefacts=5
fdi.uploadsleep=30 console=ttyS1,115200n8 BOOTIF=01-2c-60-0c-95-8d-de
net.ifnames=0
[root@fdi ~]#

Maybe because of missing method in /usr/share/fdi/facts/discovery-facts.rb,
no?

def discovery_bootif

PXELinux dash-separated hexadecimal without the leading hardware type

cmdline('BOOTIF', cmdline('fdi.pxmac', detect_first_nic_with_link)).gsub(/^[a-fA-F0-9]±/,
'').gsub('-', ':') rescue '00:00:00:00:00:00'
end

Anyway, changing above cmdline into the one below, makes things work again,
but not sure if that's what you wanted:

cmdline('BOOTIF', cmdline('fdi.pxmac',
Facter.fact("macaddress").value)).gsub(/^[a-fA-F0-9]±/, '').gsub('-', ':')
rescue '00:00:00:00:00:00'

behavior as it sets "macaddress", "ipaddress", etc. of the system to the
first IP'ed NIC it finds in a list of NICs on multi-homed systems, which is
not always a proper NIC. Personally, in situations like this, I write a
little custom fact to key off the default gateway -> device -> mac to
detect and report a proper main NIC. So, if your detect_first_nic_with_link
method was supposed to do what its name suggests, I'd do exactly that
traversing to detect a NIC the system really uses as its default device,
not just pick a first linked-up NIC.

··· From my personal experience though, I wouldn't trust default facter's

> rootflags=loop root=live:/fdi.iso rootfstype=auto ro rd.live.image
> acpi=force rd.luks=0 rd.md=0 rd.dm=0 rd.lvm=0 rd.bootif=0 rd.neednet=0
> rd.debug=1 nomodeset proxy.url=https://<foreman> proxy.type=foreman
> fdi.ssh=1 fdi.rootpw=<pw> fdi.initnet=all fdi.cachefacts=5
> fdi.uploadsleep=30 console=ttyS1,115200n8 BOOTIF=01-2c-60-0c-95-8d-de
> net.ifnames=0
>

One more question, actually - is there an option to pass a root's password
in encrypted form instead of clear-text?

Thanks!

>
>
> we have changed the FDI API, there is a new smart-proxy component to
> run
> on the image:
>
> https://github.com/theforeman/smart_proxy_discovery_image
> <https://github.com/theforeman/smart_proxy_discovery_image>
>
> Refresh Facts call now uses this new API:
>
> https://github.com/theforeman/smart_proxy_discovery_image/blob/master/lib/smart_proxy_discovery_image/inventory_api.rb
> <https://github.com/theforeman/smart_proxy_discovery_image/blob/master/lib/smart_proxy_discovery_image/inventory_api.rb>
>
>
> Its /inventory/facter path. So change your image according to this.
>
> Also not that port has changed, FDI now listens on:
>
> 8448 - HTTP
> 8443 - HTTPS
>
>
> Yes, I understand that and here's what I tried to do so far:
> - install foreman-proxy 1.10-rc3 (as DEB package)
> - install smart_proxy_discovery_image-1.0.5.gem (since there's no DEB
> package for this component)
> - settings.yml is configured with the ports you specified above

You also need a settings.d file for the plugin. It's supplied in the
gem but it needs copying to the right location:

https://github.com/theforeman/smart_proxy_discovery_image/blob/master/settings.d/discovery_image.yml.example

and change enabled to true.

> I've compared /usr/share/foreman-proxy/modules/root/root_api.rb on a
> machine running FDI 3.0.5 to the one I run and they are exactly
> identical, so I'm missing something else. If smart_proxy_discovery_image
> was a DEB package, maybe that dependency would have been satisfied.

If you have a moment to open a PR to our packaging repo, we could
publish a deb package for it too.

https://github.com/theforeman/foreman-packaging/tree/deb/develop/plugins
is where they're kept, copying and renaming the smart_proxy_discovery
dir would do.

··· On 23/11/15 21:53, 'Konstantin Orekhov' via Foreman users wrote:


Dominic Cleal
dominic@cleal.org

> Maybe because of missing method in /usr/share/fdi/facts/discovery-facts.rb,
> no?

You're right, not sure why I was not able to repro this. Perhaps wrong
branch.

http://projects.theforeman.org/issues/12610

We need to fix this.

··· -- Later, Lukas #lzap Zapletal

> One more question, actually - is there an option to pass a root's password
> in encrypted form instead of clear-text?

No.

··· -- Later, Lukas #lzap Zapletal

>
> You also need a settings.d file for the plugin. It's supplied in the
> gem but it needs copying to the right location:
>
>
> https://github.com/theforeman/smart_proxy_discovery_image/blob/master/settings.d/discovery_image.yml.example
>
> and change enabled to true.
>
>
Yes, I had that, just forgot to mention it.

What I found missing is
/usr/share/foreman-proxy/bundler.d/discovery_image.rb which is not
installed by the smart_proxy_discovery_image-1.0.5.gem. Doing this after
gem installation fixed things for me:

echo "gem 'smart_proxy_discovery_image'" >
/usr/share/foreman-proxy/bundler.d/discovery_image.rb

>
> If you have a moment to open a PR to our packaging repo, we could
> publish a deb package for it too.
>
> https://github.com/theforeman/foreman-packaging/tree/deb/develop/plugins
> <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Ftheforeman%2Fforeman-packaging%2Ftree%2Fdeb%2Fdevelop%2Fplugins&sa=D&sntz=1&usg=AFQjCNH28oDUuHKIEMmRwWSk5vUoTo3_Ug>
> is where they're kept, copying and renaming the smart_proxy_discovery
> dir would do.
>
>
Here it is - https://github.com/theforeman/foreman-packaging/pull/903
Not being familiar with your packaging process, I tried to mimic things as
much as I could but I'm pretty sure something was missed/messed up, so
please double-check that - I don't want anything to be broken :slight_smile:

> > One more question, actually - is there an option to pass a root's
> password
> > in encrypted form instead of clear-text?
>
> No.
>
>
https://github.com/theforeman/foreman-discovery-image/pull/57