Auto deletion of HostParameters on Foreman 1.15 causing non-consistent rebuilds

Support
1

Good day folks!

I didn't see this discussed earlier or see a bug for this just yet. After
updating to Foreman 1.15 I no longer saw deterministic builds of my hosts.
Turns out "HostParameters" for that host are disappearing when a build call
is triggered on that host. This worked on Foreman 1.14.3 and below for as
long as I could remember. After updating to Foreman 1.15.3 these values
get removed. I do rely values set in host parameters in my development
domain and moreso in production so have reverted my development domain back
to 1.14.3. I have a system in my test domain to demonstrate the behavior.

Reproduction cases follow showing that the parameters table gets entries
deleted. These are the HostParameters for the host that I set build
against.

Anyone have any insight into what I am seeing? I'll be on vacation from
tomorrow until the 22nd but intend to start digging at this when I get
back. Any insight in the meantime from the group out there would be
appreciated!

Thanks all!

Jason

Repro cases follow:

¡¡¡ ############### # GOOD SYSTEM # ###############

rpm -qa foreman tfm-rubygem-hammer_cli

tfm-rubygem-hammer_cli-0.8.0-1.el7.noarch
foreman-1.14.3-1.el7.noarch

sudo -u postgres psql foreman -c ‘select count(*) from parameters’

count

176
(1 row)

hammer host update --name devsystem.devdomain.com --build false

Host updated

sudo -u postgres psql foreman -c ‘select count(*) from parameters’

count

176
(1 row)

hammer host update --name devsystem.devdomain.com --build true

Host updated

sudo -u postgres psql foreman -c ‘select count(*) from parameters’

count

176
(1 row)

hammer host update --name devsystem.devdomain.com --build false

Host updated

sudo -u postgres psql foreman -c ‘select count(*) from parameters’

count

176
(1 row)

##############

BAD SYSTEM

##############

rpm -qa foreman tfm-rubygem-hammer_cli

foreman-1.15.3-1.el7.noarch
tfm-rubygem-hammer_cli-0.10.2-1.el7.noarch

sudo -u postgres psql foreman -c ‘select count(*) from parameters’

could not change directory to "/root"
count

28

(1 row)

hammer host update --name testserver.testdomain.com --build false

Host updated

sudo -u postgres psql foreman -c ‘select count(*) from parameters’

count

27

(1 row)

RE-ADD THROUGH FOREMAN

sudo -u postgres psql foreman -c ‘select count(*) from parameters’

count

28

(1 row)

hammer host update --name testserver.testdomain.com --build true

Host updated

sudo -u postgres psql foreman -c ‘select count(*) from parameters’

count

27

(1 row)

RE-ADD VIA CLI

hammer host set-parameter --host testserver.testdomain.com --name

test_value --value test_value
New host parameter created

sudo -u postgres psql foreman -c ‘select count(*) from parameters’

count

28

(1 row)

hammer host update --name testserver.testdomain.com --build false

Host updated

sudo -u postgres psql foreman -c ‘select count(*) from parameters’

count

27

(1 row)

2

//cc Ori for some thoughts on this

– Ivan

¡¡¡ On Wed, Sep 13, 2017 at 6:59 PM, Jason Unovitch wrote: > Good day folks! > > I didn't see this discussed earlier or see a bug for this just yet. After > updating to Foreman 1.15 I no longer saw deterministic builds of my hosts. > Turns out "HostParameters" for that host are disappearing when a build call > is triggered on that host. This worked on Foreman 1.14.3 and below for as > long as I could remember. After updating to Foreman 1.15.3 these values get > removed. I do rely values set in host parameters in my development domain > and moreso in production so have reverted my development domain back to > 1.14.3. I have a system in my test domain to demonstrate the behavior. > > Reproduction cases follow showing that the parameters table gets entries > deleted. These are the HostParameters for the host that I set build > against. > > Anyone have any insight into what I am seeing? I'll be on vacation from > tomorrow until the 22nd but intend to start digging at this when I get back. > Any insight in the meantime from the group out there would be appreciated! > > Thanks all! > > Jason > > Repro cases follow: > > ############### > # GOOD SYSTEM # > ############### > > # rpm -qa foreman tfm-rubygem-hammer_cli > tfm-rubygem-hammer_cli-0.8.0-1.el7.noarch > foreman-1.14.3-1.el7.noarch > > # sudo -u postgres psql foreman -c 'select count(*) from parameters' > count > ------- > 176 > (1 row) > > # hammer host update --name devsystem.devdomain.com --build false > Host updated > # sudo -u postgres psql foreman -c 'select count(*) from parameters' > count > ------- > 176 > (1 row) > > # hammer host update --name devsystem.devdomain.com --build true > Host updated > # sudo -u postgres psql foreman -c 'select count(*) from parameters' > count > ------- > 176 > (1 row) > > # hammer host update --name devsystem.devdomain.com --build false > Host updated > # sudo -u postgres psql foreman -c 'select count(*) from parameters' > count > ------- > 176 > (1 row) > > ############## > # BAD SYSTEM # > ############## > > # rpm -qa foreman tfm-rubygem-hammer_cli > foreman-1.15.3-1.el7.noarch > tfm-rubygem-hammer_cli-0.10.2-1.el7.noarch > > # sudo -u postgres psql foreman -c 'select count(*) from parameters' > could not change directory to "/root" > count > ------- > 28 > (1 row) > > # hammer host update --name testserver.testdomain.com --build false > Host updated > # sudo -u postgres psql foreman -c 'select count(*) from parameters' > count > ------- > 27 > (1 row) > > ## RE-ADD THROUGH FOREMAN > > # sudo -u postgres psql foreman -c 'select count(*) from parameters' > count > ------- > 28 > (1 row) > > # hammer host update --name testserver.testdomain.com --build true > Host updated > # sudo -u postgres psql foreman -c 'select count(*) from parameters' > count > ------- > 27 > (1 row) > > ## RE-ADD VIA CLI > > # hammer host set-parameter --host testserver.testdomain.com --name > test_value --value test_value > New host parameter created > > # sudo -u postgres psql foreman -c 'select count(*) from parameters' > count > ------- > 28 > (1 row) > > # hammer host update --name testserver.testdomain.com --build false > Host updated > # sudo -u postgres psql foreman -c 'select count(*) from parameters' > count > ------- > 27 > (1 row) > > -- > You received this message because you are subscribed to the Google Groups > "Foreman users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to foreman-users+unsubscribe@googlegroups.com. > To post to this group, send email to foreman-users@googlegroups.com. > Visit this group at https://groups.google.com/group/foreman-users. > For more options, visit https://groups.google.com/d/optout.
3

Hi Jason,

It would be great if you could attach some logs around the time you build
and count the parameters.
Also, are you sure which parameter is being deleted? Maybe send the results
of selecting from the parameters table instead of the count?

Thanks
Ori

¡¡¡ On Fri, Sep 15, 2017 at 6:14 PM, Ivan Necas wrote:

//cc Ori for some thoughts on this

– Ivan

On Wed, Sep 13, 2017 at 6:59 PM, Jason Unovitch > jason.unovitch@gmail.com wrote:

Good day folks!

I didn’t see this discussed earlier or see a bug for this just yet. After
updating to Foreman 1.15 I no longer saw deterministic builds of my
hosts.
Turns out “HostParameters” for that host are disappearing when a build
call
is triggered on that host. This worked on Foreman 1.14.3 and below for
as
long as I could remember. After updating to Foreman 1.15.3 these values
get
removed. I do rely values set in host parameters in my development
domain
and moreso in production so have reverted my development domain back to
1.14.3. I have a system in my test domain to demonstrate the behavior.

Reproduction cases follow showing that the parameters table gets entries
deleted. These are the HostParameters for the host that I set build
against.

Anyone have any insight into what I am seeing? I’ll be on vacation from
tomorrow until the 22nd but intend to start digging at this when I get
back.
Any insight in the meantime from the group out there would be
appreciated!

Thanks all!

Jason

Repro cases follow:

###############

GOOD SYSTEM

###############

rpm -qa foreman tfm-rubygem-hammer_cli

tfm-rubygem-hammer_cli-0.8.0-1.el7.noarch
foreman-1.14.3-1.el7.noarch

sudo -u postgres psql foreman -c ‘select count(*) from parameters’

count

176
(1 row)

hammer host update --name devsystem.devdomain.com --build false

Host updated

sudo -u postgres psql foreman -c ‘select count(*) from parameters’

count

176
(1 row)

hammer host update --name devsystem.devdomain.com --build true

Host updated

sudo -u postgres psql foreman -c ‘select count(*) from parameters’

count

176
(1 row)

hammer host update --name devsystem.devdomain.com --build false

Host updated

sudo -u postgres psql foreman -c ‘select count(*) from parameters’

count

176
(1 row)

##############

BAD SYSTEM

##############

rpm -qa foreman tfm-rubygem-hammer_cli

foreman-1.15.3-1.el7.noarch
tfm-rubygem-hammer_cli-0.10.2-1.el7.noarch

sudo -u postgres psql foreman -c ‘select count(*) from parameters’

could not change directory to "/root"
count

28

(1 row)

hammer host update --name testserver.testdomain.com --build false

Host updated

sudo -u postgres psql foreman -c ‘select count(*) from parameters’

count

27

(1 row)

RE-ADD THROUGH FOREMAN

sudo -u postgres psql foreman -c ‘select count(*) from parameters’

count

28

(1 row)

hammer host update --name testserver.testdomain.com --build true

Host updated

sudo -u postgres psql foreman -c ‘select count(*) from parameters’

count

27

(1 row)

RE-ADD VIA CLI

hammer host set-parameter --host testserver.testdomain.com --name

test_value --value test_value
New host parameter created

sudo -u postgres psql foreman -c ‘select count(*) from parameters’

count

28

(1 row)

hammer host update --name testserver.testdomain.com --build false

Host updated

sudo -u postgres psql foreman -c ‘select count(*) from parameters’

count

27

(1 row)

–
You received this message because you are subscribed to the Google Groups
"Foreman users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at https://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.

4

Hi there, I didn’t get the original reply with the prior group email system but didn’t forget about this.

The additional information is in bugzilla at https://bugzilla.redhat.com/show_bug.cgi?id=1486696

I never use the --parameters option to set parameters but do make use of hammer calls to reset builds often. So I’ve commented out the process_destroy call as mentioned in the bugzilla to mitigate the issue. I’ve barely worked with ruby before so it takes me a while to grok the code… any idea on the best place to look at a real fix for this?

5

Unfortunately this post was made while I had outgoing email disabled (for the Data restore progress work), so I’m going to quote the whole thing just to make sure our mailing-list-mode users get a copy :wink: