Duplicated IP addresses that can't be removed following upgrade

Support
1

Following an upgrade of our Foreman environment from 1.7.1->1.8.x->1.9.3,
I'm seeing network interfaces with duplicate IP's in my host records.

It seems to be related to the following issue:

I'm unable to delete the duplicate IP's in the web interface, because the
validation checks for a duplicate on the deleted NIC interface. In an
attempt to fix this, I introduced the changes from here:

I created patches for the three modified files (excluding test files) as
follows:

cd /usr/share/foreman
git show 35241dd65a6b4f426374a03e83d6f796f59a9d35 app/models/host/managed.rb
> patch1.txt
git show 35241dd65a6b4f426374a03e83d6f796f59a9d35 app/models/nic/base.rb >
patch2.txt
git show 35241dd65a6b4f426374a03e83d6f796f59a9d35 app/models/nic/interface.rb
> patch3.txt

and then applied them to our 1.9.3 files like so:

cd /usr/share/foreman
patch -p1 < patch1.txt
patch -p1 < patch2.txt
patch -p1 < patch3.txt

This worked in our staging environment. I'm able to remove the duplicate
IP's through the Web UI.

However, on two separate occasions, I've moved the three files over to
production and it's failed. It doesn't break anything per se, but it just
doesn't seem to "work". When I try to go into a host record and delete the
duplicate IP, I still get a validation error. I went so far as to
completely reboot both Foreman application servers (we run two in a load
balanced pair) to ensure there were no apache/passenger connections left
open.

Our stage and production environments are both running 1.9.3 and are
mostly the same, including running a copy of the production database.
The staging environment is not running hooks present in production, and has
some slightly different configuration to reflect it being on a different
host, but that's about it.

Is there anything else that could be causing this to work in our stage
environment but not production?

Has anyone had to deal with this duplicate IP issue following an upgrade
and come up with a reasonable fix? Ideally, I'd like to clean up all
duplicates in our database, but I'm a little nervous about trying to do so
in our production environment.

I've attached the three patched files, if it's helpful.

20151214.patch-base.rb (7.87 KB)

2

Hello

once the DB gets into inconsistent state validations do not work properly. You
can try using direct interface API to delete the invalid interface, this
should skip the validation unique IP validation since it happens outside of
host context.

See API documentation [1] for more details.

[1] http://theforeman.org/api/1.9/apidoc/v2/interfaces.html

Hope this helps

··· -- Marek

On Wednesday 16 of December 2015 16:56:46 Kyle Flavin wrote:

Following an upgrade of our Foreman environment from 1.7.1->1.8.x->1.9.3,
I’m seeing network interfaces with duplicate IP’s in my host records.

It seems to be related to the following issue:
http://projects.theforeman.org/issues/12486

I’m unable to delete the duplicate IP’s in the web interface, because the
validation checks for a duplicate on the deleted NIC interface. In an
attempt to fix this, I introduced the changes from here:
Fixes #11034 - custom uniqueness validations for interface attributes · theforeman/foreman@35241dd · GitHub
6f59a9d35

I created patches for the three modified files (excluding test files) as
follows:

cd /usr/share/foreman
git show 35241dd65a6b4f426374a03e83d6f796f59a9d35 app/models/host/managed.rb

patch1.txt

git show 35241dd65a6b4f426374a03e83d6f796f59a9d35 app/models/nic/base.rb >
patch2.txt
git show 35241dd65a6b4f426374a03e83d6f796f59a9d35
app/models/nic/interface.rb

patch3.txt

and then applied them to our 1.9.3 files like so:

cd /usr/share/foreman
patch -p1 < patch1.txt
patch -p1 < patch2.txt
patch -p1 < patch3.txt

This worked in our staging environment. I’m able to remove the duplicate
IP’s through the Web UI.

However, on two separate occasions, I’ve moved the three files over to
production and it’s failed. It doesn’t break anything per se, but it just
doesn’t seem to “work”. When I try to go into a host record and delete the
duplicate IP, I still get a validation error. I went so far as to
completely reboot both Foreman application servers (we run two in a load
balanced pair) to ensure there were no apache/passenger connections left
open.

Our stage and production environments are both running 1.9.3 and are
mostly the same, including running a copy of the production database.
The staging environment is not running hooks present in production, and has
some slightly different configuration to reflect it being on a different
host, but that’s about it.

Is there anything else that could be causing this to work in our stage
environment but not production?

Has anyone had to deal with this duplicate IP issue following an upgrade
and come up with a reasonable fix? Ideally, I’d like to clean up all
duplicates in our database, but I’m a little nervous about trying to do so
in our production environment.

I’ve attached the three patched files, if it’s helpful.

–
Marek

3

Marek,
It doesn't appear to work through the API either. It still performs the
validation. Here's an example:

>>> import requests
>>> from requests.auth import HTTPBasicAuth
>>> requests.packages.urllib3.disable_warnings()
>>> headers = {'Accept': 'version=2, application/json', 'ContentType':
'application/json'}
>>> url = "https://<myserver>/api/hosts/13799/interfaces/3345"
>>> requests.put(url, auth=HTTPBasicAuth('<user>', '<password>'), headers=
headers, verify=False).text
u'{\n "error": {"id":3345,"errors":{"ip":["has already been taken","has
already been taken"]},"full_messages":["IP address has already been
taken","IP address has already been taken"]}\n}\n'

Do I have any options here, other than mucking around in the database, to
fix this?

··· On Thursday, December 17, 2015 at 12:01:01 AM UTC-8, Marek Hulan wrote: > > Hello > > once the DB gets into inconsistent state validations do not work properly. > You > can try using direct interface API to delete the invalid interface, this > should skip the validation unique IP validation since it happens outside > of > host context. > > See API documentation [1] for more details. > > [1] http://theforeman.org/api/1.9/apidoc/v2/interfaces.html > > Hope this helps > > -- > Marek > > On Wednesday 16 of December 2015 16:56:46 Kyle Flavin wrote: > > Following an upgrade of our Foreman environment from > 1.7.1->1.8.x->1.9.3, > > I'm seeing network interfaces with duplicate IP's in my host records. > > > > It seems to be related to the following issue: > > http://projects.theforeman.org/issues/12486 > > > > I'm unable to delete the duplicate IP's in the web interface, because > the > > validation checks for a duplicate on the deleted NIC interface. In an > > attempt to fix this, I introduced the changes from here: > > > https://github.com/theforeman/foreman/commit/35241dd65a6b4f426374a03e83d6f79 > > 6f59a9d35 > > > > I created patches for the three modified files (excluding test files) as > > follows: > > > > cd /usr/share/foreman > > git show 35241dd65a6b4f426374a03e83d6f796f59a9d35 > app/models/host/managed.rb > > > patch1.txt > > > > git show 35241dd65a6b4f426374a03e83d6f796f59a9d35 app/models/nic/base.rb > > > > patch2.txt > > git show 35241dd65a6b4f426374a03e83d6f796f59a9d35 > > app/models/nic/interface.rb > > > patch3.txt > > > > and then applied them to our 1.9.3 files like so: > > > > cd /usr/share/foreman > > patch -p1 < patch1.txt > > patch -p1 < patch2.txt > > patch -p1 < patch3.txt > > > > This worked in our staging environment. I'm able to remove the > duplicate > > IP's through the Web UI. > > > > However, on two separate occasions, I've moved the three files over to > > production and it's failed. It doesn't break anything per se, but it > just > > doesn't seem to "work". When I try to go into a host record and delete > the > > duplicate IP, I still get a validation error. I went so far as to > > completely reboot both Foreman application servers (we run two in a load > > balanced pair) to ensure there were no apache/passenger connections left > > open. > > > > Our stage and production environments are both running 1.9.3 and are > > *mostly* the same, including running a copy of the production database. > > The staging environment is not running hooks present in production, and > has > > some slightly different configuration to reflect it being on a different > > host, but that's about it. > > > > Is there anything else that could be causing this to work in our stage > > environment but not production? > > > > Has anyone had to deal with this duplicate IP issue following an upgrade > > and come up with a reasonable fix? Ideally, I'd like to clean up all > > duplicates in our database, but I'm a little nervous about trying to do > so > > in our production environment. > > > > I've attached the three patched files, if it's helpful. > > -- > Marek >
4

I think you might need to remove the duplicated IPs via the API. I'm
struggling with this as well.

··· On Thursday, December 17, 2015 at 5:00:29 PM UTC-8, Kyle Flavin wrote: > > Marek, > It doesn't appear to work through the API either. It still performs the > validation. Here's an example: > > >>> import requests > >>> from requests.auth import HTTPBasicAuth > >>> requests.packages.urllib3.disable_warnings() > >>> headers = {'Accept': 'version=2, application/json', 'ContentType': > 'application/json'} > >>> url = "https:///api/hosts/13799/interfaces/3345" > >>> requests.put(url, auth=HTTPBasicAuth('', ''), headers= > headers, verify=False).text > u'{\n "error": {"id":3345,"errors":{"ip":["has already been taken","has > already been taken"]},"full_messages":["IP address has already been > taken","IP address has already been taken"]}\n}\n' > > Do I have any options here, other than mucking around in the database, to > fix this? > > > On Thursday, December 17, 2015 at 12:01:01 AM UTC-8, Marek Hulan wrote: >> >> Hello >> >> once the DB gets into inconsistent state validations do not work >> properly. You >> can try using direct interface API to delete the invalid interface, this >> should skip the validation unique IP validation since it happens outside >> of >> host context. >> >> See API documentation [1] for more details. >> >> [1] http://theforeman.org/api/1.9/apidoc/v2/interfaces.html >> >> Hope this helps >> >> -- >> Marek >> >> On Wednesday 16 of December 2015 16:56:46 Kyle Flavin wrote: >> > Following an upgrade of our Foreman environment from >> 1.7.1->1.8.x->1.9.3, >> > I'm seeing network interfaces with duplicate IP's in my host records. >> > >> > It seems to be related to the following issue: >> > http://projects.theforeman.org/issues/12486 >> > >> > I'm unable to delete the duplicate IP's in the web interface, because >> the >> > validation checks for a duplicate on the deleted NIC interface. In an >> > attempt to fix this, I introduced the changes from here: >> > >> https://github.com/theforeman/foreman/commit/35241dd65a6b4f426374a03e83d6f79 >> > 6f59a9d35 >> > >> > I created patches for the three modified files (excluding test files) >> as >> > follows: >> > >> > cd /usr/share/foreman >> > git show 35241dd65a6b4f426374a03e83d6f796f59a9d35 >> app/models/host/managed.rb >> > > patch1.txt >> > >> > git show 35241dd65a6b4f426374a03e83d6f796f59a9d35 >> app/models/nic/base.rb > >> > patch2.txt >> > git show 35241dd65a6b4f426374a03e83d6f796f59a9d35 >> > app/models/nic/interface.rb >> > > patch3.txt >> > >> > and then applied them to our 1.9.3 files like so: >> > >> > cd /usr/share/foreman >> > patch -p1 < patch1.txt >> > patch -p1 < patch2.txt >> > patch -p1 < patch3.txt >> > >> > This worked in our staging environment. I'm able to remove the >> duplicate >> > IP's through the Web UI. >> > >> > However, on two separate occasions, I've moved the three files over to >> > production and it's failed. It doesn't break anything per se, but it >> just >> > doesn't seem to "work". When I try to go into a host record and delete >> the >> > duplicate IP, I still get a validation error. I went so far as to >> > completely reboot both Foreman application servers (we run two in a >> load >> > balanced pair) to ensure there were no apache/passenger connections >> left >> > open. >> > >> > Our stage and production environments are both running 1.9.3 and are >> > *mostly* the same, including running a copy of the production database. >> > The staging environment is not running hooks present in production, and >> has >> > some slightly different configuration to reflect it being on a >> different >> > host, but that's about it. >> > >> > Is there anything else that could be causing this to work in our stage >> > environment but not production? >> > >> > Has anyone had to deal with this duplicate IP issue following an >> upgrade >> > and come up with a reasonable fix? Ideally, I'd like to clean up all >> > duplicates in our database, but I'm a little nervous about trying to do >> so >> > in our production environment. >> > >> > I've attached the three patched files, if it's helpful. >> >> -- >> Marek >> >
5

Hi Alyssa. Per my previous post, you can't remove the IP's through the
API; it still does the validation. It looks like you have to go directly
into the database.

··· On Friday, December 18, 2015 at 8:13:17 AM UTC-8, Alyssa wrote: > > I think you might need to remove the duplicated IPs via the API. I'm > struggling with this as well. > > On Thursday, December 17, 2015 at 5:00:29 PM UTC-8, Kyle Flavin wrote: >> >> Marek, >> It doesn't appear to work through the API either. It still performs the >> validation. Here's an example: >> >> >>> import requests >> >>> from requests.auth import HTTPBasicAuth >> >>> requests.packages.urllib3.disable_warnings() >> >>> headers = {'Accept': 'version=2, application/json', 'ContentType': >> 'application/json'} >> >>> url = "https:///api/hosts/13799/interfaces/3345" >> >>> requests.put(url, auth=HTTPBasicAuth('', ''), headers >> =headers, verify=False).text >> u'{\n "error": {"id":3345,"errors":{"ip":["has already been taken","has >> already been taken"]},"full_messages":["IP address has already been >> taken","IP address has already been taken"]}\n}\n' >> >> Do I have any options here, other than mucking around in the database, to >> fix this? >> >> >> On Thursday, December 17, 2015 at 12:01:01 AM UTC-8, Marek Hulan wrote: >>> >>> Hello >>> >>> once the DB gets into inconsistent state validations do not work >>> properly. You >>> can try using direct interface API to delete the invalid interface, this >>> should skip the validation unique IP validation since it happens outside >>> of >>> host context. >>> >>> See API documentation [1] for more details. >>> >>> [1] http://theforeman.org/api/1.9/apidoc/v2/interfaces.html >>> >>> Hope this helps >>> >>> -- >>> Marek >>> >>> On Wednesday 16 of December 2015 16:56:46 Kyle Flavin wrote: >>> > Following an upgrade of our Foreman environment from >>> 1.7.1->1.8.x->1.9.3, >>> > I'm seeing network interfaces with duplicate IP's in my host records. >>> > >>> > It seems to be related to the following issue: >>> > http://projects.theforeman.org/issues/12486 >>> > >>> > I'm unable to delete the duplicate IP's in the web interface, because >>> the >>> > validation checks for a duplicate on the deleted NIC interface. In an >>> > attempt to fix this, I introduced the changes from here: >>> > >>> https://github.com/theforeman/foreman/commit/35241dd65a6b4f426374a03e83d6f79 >>> > 6f59a9d35 >>> > >>> > I created patches for the three modified files (excluding test files) >>> as >>> > follows: >>> > >>> > cd /usr/share/foreman >>> > git show 35241dd65a6b4f426374a03e83d6f796f59a9d35 >>> app/models/host/managed.rb >>> > > patch1.txt >>> > >>> > git show 35241dd65a6b4f426374a03e83d6f796f59a9d35 >>> app/models/nic/base.rb > >>> > patch2.txt >>> > git show 35241dd65a6b4f426374a03e83d6f796f59a9d35 >>> > app/models/nic/interface.rb >>> > > patch3.txt >>> > >>> > and then applied them to our 1.9.3 files like so: >>> > >>> > cd /usr/share/foreman >>> > patch -p1 < patch1.txt >>> > patch -p1 < patch2.txt >>> > patch -p1 < patch3.txt >>> > >>> > This worked in our staging environment. I'm able to remove the >>> duplicate >>> > IP's through the Web UI. >>> > >>> > However, on two separate occasions, I've moved the three files over to >>> > production and it's failed. It doesn't break anything per se, but it >>> just >>> > doesn't seem to "work". When I try to go into a host record and >>> delete the >>> > duplicate IP, I still get a validation error. I went so far as to >>> > completely reboot both Foreman application servers (we run two in a >>> load >>> > balanced pair) to ensure there were no apache/passenger connections >>> left >>> > open. >>> > >>> > Our stage and production environments are both running 1.9.3 and are >>> > *mostly* the same, including running a copy of the production >>> database. >>> > The staging environment is not running hooks present in production, >>> and has >>> > some slightly different configuration to reflect it being on a >>> different >>> > host, but that's about it. >>> > >>> > Is there anything else that could be causing this to work in our stage >>> > environment but not production? >>> > >>> > Has anyone had to deal with this duplicate IP issue following an >>> upgrade >>> > and come up with a reasonable fix? Ideally, I'd like to clean up all >>> > duplicates in our database, but I'm a little nervous about trying to >>> do so >>> > in our production environment. >>> > >>> > I've attached the three patched files, if it's helpful. >>> >>> -- >>> Marek >>> >>
6

I believe I've used hammer to directly update the nics. Might give that a
try.

··· On Fri, Dec 18, 2015 at 8:42 AM, Kyle Flavin wrote:

Hi Alyssa. Per my previous post, you can’t remove the IP’s through the
API; it still does the validation. It looks like you have to go directly
into the database.

On Friday, December 18, 2015 at 8:13:17 AM UTC-8, Alyssa wrote:

I think you might need to remove the duplicated IPs via the API. I’m
struggling with this as well.

On Thursday, December 17, 2015 at 5:00:29 PM UTC-8, Kyle Flavin wrote:

Marek,
It doesn’t appear to work through the API either. It still performs the
validation. Here’s an example:

import requests
from requests.auth import HTTPBasicAuth
requests.packages.urllib3.disable_warnings()
headers = {‘Accept’: ‘version=2, application/json’, ‘ContentType’:
‘application/json’}
url = “https:///api/hosts/13799/interfaces/3345”
requests.put(url, auth=HTTPBasicAuth(‘’, ‘’),
headers=headers, verify=False).text
u’{\n “error”: {“id”:3345,“errors”:{“ip”:[“has already been
taken”,“has already been taken”]},“full_messages”:[“IP address has already
been taken”,“IP address has already been taken”]}\n}\n’

Do I have any options here, other than mucking around in the database,
to fix this?

On Thursday, December 17, 2015 at 12:01:01 AM UTC-8, Marek Hulan wrote:

Hello

once the DB gets into inconsistent state validations do not work
properly. You
can try using direct interface API to delete the invalid interface,
this
should skip the validation unique IP validation since it happens
outside of
host context.

See API documentation [1] for more details.

[1] http://theforeman.org/api/1.9/apidoc/v2/interfaces.html

Hope this helps

–
Marek

On Wednesday 16 of December 2015 16:56:46 Kyle Flavin wrote:

Following an upgrade of our Foreman environment from
1.7.1->1.8.x->1.9.3,
I’m seeing network interfaces with duplicate IP’s in my host records.

It seems to be related to the following issue:
http://projects.theforeman.org/issues/12486

I’m unable to delete the duplicate IP’s in the web interface, because
the
validation checks for a duplicate on the deleted NIC interface. In
an
attempt to fix this, I introduced the changes from here:

Fixes #11034 - custom uniqueness validations for interface attributes · theforeman/foreman@35241dd · GitHub

6f59a9d35

I created patches for the three modified files (excluding test files)
as
follows:

cd /usr/share/foreman
git show 35241dd65a6b4f426374a03e83d6f796f59a9d35
app/models/host/managed.rb

patch1.txt

git show 35241dd65a6b4f426374a03e83d6f796f59a9d35
app/models/nic/base.rb >
patch2.txt
git show 35241dd65a6b4f426374a03e83d6f796f59a9d35
app/models/nic/interface.rb

patch3.txt

and then applied them to our 1.9.3 files like so:

cd /usr/share/foreman
patch -p1 < patch1.txt
patch -p1 < patch2.txt
patch -p1 < patch3.txt

This worked in our staging environment. I’m able to remove the
duplicate
IP’s through the Web UI.

However, on two separate occasions, I’ve moved the three files over
to
production and it’s failed. It doesn’t break anything per se, but it
just
doesn’t seem to “work”. When I try to go into a host record and
delete the
duplicate IP, I still get a validation error. I went so far as to
completely reboot both Foreman application servers (we run two in a
load
balanced pair) to ensure there were no apache/passenger connections
left
open.

Our stage and production environments are both running 1.9.3 and are
mostly the same, including running a copy of the production
database.
The staging environment is not running hooks present in production,
and has
some slightly different configuration to reflect it being on a
different
host, but that’s about it.

Is there anything else that could be causing this to work in our
stage
environment but not production?

Has anyone had to deal with this duplicate IP issue following an
upgrade
and come up with a reasonable fix? Ideally, I’d like to clean up all
duplicates in our database, but I’m a little nervous about trying to
do so
in our production environment.

I’ve attached the three patched files, if it’s helpful.

–
Marek

–
You received this message because you are subscribed to a topic in the
Google Groups “Foreman users” group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/foreman-users/fL-oTiRjHy0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at https://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.

7

Hammer worked. Interesting, I wonder why it didn't work when I manually
call the API.

I ended up working on a solution this afternoon to clean the database
manually. I'll be running it in the next few hours. I'll post what I did
afterwards as long as it doesn't blow anything up… maybe it will help
someone.

··· On Friday, December 18, 2015 at 10:03:18 AM UTC-8, Alyssa wrote: > > I believe I've used hammer to directly update the nics. Might give that a > try. > > On Fri, Dec 18, 2015 at 8:42 AM, Kyle Flavin > wrote: > >> Hi Alyssa. Per my previous post, you can't remove the IP's through the >> API; it still does the validation. It looks like you have to go directly >> into the database. >> >> >> On Friday, December 18, 2015 at 8:13:17 AM UTC-8, Alyssa wrote: >>> >>> I think you might need to remove the duplicated IPs via the API. I'm >>> struggling with this as well. >>> >>> On Thursday, December 17, 2015 at 5:00:29 PM UTC-8, Kyle Flavin wrote: >>>> >>>> Marek, >>>> It doesn't appear to work through the API either. It still performs >>>> the validation. Here's an example: >>>> >>>> >>> import requests >>>> >>> from requests.auth import HTTPBasicAuth >>>> >>> requests.packages.urllib3.disable_warnings() >>>> >>> headers = {'Accept': 'version=2, application/json', 'ContentType': >>>> 'application/json'} >>>> >>> url = "https:///api/hosts/13799/interfaces/3345" >>>> >>> requests.put(url, auth=HTTPBasicAuth('', ''), >>>> headers=headers, verify=False).text >>>> u'{\n "error": {"id":3345,"errors":{"ip":["has already been >>>> taken","has already been taken"]},"full_messages":["IP address has already >>>> been taken","IP address has already been taken"]}\n}\n' >>>> >>>> Do I have any options here, other than mucking around in the database, >>>> to fix this? >>>> >>>> >>>> On Thursday, December 17, 2015 at 12:01:01 AM UTC-8, Marek Hulan wrote: >>>>> >>>>> Hello >>>>> >>>>> once the DB gets into inconsistent state validations do not work >>>>> properly. You >>>>> can try using direct interface API to delete the invalid interface, >>>>> this >>>>> should skip the validation unique IP validation since it happens >>>>> outside of >>>>> host context. >>>>> >>>>> See API documentation [1] for more details. >>>>> >>>>> [1] http://theforeman.org/api/1.9/apidoc/v2/interfaces.html >>>>> >>>>> Hope this helps >>>>> >>>>> -- >>>>> Marek >>>>> >>>>> On Wednesday 16 of December 2015 16:56:46 Kyle Flavin wrote: >>>>> > Following an upgrade of our Foreman environment from >>>>> 1.7.1->1.8.x->1.9.3, >>>>> > I'm seeing network interfaces with duplicate IP's in my host >>>>> records. >>>>> > >>>>> > It seems to be related to the following issue: >>>>> > http://projects.theforeman.org/issues/12486 >>>>> > >>>>> > I'm unable to delete the duplicate IP's in the web interface, >>>>> because the >>>>> > validation checks for a duplicate on the deleted NIC interface. In >>>>> an >>>>> > attempt to fix this, I introduced the changes from here: >>>>> > >>>>> https://github.com/theforeman/foreman/commit/35241dd65a6b4f426374a03e83d6f79 >>>>> > 6f59a9d35 >>>>> > >>>>> > I created patches for the three modified files (excluding test >>>>> files) as >>>>> > follows: >>>>> > >>>>> > cd /usr/share/foreman >>>>> > git show 35241dd65a6b4f426374a03e83d6f796f59a9d35 >>>>> app/models/host/managed.rb >>>>> > > patch1.txt >>>>> > >>>>> > git show 35241dd65a6b4f426374a03e83d6f796f59a9d35 >>>>> app/models/nic/base.rb > >>>>> > patch2.txt >>>>> > git show 35241dd65a6b4f426374a03e83d6f796f59a9d35 >>>>> > app/models/nic/interface.rb >>>>> > > patch3.txt >>>>> > >>>>> > and then applied them to our 1.9.3 files like so: >>>>> > >>>>> > cd /usr/share/foreman >>>>> > patch -p1 < patch1.txt >>>>> > patch -p1 < patch2.txt >>>>> > patch -p1 < patch3.txt >>>>> > >>>>> > This worked in our staging environment. I'm able to remove the >>>>> duplicate >>>>> > IP's through the Web UI. >>>>> > >>>>> > However, on two separate occasions, I've moved the three files over >>>>> to >>>>> > production and it's failed. It doesn't break anything per se, but >>>>> it just >>>>> > doesn't seem to "work". When I try to go into a host record and >>>>> delete the >>>>> > duplicate IP, I still get a validation error. I went so far as to >>>>> > completely reboot both Foreman application servers (we run two in a >>>>> load >>>>> > balanced pair) to ensure there were no apache/passenger connections >>>>> left >>>>> > open. >>>>> > >>>>> > Our stage and production environments are both running 1.9.3 and are >>>>> > *mostly* the same, including running a copy of the production >>>>> database. >>>>> > The staging environment is not running hooks present in production, >>>>> and has >>>>> > some slightly different configuration to reflect it being on a >>>>> different >>>>> > host, but that's about it. >>>>> > >>>>> > Is there anything else that could be causing this to work in our >>>>> stage >>>>> > environment but not production? >>>>> > >>>>> > Has anyone had to deal with this duplicate IP issue following an >>>>> upgrade >>>>> > and come up with a reasonable fix? Ideally, I'd like to clean up >>>>> all >>>>> > duplicates in our database, but I'm a little nervous about trying to >>>>> do so >>>>> > in our production environment. >>>>> > >>>>> > I've attached the three patched files, if it's helpful. >>>>> >>>>> -- >>>>> Marek >>>>> >>>> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "Foreman users" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/foreman-users/fL-oTiRjHy0/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> foreman-user...@googlegroups.com . >> To post to this group, send email to forema...@googlegroups.com >> . >> Visit this group at https://groups.google.com/group/foreman-users. >> For more options, visit https://groups.google.com/d/optout. >> > >
8

IIRC the debug flag to hammer will show the exact API call it makes -
might be interesting for a comparison.

Greg

··· On 18 December 2015 at 22:32, Kyle Flavin wrote: > Hammer worked. Interesting, I wonder why it didn't work when I manually > call the API.
9

Nice, Greg! Thanks for the tip. :slight_smile:

··· On Fri, Dec 18, 2015 at 2:50 PM, Greg Sutcliffe wrote:

On 18 December 2015 at 22:32, Kyle Flavin kyle.flavin@gmail.com wrote:

Hammer worked. Interesting, I wonder why it didn’t work when I manually
call the API.

IIRC the debug flag to hammer will show the exact API call it makes -
might be interesting for a comparison.

Greg

–
You received this message because you are subscribed to a topic in the
Google Groups “Foreman users” group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/foreman-users/fL-oTiRjHy0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at https://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.

10

Cool, thanks Greg, I'll have to see what was different.

I ended up cleaning the database with a quick 'n dirty Python script. The
script dumps down the host id + nic id to a file, where any non-primary
NIC's IP or MAC matches the primary NIC. I then used a bash script (not
included) to turn the id's into SQL statements in the form of:
delete from nics where host_id='<hostid>' and id='<nicid>';
…

I ended up with about 10,000 records that needed to be removed. Nothing
has imploded… yet… and I'm able to modify the host records now.

import requests
import json
from requests.auth import HTTPBasicAuth
requests.packages.urllib3.disable_warnings()
headers = {'Accept': 'version=2, application/json'}
server = "http://<myserver>"
outfile = "./prod_dupe_ips"
url = "%s/api/hosts" % server
user = '<user>'
password = '<password>'
params = {'per_page': '8000'}
hosts = json.loads(requests.get(url, auth=HTTPBasicAuth(user, password),
headers=headers, verify=False, params=params).text)
ids = {}
for i in hosts['results']:
ids["%s" % i['id']] = i['name']

count = 1
with open(outfile, "w") as f:
for id,hostname in ids.items():
count += 1
print count, id, hostname
url = "%s/api/hosts/%s/interfaces" % (server, id,)
body = json.loads(requests.get(url, auth=HTTPBasicAuth(user, password),
headers=headers, verify=False).text)
interfaces = []
primary = ""
for int in body['results']:
if not int['primary']:
interfaces.append(int)
else:
primary = int
for i in interfaces:
skip = 0
if i['ip'] == primary['ip']:
f.write("%s %s IP match\n" % (id, i['id'],))
skip = 1
if not skip and i['mac'] == primary['mac']:
f.write("%s %s MAC match\n" % (id, i['id'],))

··· On Friday, December 18, 2015 at 4:09:20 PM UTC-8, Alyssa wrote: > > Nice, Greg! Thanks for the tip. :) > > On Fri, Dec 18, 2015 at 2:50 PM, Greg Sutcliffe > wrote: > >> On 18 December 2015 at 22:32, Kyle Flavin > > wrote: >> > Hammer worked. Interesting, I wonder why it didn't work when I manually >> > call the API. >> >> IIRC the debug flag to hammer will show the exact API call it makes - >> might be interesting for a comparison. >> >> Greg >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "Foreman users" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/foreman-users/fL-oTiRjHy0/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> foreman-user...@googlegroups.com . >> To post to this group, send email to forema...@googlegroups.com >> . >> Visit this group at https://groups.google.com/group/foreman-users. >> For more options, visit https://groups.google.com/d/optout. >> > >
11

I tried something like that on a smaller scale, only to have it wiped out
next runs because I have the discovery plugin installed.

··· On Fri, Dec 18, 2015 at 5:48 PM, Kyle Flavin wrote:

Cool, thanks Greg, I’ll have to see what was different.

I ended up cleaning the database with a quick ‘n dirty Python script. The
script dumps down the host id + nic id to a file, where any non-primary
NIC’s IP or MAC matches the primary NIC. I then used a bash script (not
included) to turn the id’s into SQL statements in the form of:
delete from nics where host_id=’’ and id=‘’;
…

I ended up with about 10,000 records that needed to be removed. Nothing
has imploded… yet… and I’m able to modify the host records now.

import requests
import json
from requests.auth import HTTPBasicAuth
requests.packages.urllib3.disable_warnings()
headers = {‘Accept’: ‘version=2, application/json’}
server = “http://”
outfile = “./prod_dupe_ips”
url = “%s/api/hosts” % server
user = ‘’
password = ‘’
params = {‘per_page’: ‘8000’}
hosts = json.loads(requests.get(url, auth=HTTPBasicAuth(user, password),
headers=headers, verify=False, params=params).text)
ids = {}
for i in hosts[‘results’]:
ids[“%s” % i[‘id’]] = i[‘name’]

count = 1
with open(outfile, “w”) as f:
for id,hostname in ids.items():
count += 1
print count, id, hostname
url = “%s/api/hosts/%s/interfaces” % (server, id,)
body = json.loads(requests.get(url, auth=HTTPBasicAuth(user, password
), headers=headers, verify=False).text)
interfaces =
primary = “”
for int in body[‘results’]:
if not int[‘primary’]:
interfaces.append(int)
else:
primary = int
for i in interfaces:
skip = 0
if i[‘ip’] == primary[‘ip’]:
f.write(“%s %s IP match\n” % (id, i[‘id’],))
skip = 1
if not skip and i[‘mac’] == primary[‘mac’]:
f.write(“%s %s MAC match\n” % (id, i[‘id’],))

On Friday, December 18, 2015 at 4:09:20 PM UTC-8, Alyssa wrote:

Nice, Greg! Thanks for the tip. :slight_smile:

On Fri, Dec 18, 2015 at 2:50 PM, Greg Sutcliffe greg.su...@gmail.com >> wrote:

On 18 December 2015 at 22:32, Kyle Flavin kyle....@gmail.com wrote:

Hammer worked. Interesting, I wonder why it didn’t work when I
manually
call the API.

IIRC the debug flag to hammer will show the exact API call it makes -
might be interesting for a comparison.

Greg

–
You received this message because you are subscribed to a topic in the
Google Groups “Foreman users” group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/foreman-users/fL-oTiRjHy0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
foreman-user...@googlegroups.com.
To post to this group, send email to forema...@googlegroups.com.
Visit this group at https://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.

–
You received this message because you are subscribed to a topic in the
Google Groups “Foreman users” group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/foreman-users/fL-oTiRjHy0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at https://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.

12

Oh, really? Having the discovery plugin installed causes all the dupes to
come back? Or just on new installs?

··· On Saturday, December 19, 2015 at 8:34:26 AM UTC-8, Alyssa wrote: > > I tried something like that on a smaller scale, only to have it wiped out > next runs because I have the discovery plugin installed. > > On Fri, Dec 18, 2015 at 5:48 PM, Kyle Flavin > wrote: > >> Cool, thanks Greg, I'll have to see what was different. >> >> I ended up cleaning the database with a quick 'n dirty Python script. >> The script dumps down the host id + nic id to a file, where any non-primary >> NIC's IP or MAC matches the primary NIC. I then used a bash script (not >> included) to turn the id's into SQL statements in the form of: >> delete from nics where host_id='' and id=''; >> ... >> >> I ended up with about 10,000 records that needed to be removed. Nothing >> has imploded... yet.... and I'm able to modify the host records now. >> >> import requests >> import json >> from requests.auth import HTTPBasicAuth >> requests.packages.urllib3.disable_warnings() >> headers = {'Accept': 'version=2, application/json'} >> server = "http://" >> outfile = "./prod_dupe_ips" >> url = "%s/api/hosts" % server >> user = '' >> password = '' >> params = {'per_page': '8000'} >> hosts = json.loads(requests.get(url, auth=HTTPBasicAuth(user, password), >> headers=headers, verify=False, params=params).text) >> ids = {} >> for i in hosts['results']: >> ids["%s" % i['id']] = i['name'] >> >> count = 1 >> with open(outfile, "w") as f: >> for id,hostname in ids.items(): >> count += 1 >> print count, id, hostname >> url = "%s/api/hosts/%s/interfaces" % (server, id,) >> body = json.loads(requests.get(url, auth=HTTPBasicAuth(user, password >> ), headers=headers, verify=False).text) >> interfaces = [] >> primary = "" >> for int in body['results']: >> if not int['primary']: >> interfaces.append(int) >> else: >> primary = int >> for i in interfaces: >> skip = 0 >> if i['ip'] == primary['ip']: >> f.write("%s %s IP match\n" % (id, i['id'],)) >> skip = 1 >> if not skip and i['mac'] == primary['mac']: >> f.write("%s %s MAC match\n" % (id, i['id'],)) >> >> >> >> >> On Friday, December 18, 2015 at 4:09:20 PM UTC-8, Alyssa wrote: >>> >>> Nice, Greg! Thanks for the tip. :) >>> >>> On Fri, Dec 18, 2015 at 2:50 PM, Greg Sutcliffe >>> wrote: >>> >>>> On 18 December 2015 at 22:32, Kyle Flavin wrote: >>>> > Hammer worked. Interesting, I wonder why it didn't work when I >>>> manually >>>> > call the API. >>>> >>>> IIRC the debug flag to hammer will show the exact API call it makes - >>>> might be interesting for a comparison. >>>> >>>> Greg >>>> >>>> -- >>>> You received this message because you are subscribed to a topic in the >>>> Google Groups "Foreman users" group. >>>> To unsubscribe from this topic, visit >>>> https://groups.google.com/d/topic/foreman-users/fL-oTiRjHy0/unsubscribe >>>> . >>>> To unsubscribe from this group and all its topics, send an email to >>>> foreman-user...@googlegroups.com. >>>> To post to this group, send email to forema...@googlegroups.com. >>>> Visit this group at https://groups.google.com/group/foreman-users. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "Foreman users" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/foreman-users/fL-oTiRjHy0/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> foreman-user...@googlegroups.com . >> To post to this group, send email to forema...@googlegroups.com >> . >> Visit this group at https://groups.google.com/group/foreman-users. >> For more options, visit https://groups.google.com/d/optout. >> > >
13

Having discovery plugin installed causes the nics to be pushed up from
puppet run facts. If you have a setup, say with HA that migrates between
systems, this can really screw you up.

··· On Mon, Dec 21, 2015 at 9:41 AM, Kyle Flavin wrote:

Oh, really? Having the discovery plugin installed causes all the dupes to
come back? Or just on new installs?

On Saturday, December 19, 2015 at 8:34:26 AM UTC-8, Alyssa wrote:

I tried something like that on a smaller scale, only to have it wiped out
next runs because I have the discovery plugin installed.

On Fri, Dec 18, 2015 at 5:48 PM, Kyle Flavin kyle....@gmail.com wrote:

Cool, thanks Greg, I’ll have to see what was different.

I ended up cleaning the database with a quick ‘n dirty Python script.
The script dumps down the host id + nic id to a file, where any non-primary
NIC’s IP or MAC matches the primary NIC. I then used a bash script (not
included) to turn the id’s into SQL statements in the form of:
delete from nics where host_id=’’ and id=‘’;
…

I ended up with about 10,000 records that needed to be removed. Nothing
has imploded… yet… and I’m able to modify the host records now.

import requests
import json
from requests.auth import HTTPBasicAuth
requests.packages.urllib3.disable_warnings()
headers = {‘Accept’: ‘version=2, application/json’}
server = “http://”
outfile = “./prod_dupe_ips”
url = “%s/api/hosts” % server
user = ‘’
password = ‘’
params = {‘per_page’: ‘8000’}
hosts = json.loads(requests.get(url, auth=HTTPBasicAuth(user, password),
headers=headers, verify=False, params=params).text)
ids = {}
for i in hosts[‘results’]:
ids[“%s” % i[‘id’]] = i[‘name’]

count = 1
with open(outfile, “w”) as f:
for id,hostname in ids.items():
count += 1
print count, id, hostname
url = “%s/api/hosts/%s/interfaces” % (server, id,)
body = json.loads(requests.get(url, auth=HTTPBasicAuth(user,
password), headers=headers, verify=False).text)
interfaces =
primary = “”
for int in body[‘results’]:
if not int[‘primary’]:
interfaces.append(int)
else:
primary = int
for i in interfaces:
skip = 0
if i[‘ip’] == primary[‘ip’]:
f.write(“%s %s IP match\n” % (id, i[‘id’],))
skip = 1
if not skip and i[‘mac’] == primary[‘mac’]:
f.write(“%s %s MAC match\n” % (id, i[‘id’],))

On Friday, December 18, 2015 at 4:09:20 PM UTC-8, Alyssa wrote:

Nice, Greg! Thanks for the tip. :slight_smile:

On Fri, Dec 18, 2015 at 2:50 PM, Greg Sutcliffe greg.su...@gmail.com >>>> wrote:

On 18 December 2015 at 22:32, Kyle Flavin kyle....@gmail.com wrote:

Hammer worked. Interesting, I wonder why it didn’t work when I
manually
call the API.

IIRC the debug flag to hammer will show the exact API call it makes -
might be interesting for a comparison.

Greg

–
You received this message because you are subscribed to a topic in the
Google Groups “Foreman users” group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/foreman-users/fL-oTiRjHy0/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
foreman-user...@googlegroups.com.
To post to this group, send email to forema...@googlegroups.com.
Visit this group at https://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.

–
You received this message because you are subscribed to a topic in the
Google Groups “Foreman users” group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/foreman-users/fL-oTiRjHy0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
foreman-user...@googlegroups.com.
To post to this group, send email to forema...@googlegroups.com.
Visit this group at https://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.

–
You received this message because you are subscribed to a topic in the
Google Groups “Foreman users” group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/foreman-users/fL-oTiRjHy0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at https://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.

14

That sounds like a major bug. Do you know if there Is there a current
issue open for this? I was planning to install the discovery plugin (I
upgraded from 1.7 to 1.9 to do so…), and now it sounds like I can't if
it's going to whack out all of my host records.

··· On Monday, December 21, 2015 at 10:03:46 AM UTC-8, Alyssa wrote: > > Having discovery plugin installed causes the nics to be pushed up from > puppet run facts. If you have a setup, say with HA that migrates between > systems, this can really screw you up. > > On Mon, Dec 21, 2015 at 9:41 AM, Kyle Flavin > wrote: > >> Oh, really? Having the discovery plugin installed causes all the dupes >> to come back? Or just on new installs? >> >> On Saturday, December 19, 2015 at 8:34:26 AM UTC-8, Alyssa wrote: >>> >>> I tried something like that on a smaller scale, only to have it wiped >>> out next runs because I have the discovery plugin installed. >>> >>> On Fri, Dec 18, 2015 at 5:48 PM, Kyle Flavin wrote: >>> >>>> Cool, thanks Greg, I'll have to see what was different. >>>> >>>> I ended up cleaning the database with a quick 'n dirty Python script. >>>> The script dumps down the host id + nic id to a file, where any non-primary >>>> NIC's IP or MAC matches the primary NIC. I then used a bash script (not >>>> included) to turn the id's into SQL statements in the form of: >>>> delete from nics where host_id='' and id=''; >>>> ... >>>> >>>> I ended up with about 10,000 records that needed to be removed. >>>> Nothing has imploded... yet.... and I'm able to modify the host records >>>> now. >>>> >>>> import requests >>>> import json >>>> from requests.auth import HTTPBasicAuth >>>> requests.packages.urllib3.disable_warnings() >>>> headers = {'Accept': 'version=2, application/json'} >>>> server = "http://" >>>> outfile = "./prod_dupe_ips" >>>> url = "%s/api/hosts" % server >>>> user = '' >>>> password = '' >>>> params = {'per_page': '8000'} >>>> hosts = json.loads(requests.get(url, auth=HTTPBasicAuth(user, password >>>> ), headers=headers, verify=False, params=params).text) >>>> ids = {} >>>> for i in hosts['results']: >>>> ids["%s" % i['id']] = i['name'] >>>> >>>> count = 1 >>>> with open(outfile, "w") as f: >>>> for id,hostname in ids.items(): >>>> count += 1 >>>> print count, id, hostname >>>> url = "%s/api/hosts/%s/interfaces" % (server, id,) >>>> body = json.loads(requests.get(url, auth=HTTPBasicAuth(user, >>>> password), headers=headers, verify=False).text) >>>> interfaces = [] >>>> primary = "" >>>> for int in body['results']: >>>> if not int['primary']: >>>> interfaces.append(int) >>>> else: >>>> primary = int >>>> for i in interfaces: >>>> skip = 0 >>>> if i['ip'] == primary['ip']: >>>> f.write("%s %s IP match\n" % (id, i['id'],)) >>>> skip = 1 >>>> if not skip and i['mac'] == primary['mac']: >>>> f.write("%s %s MAC match\n" % (id, i['id'],)) >>>> >>>> >>>> >>>> >>>> On Friday, December 18, 2015 at 4:09:20 PM UTC-8, Alyssa wrote: >>>>> >>>>> Nice, Greg! Thanks for the tip. :) >>>>> >>>>> On Fri, Dec 18, 2015 at 2:50 PM, Greg Sutcliffe >>>>> wrote: >>>>> >>>>>> On 18 December 2015 at 22:32, Kyle Flavin wrote: >>>>>> > Hammer worked. Interesting, I wonder why it didn't work when I >>>>>> manually >>>>>> > call the API. >>>>>> >>>>>> IIRC the debug flag to hammer will show the exact API call it makes - >>>>>> might be interesting for a comparison. >>>>>> >>>>>> Greg >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to a topic in >>>>>> the Google Groups "Foreman users" group. >>>>>> To unsubscribe from this topic, visit >>>>>> https://groups.google.com/d/topic/foreman-users/fL-oTiRjHy0/unsubscribe >>>>>> . >>>>>> To unsubscribe from this group and all its topics, send an email to >>>>>> foreman-user...@googlegroups.com. >>>>>> To post to this group, send email to forema...@googlegroups.com. >>>>>> Visit this group at https://groups.google.com/group/foreman-users. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> -- >>>> You received this message because you are subscribed to a topic in the >>>> Google Groups "Foreman users" group. >>>> To unsubscribe from this topic, visit >>>> https://groups.google.com/d/topic/foreman-users/fL-oTiRjHy0/unsubscribe >>>> . >>>> To unsubscribe from this group and all its topics, send an email to >>>> foreman-user...@googlegroups.com. >>>> To post to this group, send email to forema...@googlegroups.com. >>>> Visit this group at https://groups.google.com/group/foreman-users. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "Foreman users" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/foreman-users/fL-oTiRjHy0/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> foreman-user...@googlegroups.com . >> To post to this group, send email to forema...@googlegroups.com >> . >> Visit this group at https://groups.google.com/group/foreman-users. >> For more options, visit https://groups.google.com/d/optout. >> > >
15

I know Kyle, I did the same.

··· On Mon, Dec 21, 2015 at 11:15 AM, Kyle Flavin wrote:

That sounds like a major bug. Do you know if there Is there a current
issue open for this? I was planning to install the discovery plugin (I
upgraded from 1.7 to 1.9 to do so…), and now it sounds like I can’t if
it’s going to whack out all of my host records.

On Monday, December 21, 2015 at 10:03:46 AM UTC-8, Alyssa wrote:

Having discovery plugin installed causes the nics to be pushed up from
puppet run facts. If you have a setup, say with HA that migrates between
systems, this can really screw you up.

On Mon, Dec 21, 2015 at 9:41 AM, Kyle Flavin kyle....@gmail.com wrote:

Oh, really? Having the discovery plugin installed causes all the dupes
to come back? Or just on new installs?

On Saturday, December 19, 2015 at 8:34:26 AM UTC-8, Alyssa wrote:

I tried something like that on a smaller scale, only to have it wiped
out next runs because I have the discovery plugin installed.

On Fri, Dec 18, 2015 at 5:48 PM, Kyle Flavin kyle....@gmail.com >>>> wrote:

Cool, thanks Greg, I’ll have to see what was different.

I ended up cleaning the database with a quick ‘n dirty Python script.
The script dumps down the host id + nic id to a file, where any non-primary
NIC’s IP or MAC matches the primary NIC. I then used a bash script (not
included) to turn the id’s into SQL statements in the form of:
delete from nics where host_id=’’ and id=‘’;
…

I ended up with about 10,000 records that needed to be removed.
Nothing has imploded… yet… and I’m able to modify the host records
now.

import requests
import json
from requests.auth import HTTPBasicAuth
requests.packages.urllib3.disable_warnings()
headers = {‘Accept’: ‘version=2, application/json’}
server = “http://”
outfile = “./prod_dupe_ips”
url = “%s/api/hosts” % server
user = ‘’
password = ‘’
params = {‘per_page’: ‘8000’}
hosts = json.loads(requests.get(url, auth=HTTPBasicAuth(user, password
), headers=headers, verify=False, params=params).text)
ids = {}
for i in hosts[‘results’]:
ids[“%s” % i[‘id’]] = i[‘name’]

count = 1
with open(outfile, “w”) as f:
for id,hostname in ids.items():
count += 1
print count, id, hostname
url = “%s/api/hosts/%s/interfaces” % (server, id,)
body = json.loads(requests.get(url, auth=HTTPBasicAuth(user,
password), headers=headers, verify=False).text)
interfaces =
primary = “”
for int in body[‘results’]:
if not int[‘primary’]:
interfaces.append(int)
else:
primary = int
for i in interfaces:
skip = 0
if i[‘ip’] == primary[‘ip’]:
f.write(“%s %s IP match\n” % (id, i[‘id’],))
skip = 1
if not skip and i[‘mac’] == primary[‘mac’]:
f.write(“%s %s MAC match\n” % (id, i[‘id’],))

On Friday, December 18, 2015 at 4:09:20 PM UTC-8, Alyssa wrote:

Nice, Greg! Thanks for the tip. :slight_smile:

On Fri, Dec 18, 2015 at 2:50 PM, Greg Sutcliffe <greg.su...@gmail.com >>>>>> > wrote:

On 18 December 2015 at 22:32, Kyle Flavin kyle....@gmail.com >>>>>>> wrote:

Hammer worked. Interesting, I wonder why it didn’t work when I
manually
call the API.

IIRC the debug flag to hammer will show the exact API call it makes -
might be interesting for a comparison.

Greg

–
You received this message because you are subscribed to a topic in
the Google Groups “Foreman users” group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/foreman-users/fL-oTiRjHy0/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
foreman-user...@googlegroups.com.
To post to this group, send email to forema...@googlegroups.com.
Visit this group at https://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.

–
You received this message because you are subscribed to a topic in the
Google Groups “Foreman users” group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/foreman-users/fL-oTiRjHy0/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
foreman-user...@googlegroups.com.
To post to this group, send email to forema...@googlegroups.com.
Visit this group at https://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.

–
You received this message because you are subscribed to a topic in the
Google Groups “Foreman users” group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/foreman-users/fL-oTiRjHy0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
foreman-user...@googlegroups.com.
To post to this group, send email to forema...@googlegroups.com.
Visit this group at https://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.

–
You received this message because you are subscribed to a topic in the
Google Groups “Foreman users” group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/foreman-users/fL-oTiRjHy0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at https://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.

16

Ack, okay thanks for the heads up!

··· On Monday, December 21, 2015 at 11:37:57 AM UTC-8, Alyssa wrote: > > I know Kyle, I did the same. > > http://projects.theforeman.org/issues/12391 > http://projects.theforeman.org/issues/11047 > > > On Mon, Dec 21, 2015 at 11:15 AM, Kyle Flavin > wrote: > >> That sounds like a major bug. Do you know if there Is there a current >> issue open for this? I was planning to install the discovery plugin (I >> upgraded from 1.7 to 1.9 to do so...), and now it sounds like I can't if >> it's going to whack out all of my host records. >> >> On Monday, December 21, 2015 at 10:03:46 AM UTC-8, Alyssa wrote: >>> >>> Having discovery plugin installed causes the nics to be pushed up from >>> puppet run facts. If you have a setup, say with HA that migrates between >>> systems, this can really screw you up. >>> >>> On Mon, Dec 21, 2015 at 9:41 AM, Kyle Flavin wrote: >>> >>>> Oh, really? Having the discovery plugin installed causes all the dupes >>>> to come back? Or just on new installs? >>>> >>>> On Saturday, December 19, 2015 at 8:34:26 AM UTC-8, Alyssa wrote: >>>>> >>>>> I tried something like that on a smaller scale, only to have it wiped >>>>> out next runs because I have the discovery plugin installed. >>>>> >>>>> On Fri, Dec 18, 2015 at 5:48 PM, Kyle Flavin >>>>> wrote: >>>>> >>>>>> Cool, thanks Greg, I'll have to see what was different. >>>>>> >>>>>> I ended up cleaning the database with a quick 'n dirty Python >>>>>> script. The script dumps down the host id + nic id to a file, where any >>>>>> non-primary NIC's IP or MAC matches the primary NIC. I then used a bash >>>>>> script (not included) to turn the id's into SQL statements in the form of: >>>>>> delete from nics where host_id='' and id=''; >>>>>> ... >>>>>> >>>>>> I ended up with about 10,000 records that needed to be removed. >>>>>> Nothing has imploded... yet.... and I'm able to modify the host records >>>>>> now. >>>>>> >>>>>> import requests >>>>>> import json >>>>>> from requests.auth import HTTPBasicAuth >>>>>> requests.packages.urllib3.disable_warnings() >>>>>> headers = {'Accept': 'version=2, application/json'} >>>>>> server = "http://" >>>>>> outfile = "./prod_dupe_ips" >>>>>> url = "%s/api/hosts" % server >>>>>> user = '' >>>>>> password = '' >>>>>> params = {'per_page': '8000'} >>>>>> hosts = json.loads(requests.get(url, auth=HTTPBasicAuth(user, >>>>>> password), headers=headers, verify=False, params=params).text) >>>>>> ids = {} >>>>>> for i in hosts['results']: >>>>>> ids["%s" % i['id']] = i['name'] >>>>>> >>>>>> count = 1 >>>>>> with open(outfile, "w") as f: >>>>>> for id,hostname in ids.items(): >>>>>> count += 1 >>>>>> print count, id, hostname >>>>>> url = "%s/api/hosts/%s/interfaces" % (server, id,) >>>>>> body = json.loads(requests.get(url, auth=HTTPBasicAuth(user, >>>>>> password), headers=headers, verify=False).text) >>>>>> interfaces = [] >>>>>> primary = "" >>>>>> for int in body['results']: >>>>>> if not int['primary']: >>>>>> interfaces.append(int) >>>>>> else: >>>>>> primary = int >>>>>> for i in interfaces: >>>>>> skip = 0 >>>>>> if i['ip'] == primary['ip']: >>>>>> f.write("%s %s IP match\n" % (id, i['id'],)) >>>>>> skip = 1 >>>>>> if not skip and i['mac'] == primary['mac']: >>>>>> f.write("%s %s MAC match\n" % (id, i['id'],)) >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Friday, December 18, 2015 at 4:09:20 PM UTC-8, Alyssa wrote: >>>>>>> >>>>>>> Nice, Greg! Thanks for the tip. :) >>>>>>> >>>>>>> On Fri, Dec 18, 2015 at 2:50 PM, Greg Sutcliffe < >>>>>>> greg.su...@gmail.com> wrote: >>>>>>> >>>>>>>> On 18 December 2015 at 22:32, Kyle Flavin >>>>>>>> wrote: >>>>>>>> > Hammer worked. Interesting, I wonder why it didn't work when I >>>>>>>> manually >>>>>>>> > call the API. >>>>>>>> >>>>>>>> IIRC the debug flag to hammer will show the exact API call it makes >>>>>>>> - >>>>>>>> might be interesting for a comparison. >>>>>>>> >>>>>>>> Greg >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to a topic in >>>>>>>> the Google Groups "Foreman users" group. >>>>>>>> To unsubscribe from this topic, visit >>>>>>>> https://groups.google.com/d/topic/foreman-users/fL-oTiRjHy0/unsubscribe >>>>>>>> . >>>>>>>> To unsubscribe from this group and all its topics, send an email to >>>>>>>> foreman-user...@googlegroups.com. >>>>>>>> To post to this group, send email to forema...@googlegroups.com. >>>>>>>> Visit this group at https://groups.google.com/group/foreman-users. >>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>> >>>>>>> >>>>>>> -- >>>>>> You received this message because you are subscribed to a topic in >>>>>> the Google Groups "Foreman users" group. >>>>>> To unsubscribe from this topic, visit >>>>>> https://groups.google.com/d/topic/foreman-users/fL-oTiRjHy0/unsubscribe >>>>>> . >>>>>> To unsubscribe from this group and all its topics, send an email to >>>>>> foreman-user...@googlegroups.com. >>>>>> To post to this group, send email to forema...@googlegroups.com. >>>>>> Visit this group at https://groups.google.com/group/foreman-users. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> -- >>>> You received this message because you are subscribed to a topic in the >>>> Google Groups "Foreman users" group. >>>> To unsubscribe from this topic, visit >>>> https://groups.google.com/d/topic/foreman-users/fL-oTiRjHy0/unsubscribe >>>> . >>>> To unsubscribe from this group and all its topics, send an email to >>>> foreman-user...@googlegroups.com. >>>> To post to this group, send email to forema...@googlegroups.com. >>>> Visit this group at https://groups.google.com/group/foreman-users. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "Foreman users" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/foreman-users/fL-oTiRjHy0/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> foreman-user...@googlegroups.com . >> To post to this group, send email to forema...@googlegroups.com >> . >> Visit this group at https://groups.google.com/group/foreman-users. >> For more options, visit https://groups.google.com/d/optout. >> > >