FQDN of Puppet Agents gets lost and hosts can't be edited

Problem:

Recently, some of our hosts started to show up without their domain in the hosts list. On there nodes, I noticed, that the environment variable “fqdn” does not work on puppet runs anymore. I asked in the Puppet Support and modified our modules to use facts[fqdn] instead, which works.
With some other nodes, this seems to lead to the following error in Foreman, when trying to edit a hosts parameters or add classes:

**Oops, we're sorry but something went wrong** PG::UniqueViolation: ERROR: duplicate key value violates unique constraint "index_lookup_values_on_lookup_key_id_and_match" DETAIL: Key (lookup_key_id, match)=(132, fqdn=ibs-12-au) already exists.

Expected outcome:

I expected to still get the FQDN on puppet runs and that I could edit my hosts without running into this error.

Foreman and Proxy versions:

Tested on 2.4 and 2.5

Distribution and version:

Foreman-Server: Ubuntu 18.04
Nodes: Ubuntu 20.04, Puppet Agent 6.22.1

Other relevant data:

[I|app|57b947e2] Backtrace for 'Rolling back due to exception during save' error (ActiveRecord::RecordNotUnique): PG::UniqueViolation: ERROR: duplicate key value violates unique constraint "index_lookup_values_on_lookup_key_id_and_match" 57b947e2 | DETAIL: Key (lookup_key_id, match)=(132, fqdn=ibs-12-au) already exists.

I’m not sure, what has changed and don’t fully understand your description. But hopefulluly it helps further debugging that the Foreman error message basically means, you’re trying to set a matcher value (smart class parameter value) for the parameter, for the fqdn that already has a value. Meaning you are setting the value for a single host for the second time. Which fails, since each param can have a value defined on the host level just once.

Thank you for your reply!
I checked the list of Smart Class Parameters and there is no parameter “fqdn”.
Is there another place where parameters like this one could get defined on the host level?

Additionally, I am not sure, if this error is even connected to the missing “$fqdn” variable on Puppet runs. I am just assuming this, as it’s both referring to the “fqdn”.
Right now, I am not able to change any configuration or add/remove classes to these nodes.

Is there a way to find useful information in the foreman postgresql database or where could I search for further information?

Well, I found a solution to my problem:

We are using a git project to store our Puppet modules. Our Foreman instance checks this out and uses “r10k” to create environments from the git branches in the project. We use this to seperate our different deployments, like testing, rc, … .
A few weeks ago, one of our developers created a feature branch “feature-branch”, another one merged it into our master and did not delete the source branch.
r10k created an environment directory for “feature-branch” with the handful modules.
One of our guys then imported the new modules in our Foreman instance and mistakenly created a new environment “feature-branch”.
About a week later, I noticed the “feature-branch” and deleted it from the git project. I’m not sure, if I did the “Import from Foreman” again, to delete the (now missing) modules.
I guess, that this was the source of my problems.
Today I deleted the “feature-branch” environment from Foreman and immediately it worked again. Even the FQDN is working as before.

1 Like