Load inherited parameters on class import?

Support
1

Hi all,

I'm running into the issue (mentioned on the wiki) where on class import,
Foreman interprets inherited class parameters as strings with values that
look similar to "${$original::param::name}". Is there no way to configure
foreman to interpolate these values on class import?

Although it might be easy enough to simply set the defaults in both the
Puppet modules and in the web UI one time around, there are several
maintenance issues here:

  • Changes to the class defaults require changes in two places (the
    module and in Foreman). This isn't DRY and can lead to error.
  • Although it may be simple enough to set the defaults in the foreman UI
    if you're managing a single Foreman instance, if you have multiple Foreman
    instances (as I do in my environment, due to my org's support of several
    disconnected, private LANs), the maintenance issues become worse.
  • This may just be ignorance on my part, but with the current
    implementation, I don't have a great way to detect where Foreman is placing
    these "${$parameter}" values except manual inspection of each class.
  • I've had some issues doing class reimports after changing the data
    type of a parameter from its loaded value of "string" ("Validation failed:
    Default value is invalid").
  • If I don't override and set a default (which would mean foreman isn't
    managing the parameter, thus the default I've already defined in the class
    is used) then I can't do overrides at the host level.

I appreciate any advice you can give me !

Thanks,
Jon

2

> Hi all,
>
> I'm running into the issue (mentioned on the wiki) where on class
> import, Foreman interprets inherited class parameters as strings with
> values that look similar to "${$original::param::name}". Is there no way
> to configure foreman to interpolate these values on class import?
>
> Although it might be easy enough to simply set the defaults in both the
> Puppet modules and in the web UI one time around, there are several
> maintenance issues here:
>
> * Changes to the class defaults require changes in two places (the
> module and in Foreman). This isn't DRY and can lead to error.

Yes, this is a problem. The reason it happens is because we use a
parser on the class, so it can only provide the data written in the
manifest - to interpolate the value would be impossible without the
context of a host to run it on, since values can change based on facts.

> * Although it may be simple enough to set the defaults in the foreman
> UI if you're managing a single Foreman instance, if you have
> multiple Foreman instances (as I do in my environment, due to my
> org's support of several disconnected, private LANs), the
> maintenance issues become worse.

Synchronising data between instances has been mentioned by people
before, would some sort of continuous import/export mechanism solve this
for you?

> * This may just be ignorance on my part, but with the current
> implementation, I don't have a great way to detect where Foreman is
> placing these "${$parameter}" values except manual inspection of
> each class.
> * I've had some issues doing class reimports after changing the data
> type of a parameter from its loaded value of "string" ("Validation
> failed: Default value is invalid").

I've seen this reported in IRC before, but I didn't know what caused it.
It sounds like you've figured out how to reproduce it, would you mind
filing a bug with the steps?

http://projects.theforeman.org/projects/foreman/issues/new

> * If I don't override and set a default (which would mean foreman
> isn't managing the parameter, thus the default I've already defined
> in the class is used) then I can't do overrides at the host level.

So the default is there so Foreman can always supply the parameter to
Puppet via the ENC. Perhaps we need an additional setting on a
parameter so the default is actually for Foreman not to supply the
parameter and instead, let Puppet resolve it or error if appropriate.

··· On 03/07/13 22:53, jcmcken@gmail.com wrote:

–
Dominic Cleal
Red Hat Engineering

3

Thanks Dominic,

Per your first point (and sorry to speculate without providing code), but
would it be possible to have a "variable" type that has no particular type
validation? The ENC routines could fetch the node's last known fact scope,
inject all of the usual data, then run a variable interpolation pass as a
last step (Hiera does a similar thing, I believe).

On your question about import/export, I think that would definitely be
handy! I understand it's a bit of a difficult (well, annoying) problem, if
you have hosts relying on module set v.1, and you try to import v.2 which
changes the API in some way, how do you rectify the differences in a sane
way? Maybe Foreman can implement some sort of MVCC for class imports
(although I suspect this is something Puppet itself would have to implement
first, otherwise Foreman would have to take a much more active role in
managing your Puppet code).

As to the reimport issue, when I get into work tomorrow (or Monday), I'll
try to get a good baseline for reproducing it. I believe the issue arose
when I enabled override on an inherited parameter, changed its value,
changed its type from string to something else, disabled the override, and
then later tried to reimport after I made a class change. It almost looked
like it was still performing the type validation during import despite the
override being disabled. I do remember that it was this line
(https://github.com/theforeman/foreman/blob/develop/app/models/puppet_class_importer.rb#L210)
throwing the validation error. To fix, I had to reenable override and
convert the parameter back to a string.

Thanks for the help

  • Jon
··· On Wednesday, July 3, 2013 5:53:24 PM UTC-4, jcm...@gmail.com wrote: > > Hi all, > > I'm running into the issue (mentioned on the wiki) where on class import, > Foreman interprets inherited class parameters as strings with values that > look similar to "${$original::param::name}". Is there no way to configure > foreman to interpolate these values on class import? > > Although it might be easy enough to simply set the defaults in both the > Puppet modules and in the web UI one time around, there are several > maintenance issues here: > > > - Changes to the class defaults require changes in two places (the > module and in Foreman). This isn't DRY and can lead to error. > - Although it may be simple enough to set the defaults in the foreman > UI if you're managing a single Foreman instance, if you have multiple > Foreman instances (as I do in my environment, due to my org's support of > several disconnected, private LANs), the maintenance issues become worse. > - This may just be ignorance on my part, but with the current > implementation, I don't have a great way to detect where Foreman is placing > these "${$parameter}" values except manual inspection of each class. > - I've had some issues doing class reimports after changing the data > type of a parameter from its loaded value of "string" ("Validation failed: > Default value is invalid"). > - If I don't override and set a default (which would mean foreman > isn't managing the parameter, thus the default I've already defined in the > class is used) then I can't do overrides at the host level. > > I appreciate any advice you can give me ! > > Thanks, > Jon
4

Well, I'm not sure what happened, but my account in your Redmine is locked.
Successfully reset password, but it's giving me an "Invalid user or
password" error. Email is the same I'm posting with

··· On Wednesday, July 3, 2013 5:53:24 PM UTC-4, jcm...@gmail.com wrote: > > Hi all, > > I'm running into the issue (mentioned on the wiki) where on class import, > Foreman interprets inherited class parameters as strings with values that > look similar to "${$original::param::name}". Is there no way to configure > foreman to interpolate these values on class import? > > Although it might be easy enough to simply set the defaults in both the > Puppet modules and in the web UI one time around, there are several > maintenance issues here: > > > - Changes to the class defaults require changes in two places (the > module and in Foreman). This isn't DRY and can lead to error. > - Although it may be simple enough to set the defaults in the foreman > UI if you're managing a single Foreman instance, if you have multiple > Foreman instances (as I do in my environment, due to my org's support of > several disconnected, private LANs), the maintenance issues become worse. > - This may just be ignorance on my part, but with the current > implementation, I don't have a great way to detect where Foreman is placing > these "${$parameter}" values except manual inspection of each class. > - I've had some issues doing class reimports after changing the data > type of a parameter from its loaded value of "string" ("Validation failed: > Default value is invalid"). > - If I don't override and set a default (which would mean foreman > isn't managing the parameter, thus the default I've already defined in the > class is used) then I can't do overrides at the host level. > > I appreciate any advice you can give me ! > > Thanks, > Jon
5

> Thanks Dominic,
>
> Per your first point (and sorry to speculate without providing code),
> but would it be possible to have a "variable" type that has no
> particular type validation? The ENC routines could fetch the node's last
> known fact scope, inject all of the usual data, then run a variable
> interpolation pass as a last step (Hiera does a similar thing, I believe).

Greg's done something similar to allow ERB template syntax in parameter
values, but it doesn't parse Puppet syntax. It also wouldn't be able to
evaluate something like $foreman::params::foo without evaluating the
classes, which needs facts. It would get far too complex I think, you'd
have to compile an entire catalog just to work out a few values.

> On your question about import/export, I think that would definitely be
> handy! I understand it's a bit of a difficult (well, annoying) problem,
> if you have hosts relying on module set v.1, and you try to import v.2
> which changes the API in some way, how do you rectify the differences in
> a sane way? Maybe Foreman can implement some sort of MVCC for class
> imports (although I suspect this is something Puppet itself would have
> to implement first, otherwise Foreman would have to take a much more
> active role in managing your Puppet code).
>
> As to the reimport issue, when I get into work tomorrow (or Monday),
> I'll try to get a good baseline for reproducing it. I believe the issue
> arose when I enabled override on an inherited parameter, changed its
> value, changed its type from string to something else, disabled the
> override, and then later tried to reimport after I made a class change.
> It almost looked like it was still performing the type validation during
> import despite the override being disabled. I do remember that it was
> this line
> (https://github.com/theforeman/foreman/blob/develop/app/models/puppet_class_importer.rb#L210)
> throwing the validation error. To fix, I had to reenable override and
> convert the parameter back to a string.

Ah, that process makes sense, I think I see why that's happening.
Please file a bug and we can get that fixed.

Cheers,

··· On 05/07/13 01:07, jcmcken@gmail.com wrote:

–
Dominic Cleal
Red Hat Engineering

6

Try again now, the account hadn't been activated (e-mail was probably
swallowed by gmail's spam filters). I've activated it.

··· -- Dominic Cleal Red Hat Engineering

On 05/07/13 15:55, jcmcken@gmail.com wrote:

Well, I’m not sure what happened, but my account in your Redmine is
locked. Successfully reset password, but it’s giving me an “Invalid user
or password” error. Email is the same I’m posting with

On Wednesday, July 3, 2013 5:53:24 PM UTC-4, jcm...@gmail.com wrote:

Hi all,

I'm running into the issue (mentioned on the wiki) where on class
import, Foreman interprets inherited class parameters as strings
with values that look similar to "${$original::param::name}". Is
there no way to configure foreman to interpolate these values on
class import?

Although it might be easy enough to simply set the defaults in both
the Puppet modules and in the web UI one time around, there are
several maintenance issues here:

  * Changes to the class defaults require changes in two places (the
    module and in Foreman). This isn't DRY and can lead to error.
  * Although it may be simple enough to set the defaults in the
    foreman UI if you're managing a single Foreman instance, if you
    have multiple Foreman instances (as I do in my environment, due
    to my org's support of several disconnected, private LANs), the
    maintenance issues become worse.
  * This may just be ignorance on my part, but with the current
    implementation, I don't have a great way to detect where Foreman
    is placing these "${$parameter}" values except manual inspection
    of each class.
  * I've had some issues doing class reimports after changing the
    data type of a parameter from its loaded value of "string"
    ("Validation failed: Default value is invalid").
  * If I don't override and set a default (which would mean foreman
    isn't managing the parameter, thus the default I've already
    defined in the class is used) then I can't do overrides at the
    host level.

I appreciate any advice you can give me !

Thanks,
Jon

–
You received this message because you are subscribed to the Google
Groups “Foreman users” group.
To unsubscribe from this group and stop receiving emails from it, send
an email to foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at http://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/groups/opt_out.

7

Thanks -

··· On Friday, July 5, 2013 10:59:31 AM UTC-4, Dominic Cleal wrote: > > Try again now, the account hadn't been activated (e-mail was probably > swallowed by gmail's spam filters). I've activated it. > > -- > Dominic Cleal > Red Hat Engineering > > On 05/07/13 15:55, jcm...@gmail.com wrote: > > Well, I'm not sure what happened, but my account in your Redmine is > > locked. Successfully reset password, but it's giving me an "Invalid user > > or password" error. Email is the same I'm posting with > > > > On Wednesday, July 3, 2013 5:53:24 PM UTC-4, jcm...@gmail.com wrote: > > > > Hi all, > > > > I'm running into the issue (mentioned on the wiki) where on class > > import, Foreman interprets inherited class parameters as strings > > with values that look similar to "${$original::param::name}". Is > > there no way to configure foreman to interpolate these values on > > class import? > > > > Although it might be easy enough to simply set the defaults in both > > the Puppet modules and in the web UI one time around, there are > > several maintenance issues here: > > > > * Changes to the class defaults require changes in two places (the > > module and in Foreman). This isn't DRY and can lead to error. > > * Although it may be simple enough to set the defaults in the > > foreman UI if you're managing a single Foreman instance, if you > > have multiple Foreman instances (as I do in my environment, due > > to my org's support of several disconnected, private LANs), the > > maintenance issues become worse. > > * This may just be ignorance on my part, but with the current > > implementation, I don't have a great way to detect where Foreman > > is placing these "${$parameter}" values except manual inspection > > of each class. > > * I've had some issues doing class reimports after changing the > > data type of a parameter from its loaded value of "string" > > ("Validation failed: Default value is invalid"). > > * If I don't override and set a default (which would mean foreman > > isn't managing the parameter, thus the default I've already > > defined in the class is used) then I can't do overrides at the > > host level. > > > > I appreciate any advice you can give me ! > > > > Thanks, > > Jon > > > > -- > > You received this message because you are subscribed to the Google > > Groups "Foreman users" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to foreman-user...@googlegroups.com . > > To post to this group, send email to forema...@googlegroups.com. > > > Visit this group at http://groups.google.com/group/foreman-users. > > For more options, visit https://groups.google.com/groups/opt_out. > > > > > >
8

Hi,

I'm having the same issue with importing the parameters value at the moment.

What is most wise at the moment ? Change your manifests or just live with
it ?

What have you done wtih this Jon ?

Cheers,

Matt

··· Op zaterdag 6 juli 2013 04:11:05 UTC+2 schreef jcm...@gmail.com: > > Thanks - > > On Friday, July 5, 2013 10:59:31 AM UTC-4, Dominic Cleal wrote: >> >> Try again now, the account hadn't been activated (e-mail was probably >> swallowed by gmail's spam filters). I've activated it. >> >> -- >> Dominic Cleal >> Red Hat Engineering >> >> On 05/07/13 15:55, jcm...@gmail.com wrote: >> > Well, I'm not sure what happened, but my account in your Redmine is >> > locked. Successfully reset password, but it's giving me an "Invalid >> user >> > or password" error. Email is the same I'm posting with >> > >> > On Wednesday, July 3, 2013 5:53:24 PM UTC-4, jcm...@gmail.com wrote: >> > >> > Hi all, >> > >> > I'm running into the issue (mentioned on the wiki) where on class >> > import, Foreman interprets inherited class parameters as strings >> > with values that look similar to "${$original::param::name}". Is >> > there no way to configure foreman to interpolate these values on >> > class import? >> > >> > Although it might be easy enough to simply set the defaults in both >> > the Puppet modules and in the web UI one time around, there are >> > several maintenance issues here: >> > >> > * Changes to the class defaults require changes in two places >> (the >> > module and in Foreman). This isn't DRY and can lead to error. >> > * Although it may be simple enough to set the defaults in the >> > foreman UI if you're managing a single Foreman instance, if you >> > have multiple Foreman instances (as I do in my environment, due >> > to my org's support of several disconnected, private LANs), the >> > maintenance issues become worse. >> > * This may just be ignorance on my part, but with the current >> > implementation, I don't have a great way to detect where >> Foreman >> > is placing these "${$parameter}" values except manual >> inspection >> > of each class. >> > * I've had some issues doing class reimports after changing the >> > data type of a parameter from its loaded value of "string" >> > ("Validation failed: Default value is invalid"). >> > * If I don't override and set a default (which would mean foreman >> > isn't managing the parameter, thus the default I've already >> > defined in the class is used) then I can't do overrides at the >> > host level. >> > >> > I appreciate any advice you can give me ! >> > >> > Thanks, >> > Jon >> > >> > -- >> > You received this message because you are subscribed to the Google >> > Groups "Foreman users" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> > an email to foreman-user...@googlegroups.com. >> > To post to this group, send email to forema...@googlegroups.com. >> > Visit this group at http://groups.google.com/group/foreman-users. >> > For more options, visit https://groups.google.com/groups/opt_out. >> > >> > >> >>