Best way to set variable on something that hasn't been parameterized

Support
1

I'm trying to use https://github.com/elasticsearch/puppet-logstashforwarder
to manage the files that are being pushed to my logstash server but that
module has logstashforwarder::file (see
https://github.com/elasticsearch/puppet-logstashforwarder#files) as a
define and not a class so it isn't shown in Foreman. I was hoping that I'd
able to create a smart variable and set the content as yaml to something
like:

paths:
  - /var/log/messages
fields:
  type: log

but that results in a global setting:

parameters:
  ...
  logstashforwarder::file:
    paths:

      - /var/log/messages

    fields:
      type: log

and the "files": [ ] section in the resulting config.json is empty.

This may have something to do with the fact that after creating the variable, despite my assigning the class to logstashforwarder it appears empty in the list of variables. Nonetheless, my question is what's the best way to do what I'm trying? I also tried to create a corresponding class variable:

class logstashforwarder::files {

  create_resources('logstashforwarder::file', $paths, $fields)

}

but that didn't work either.

Thanks,

Geoff

2

> I'm trying to use https://github.com/elasticsearch/puppet-logstashforwarder
> to manage the files that are being pushed to my logstash server but that
> module has logstashforwarder::file (see
> https://github.com/elasticsearch/puppet-logstashforwarder#files) as a define
> and not a class so it isn't shown in Foreman. I was hoping that I'd able to
> create a smart variable and set the content as yaml to something like:
>
> paths:
> - /var/log/messages
> fields:
> type: log
>
> but that results in a global setting:

That's how smart variables work - they always end up in the parameters
section of the ENC. They're not widely used these days; they were our
first attempt to add some form of configurable parameters before
proper class parameter support was added.

> This may have something to do with the fact that after creating the
> variable, despite my assigning the class to logstashforwarder it appears
> empty in the list of variables. Nonetheless, my question is what's the best
> way to do what I'm trying? I also tried to create a corresponding class
> variable:
>
> class logstashforwarder::files {
>
> create_resources('logstashforwarder::file', $paths, $fields)
>
> }

You're almost there :). You want to pass through your hash of data
as a class parameter to the wrapper class. In other words:

class foo ($bar={}) {
create_resources(foo::quux,$bar)
}

Importing this class to Foreman will allow you to set $bar in Foreman
(or Hiera, etc) and have it call the define. See
http://projects.theforeman.org/projects/foreman/wiki/Instantiate_Puppet_resources
for more details.

Greg

··· On 10 March 2015 at 22:46, Geoff Johnson wrote:
3

Thanks for the link, that's helpful. After making a couple of changes I now
have this in my resulting yaml:

files:
  messages:
    paths:
    - /var/log/messages
    fields:
      type: messages

but I think what I need is:

logstashforwarder::file:
  paths:
  - /var/log/messages
  fields:
    type: messages

of course I can't call my wrapper class logstashforwarder::file because
that already exists as a define. This results in the following error by the
agent:

Error: Could not retrieve catalog from remote server: Error 400 on

SERVER: Could not find class file for client.domain.net on node
client.domain.net

What I'm using in the file.pp manifest is:

class file($paths = {}, $fields = {}) {
  create_resources(logstashforwarder::file, $paths, $fields)
}

I don't suppose there's a way to force the output for a specific class for
testing? Or, and potentially better, is there a standard/good way that I
can add a file in my system that will be picked up as partial configuration
after the foreman config gets applied. The module is meant to be used with
puppet so I assume that you could just manage it that way directly, but I
haven't had to do this since switching to foreman so I don't know if that's
something that should be done.

Thanks again,
Geoff

··· On Wednesday, 11 March 2015 04:12:26 UTC-7, Greg Sutcliffe wrote: > > On 10 March 2015 at 22:46, Geoff Johnson > wrote: > > I'm trying to use > https://github.com/elasticsearch/puppet-logstashforwarder > > to manage the files that are being pushed to my logstash server but that > > module has logstashforwarder::file (see > > https://github.com/elasticsearch/puppet-logstashforwarder#files) as a > define > > and not a class so it isn't shown in Foreman. I was hoping that I'd able > to > > create a smart variable and set the content as yaml to something like: > > > > paths: > > - /var/log/messages > > fields: > > type: log > > > > but that results in a global setting: > > That's how smart variables work - they always end up in the parameters > section of the ENC. They're not widely used these days; they were our > first attempt to add some form of configurable parameters before > proper class parameter support was added. > > > This may have something to do with the fact that after creating the > > variable, despite my assigning the class to logstashforwarder it appears > > empty in the list of variables. Nonetheless, my question is what's the > best > > way to do what I'm trying? I also tried to create a corresponding class > > variable: > > > > class logstashforwarder::files { > > > > create_resources('logstashforwarder::file', $paths, $fields) > > > > } > > You're *almost* there :). You want to pass through your hash of data > as a class parameter to the wrapper class. In other words: > > class foo ($bar={}) { > create_resources(foo::quux,$bar) > } > > Importing this class to Foreman will allow you to set $bar in Foreman > (or Hiera, etc) and have it call the define. See > > http://projects.theforeman.org/projects/foreman/wiki/Instantiate_Puppet_resources > for more details. > > Greg >
4

Hmm, I'm not sure why I never thought to look into using hiera before, but
I read your reply again and saw that mentioned so I went and looked into
it. Apparently that and the manifests/site.pp is exactly what I want. I'm
still very fresh with puppet so while this may seem old hat for most it
wasn't for me.

Thanks again,
Geoff

··· On Wednesday, 11 March 2015 12:05:11 UTC-7, Geoff Johnson wrote: > > Thanks for the link, that's helpful. After making a couple of changes I > now have this in my resulting yaml: > > files: > messages: > paths: > - /var/log/messages > fields: > type: messages > > > but I think what I need is: > > logstashforwarder::file: > paths: > - /var/log/messages > fields: > type: messages > > > of course I can't call my wrapper class logstashforwarder::file because > that already exists as a define. This results in the following error by the > agent: > > Error: Could not retrieve catalog from remote server: Error 400 on > SERVER: Could not find class file for client.domain.net on node > client.domain.net > > What I'm using in the file.pp manifest is: > > class file($paths = {}, $fields = {}) { > create_resources(logstashforwarder::file, $paths, $fields) > } > > I don't suppose there's a way to force the output for a specific class for > testing? Or, and potentially better, is there a standard/good way that I > can add a file in my system that will be picked up as partial configuration > after the foreman config gets applied. The module is meant to be used with > puppet so I assume that you could just manage it that way directly, but I > haven't had to do this since switching to foreman so I don't know if that's > something that should be done. > > Thanks again, > Geoff > > On Wednesday, 11 March 2015 04:12:26 UTC-7, Greg Sutcliffe wrote: >> >> On 10 March 2015 at 22:46, Geoff Johnson wrote: >> > I'm trying to use >> https://github.com/elasticsearch/puppet-logstashforwarder >> > to manage the files that are being pushed to my logstash server but >> that >> > module has logstashforwarder::file (see >> > https://github.com/elasticsearch/puppet-logstashforwarder#files) as a >> define >> > and not a class so it isn't shown in Foreman. I was hoping that I'd >> able to >> > create a smart variable and set the content as yaml to something like: >> > >> > paths: >> > - /var/log/messages >> > fields: >> > type: log >> > >> > but that results in a global setting: >> >> That's how smart variables work - they always end up in the parameters >> section of the ENC. They're not widely used these days; they were our >> first attempt to add some form of configurable parameters before >> proper class parameter support was added. >> >> > This may have something to do with the fact that after creating the >> > variable, despite my assigning the class to logstashforwarder it >> appears >> > empty in the list of variables. Nonetheless, my question is what's the >> best >> > way to do what I'm trying? I also tried to create a corresponding class >> > variable: >> > >> > class logstashforwarder::files { >> > >> > create_resources('logstashforwarder::file', $paths, $fields) >> > >> > } >> >> You're *almost* there :). You want to pass through your hash of data >> as a class parameter to the wrapper class. In other words: >> >> class foo ($bar={}) { >> create_resources(foo::quux,$bar) >> } >> >> Importing this class to Foreman will allow you to set $bar in Foreman >> (or Hiera, etc) and have it call the define. See >> >> http://projects.theforeman.org/projects/foreman/wiki/Instantiate_Puppet_resources >> for more details. >> >> Greg >> >
5

If Hiera works for you, go ahead. Foreman won't mind either way :slight_smile:

Do be careful with site.pp - while you can mix it with an ENC (which
Foreman is), there are gotchas. In general, resource defaults (such as
File, Package etc, note the capitalizaton) is fine, avoid using node{}
blocks in site.pp unless you know what you're doing :wink:

As for your code - create_resources only accepts a single hash; the
3rd argument of create_resources is just a set of defaults, so I don't
think what you have there can work. I can't see your puppet code to be
sure, but I suspect what you want in the ENC is

classes:
mywrapper:
logstashstuff:
fields:
- foo
paths:
- bar

and then code like

class mywrapper($logstashstuff={}) {
create_resources(logstashforwarder::file, $logstashsuff)
}

Cheers,
Greg

··· On 11 March 2015 at 20:13, Geoff Johnson wrote: > Hmm, I'm not sure why I never thought to look into using hiera before, but I > read your reply again and saw that mentioned so I went and looked into it. > Apparently that and the manifests/site.pp is exactly what I want. I'm still > very fresh with puppet so while this may seem old hat for most it wasn't for > me.
6

I found out quickly that adding a node{} to site.pp broke everything
because I don't know what I'm doing, what I did manage to get working
though is an include in the site file and then node/* files. The only
unfortunate thing is that there needs to be a match for every node
otherwise the agent borks. That isn't ideal but it can be worked around
easily enough.

I'm likely to come back to this and try again in the future but for now I
think the mixed setup will suffice.

Cheers,
Geoff

··· On Wednesday, 11 March 2015 17:16:11 UTC-7, Greg Sutcliffe wrote: > > On 11 March 2015 at 20:13, Geoff Johnson > wrote: > > Hmm, I'm not sure why I never thought to look into using hiera before, > but I > > read your reply again and saw that mentioned so I went and looked into > it. > > Apparently that and the manifests/site.pp is exactly what I want. I'm > still > > very fresh with puppet so while this may seem old hat for most it wasn't > for > > me. > > If Hiera works for you, go ahead. Foreman won't mind either way :) > > Do be careful with site.pp - while you can mix it with an ENC (which > Foreman is), there are gotchas. In general, resource defaults (such as > File, Package etc, note the capitalizaton) is fine, avoid using node{} > blocks in site.pp unless you know what you're doing ;) > > As for your code - create_resources only accepts a single hash; the > 3rd argument of create_resources is just a set of defaults, so I don't > think what you have there can work. I can't see your puppet code to be > sure, but I suspect what you want in the ENC is > > classes: > mywrapper: > logstashstuff: > fields: > - foo > paths: > - bar > > and then code like > > class mywrapper($logstashstuff={}) { > create_resources(logstashforwarder::file, $logstashsuff) > } > > Cheers, > Greg >
7

A simple "node default {}" declaration should be sufficient to handle
machines you haven't declared explicitly, just for info :slight_smile:

Greg