Documentation for Reporting

Support
1

Hello Group!

I have installed and am using foreman (with puppet) on linux servers.
The puppetmaster is installed on a Linux Debian server as well. I have
installed according to the documentation by running "foreman-installer",
which worked great!
However, that has the effect that I don't understand all the internals,
since it all got up set automatically. I am especially curious about the
exact interaction with puppet…

Well, now I am trying to track down an issue, and that missing knowledge
about the internals is screwing that up.
I have the effect that for some changes (but not all of them!) I am only
getting rudimentary reports. It's dependent on the change, not on the node
it gets deployed on. All The Info Level stuff is missing, and the notice
which contains the diff. I do get the notice about the old and new file md5
sums though.

Now I'd like to dive into the configuration and log files to figure out
what is going wrong. But I don't even find a proper starting point:
/var/log/puppet/masterhttp.log is empty, puppet has no other log files
/var/log/foreman/production.log is very detailed, but I can not see any
problems reported when such a report happens. Log entries:

2016-06-21T08:36:09 [app] [I] Started POST "/api/reports" for <ip> at
2016-06-21 08:36:09 +0200
2016-06-21T08:36:09 [app] [I] Processing by
Api::V2::ReportsController#create as JSON
2016-06-21T08:36:09 [app] [I] Parameters: {"report"=>"[FILTERED]",
"apiv"=>"v2"}
2016-06-21T08:36:09 [app] [I] processing report for <host>
2016-06-21T08:36:09 [app] [I] Imported report for <host> in 0.04 seconds

This all looks fine to me. But what does foreman actually DO? How is this
structured, where are the reports saved on disk, which configuration files
are used, …
I have the feeling I am completely casting about in the dark here.

So, is there any documentation that explains how this stuff works? The
foreman documentation doesn't lose a single word about it as far as I can
find. And I have no idea how much puppet documentation is applicable.
I read that reports should be stored in /var/lib/puppet/reports/. (vardir
appears to be set to /var/lib/puppet, according to /etc/puppet/puppet.conf)
But there are only reports from the time of setting the server up in there,
so it's apparently not used.
"/etc/init.d/puppetmaster status" says that puppetmaster is not running,
though everything is working fine, which is just an example that I have no
idea how this stuff is organized and set up. :frowning:

So, can someone throw me a bone here and point me to documentation
explaining the big picture here? Either that, or try to answer the concrete
questions I have asked.

2

> Well, now I am trying to track down an issue, and that missing knowledge
> about the internals is screwing that up.
> I have the effect that for some changes (but not all of them!) I am only
> getting rudimentary reports. It's dependent on the change, not on the
> node it gets deployed on. All The Info Level stuff is missing, and the
> notice which contains the diff. I do get the notice about the old and
> new file md5 sums though.

Diffs are controlled by Puppet's show_diff configuration parameter:
https://docs.puppet.com/puppet/latest/reference/configuration.html#showdiff,
which does default to false as it can reveal sensitive information. This
has to be set on the agent.

I think other logging in reports is still controlled by log_level
(https://docs.puppet.com/puppet/latest/reference/configuration.html#loglevel).

> Now I'd like to dive into the configuration and log files to figure out
> what is going wrong. But I don't even find a proper starting point:
> /var/log/puppet/masterhttp.log is empty, puppet has no other log files

Puppet defaults to syslog
(https://docs.puppet.com/puppet/latest/reference/man/agent.html), which
will probably be /var/log/messages, /var/log/syslog, daemon.log or
similar depending on your config. You should find messages from
puppet-agent and puppet-master.

Puppet 4 with Puppet Server has its own log file under
/var/log/puppetlabs for the master.

> This all looks fine to me. But what does foreman actually DO? How is
> this structured, where are the reports saved on disk, which
> configuration files are used, …
> I have the feeling I am completely casting about in the dark here.

With reports, the agent generates all of the logs and diffs as it runs
through the catalog, which it uploads to the Puppet master at the very
end of the Puppet run. This is passed to the configured report
processors (foreman.rb) based on the "reports" setting in the master's
puppet.conf, which in this case uploads it to Foreman's API where it's
stored in the DB.

The report processor (foreman.rb,
Foreman :: Manual),
rewrites the internal Puppet report object into JSON and uploads it to
Foreman's API. This processor code is stored on the Puppet master.

Foreman is receiving a JSON upload of a report from the Puppet master,
and stores the report in its database, which if you used the installer
will be a local PostgreSQL DB. This is then used to re-display it in the
UI. It doesn't do much processing of the report here.

> So, is there any documentation that explains how this stuff works? The
> foreman documentation doesn't lose a single word about it as far as I
> can find. And I have no idea how much puppet documentation is applicable.
> I read that reports should be stored in /var/lib/puppet/reports/.
> (vardir appears to be set to /var/lib/puppet, according to
> /etc/puppet/puppet.conf) But there are only reports from the time of
> setting the server up in there, so it's apparently not used.

This depends on which report processors the master has configured in the
puppet.conf "reports"
(https://docs.puppet.com/puppet/latest/reference/configuration.html#reports).
Our installer only sets it to "foreman", but if "store" is listed too
then it will save a copy of the report to the master's vardir
(https://docs.puppet.com/puppet/latest/reference/report.html#store).
This adds up!

Enabling the store report processor though would show you what
information is contained in the raw report from the agent, so might be
worth trying.

> "/etc/init.d/puppetmaster status" says that puppetmaster is not running,
> though everything is working fine, which is just an example that I have
> no idea how this stuff is organized and set up. :frowning:

It's probably running under Apache (with Passenger), so restart httpd or
apache2.

Hope that helps.

··· On 23/06/16 11:10, Martin Gerdes wrote:

–
Dominic Cleal
dominic@cleal.org

3

Thanks a ton!

I have already tried out the "show_diff" option, and after modifying the
same file where I only got an md5sum before, I now get a propper diff! :slight_smile:
Unfortunately I also got a proper diff when I tried turning that option off
again, so I'll have to see whether I get that md5sum only effect again or
not.

But thank you also for the thorough explanation of how reporting in
foreman+puppet works! Even if my problem turns out not te be fixed, I now
know where to look for logs and raw report data.
I was completely unable to find any similar explanation before posting
here, so that information is extremely helpful.

Thank you, thank you, thank you! :smiley: