Upgrade to puppet 3.3.0: reports appear to be gone

I’m running foreman 1.2.2 and, up until I upgraded to puppet 3.3.0 from puppet 2.6, reports were going fine, both to Foreman and to Puppet Dashboard. Since upgrading to puppet 3.3.0, I no longer see reports in either, however the puppet nodes are running on-schedule and puppet seems to be running without any errors.

Has anyone else seen this behavior, or know how to debug the reporting system?

Many thanks,
Dan

··· Daniel Alex Finkelstein • Sr Dev Ops Engineer • SunGard Financial Systems • Kiodex • 59 Maiden Lane Fl 32, New York, NY 10038 • tel +1-646-445-1228 • cell +1-718-246-7421 • fax +1-212-682-5892 • daniel.finkelstein@sungard.com

[http://my.knowhow.sungard.com/_layouts/images/SunGard%20Signature/Signature1.png]http://www.sungard.com/ten

Join the online conversation with SunGard’s customers, partners and Industry experts and find an event near you at: www.sungard.com/tenhttp://www.sungard.com/ten.

P Think before you print

CONFIDENTIALITY: This e-mail (including any attachments) may contain confidential, proprietary and privileged information, and unauthorized disclosure or use is prohibited. If you received this e-mail in error, please notify the sender and delete this e-mail from your system.

Start with the foreman logs. You should requests coming in from the
puppetmaster to /reports/create (or in 1.3, /api/reports). If they are, see
if they're 404'ing for some reason. If not, check your puppetmaster
logs/config (it usually logs to syslog) for why it's not sending them.

Greg

··· On 28 September 2013 18:40, wrote:

I’m running foreman 1.2.2 and, up until I upgraded to puppet 3.3.0 from
puppet 2.6, reports were going fine, both to Foreman and to Puppet
Dashboard. Since upgrading to puppet 3.3.0, I no longer see reports in
either, however the puppet nodes are running on-schedule and puppet seems
to be running without any errors.****


Has anyone else seen this behavior, or know how to debug the reporting
system?

I had similar problem this week and found a bug report @ puppet labs on it.
Sorry, no link handy. I didn't have the time or patience to dive in, so I
did a yum downgrade on puppet to the Epel version, and set yum version lock
to keep it there.

I should be able to post my logs on Monday if it will help to compare.

  • Sean Alderman
    Senior Engineer, UDit Systems Integration

This message had been brought to you by Android Bionic.

··· On Sep 28, 2013 4:25 PM, "Greg Sutcliffe" wrote:

On 28 September 2013 18:40, Daniel.Finkelstein@sungard.com wrote:

I’m running foreman 1.2.2 and, up until I upgraded to puppet 3.3.0 from
puppet 2.6, reports were going fine, both to Foreman and to Puppet
Dashboard. Since upgrading to puppet 3.3.0, I no longer see reports in
either, however the puppet nodes are running on-schedule and puppet seems
to be running without any errors.****


Has anyone else seen this behavior, or know how to debug the reporting
system?

Start with the foreman logs. You should requests coming in from the
puppetmaster to /reports/create (or in 1.3, /api/reports). If they are, see
if they’re 404’ing for some reason. If not, check your puppetmaster
logs/config (it usually logs to syslog) for why it’s not sending them.

Greg


You received this message because you are subscribed to the Google Groups
"Foreman users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at http://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/groups/opt_out.

I found the same issues online, but even odder is that I'm getting no 404
errors. Just normal 200 status and no complaints at all from the nodes, yet
no updates on the Foreman or Puppet Dashboard servers.

··· On Saturday, September 28, 2013 6:08:04 PM UTC-4, Sean Alderman wrote: > > I had similar problem this week and found a bug report @ puppet labs on > it. Sorry, no link handy. I didn't have the time or patience to dive in, so > I did a yum downgrade on puppet to the Epel version, and set yum version > lock to keep it there. > > I should be able to post my logs on Monday if it will help to compare. > > - Sean Alderman > Senior Engineer, UDit Systems Integration > > This message had been brought to you by Android Bionic. > On Sep 28, 2013 4:25 PM, "Greg Sutcliffe" <greg.su...@gmail.com> > wrote: > >> On 28 September 2013 18:40, <Daniel.Fi...@sungard.com >wrote: >> >>> I’m running foreman 1.2.2 and, up until I upgraded to puppet 3.3.0 >>> from puppet 2.6, reports were going fine, both to Foreman and to Puppet >>> Dashboard. Since upgrading to puppet 3.3.0, I no longer see reports in >>> either, however the puppet nodes are running on-schedule and puppet seems >>> to be running without any errors.**** >>> >>> ** ** >>> >>> Has anyone else seen this behavior, or know how to debug the reporting >>> system? >>> >> >> Start with the foreman logs. You should requests coming in from the >> puppetmaster to /reports/create (or in 1.3, /api/reports). If they are, see >> if they're 404'ing for some reason. If not, check your puppetmaster >> logs/config (it usually logs to syslog) for why it's not sending them. >> >> Greg >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Foreman users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to foreman-user...@googlegroups.com . >> To post to this group, send email to forema...@googlegroups.com >> . >> Visit this group at http://groups.google.com/group/foreman-users. >> For more options, visit https://groups.google.com/groups/opt_out. >> >

Temporary solution was to downgrade puppet to 2.6.18, which makes
everything magically work again.

··· On Monday, September 30, 2013 10:44:56 AM UTC-4, Daniel Finkelstein wrote: > > I found the same issues online, but even odder is that I'm getting no 404 > errors. Just normal 200 status and no complaints at all from the nodes, yet > no updates on the Foreman or Puppet Dashboard servers. > > On Saturday, September 28, 2013 6:08:04 PM UTC-4, Sean Alderman wrote: >> >> I had similar problem this week and found a bug report @ puppet labs on >> it. Sorry, no link handy. I didn't have the time or patience to dive in, so >> I did a yum downgrade on puppet to the Epel version, and set yum version >> lock to keep it there. >> >> I should be able to post my logs on Monday if it will help to compare. >> >> - Sean Alderman >> Senior Engineer, UDit Systems Integration >> >> This message had been brought to you by Android Bionic. >> On Sep 28, 2013 4:25 PM, "Greg Sutcliffe" wrote: >> >>> On 28 September 2013 18:40, wrote: >>> >>>> I’m running foreman 1.2.2 and, up until I upgraded to puppet 3.3.0 >>>> from puppet 2.6, reports were going fine, both to Foreman and to Puppet >>>> Dashboard. Since upgrading to puppet 3.3.0, I no longer see reports in >>>> either, however the puppet nodes are running on-schedule and puppet seems >>>> to be running without any errors.**** >>>> >>>> ** ** >>>> >>>> Has anyone else seen this behavior, or know how to debug the reporting >>>> system? >>>> >>> >>> Start with the foreman logs. You should requests coming in from the >>> puppetmaster to /reports/create (or in 1.3, /api/reports). If they are, see >>> if they're 404'ing for some reason. If not, check your puppetmaster >>> logs/config (it usually logs to syslog) for why it's not sending them. >>> >>> Greg >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Foreman users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to foreman-user...@googlegroups.com. >>> To post to this group, send email to forema...@googlegroups.com. >>> Visit this group at http://groups.google.com/group/foreman-users. >>> For more options, visit https://groups.google.com/groups/opt_out. >>> >>

I've splunked back into my logs, and only found the following messages like
these on the client side (for agents with versions 2.6.18, 2.7.14, and
2.7.21):

Sep 23 12:59:01 host puppet-agent[2205]: Could not send report: Error 406 on SERVER: Not Acceptable: No supported formats are acceptable (Accept: b64_zlib_yaml, yaml, raw)
>
> Sep 23 12:59:01 host puppet-agent[2205]: Finished catalog run in 0.93 seconds
>
>
The server side log stream is whacky. The time stamps don't match up
between the apache and foreman logs…

/var/log/httpd/access_log:

> 10.0.0.82 - - [23/Sep/2013:12:58:55 -0400] "GET
> /default/node/host.example.edu? HTTP/1.1" 200 4470 "-" "-"
> 10.0.0.82 - - [23/Sep/2013:12:58:56 -0400] "GET
> /production/file_metadatas/plugins?recurse=true&links=manage&ignore=—+%0A+±+%22.svn%22%0A+±+CVS%0A+±+%22.git%22&checksum_type=md5&
> HTTP/1.1" 200 45991 "-" "-"
> 10.0.0.82 - - [23/Sep/2013:12:58:58 -0400] "POST
> /production/catalog/host.example.edu HTTP/1.1" 200 23663 "-" "-"
> 10.0.0.82 - - [23/Sep/2013:12:59:01 -0400] "GET
> /production/file_metadata/modules/nagios/nagios-plugins/check_ageandcontent.pl?
> HTTP/1.1" 200 338 "-" "-"
> 10.0.0.82 - - [23/Sep/2013:12:59:01 -0400] "GET
> /production/file_metadata/modules/nagios/nagios-plugins/check_yum?
> HTTP/1.1" 200 325 "-" "-"
> 10.0.0.82 - - [23/Sep/2013:12:59:01 -0400 "GET
> /production/file_metadata/modules/nagios/nagios-plugins/check_mount?
> HTTP/1.1" 200 327 "-" "-"
> 10.0.0.82 - - [23/Sep/2013:12:59:01 -0400] "PUT
> /production/report/host.example.edu HTTP/1.1" 406 86 "-" "-"
>

/var/log/httpd/error_log: no logs for this client

/var/log/foreman/production.log

··· > Started GET "/node/host.example.edu?format=yml" for 10.0.0.186 at > 2013-09-22 12:52:44 -0400 > Processing by HostsController#externalNodes as YML > Parameters: {"name"=>"host.example.edu"} > Rendered text template (0.0ms) > Completed 200 OK in 29ms (Views: 0.5ms | ActiveRecord: 6.4ms) > > Started POST "/fact_values/create?format=yml" for 10.0.0.186 at 2013-09-22 > 12:52:47 -0400 > Processing by FactValuesController#create as YML > Parameters: {"facts"=>"[FILTERED]"} > Rendered text template (0.0ms) > Completed 200 OK in 35ms (Views: 0.4ms | ActiveRecord: 7.8ms) > > Started GET "/node/host.example.edu?format=yml" for 10.0.0.186 at > 2013-09-22 12:52:47 -0400 > Processing by HostsController#externalNodes as YML > Parameters: {"name"=>"host.example.edu"} > Rendered text template (0.0ms) > Completed 200 OK in 29ms (Views: 0.4ms | ActiveRecord: 6.7ms) > > Started POST "/reports/create?format=yml" for 10.0.0.186 at 2013-09-22 > 12:52:50 -0400 > Processing by ReportsController#create as YML > Parameters: {"report"=>"[FILTERED]"} > processing report for host.example.edu > Rendered text template (0.0ms) > Completed 200 OK in 44ms (Views: 0.5ms | ActiveRecord: 8.7ms) >

On Monday, September 30, 2013 11:45:54 AM UTC-4, Daniel Finkelstein wrote:

Temporary solution was to downgrade puppet to 2.6.18, which makes
everything magically work again.

On Monday, September 30, 2013 10:44:56 AM UTC-4, Daniel Finkelstein wrote:

I found the same issues online, but even odder is that I’m getting no 404
errors. Just normal 200 status and no complaints at all from the nodes, yet
no updates on the Foreman or Puppet Dashboard servers.

On Saturday, September 28, 2013 6:08:04 PM UTC-4, Sean Alderman wrote:

I had similar problem this week and found a bug report @ puppet labs on
it. Sorry, no link handy. I didn’t have the time or patience to dive in, so
I did a yum downgrade on puppet to the Epel version, and set yum version
lock to keep it there.

I should be able to post my logs on Monday if it will help to compare.

  • Sean Alderman
    Senior Engineer, UDit Systems Integration

This message had been brought to you by Android Bionic.
On Sep 28, 2013 4:25 PM, “Greg Sutcliffe” greg.su...@gmail.com wrote:

On 28 September 2013 18:40, Daniel.Fi...@sungard.com wrote:

I’m running foreman 1.2.2 and, up until I upgraded to puppet 3.3.0
from puppet 2.6, reports were going fine, both to Foreman and to Puppet
Dashboard. Since upgrading to puppet 3.3.0, I no longer see reports in
either, however the puppet nodes are running on-schedule and puppet seems
to be running without any errors.****


Has anyone else seen this behavior, or know how to debug the reporting
system?

Start with the foreman logs. You should requests coming in from the
puppetmaster to /reports/create (or in 1.3, /api/reports). If they are, see
if they’re 404’ing for some reason. If not, check your puppetmaster
logs/config (it usually logs to syslog) for why it’s not sending them.

Greg


You received this message because you are subscribed to the Google
Groups “Foreman users” group.
To unsubscribe from this group and stop receiving emails from it, send
an email to foreman-user...@googlegroups.com.
To post to this group, send email to forema...@googlegroups.com.
Visit this group at http://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/groups/opt_out.

> From: "Sean Alderman" <salderman1@udayton.edu>
> To: foreman-users@googlegroups.com
> Sent: Monday, September 30, 2013 1:21:47 PM
> Subject: Re: [foreman-users] Upgrade to puppet 3.3.0: reports appear to be gone
>
> I've splunked back into my logs, and only found the following messages like
> these on the client side (for agents with versions 2.6.18, 2.7.14, and
> 2.7.21):
>
> Sep 23 12:59:01 host puppet-agent[2205]: Could not send report: Error 406 on
> SERVER: Not Acceptable: No supported formats are acceptable (Accept:
> b64_zlib_yaml, yaml, raw)

Sorry for not responding sooner, I just saw this thread. This is a known issue with reporting when using a 3.3.0 master with versions below 3.3.0 [1].

-Sam

  1. http://projects.puppetlabs.com/issues/22530
··· ----- Original Message -----

Sep 23 12:59:01 host puppet-agent[2205]: Finished catalog run in 0.93
seconds

The server side log stream is whacky. The time stamps don’t match up
between the apache and foreman logs…

/var/log/httpd/access_log:

10.0.0.82 - - [23/Sep/2013:12:58:55 -0400] “GET
/default/node/host.example.edu? HTTP/1.1” 200 4470 “-” "-"
10.0.0.82 - - [23/Sep/2013:12:58:56 -0400] “GET
/production/file_metadatas/plugins?recurse=true&links=manage&ignore=—+%0A+±+%22.svn%22%0A+±+CVS%0A+±+%22.git%22&checksum_type=md5&
HTTP/1.1” 200 45991 “-” "-"
10.0.0.82 - - [23/Sep/2013:12:58:58 -0400] “POST
/production/catalog/host.example.edu HTTP/1.1” 200 23663 “-” "-"
10.0.0.82 - - [23/Sep/2013:12:59:01 -0400] “GET
/production/file_metadata/modules/nagios/nagios-plugins/check_ageandcontent.pl?
HTTP/1.1” 200 338 “-” "-"
10.0.0.82 - - [23/Sep/2013:12:59:01 -0400] “GET
/production/file_metadata/modules/nagios/nagios-plugins/check_yum?
HTTP/1.1” 200 325 “-” "-"
10.0.0.82 - - [23/Sep/2013:12:59:01 -0400 “GET
/production/file_metadata/modules/nagios/nagios-plugins/check_mount?
HTTP/1.1” 200 327 “-” "-"
10.0.0.82 - - [23/Sep/2013:12:59:01 -0400] “PUT
/production/report/host.example.edu HTTP/1.1” 406 86 “-” “-”

/var/log/httpd/error_log: no logs for this client

/var/log/foreman/production.log

Started GET “/node/host.example.edu?format=yml” for 10.0.0.186 at
2013-09-22 12:52:44 -0400
Processing by HostsController#externalNodes as YML
Parameters: {“name”=>“host.example.edu”}
Rendered text template (0.0ms)
Completed 200 OK in 29ms (Views: 0.5ms | ActiveRecord: 6.4ms)

Started POST “/fact_values/create?format=yml” for 10.0.0.186 at 2013-09-22
12:52:47 -0400
Processing by FactValuesController#create as YML
Parameters: {“facts”=>"[FILTERED]"}
Rendered text template (0.0ms)
Completed 200 OK in 35ms (Views: 0.4ms | ActiveRecord: 7.8ms)

Started GET “/node/host.example.edu?format=yml” for 10.0.0.186 at
2013-09-22 12:52:47 -0400
Processing by HostsController#externalNodes as YML
Parameters: {“name”=>“host.example.edu”}
Rendered text template (0.0ms)
Completed 200 OK in 29ms (Views: 0.4ms | ActiveRecord: 6.7ms)

Started POST “/reports/create?format=yml” for 10.0.0.186 at 2013-09-22
12:52:50 -0400
Processing by ReportsController#create as YML
Parameters: {“report”=>"[FILTERED]"}
processing report for host.example.edu
Rendered text template (0.0ms)
Completed 200 OK in 44ms (Views: 0.5ms | ActiveRecord: 8.7ms)

On Monday, September 30, 2013 11:45:54 AM UTC-4, Daniel Finkelstein wrote:

Temporary solution was to downgrade puppet to 2.6.18, which makes
everything magically work again.

On Monday, September 30, 2013 10:44:56 AM UTC-4, Daniel Finkelstein wrote:

I found the same issues online, but even odder is that I’m getting no 404
errors. Just normal 200 status and no complaints at all from the nodes,
yet
no updates on the Foreman or Puppet Dashboard servers.

On Saturday, September 28, 2013 6:08:04 PM UTC-4, Sean Alderman wrote:

I had similar problem this week and found a bug report @ puppet labs on
it. Sorry, no link handy. I didn’t have the time or patience to dive in,
so
I did a yum downgrade on puppet to the Epel version, and set yum version
lock to keep it there.

I should be able to post my logs on Monday if it will help to compare.

  • Sean Alderman
    Senior Engineer, UDit Systems Integration

This message had been brought to you by Android Bionic.
On Sep 28, 2013 4:25 PM, “Greg Sutcliffe” greg.su...@gmail.com wrote:

On 28 September 2013 18:40, Daniel.Fi...@sungard.com wrote:

I’m running foreman 1.2.2 and, up until I upgraded to puppet 3.3.0
from puppet 2.6, reports were going fine, both to Foreman and to Puppet
Dashboard. Since upgrading to puppet 3.3.0, I no longer see reports in
either, however the puppet nodes are running on-schedule and puppet
seems
to be running without any errors.****


Has anyone else seen this behavior, or know how to debug the reporting
system?

Start with the foreman logs. You should requests coming in from the
puppetmaster to /reports/create (or in 1.3, /api/reports). If they are,
see
if they’re 404’ing for some reason. If not, check your puppetmaster
logs/config (it usually logs to syslog) for why it’s not sending them.

Greg


You received this message because you are subscribed to the Google
Groups “Foreman users” group.
To unsubscribe from this group and stop receiving emails from it, send
an email to foreman-user...@googlegroups.com.
To post to this group, send email to forema...@googlegroups.com.
Visit this group at http://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/groups/opt_out.


You received this message because you are subscribed to the Google Groups
"Foreman users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at http://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/groups/opt_out.

The weird part is that I was using 3.3.0 for puppet across the board,
master and nodes, and was not getting report updates in Foreman or the
Puppet Dashboard. There were no HTTP errors, either, and I saw status 200
in all the logs. It was bizarre. After downgrading to 2.6.18, I was back in
business.

··· On Monday, September 30, 2013 1:32:23 PM UTC-4, Sam Kottler wrote: > > > > ----- Original Message ----- > > From: "Sean Alderman" <salde...@udayton.edu > > > To: forema...@googlegroups.com > > Sent: Monday, September 30, 2013 1:21:47 PM > > Subject: Re: [foreman-users] Upgrade to puppet 3.3.0: reports appear to > be gone > > > > I've splunked back into my logs, and only found the following messages > like > > these on the client side (for agents with versions 2.6.18, 2.7.14, and > > 2.7.21): > > > > Sep 23 12:59:01 host puppet-agent[2205]: Could not send report: Error > 406 on > > SERVER: Not Acceptable: No supported formats are acceptable (Accept: > > b64_zlib_yaml, yaml, raw) > > Sorry for not responding sooner, I just saw this thread. This is a known > issue with reporting when using a 3.3.0 master with versions below 3.3.0 > [1]. > > -Sam > > 1. http://projects.puppetlabs.com/issues/22530 > > > > > > > Sep 23 12:59:01 host puppet-agent[2205]: Finished catalog run in 0.93 > > > seconds > > > > > > > > The server side log stream is whacky. The time stamps don't match up > > between the apache and foreman logs... > > > > /var/log/httpd/access_log: > > > > > 10.0.0.82 - - [23/Sep/2013:12:58:55 -0400] "GET > > > /default/node/host.example.edu? HTTP/1.1" 200 4470 "-" "-" > > > 10.0.0.82 - - [23/Sep/2013:12:58:56 -0400] "GET > > > > /production/file_metadatas/plugins?recurse=true&links=manage&ignore=---+%0A++-+%22.svn%22%0A++-+CVS%0A++-+%22.git%22&checksum_type=md5& > > > > HTTP/1.1" 200 45991 "-" "-" > > > 10.0.0.82 - - [23/Sep/2013:12:58:58 -0400] "POST > > > /production/catalog/host.example.edu HTTP/1.1" 200 23663 "-" "-" > > > 10.0.0.82 - - [23/Sep/2013:12:59:01 -0400] "GET > > > /production/file_metadata/modules/nagios/nagios-plugins/ > check_ageandcontent.pl? > > > HTTP/1.1" 200 338 "-" "-" > > > 10.0.0.82 - - [23/Sep/2013:12:59:01 -0400] "GET > > > /production/file_metadata/modules/nagios/nagios-plugins/check_yum? > > > HTTP/1.1" 200 325 "-" "-" > > > 10.0.0.82 - - [23/Sep/2013:12:59:01 -0400 "GET > > > /production/file_metadata/modules/nagios/nagios-plugins/check_mount? > > > HTTP/1.1" 200 327 "-" "-" > > > 10.0.0.82 - - [23/Sep/2013:12:59:01 -0400] "PUT > > > /production/report/host.example.edu HTTP/1.1" 406 86 "-" "-" > > > > > > > /var/log/httpd/error_log: no logs for this client > > > > /var/log/foreman/production.log > > > > > Started GET "/node/host.example.edu?format=yml" for 10.0.0.186 at > > > 2013-09-22 12:52:44 -0400 > > > Processing by HostsController#externalNodes as YML > > > Parameters: {"name"=>"host.example.edu"} > > > Rendered text template (0.0ms) > > > Completed 200 OK in 29ms (Views: 0.5ms | ActiveRecord: 6.4ms) > > > > > > Started POST "/fact_values/create?format=yml" for 10.0.0.186 at > 2013-09-22 > > > 12:52:47 -0400 > > > Processing by FactValuesController#create as YML > > > Parameters: {"facts"=>"[FILTERED]"} > > > Rendered text template (0.0ms) > > > Completed 200 OK in 35ms (Views: 0.4ms | ActiveRecord: 7.8ms) > > > > > > Started GET "/node/host.example.edu?format=yml" for 10.0.0.186 at > > > 2013-09-22 12:52:47 -0400 > > > Processing by HostsController#externalNodes as YML > > > Parameters: {"name"=>"host.example.edu"} > > > Rendered text template (0.0ms) > > > Completed 200 OK in 29ms (Views: 0.4ms | ActiveRecord: 6.7ms) > > > > > > Started POST "/reports/create?format=yml" for 10.0.0.186 at 2013-09-22 > > > 12:52:50 -0400 > > > Processing by ReportsController#create as YML > > > Parameters: {"report"=>"[FILTERED]"} > > > processing report for host.example.edu > > > Rendered text template (0.0ms) > > > Completed 200 OK in 44ms (Views: 0.5ms | ActiveRecord: 8.7ms) > > > > > > > > > > > > > > > On Monday, September 30, 2013 11:45:54 AM UTC-4, Daniel Finkelstein > wrote: > > > > > > Temporary solution was to downgrade puppet to 2.6.18, which makes > > > everything magically work again. > > > > > > On Monday, September 30, 2013 10:44:56 AM UTC-4, Daniel Finkelstein > wrote: > > >> > > >> I found the same issues online, but even odder is that I'm getting no > 404 > > >> errors. Just normal 200 status and no complaints at all from the > nodes, > > >> yet > > >> no updates on the Foreman or Puppet Dashboard servers. > > >> > > >> On Saturday, September 28, 2013 6:08:04 PM UTC-4, Sean Alderman > wrote: > > >>> > > >>> I had similar problem this week and found a bug report @ puppet labs > on > > >>> it. Sorry, no link handy. I didn't have the time or patience to dive > in, > > >>> so > > >>> I did a yum downgrade on puppet to the Epel version, and set yum > version > > >>> lock to keep it there. > > >>> > > >>> I should be able to post my logs on Monday if it will help to > compare. > > >>> > > >>> - Sean Alderman > > >>> Senior Engineer, UDit Systems Integration > > >>> > > >>> This message had been brought to you by Android Bionic. > > >>> On Sep 28, 2013 4:25 PM, "Greg Sutcliffe" > wrote: > > >>> > > >>>> On 28 September 2013 18:40, wrote: > > >>>> > > >>>>> I’m running foreman 1.2.2 and, up until I upgraded to puppet > 3.3.0 > > >>>>> from puppet 2.6, reports were going fine, both to Foreman and to > Puppet > > >>>>> Dashboard. Since upgrading to puppet 3.3.0, I no longer see > reports in > > >>>>> either, however the puppet nodes are running on-schedule and > puppet > > >>>>> seems > > >>>>> to be running without any errors.**** > > >>>>> > > >>>>> ** ** > > >>>>> > > >>>>> Has anyone else seen this behavior, or know how to debug the > reporting > > >>>>> system? > > >>>>> > > >>>> > > >>>> Start with the foreman logs. You should requests coming in from the > > >>>> puppetmaster to /reports/create (or in 1.3, /api/reports). If they > are, > > >>>> see > > >>>> if they're 404'ing for some reason. If not, check your puppetmaster > > >>>> logs/config (it usually logs to syslog) for why it's not sending > them. > > >>>> > > >>>> Greg > > >>>> > > >>>> -- > > >>>> You received this message because you are subscribed to the Google > > >>>> Groups "Foreman users" group. > > >>>> To unsubscribe from this group and stop receiving emails from it, > send > > >>>> an email to foreman-user...@googlegroups.com. > > >>>> To post to this group, send email to forema...@googlegroups.com. > > >>>> Visit this group at http://groups.google.com/group/foreman-users. > > >>>> For more options, visit https://groups.google.com/groups/opt_out. > > >>>> > > >>> > > > > -- > > You received this message because you are subscribed to the Google > Groups > > "Foreman users" group. > > To unsubscribe from this group and stop receiving emails from it, send > an > > email to foreman-user...@googlegroups.com . > > To post to this group, send email to forema...@googlegroups.com. > > > Visit this group at http://groups.google.com/group/foreman-users. > > For more options, visit https://groups.google.com/groups/opt_out. > > >

I have similar issue but only with the facts.

https://projects.puppetlabs.com/issues/22713

··· On Monday, September 30, 2013 1:32:38 PM UTC-7, Daniel Finkelstein wrote: > > The weird part is that I was using 3.3.0 for puppet across the board, > master and nodes, and was not getting report updates in Foreman or the > Puppet Dashboard. There were no HTTP errors, either, and I saw status 200 > in all the logs. It was bizarre. After downgrading to 2.6.18, I was back in > business. > > On Monday, September 30, 2013 1:32:23 PM UTC-4, Sam Kottler wrote: >> >> >> >> ----- Original Message ----- >> > From: "Sean Alderman" >> > To: forema...@googlegroups.com >> > Sent: Monday, September 30, 2013 1:21:47 PM >> > Subject: Re: [foreman-users] Upgrade to puppet 3.3.0: reports appear to >> be gone >> > >> > I've splunked back into my logs, and only found the following messages >> like >> > these on the client side (for agents with versions 2.6.18, 2.7.14, and >> > 2.7.21): >> > >> > Sep 23 12:59:01 host puppet-agent[2205]: Could not send report: Error >> 406 on >> > SERVER: Not Acceptable: No supported formats are acceptable (Accept: >> > b64_zlib_yaml, yaml, raw) >> >> Sorry for not responding sooner, I just saw this thread. This is a known >> issue with reporting when using a 3.3.0 master with versions below 3.3.0 >> [1]. >> >> -Sam >> >> 1. http://projects.puppetlabs.com/issues/22530 >> >> > > >> > > Sep 23 12:59:01 host puppet-agent[2205]: Finished catalog run in 0.93 >> > > seconds >> > > >> > > >> > The server side log stream is whacky. The time stamps don't match up >> > between the apache and foreman logs... >> > >> > /var/log/httpd/access_log: >> > >> > > 10.0.0.82 - - [23/Sep/2013:12:58:55 -0400] "GET >> > > /default/node/host.example.edu? HTTP/1.1" 200 4470 "-" "-" >> > > 10.0.0.82 - - [23/Sep/2013:12:58:56 -0400] "GET >> > > >> /production/file_metadatas/plugins?recurse=true&links=manage&ignore=---+%0A++-+%22.svn%22%0A++-+CVS%0A++-+%22.git%22&checksum_type=md5& >> >> > > HTTP/1.1" 200 45991 "-" "-" >> > > 10.0.0.82 - - [23/Sep/2013:12:58:58 -0400] "POST >> > > /production/catalog/host.example.edu HTTP/1.1" 200 23663 "-" "-" >> > > 10.0.0.82 - - [23/Sep/2013:12:59:01 -0400] "GET >> > > /production/file_metadata/modules/nagios/nagios-plugins/ >> check_ageandcontent.pl? >> > > HTTP/1.1" 200 338 "-" "-" >> > > 10.0.0.82 - - [23/Sep/2013:12:59:01 -0400] "GET >> > > /production/file_metadata/modules/nagios/nagios-plugins/check_yum? >> > > HTTP/1.1" 200 325 "-" "-" >> > > 10.0.0.82 - - [23/Sep/2013:12:59:01 -0400 "GET >> > > /production/file_metadata/modules/nagios/nagios-plugins/check_mount? >> > > HTTP/1.1" 200 327 "-" "-" >> > > 10.0.0.82 - - [23/Sep/2013:12:59:01 -0400] "PUT >> > > /production/report/host.example.edu HTTP/1.1" 406 86 "-" "-" >> > > >> > >> > /var/log/httpd/error_log: no logs for this client >> > >> > /var/log/foreman/production.log >> > >> > > Started GET "/node/host.example.edu?format=yml" for 10.0.0.186 at >> > > 2013-09-22 12:52:44 -0400 >> > > Processing by HostsController#externalNodes as YML >> > > Parameters: {"name"=>"host.example.edu"} >> > > Rendered text template (0.0ms) >> > > Completed 200 OK in 29ms (Views: 0.5ms | ActiveRecord: 6.4ms) >> > > >> > > Started POST "/fact_values/create?format=yml" for 10.0.0.186 at >> 2013-09-22 >> > > 12:52:47 -0400 >> > > Processing by FactValuesController#create as YML >> > > Parameters: {"facts"=>"[FILTERED]"} >> > > Rendered text template (0.0ms) >> > > Completed 200 OK in 35ms (Views: 0.4ms | ActiveRecord: 7.8ms) >> > > >> > > Started GET "/node/host.example.edu?format=yml" for 10.0.0.186 at >> > > 2013-09-22 12:52:47 -0400 >> > > Processing by HostsController#externalNodes as YML >> > > Parameters: {"name"=>"host.example.edu"} >> > > Rendered text template (0.0ms) >> > > Completed 200 OK in 29ms (Views: 0.4ms | ActiveRecord: 6.7ms) >> > > >> > > Started POST "/reports/create?format=yml" for 10.0.0.186 at >> 2013-09-22 >> > > 12:52:50 -0400 >> > > Processing by ReportsController#create as YML >> > > Parameters: {"report"=>"[FILTERED]"} >> > > processing report for host.example.edu >> > > Rendered text template (0.0ms) >> > > Completed 200 OK in 44ms (Views: 0.5ms | ActiveRecord: 8.7ms) >> > > >> > >> > >> > >> > >> > >> > On Monday, September 30, 2013 11:45:54 AM UTC-4, Daniel Finkelstein >> wrote: >> > > >> > > Temporary solution was to downgrade puppet to 2.6.18, which makes >> > > everything magically work again. >> > > >> > > On Monday, September 30, 2013 10:44:56 AM UTC-4, Daniel Finkelstein >> wrote: >> > >> >> > >> I found the same issues online, but even odder is that I'm getting >> no 404 >> > >> errors. Just normal 200 status and no complaints at all from the >> nodes, >> > >> yet >> > >> no updates on the Foreman or Puppet Dashboard servers. >> > >> >> > >> On Saturday, September 28, 2013 6:08:04 PM UTC-4, Sean Alderman >> wrote: >> > >>> >> > >>> I had similar problem this week and found a bug report @ puppet >> labs on >> > >>> it. Sorry, no link handy. I didn't have the time or patience to >> dive in, >> > >>> so >> > >>> I did a yum downgrade on puppet to the Epel version, and set yum >> version >> > >>> lock to keep it there. >> > >>> >> > >>> I should be able to post my logs on Monday if it will help to >> compare. >> > >>> >> > >>> - Sean Alderman >> > >>> Senior Engineer, UDit Systems Integration >> > >>> >> > >>> This message had been brought to you by Android Bionic. >> > >>> On Sep 28, 2013 4:25 PM, "Greg Sutcliffe" >> wrote: >> > >>> >> > >>>> On 28 September 2013 18:40, wrote: >> > >>>> >> > >>>>> I’m running foreman 1.2.2 and, up until I upgraded to puppet >> 3.3.0 >> > >>>>> from puppet 2.6, reports were going fine, both to Foreman and to >> Puppet >> > >>>>> Dashboard. Since upgrading to puppet 3.3.0, I no longer see >> reports in >> > >>>>> either, however the puppet nodes are running on-schedule and >> puppet >> > >>>>> seems >> > >>>>> to be running without any errors.**** >> > >>>>> >> > >>>>> ** ** >> > >>>>> >> > >>>>> Has anyone else seen this behavior, or know how to debug the >> reporting >> > >>>>> system? >> > >>>>> >> > >>>> >> > >>>> Start with the foreman logs. You should requests coming in from >> the >> > >>>> puppetmaster to /reports/create (or in 1.3, /api/reports). If they >> are, >> > >>>> see >> > >>>> if they're 404'ing for some reason. If not, check your >> puppetmaster >> > >>>> logs/config (it usually logs to syslog) for why it's not sending >> them. >> > >>>> >> > >>>> Greg >> > >>>> >> > >>>> -- >> > >>>> You received this message because you are subscribed to the Google >> > >>>> Groups "Foreman users" group. >> > >>>> To unsubscribe from this group and stop receiving emails from it, >> send >> > >>>> an email to foreman-user...@googlegroups.com. >> > >>>> To post to this group, send email to forema...@googlegroups.com. >> > >>>> Visit this group at http://groups.google.com/group/foreman-users. >> > >>>> For more options, visit https://groups.google.com/groups/opt_out. >> > >>>> >> > >>> >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups >> > "Foreman users" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> an >> > email to foreman-user...@googlegroups.com. >> > To post to this group, send email to forema...@googlegroups.com. >> > Visit this group at http://groups.google.com/group/foreman-users. >> > For more options, visit https://groups.google.com/groups/opt_out. >> > >> >