Problem: I’ve been working on deploying Foreman, Foreman Proxy, Puppetserver, Puppetdb, etc from scratch, and replicating what the foreman-installer is doing via the set of puppet modules that manage each component. Everything works as expected when using the foreman-installer, but when configuring the components using the supported modules, I am unable to view puppet reports from the Foreman UI. I can see them being successfully created in the foreman logs, but am unable to retrieve or view them. I’ve gone through Foreman :: Manual to verify all the report configuration params and plugins are available (mostly default params from the modules), enabled debug logging on multiple pieces and have been unable to determine why I’m unable to view reports through the UI. Any help would be greatly appreciated!
Expected outcome: Viewing puppet reports through the Foreman UI
Foreman and Proxy versions:
foreman-3.0.0-1.el7.noarch
foreman-proxy-3.0.0-1.el7.noarch
Foreman and Proxy plugin versions:
foreman-debug-3.0.0-1.el7.noarch
tfm-rubygem-puppetdb_foreman-5.0.0-3.fm2_5.el7.noarch
foreman-service-3.0.0-1.el7.noarch
tfm-rubygem-hammer_cli_foreman-3.0.0-1.el7.noarch
foreman-dynflow-sidekiq-3.0.0-1.el7.noarch
tfm-rubygem-hammer_cli_foreman_puppet-0.0.3-1.fm3_0.el7.noarch
foreman-postgresql-3.0.0-1.el7.noarch
foreman-cli-3.0.0-1.el7.noarch
tfm-rubygem-foreman_puppet-1.0.1-1.fm3_0.el7.noarch
Distribution and version:
CentOS 7
x
Other relevant data:
Log requests from foreman / httpd during report submissions (successful 201 Created)
2021-09-29T13:17:47 [I|app|e58e44be] Started POST "/api/config_reports" for 10.2.3.4.5 at 2021-09-29 13:17:47 -0700
2021-09-29T13:17:47 [I|app|e58e44be] Processing by Api::V2::ConfigReportsController#create as JSON
2021-09-29T13:17:47 [I|app|e58e44be] Parameters: {"config_report"=>"[FILTERED]", "apiv"=>"v2"}
2021-09-29T13:17:47 [D|app|e58e44be] Client sent certificate with subject 'puppettester2.example.com' and subject alt names '["puppettester2.example.com", "puppet.datacenter1.example.com", "puppet.datacenter2.example.com", "puppet.example.com", "puppet.datacenter3.example.com", "puppet.datacenter4.example.com", "puppetdb.example.com"]'
2021-09-29T13:17:47 [D|app|e58e44be] CN and SANs were extracted from a client certificate.
2021-09-29T13:17:47 [D|app|e58e44be] Verifying request from ["puppettester2.example.com", "puppet.datacenter1.example.com", "puppet.datacenter2.example.com", "puppet.example.com", "puppet.datacenter3.example.com", "puppet.datacenter14.example.com", "puppetdb.example.com"] against ["puppettester2.example.com"]
2021-09-29T13:17:47 [D|app|e58e44be] Processing report: {"host"=>"puppettester2.example.com", "reported_at"=>"2021-09-29 20:17:01 UTC", "status"=>{"applied"=>0, "restarted"=>0, "failed"=>0, "failed_restarts"=>0, "skipped"=>0, "pending"=>1}, "metrics"=>{"resources"=>{"changed"=>0, "corrective_change"=>0, "failed"=>0, "failed_to_restart"=>0, "out_of_sync"=>1, "restarted"=>0, "scheduled"=>0, "skipped"=>0, "total"=>481}, "time"=>{"anchor"=>0.000508764, "augeas"=>0.293724531, "catalog_application"=>46.1207558340102, "concat_file"=>0.001423718, "concat_fragment"=>0.008948491, "config_retrieval"=>6.385189932, "cron"=>0.001091673, "datacat_collector"=>0.004371801, "datacat_fragment"=>0.00039958100000000004, "exec"=>19.242628728, "file"=>0.22054999699999991, "filebucket"=>6.8428e-05, "foreman_config_entry"=>0.001882493, "foreman_smartproxy"=>2.220397818, "group"=>0.0011215510000000001, "ini_setting"=>0.023123105000000005, "ini_subsetting"=>0.000683065, "package"=>0.12255274000000001, "postgresql_conf"=>0.000517414, "postgresql_conn_validator"=>0.136301401, "postgresql_psql"=>2.2505175860000004, "puppetdb_conn_validator"=>0.134223167, "schedule"=>0.000622721, "service"=>0.429491886, "total"=>46.180033838, "transaction_evaluation"=>45.98922192200553, "user"=>0.042544081}, "changes"=>{"total"=>0}, "events"=>{"failure"=>0, "noop"=>1, "success"=>0, "total"=>1}}, "logs"=>[{"log"=>{"sources"=>{"source"=>"Puppet"}, "messages"=>{"message"=>"Using environment 'production'"}, "level"=>"info"}}, {"log"=>{"sources"=>{"source"=>"Puppet"}, "messages"=>{"message"=>"Applying configuration version '1632946615'"}, "level"=>"info"}}, {"log"=>{"sources"=>{"source"=>"/Stage[main]/Puppet::Config/Concat[/etc/puppetlabs/puppet/puppet.conf]/File[/etc/puppetlabs/puppet/puppet.conf]/content"}, "messages"=>{"message"=>"current_value '{sha256}732a10a3bf027b93395a9a7d95e075f5e412b92435aafa6ad7be2dc3c0ecad7c', should be '{sha256}76ee5260b370e59aa1f53fbfceadee663797ac87cbbb6d588c88f23a064e898f' (noop)"}, "level"=>"notice"}}, {"log"=>{"sources"=>{"source"=>"/etc/puppetlabs/puppet/puppet.conf"}, "messages"=>{"message"=>"Would have triggered 'refresh' from 1 event"}, "level"=>"notice"}}, {"log"=>{"sources"=>{"source"=>"Concat[/etc/puppetlabs/puppet/puppet.conf]"}, "messages"=>{"message"=>"Would have triggered 'refresh' from 2 events"}, "level"=>"notice"}}, {"log"=>{"sources"=>{"source"=>"Class[Puppet::Config]"}, "messages"=>{"message"=>"Would have triggered 'refresh' from 1 event"}, "level"=>"notice"}}, {"log"=>{"sources"=>{"source"=>"Class[Puppet::Config]"}, "messages"=>{"message"=>"Scheduling refresh of Class[Puppet::Agent::Service]"}, "level"=>"info"}}, {"log"=>{"sources"=>{"source"=>"Class[Puppet::Config]"}, "messages"=>{"message"=>"Scheduling refresh of Class[Puppet::Server::Service]"}, "level"=>"info"}}, {"log"=>{"sources"=>{"source"=>"Class[Puppet::Server::Service]"}, "messages"=>{"message"=>"Would have triggered 'refresh' from 1 event"}, "level"=>"notice"}}, {"log"=>{"sources"=>{"source"=>"Class[Puppet::Server::Service]"}, "messages"=>{"message"=>"Scheduling refresh of Service[puppetserver]"}, "level"=>"info"}}, {"log"=>{"sources"=>{"source"=>"/Service[puppetserver]"}, "messages"=>{"message"=>"Would have triggered 'refresh' from 1 event"}, "level"=>"notice"}}, {"log"=>{"sources"=>{"source"=>"Class[Puppet::Server::Service]"}, "messages"=>{"message"=>"Would have triggered 'refresh' from 1 event"}, "level"=>"notice"}}, {"log"=>{"sources"=>{"source"=>"Class[Puppet::Server]"}, "messages"=>{"message"=>"Would have triggered 'refresh' from 1 event"}, "level"=>"notice"}}, {"log"=>{"sources"=>{"source"=>"Class[Puppet::Agent::Service]"}, "messages"=>{"message"=>"Would have triggered 'refresh' from 1 event"}, "level"=>"notice"}}, {"log"=>{"sources"=>{"source"=>"Class[Puppet::Agent::Service]"}, "messages"=>{"message"=>"Scheduling refresh of Class[Puppet::Agent::Service::Daemon]"}, "level"=>"info"}}, {"log"=>{"sources"=>{"source"=>"Class[Puppet::Agent::Service]"}, "messages"=>{"message"=>"Scheduling refresh of Class[Puppet::Agent::Service::Systemd]"}, "level"=>"info"}}, {"log"=>{"sources"=>{"source"=>"Class[Puppet::Agent::Service]"}, "messages"=>{"message"=>"Scheduling refresh of Class[Puppet::Agent::Service::Cron]"}, "level"=>"info"}}, {"log"=>{"sources"=>{"source"=>"Class[Puppet::Agent::Service::Daemon]"}, "messages"=>{"message"=>"Would have triggered 'refresh' from 1 event"}, "level"=>"notice"}}, {"log"=>{"sources"=>{"source"=>"Class[Puppet::Agent::Service::Daemon]"}, "messages"=>{"message"=>"Scheduling refresh of Service[puppet]"}, "level"=>"info"}}, {"log"=>{"sources"=>{"source"=>"Class[Puppet::Agent::Service::Systemd]"}, "messages"=>{"message"=>"Would have triggered 'refresh' from 1 event"}, "level"=>"notice"}}, {"log"=>{"sources"=>{"source"=>"Class[Puppet::Agent::Service::Systemd]"}, "messages"=>{"message"=>"Scheduling refresh of Systemd::Timer[puppet-run.timer]"}, "level"=>"info"}}, {"log"=>{"sources"=>{"source"=>"Systemd::Timer[puppet-run.timer]"}, "messages"=>{"message"=>"Would have triggered 'refresh' from 1 event"}, "level"=>"notice"}}, {"log"=>{"sources"=>{"source"=>"Systemd::Timer[puppet-run.timer]"}, "messages"=>{"message"=>"Scheduling refresh of Systemd::Unit_file[puppet-run.service]"}, "level"=>"info"}}, {"log"=>{"sources"=>{"source"=>"Systemd::Timer[puppet-run.timer]"}, "messages"=>{"message"=>"Scheduling refresh of Systemd::Unit_file[puppet-run.timer]"}, "level"=>"info"}}, {"log"=>{"sources"=>{"source"=>"Class[Puppet::Agent::Service::Cron]"}, "messages"=>{"message"=>"Would have triggered 'refresh' from 1 event"}, "level"=>"notice"}}, {"log"=>{"sources"=>{"source"=>"Systemd::Unit_file[puppet-run.service]"}, "messages"=>{"message"=>"Would have triggered 'refresh' from 1 event"}, "level"=>"notice"}}, {"log"=>{"sources"=>{"source"=>"Systemd::Unit_file[puppet-run.service]"}, "messages"=>{"message"=>"Scheduling refresh of Exec[puppet-run.service-systemctl-daemon-reload]"}, "level"=>"info"}}, {"log"=>{"sources"=>{"source"=>"Systemd::Unit_file[puppet-run.timer]"}, "messages"=>{"message"=>"Would have triggered 'refresh' from 1 event"}, "level"=>"notice"}}, {"log"=>{"sources"=>{"source"=>"Systemd::Unit_file[puppet-run.timer]"}, "messages"=>{"message"=>"Scheduling refresh of Service[puppet-run.timer]"}, "level"=>"info"}}, {"log"=>{"sources"=>{"source"=>"/Stage[main]/Puppet::Agent::Service::Systemd/Systemd::Timer[puppet-run.timer]/Systemd::Unit_file[puppet-run.service]/Exec[puppet-run.service-systemctl-daemon-reload]"}, "messages"=>{"message"=>"Would have triggered 'refresh' from 1 event"}, "level"=>"notice"}}, {"log"=>{"sources"=>{"source"=>"Systemd::Unit_file[puppet-run.service]"}, "messages"=>{"message"=>"Would have triggered 'refresh' from 1 event"}, "level"=>"notice"}}, {"log"=>{"sources"=>{"source"=>"/Service[puppet]"}, "messages"=>{"message"=>"Would have triggered 'refresh' from 1 event"}, "level"=>"notice"}}, {"log"=>{"sources"=>{"source"=>"Class[Puppet::Agent::Service::Daemon]"}, "messages"=>{"message"=>"Would have triggered 'refresh' from 1 event"}, "level"=>"notice"}}, {"log"=>{"sources"=>{"source"=>"/Service[puppet-run.timer]"}, "messages"=>{"message"=>"Would have triggered 'refresh' from 1 event"}, "level"=>"notice"}}, {"log"=>{"sources"=>{"source"=>"Systemd::Unit_file[puppet-run.timer]"}, "messages"=>{"message"=>"Would have triggered 'refresh' from 1 event"}, "level"=>"notice"}}, {"log"=>{"sources"=>{"source"=>"Systemd::Timer[puppet-run.timer]"}, "messages"=>{"message"=>"Would have triggered 'refresh' from 2 events"}, "level"=>"notice"}}, {"log"=>{"sources"=>{"source"=>"Class[Puppet::Agent::Service::Systemd]"}, "messages"=>{"message"=>"Would have triggered 'refresh' from 1 event"}, "level"=>"notice"}}, {"log"=>{"sources"=>{"source"=>"Class[Puppet::Agent::Service]"}, "messages"=>{"message"=>"Would have triggered 'refresh' from 2 events"}, "level"=>"notice"}}, {"log"=>{"sources"=>{"source"=>"Class[Puppet::Agent]"}, "messages"=>{"message"=>"Would have triggered 'refresh' from 1 event"}, "level"=>"notice"}}, {"log"=>{"sources"=>{"source"=>"Class[Puppet]"}, "messages"=>{"message"=>"Would have triggered 'refresh' from 3 events"}, "level"=>"notice"}}, {"log"=>{"sources"=>{"source"=>"Stage[main]"}, "messages"=>{"message"=>"Would have triggered 'refresh' from 8 events"}, "level"=>"notice"}}, {"log"=>{"sources"=>{"source"=>"Puppet"}, "messages"=>{"message"=>"Applied catalog in 46.12 seconds"}, "level"=>"notice"}}]}
2021-09-29T13:17:47 [I|app|e58e44be] Scanning report with: Foreman::PuppetReportScanner
2021-09-29T13:17:47 [D|app|e58e44be] Changes after reporter specific data added: {"host_id"=>[nil, 1], "reported_at"=>[nil, Wed, 29 Sep 2021 20:17:01 UTC +00:00], "status"=>[nil, 1125899906842624], "metrics"=>[nil, "---\nresources: !ruby/hash:ActiveSupport::HashWithIndifferentAccess\n changed: 0\n corrective_change: 0\n failed: 0\n failed_to_restart: 0\n out_of_sync: 1\n restarted: 0\n scheduled: 0\n skipped: 0\n total: 481\ntime: !ruby/hash:ActiveSupport::HashWithIndifferentAccess\n anchor: 0.000508764\n augeas: 0.293724531\n catalog_application: 46.1207558340102\n concat_file: 0.001423718\n concat_fragment: 0.008948491\n config_retrieval: 6.385189932\n cron: 0.001091673\n datacat_collector: 0.004371801\n datacat_fragment: 0.00039958100000000004\n exec: 19.242628728\n file: 0.22054999699999991\n filebucket: 6.8428e-05\n foreman_config_entry: 0.001882493\n foreman_smartproxy: 2.220397818\n group: 0.0011215510000000001\n ini_setting: 0.023123105000000005\n ini_subsetting: 0.000683065\n package: 0.12255274000000001\n postgresql_conf: 0.000517414\n postgresql_conn_validator: 0.136301401\n postgresql_psql: 2.2505175860000004\n puppetdb_conn_validator: 0.134223167\n schedule: 0.000622721\n service: 0.429491886\n total: 46.180033838\n transaction_evaluation: 45.98922192200553\n user: 0.042544081\nchanges: !ruby/hash:ActiveSupport::HashWithIndifferentAccess\n total: 0\nevents: !ruby/hash:ActiveSupport::HashWithIndifferentAccess\n failure: 0\n noop: 1\n success: 0\n total: 1\n"], "origin"=>[nil, "Puppet"]}
2021-09-29T13:17:48 [I|app|e58e44be] Imported report for puppettester2.example.com in 294.8 ms, status refreshed in 24.9 ms
2021-09-29T13:17:48 [I|app|e58e44be] Rendering api/v2/config_reports/create.json.rabl
2021-09-29T13:17:48 [I|app|e58e44be] Rendered api/v2/config_reports/create.json.rabl (Duration: 13.0ms | Allocations: 1478)
2021-09-29T13:17:48 [D|app|e58e44be] Body: {"metrics":{"resources":{"changed":0,"corrective_change":0,"failed":0,"failed_to_restart":0,"out_of_sync":1,"restarted":0,"scheduled":0,"skipped":0,"total":481},"time":{"anchor":0.000508764,"augeas":0.293724531,"catalog_application":46.1207558340102,"concat_file":0.001423718,"concat_fragment":0.008948491,"config_retrieval":6.385189932,"cron":0.001091673,"datacat_collector":0.004371801,"datacat_fragment":0.00039958100000000004,"exec":19.242628728,"file":0.22054999699999991,"filebucket":6.8428e-05,"foreman_config_entry":0.001882493,"foreman_smartproxy":2.220397818,"group":0.0011215510000000001,"ini_setting":0.023123105000000005,"ini_subsetting":0.000683065,"package":0.12255274000000001,"postgresql_conf":0.000517414,"postgresql_conn_validator":0.136301401,"postgresql_psql":2.2505175860000004,"puppetdb_conn_validator":0.134223167,"schedule":0.000622721,"service":0.429491886,"total":46.180033838,"transaction_evaluation":45.98922192200553,"user":0.042544081},"changes":{"total":0},"events":{"failure":0,"noop":1,"success":0,"total":1}},"created_at":"2021-09-29 20:17:47 UTC","updated_at":"2021-09-29 20:17:47 UTC","id":51,"host_id":1,"host_name":"puppettester2.example.com","reported_at":"2021-09-29 20:17:01 UTC","status":{"applied":0,"restarted":0,"failed":0,"failed_restarts":0,"skipped":0,"pending":1},"origin":"Puppet"}
2021-09-29T13:17:48 [I|app|e58e44be] Completed 201 Created in 517ms (Views: 26.6ms | ActiveRecord: 147.0ms | Allocations: 118506)
2021-09-29T13:17:49 [I|app|4415ba6e] Started GET "/notification_recipients" for 10.39.71.121 at 2021-09-29 13:17:49 -0700
2021-09-29T13:17:49 [I|app|4415ba6e] Processing by NotificationRecipientsController#index as JSON
2021-09-29T13:17:49 [D|tax|4415ba6e] Current location set to Default Location
2021-09-29T13:17:49 [D|tax|4415ba6e] Current organization set to Default Organization
2021-09-29T13:17:49 [D|not|4415ba6e] Cache Hit: notification, reading cache for notification-4
2021-09-29T13:17:49 [D|app|4415ba6e] Body: {"notifications":[{"id":8,"seen":false,"level":"info","text":"Automating Systems with Foreman, AWX, and FreeIPA","created_at":"2021-09-28T22:20:06.555Z","group":"Community","actions":{"links":[{"href":"https://theforeman.org/2021/09/automating-systems-with-foreman-awx-and-freeipa.html","title":"Open","external":true}]}},{"id":4,"seen":false,"level":"info","text":"Foreman 3.0 is here!","created_at":"2021-09-28T22:20:06.498Z","group":"Community","actions":{"links":[{"href":"https://theforeman.org/2021/09/foreman-30-is-here.html","title":"Open","external":true}]}}]}
2021-09-29T13:17:49 [I|app|4415ba6e] Completed 200 OK in 98ms (Views: 0.2ms | ActiveRecord: 2.3ms | Allocations: 10249)
==> /var/log/httpd/foreman-ssl_access_ssl.log <==
10.2.3.4.5 - - [29/Sep/2021:13:17:47 -0700] "POST /api/config_reports HTTP/1.1" 201 1335 "-" "Ruby"
GET requests during report search
==> /var/log/foreman/production.log <==
2021-09-29T13:26:50 [I|app|1a41a0c9] Started GET "/config_reports?search=&page=1" for 10.1.2.3.4 at 2021-09-29 13:26:50 -0700
2021-09-29T13:26:50 [I|app|1a41a0c9] Processing by ConfigReportsController#index as HTML
2021-09-29T13:26:50 [I|app|1a41a0c9] Parameters: {"search"=>"", "page"=>"1"}
2021-09-29T13:26:50 [D|tax|1a41a0c9] Current location set to Default Location
2021-09-29T13:26:50 [D|tax|1a41a0c9] Current organization set to Default Organization
2021-09-29T13:26:50 [I|app|1a41a0c9] Rendering config_reports/index.html.erb within layouts/application
2021-09-29T13:26:50 [I|app|1a41a0c9] Rendering text template
2021-09-29T13:26:50 [I|app|1a41a0c9] Rendered text template (Duration: 0.0ms | Allocations: 1)
2021-09-29T13:26:50 [I|app|1a41a0c9] Rendered config_reports/_list.html.erb (Duration: 12.0ms | Allocations: 3455)
2021-09-29T13:26:50 [I|app|1a41a0c9] Rendered config_reports/index.html.erb within layouts/application (Duration: 15.7ms | Allocations: 3888)
2021-09-29T13:26:50 [I|app|1a41a0c9] Rendered layouts/_application_content.html.erb (Duration: 2.0ms | Allocations: 416)
2021-09-29T13:26:50 [I|app|1a41a0c9] Rendering layouts/base.html.erb
2021-09-29T13:26:50 [I|app|1a41a0c9] Rendered layouts/base.html.erb (Duration: 38.1ms | Allocations: 13726)
2021-09-29T13:26:50 [I|app|1a41a0c9] Completed 200 OK in 118ms (Views: 56.1ms | ActiveRecord: 10.8ms | Allocations: 31647)
==> /var/log/httpd/foreman-ssl_access_ssl.log <==
10.1.2.3.4 - - [29/Sep/2021:13:26:50 -0700] "GET /config_reports?search=&page=1 HTTP/1.1" 200 3441 "https://puppettester2.example.com/config_reports?search=&page=1" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4656.3 Safari/537.36"
10.1.2.3.4 - - [29/Sep/2021:13:26:50 -0700] "GET /webpack/foreman-vendor.bundle-v8.8.0-production-58389a20e841d3e5b9a0.css HTTP/1.1" 200 299710 "https://puppettester2.example.com/config_reports?search=&page=1" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4656.3 Safari/537.36"
10.1.2.3.4 - - [29/Sep/2021:13:26:50 -0700] "GET /webpack/bundle-5ba7d7debd65d3801db0.css HTTP/1.1" 200 9462 "https://puppettester2.example.com/config_reports?search=&page=1" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4656.3 Safari/537.36"
10.1.2.3.4 - - [29/Sep/2021:13:26:50 -0700] "GET /assets/application-ef998467a92cb32448e6d3751ad56b2a4de1b4cc0b219a3eb7d78b165956badd.css HTTP/1.1" 200 75052 "https://puppettester2.example.com/config_reports?search=&page=1" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4656.3 Safari/537.36"
10.1.2.3.4 - - [29/Sep/2021:13:26:50 -0700] "GET /webpack/foreman_puppet/foreman_puppet:fills-8f434cee2b573c5f57b7.css HTTP/1.1" 200 6540 "https://puppettester2.example.com/config_reports?search=&page=1" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4656.3 Safari/537.36"
That apparently was it! I had no idea that was even filtering results, had completely overlooked it. Thank you so much! Now to figure out how that data is getting applied differently between how foreman-installer is doing this vs. how the modules are. I really appreciate the help on this 