Identify config report origin

ekohl · December 5, 2022, 4:11pm

After upgrading my own Foreman instance from 3.3 to 3.4 (and now 3.5, but it was present in 3.4) I realized there’s a regression: it no longer reports the origin for Puppet reports.

So digging into this, I found that this is how we identify a Puppet report:

github.com

theforeman/foreman/blob/a0dae9e5ca5937baa4daa6db589974da6b570c95/app/services/report_scanner/puppet_report_scanner.rb#L4-L15


      
          def identify_origin(report_data)
            'Puppet' if puppet_report?(report_data['logs'] || [])
          end
          
          def add_reporter_data(report, report_data)
            # no additional data apart of origin
          end
          
          def puppet_report?(logs)
            log = logs.last
            log && log['log'].fetch('sources', {}).fetch('source', '') =~ /Puppet/
          end

In short: if there’s a log entry with a source that matches Puppet, it’s Puppet. However, in 3.4 I made this change:

This means that for uneventful reports there is no longer any log message, just metrics that Puppet ran (which is needed to ensure that the host isn’t out of sync). Now the problem becomes: how do we identify this case. Looking at the API, there’s no field for the source/origin:

https://apidocs.theforeman.org/foreman/3.4/apidoc/v2/config_reports/create.html

We can look at how foreman_ansible does it:

github.com

theforeman/foreman_ansible/blob/125e5d4b231dd7dd2a90884d2d72129eee0a454b/app/services/foreman_ansible/ansible_report_scanner.rb#L24-L38


      
          def identify_origin(raw)
            'Ansible' if ansible_report?(raw)
          end
          
          def ansible_report?(raw)
            raw['reporter'] == 'ansible' || ansible_legacy_report?(raw['logs'])
          end
          
          def ansible_legacy_report?(logs)
            return false if logs.blank?
            logs.any? do |log|
              log['log'].fetch('messages', {}).
                fetch('message', '') =~ /"_ansible_parsed"/
            end
          end

It looks like it invents the field reporter in the API, which is accepted since we don’t have strict validation, but would probably break if we turn it on.

So my question is: how do we solve this? Do we enhance the API and add a formal origin or format field to it rather than relying on autodetection?

ekohl · December 7, 2022, 11:28am

Reported this as Bug #35833: Puppet configuration reports without entries do not get an origin - Foreman

ekohl · December 7, 2022, 12:04pm

And I opened a proposal to expose the origin directly.
https://github.com/theforeman/foreman/pull/9536