We have around 800 Azure VMs hosted in foreman (v 2.5.0), out of which only few machines are showing as out of sync in foreman console. On further check, it turns out that VMs are correctly reporting at puppet server ( updated logs are there in /var/log/puppetlabs/puppetserver/puppetserver.log corresponding to each out of sync VM ). But some how foreman failed to pickup the status. When I did a grep for out of sync VM in /var/log/foreman/production.log, I managed to pull below error corresponding to each out of sync VM
2021-06-14T00:37:57 [I|app|a1038b97] Backtrace for ‘Action failed’ error (ActiveRecord::AssociationTypeMismatch): Host::Managed(#81661260) expected, got #<Host::Base id: 193, name: “xxx.domain.com”, last_compile: nil, last_report: nil, updated_at: “2021-05-17 12:50:28”, created_at: “2021-05-17 12:50:25”, root_pass: nil, architecture_id: 1, operatingsystem_id: 2, environment_id: 1, ptable_id: nil, medium_id: nil, build: false, comment: nil, disk: nil, installed_at: nil, model_id: 1, hostgroup_id: nil, owner_id: nil, owner_type: nil, enabled: true, puppet_ca_proxy_id: 1, managed: false, use_image: nil, image_file: nil, uuid: nil, compute_resource_id: nil, puppet_proxy_id: 1, certname: “xxx.domain.com”, image_id: nil, organization_id: 1, location_id: 2, type: nil, otp: nil, realm_id: nil, compute_profile_id: nil, provision_method: nil, grub_pass: “”, global_status: 0, lookup_value_matcher: [FILTERED], pxe_loader: nil, initiated_at: nil, build_errors: nil> which is an instance of Host::Base(#77610880)
Any idea why only few machines are showing this behavior while majority are reporting correctly.