Hi,
Is anyone actively using foreman and the integration with chef ?
If so, can you please share a few things about your setup and configuration ? I’m interested in finding out what versions of chef-server/chef-client you are running as well as the foreman version.
the problem i have is getting chef facts uploaded to foreman, and I’d like to see if running an old chef-server version might be the cause.
I do appreciate all replies, thanks.
I’m using it.
Chef server:
Foreman server:
Hope this helps.
Also I think it’s fair to say I no longer actively work on the plugin. If you find some compatibility issue with the new Foreman release, please report here and I can try to fix, but I’m not able to contribute much on regular basis.
@andrp if your facts are not populated, first see the debug log from chef-client, if you don’t see any foreman related messages, the chef-handler-foreman may be misconfigured. If you see something, see foreman-proxy and foreman log while the upload happens to see if there some error. It should not depend on chef-server version at all, facts are uploaded directly from chef-client via smart proxy to foreman.
@Marek_Hulan I totally understand your position.
I think the problem is on foreman side.
on the chef node everything looks fine:
Running handlers:
[2019-05-17T13:58:09+00:00] INFO: Running report handlers
[2019-05-17T13:58:09+00:00] INFO: Sending attributes to foreman
[2019-05-17T13:58:09+00:00] DEBUG: {:name=>"foreman-test005.example.com", :facts=>{"cpu::0::vendor_id"=>"GenuineIntel", "cpu::0::family"=>"6", "cpu::0::model"=>"45", "cpu::0::model_name"=>"Intel(R) Xeon(R) Gold 5115 CPU @ 2.40GHz", "cpu::0::stepping"=>"2", "cpu::0::mhz"=>"2394.375", "cpu::0::cache_size"=>"14080 KB", "cpu::0::physical_id"=>"0", "cpu::0::core_id"=>"0", "cpu::0::cores"=>"1", "cpu::0::flags::0"=>"fpu", "cpu::0::flags::1"=>"vme", "cpu::0::flags::2"=>"de", "cpu::0::flags::3"=>"pse", "cpu::0::flags::4"=>"tsc", "cpu::0::flags::5"=>"msr", "cpu::0::flags::6"=>"pae", "cpu::0::flags::7"=>"mce", "cpu::0::flags::8"=>"cx8", "cpu::0::flags::9"=>"apic", "cpu::0::flags::10"=>"sep", "cpu::0::flags::11"=>"mtrr", "cpu::0::flags::12"=>"pge", "cpu::0::flags::13"=>"mca", "cpu::0::flags::14"=>"cmov", "cpu::0::flags::15"=>"pat", "cpu::0::flags::16"=>"pse36", "cpu::0::flags::17"=>"clflush", "cpu::0::flags::18"=>"dts", "cpu::0::flags::19"=>"mmx", "cpu::0::flags::20"=>"fxsr", "cpu::0::flags::21"=>"sse", "cpu::0::flags::22"=>"sse2", "cpu::0::flags::23"=>"ss", "cpu::0::flags::24"=>"syscall", "cpu::0::flags::25"=>"nx", "cpu::0::flags::26"=>"rdtscp", "cpu::0::flags::27"=>"lm", "cpu::0::flags::28"=>"constant_tsc", "cpu::0::flags::29"=>"arch_perfmon", "cpu::0::flags::30"=>"pebs", "cpu::0::flags::31"=>"bts", "cpu::0::flags::32"=>"nopl", "cpu::0::flags::33"=>"xtopology", "cpu::0::flags::34"=>"tsc_reliable", "cpu::0::flags::35"=>"nonstop_tsc", "cpu::0::flags::36"=>"eagerfpu", "cpu::0::flags::37"=>"pni", "cpu::0::flags::38"=>"pclmulqdq", "cpu::0::flags::39"=>"ssse3", "cpu::0::flags::40"=>"cx16", "cpu::0::flags::41"=>"pcid", "cpu::0::flags::42"=>"sse4_1", "cpu::0::flags::43"=>"sse4_2", "cpu::0::flags::44"=>"x2apic", "cpu::0::flags::45"=>"popcnt", "cpu::0::flags::46"=>"tsc_deadline_timer", "cpu::0::flags::47"=>"aes", "cpu::0::flags::48"=>"xsave", "cpu::0::flags::49"=>"avx", "cpu::0::flags::50"=>"hypervisor", "cpu::0::flags::51"=>"lahf_lm", "cpu::0::flags::52"=>"ssbd", "cpu::0::flags::53"=>"ibrs", "cpu::0::flags::54"=>"ibpb", "cpu::0::flags::55"=>"stibp", "cpu::0::flags::56"=>"tsc_adjust", "cpu::0::flags::57"=>"arat", "cpu::0::flags::58"=>"spec_ctrl", "cpu::0::flags::59"=>"intel_stibp", "cpu::0::flags::60"=>"flush_l1d", "cpu::0::flags::61"=>"arch_capabilities", "cpu i::total"=>1, "cpu::real"=>1, :environment=>"_default", :chef_node_name=>"foreman-test005.example.com", :operatingsystem=>"RedHat", :operatingsystemrelease=>"7.5", :_timestamp=>2019-05-17 13:58:09 +0000, :_type=>"foreman_chef"}}
ChefHandlerForeman::ForemanFacts
Running handlers complete
on the foreman-proxy side - there’s one error that isn’t saying too much:
2019-05-17T13:58:09 4b553d44 [I] Started POST /hosts/facts
2019-05-17T13:58:09 4b553d44 [D] starting chef signature authentication
2019-05-17T13:58:09 4b553d44 [D] header HTTP_X_FOREMAN_CLIENT: foreman-test005.example.com
2019-05-17T13:58:09 4b553d44 [D] foreman-test005.example.com authenticated successfully
2019-05-17T13:58:09 4b553d44 [D] facts upload request received
2019-05-17T13:58:09 4b553d44 [E] forwarding failed, Foreman responded with 500, check Foreman access and production logs for more details
2019-05-17T13:58:09 4b553d44 [I] Finished POST /hosts/facts with 200 (122.96 ms)
foreman’s log regarding the action that failed:
[2019-05-17T13:58:09+00:00] INFO: Running report handlers
2019-05-17T13:58:09 [I|app|d3781888] Processing by Api::V2::HostsController#facts as JSON
2019-05-17T13:58:09 [I|app|d3781888] Parameters: {"name"=>"foreman-test005.example.com", "facts"=>"[FILTERED]", "apiv"=>"v2", "host"=>{"name"=>"foreman-test005.example.com"}}
2019-05-17T13:58:09 [D|app|d3781888] Examining client certificate to extract dn and sans
2019-05-17T13:58:09 [D|app|d3781888] Client sent certificate with subject 'opvitlinfor003.example.com' and subject alt names '["opvitlinfor003.example.com", "puppet", "puppet.example.com"]'
2019-05-17T13:58:09 [D|app|d3781888] Verifying request from ["opvitlinfor003.example.com", "puppet", "puppet.example.com"] against ["opvitlinfor003.example.com"]
2019-05-17T13:58:09 [D|app|d3781888] Current user set to foreman_api_admin (admin)
2019-05-17T13:58:09 [W|app|d3781888] Action failed
ActiveRecord::NotNullViolation: PG::NotNullViolation: ERROR: null value in column "fact_name_id" violates not-null constraint
DETAIL: Failing row contains (15830, null, null, 2, 2019-05-17 13:58:09.55139, 2019-05-17 13:58:09.55139).
: INSERT INTO "fact_values" ("host_id", "updated_at", "created_at") VALUES ($1, $2, $3) RETURNING "id"
Any hints on how I can troubleshoot this further ?
Packages:
foreman-release-1.21.2-1.el7.noarch
tfm-rubygem-foreman_chef-0.8.1-1.fm1_20.el7.noarch
chef_handler_foreman (0.2.0)
regarding chef-server and chef-client version i tried with both latest versions and older versions and the same error message appears, so I guess that’s not relevant.
I recall seeing this before. I think it was race condition during facts importing, the way Rails implements unique validation does not work reliably… I know there were some changes around fact import and its wrapping in transaction, which may have changed something. Since we now have active job for background tasks in core, it would be interesting to convert fact parsing from foreman-tasks and see if we can now rely on core only. Actually something that may be useful for ansible plugin too. I’ll try to find some time for it, if there is someone willing to help, I can point them to the right places in the code.
i’d gladly help, but unfortunately I’m not a developer - more of a sysadmin - so I can help only with that part.
i’d be happy with any fix or workaround for the issue.
hi again,
my company has previously paid for support / sponsored open-source projects, so if there is anyone willing to help with the chef facts uploading issue that we have, please contact me so we can discuss details.
thanks,
Andrei