Get hooks working -> after_create

Support
,
1

Hello,

i would like to add newly created hosts to monitoring system (check_mk). I have access to API so it would be no problem at all i hope. I would like to do it as simple as possible. Foreman monitoring plugin works only with Icinga and at the moment I am not going to edit this this plugin to my needs, maybe later.
I thought to use foreman hooks in this case. probably event after_create would be the best in this situation but unfortunately this action is not triggered.

I have checked host/managed with 4 (after_build, after_create, after_save and create) hooks.
after_build and after_save are working, but after_create unfortunately not.
OS is CentOS mit Puppet 5.5.14. SElinux is not a problem. My script saves data in /usr/share/foreman/tmp/ so it is also not a problem.
What else can i debug to get hooks working?

Expected outcome:
The main case is, that i would like to trigger some action/hook (execute a script) when all facts are uploaded.
Maybe would be better to watch (with inotify_tools or something similar) when new facts are uploaded to /opt/puppetlabs/server/data/puppetserver/yaml/facts/ ?

Foreman and Proxy versions:
Foreman . 1.20.2
Foreman and Proxy plugin versions:
foreman-tasks . 0.14.3
foreman_dhcp_browser . 0.0.8
foreman_discovery . 14.0.1
foreman_hooks . 0.3.15
foreman_remote_execution . 1.6.7

Other relevant data:

[root@puppet ~]# ls -laZ /usr/share/foreman/config/hooks/host/managed/after_create/
drwxr-xr-x. foreman foreman system_u:object_r:bin_t:s0       .
drwxr-xr-x. foreman foreman system_u:object_r:bin_t:s0       ..
-rwxr-xr-x. foreman foreman system_u:object_r:bin_t:s0       10_print_label.sh
-rwxr-xr-x. foreman foreman system_u:object_r:bin_t:s0       20_log.sh
-rwxr-xr-x. foreman foreman system_u:object_r:bin_t:s0       hook_functions.sh

[root@puppet ~]# grep -i hook /var/log/foreman/production.log
2019-08-12T14:30:36 [I|app|] Finished discovering 3 hooks for Host::Managed#after_create
2019-08-12T14:30:36 [I|app|] Finished discovering 3 hooks for Host::Managed#after_build
2019-08-12T14:30:36 [I|app|] Finished discovering 3 hooks for Host::Managed#create
2019-08-12T14:30:36 [I|app|] Finished discovering 3 hooks for Host::Managed#after_save
2019-08-12T14:30:43 [D|app|] Extending Host::Managed with foreman_hooks orchestration hooking support
2019-08-12T14:30:43 [D|app|] Extending Host::Managed with foreman_hooks Rails hooking support
2019-08-12T14:30:43 [D|app|] Created hook method after_create on Host::Managed
2019-08-12T14:30:43 [D|app|] Created hook method after_build on Host::Managed
2019-08-12T14:30:43 [D|app|] Created hook method after_save on Host::Managed
2019-08-12T14:30:44 [D|app|] Extending Host::Managed with foreman_hooks orchestration hooking support
2019-08-12T14:30:44 [D|app|] Extending Host::Managed with foreman_hooks Rails hooking support
2019-08-12T14:30:44 [D|app|] Created hook method after_create on Host::Managed
2019-08-12T14:30:44 [D|app|] Created hook method after_build on Host::Managed
2019-08-12T14:30:44 [D|app|] Created hook method after_save on Host::Managed
2019-08-12T14:40:26 [D|app|6fa44] Observed after_save hook on hotspot-1333794.frederix-hotspot.de
2019-08-12T14:40:26 [D|app|6fa44] Running 3 hooks for Host::Managed#after_save
2019-08-12T14:40:26 [D|app|6fa44] Running hook: /usr/share/foreman/config/hooks/host/managed/after_save/10_print_label.sh after_save hotspot-1333794.frederix-hotspot.de
2019-08-12T14:40:26 [D|app|6fa44] Running hook: /usr/share/foreman/config/hooks/host/managed/after_save/20_log.sh after_save hotspot-1333794.frederix-hotspot.de
2019-08-12T14:40:26 [D|app|6fa44] Running hook: /usr/share/foreman/config/hooks/host/managed/after_save/hook_functions.sh after_save hotspot-1333794.frederix-hotspot.de
2

There is no hook for facts events AFAIK.

You could leverage audit log, almost everything creates an audit record in our audit table but I am not sure about facts honestly - maybe we blacklist this event because it occurs so often.

One idea, if you enable telemetry (prometheus, statsd or logging) then every single API request will generate a measurement you could readout. That could do it.

:telemetry:
  :prefix: 'fm_rails'
  :prometheus:
    :enabled: false
  :statsd:
    :enabled: true
    :host: '127.0.0.1:8125'
    :protocol: 'datadog'
  :logger:
    :enabled: false
    :level: 'INFO'

Then create a statsd-compatible daemon that will listen to events:

nc -lu 8125

Warning: netcat will not end lines with CRLF so it will look like one huge line, you need to write a script that will listen to UDP 8125.

Restart foreman and process those events - these are text-based UDP packets. Throw away everything except those api/v2 controller events you are interested in (fact upload).