Triggering Foreman itself via webhooks plugin

Problem:

Hello everyone,

we have been using Foreman for some time now in our company, and it has proven to be a powerful tool. Recently, we upgraded to Foreman v3.1.1. While the “old” hooks plugin has worked nicely, we are now struggling with even the simplest use of the “new” webhooks plugin.

The goal is replace our old hook scripts with direct calls to the web API. There seems to be a lack of documentation on webhooks (see: Bug #31860: Write documentation for webhooks - Webhooks - Foreman). Also, the examples provided mainly describe how to trigger external APIs (e.g. Ansible Tower), not Foreman itself.

For instance, we need to manipulate host parameters after a host is created externally by vRealize Orchestrator. Below is a simple example of a shell script that distinguishes VM from hardware servers by host parameter.

We have hit a wall trying to rewrite this script as a webhook template with ERB. So far we are able to send a PUT request to /api/hosts/$ID, but setting one host parameter overwrites the whole array. We would probably need to split up @origin.params (Ruby) and translate it to host_parameters_attributes (API), but how to do this exactly?

Any hints on how to migrate scripts like this are appreciated. Maybe @lzap (highly recommended by @dirk) could provide more complex webhook examples?

Greetings from Germany

Foreman and Proxy versions:
foreman-release-3.1.1-1.el8.noarch

Foreman and Proxy plugin versions:
rubygem-foreman_webhooks-3.0.0-1.fm3_1.el8.noarch

Distribution and version:
RHEL 8.5

Other relevant data:

#!/bin/bash

mac="$(sudo hammer --output=csv host info --name "${2}" --fields 'Network/MAC' | tail -n 1)"

# VMware MAC range
if [[ "$mac" =~ 00:50:56(:[0-9a-fA-F]{2}){3} ]] ; then
  virtual="true"
else
  virtual="false"
fi

sudo hammer host set-parameter --host "${2}" --parameter-type 'boolean' --name 'virtual' --value "$virtual"
1 Like

Hi @dpenguins,

If I read this correctly, you want to set up a webhook that will listen to host_created event and then will be fired up to the Foreman itself to change the host’s parameter?

The are two ways, both are quite equal in this case:

  • I
    • Create a webhook to listen to host_created event with taget url https://foreman.example.com/api/v2/hosts/<%= @object.id %>/parameters
    • Create a webhook template associated to this webhook
    • In the body of the webhook template:
<%  virtual = @object.mac.match(/00:50:56(:[0-9a-fA-F]{2}){3}/) -%>
<%=
payload({
  "parameter": {
    "name": "virtual",
    "parameter_type": "boolean",
    "value": virtual ? true : false
  }
})
-%>
  • II
    • Install shellhooks [1] plugin for the Smart Proxy
    • Place your script as an executable to /var/lib/foreman-proxy/shellhooks. You can actually use the provided one.
    • Create a webhook subscribed to host_created event with target url https://<foreman|proxy>.example.com:9090/shellhook/yourscriptname
    • Set HTTP Method to be POST
    • Set Proxy Authorization in Credentials section
    • And to be able to use the hostname in your script, you need to pass it via Optional HTTP headers in Additional section, something like that should work:
{
  "X-Shellhook-Arg-1": "<%= @object.name -%>"
}

You can also find this/additional information here [2].

[1] - https://github.com/theforeman/smart_proxy_shellhooks
[2] - Administering Foreman

3 Likes

Hello!

Oh I closed the issue, it was a left over. Here is the documentation: Administering Foreman

Note that webhooks were specifically designed to call custom web hooks, not to call Foreman itself. Dealing with request parameters and headers is limited, the goal is to hand over the data so all the logic can be implemented in a custom webhook. We do provide shellhooks plugin for easy-to-do webhooks written as scripts.

2 Likes

Thanks for the detailed response, @ofedoren. It took us some time to try this out, but unfortunately we were not able to get your webhooks solution to work, even playing with different HTTP methods and API paths. But it still gave us a better understanding of the API and Ruby potential in other areas of Foreman that are revelant to us, so thanks for that.

Nevertheless, can you tell us where we can look up all available options/methods for the Ruby @object you mentioned? This would be useful for us to define additional arguments for shellhooks.

Also, thanks to @lzap for elaborating on webhooks. Indeed, we will go with the shellhooks solution now. Originally, our goal was not to leave the API/web context, but setting up shellhooks was easier than we thought (also thanks to proxy auth). Our old hook scripts seem to be working fine with only slight modifications. The only feature we are missing would be for shellhooks to integrate with Foreman templates the way webhooks do. As of now, we still have to set up an additional git repo to sync to /var/lib/foreman-proxy/shellhooks.

https://docs.theforeman.org/nightly/Administering_Red_Hat_Satellite/index-foreman-el.html#webhooks-available-events_admin

For more information about payload, go to Administer > About > Support > Templates DSL . A list of available types is provided in the following table. Some events are marked as custom , in that case, the payload is an object object but a Ruby hash (key-value data structure) so syntax is different.

Just to be clear: you literally need to go to a page on your Foreman instance to see all available objects and methods. It is generated documentation from DSL.

This is really great, webhooks are deliberately more limiting so we don’t leak internal data which often change after upgrade which breaks hooks and cause support calls.

This would be a security problem, you cannot allow external entity (Foreman Server) to modify executables on your Smart Proxy. Shellhooks are designed to be secure by default, unless you make a mistake in your shell/script code. So take a special care when writing those scripts :slight_smile:

If you can guys share some scripts with us, that would be great. We need more examples, the only working shellscript atm we have is Ansible Tower integration.

1 Like

We knew about the documentation inside of our Foreman instance, it’s been a great reference for templating. We simply didn’t think of checking it when the task at hand was webhooks, so we went for the API docs instead. Sorry for that :innocent:

Thank you for explaining the security implications of webhooks. Another hint for us to proceed with shellhooks instead. We are currently adapting our old scripts as shellhooks, using webhooks with empty templates to trigger them.

Of course, we will be happy to share our scripts with you. Due to corporate restrictions we have no public git repo we could refer you to, but we will add them to this thread as soon as we have finished tweaking and testing them again.

2 Likes

To whom it may concern, here are our shellhook examples as promised. These scripts were recently tidied up and tested successfully in our pre-production environment.

Prerequisites:

We trigger all of our shellhooks with Foreman webhooks that are subscribed to the Host Created/User Created event, using the Empty Payload template and @object.name/@object.login as X-Shellhook-Arg-1. As we rely on hammer to do the heavy lifting, the user foreman-proxy is allowed to execute hammer without a password via sudoers on the Foreman server. The scripts themselves sit in a git repository that is automatically synched to the shellhooks path.

Use case:

Again, please note that our Foreman instance is part of a larger setup in which we do not control all of the infrastructure. Therefore we use shellhooks to manipulate hosts after they have been created externally. Setting custom host parameters, and using them inside Foreman templates subsequently, has proven to be a reliable solution for us. This might be a niche case, but we hope that it serves as an inspiration to others. And thanks again to @ofedoren, @lzap and @Dirk for helping us migrate from hooks to shellhooks.

$ cat /var/lib/foreman-proxy/shellhooks/host_join_path.sh

#!/usr/bin/env bash

# Transform the host's LDAP path as received from Orchestrator into the OU path required by 'net ads' for joining.
# Save the result as a host parameter for use inside the deployment template's post install section.

vro_ldap_path="$(sudo -n hammer --output 'base' host info --name "${1}" --fields 'Parameters' | awk '$1~/vro_ldap_path/ {print $3}')"
tfm_join_path="$(echo "${vro_ldap_path}" | sed -r 's:CN=[^,]+,:: ; s:,DC=.*$:: ; s:OU=::g' | awk -F ',' '{ for (i=NF;i>0;i--) if (i!=1) printf $i"/"; else print $i; }')"

sudo -n hammer host set-parameter --host "${1}" --parameter-type 'string' --name 'tfm_join_path' --value "${tfm_join_path}"

exit 0
$ cat /var/lib/foreman-proxy/shellhooks/host_machine_type.sh

#!/usr/bin/env bash
# Check whether the host's MAC address is within the vendor range for VMware.
# Save the result as a host parameter for virtual/physical conditionals inside deployment templates.

host_mac="$(sudo -n hammer --output 'csv' host info --name "${1}" --fields 'Network/MAC' | tail -n 1)"

if [[ "${host_mac}" =~ 00:50:56(:[0-9a-fA-F]{2}){3} ]] ; then
  tfm_virtual="true"
else
  tfm_virtual="false"
fi

sudo -n hammer host set-parameter --host "${1}" --parameter-type 'boolean' --name 'tfm_virtual' --value "${tfm_virtual}"

exit 0
$ cat /var/lib/foreman-proxy/shellhooks/host_puppet_environment.sh

#!/usr/bin/env bash
# Define the host's Puppet environment based on the hostname's 3rd letter, or fall back to 'empty' environment.

envletter="$(sudo -n hammer --output 'csv' host info --name "${1}" --fields 'Name' | tail -n 1 | cut -b 3)"

case "${envletter}" in
  t)  host_env='testing' ;;
  s)  host_env='staging' ;;
  p)  host_env='production' ;;
  *)  host_env='empty' ;;
esac

sudo -n hammer host update --name "${1}" --environment "${host_env}"

exit 0
$ cat /var/lib/foreman-proxy/shellhooks/host_subnet.sh

#!/usr/bin/env bash
# Define the host's subnet based on the first two octets of its IP address, as assigned by Orchestrator.

host_ipaddr="$(sudo -n hammer --output 'csv' host info --name "${1}" --fields 'Network interfaces/IPv4 address' | tail -n 1 | cut -d '.' -f 1-2)"
host_subnet="$(sudo -n hammer --output 'csv' subnet list --fields 'Id,Network Addr' | grep "${host_ipaddr}" | cut -d ',' -f 1)"

sudo -n hammer host update --name "${1}" --subnet-id "${host_subnet}"

exit 0
$ cat /var/lib/foreman-proxy/shellhooks/user_default_settings.sh

#!/usr/bin/env bash
# Set common defaults for new users.

sudo -n hammer user update --login "${1}" --organization 'SNMP' --default-organization 'SNMP' --locations 'Springfield,Shelbyville' --locale 'en' --timezone 'Berlin'

exit 0
3 Likes