during the next sprint, I would like to extend the initial idea and implementation of Foreman core subscription API and @TimoGoebel 's foreman_webhooks plugin. We think it is the best and most flexible way of preaparing Foreman for the future in regard to integration.
The initial idea and motivation remains the same - we will maintain a simple subscription API in Foreman core and allow plugins to hook on various events similarly to the existing foreman_hooks plugin. Although I planned developing some other plugins (I mentioned Redis pub/sub), for the best user experience we would like to go forward with foreman_webhooks as the defacto implementation. For more details about the process how we got here, read:
Timo and our friends at iRonin layed the basics: we have subscription API in Foreman core and thereās the WIP prototype published at:
It provides a simple interface to create webhooks, define URL and actions on which they should trigger. It integrates with the subscription API to hook on these events and it processes them by creating ActiveJobs to deliver them to external services.
We would like to expand on that:
Add new template kind āwebhookā defining payloads in a flexible way.
Set of new macros which allows loading basic records via a database ID (couple of these already exist).
Integrate template payload to the hook definitions.
Rewrite UI in React.
Deliver API and CLI portions of the same.
Prepare some example payload templates for selected external services.
Create a smart proxy module webhook āshellā endpoint and payload template for easy testing and upgrade path for foreman_hooks users (executes a shell script).
We will not be aiming for 100% compatibility with current foreman_hooks, the plugin is not going anywhere and can be used alongside with foreman_webhooks going forward. We expect to ship it āuntil it breaksā, at least for few major releases so there is enough transition time for all users.
Iād like to ask @TimoGoebel if he is willing to contribute his WIP as a starting point for this work. It looks like 90% of the WIP code can be directly used for what we are planning to build.
Edit: Iāve described the minimum viable product, we would like to deliver other features like:
Audit support and extensive logging.
Request-session tracking via HTTP headers.
Add configurable timeout
Add re-triggering support and perhaps configurable amount of retries
Flag which can enable or disable hooks per hook item
Configurable HTTP headers (e.g. for secret tokens)
Take this as a TODO list which I will keep up to date as we dig deeper.
Sure, by all means. What do you need? Do you want to transfer the plugin to the Foreman org? Do you want me open a PR so we can move the code to core? Do you want me to just say, āyesā.
Thatās all for now, thank you. If you can help with reviews of the code changes, weād appreciate your valuable input at any point. Feel free.
@tbrisker I would like to request creating of new repository theforeman/foreman_webhooks where I would push the original Timoās WIP prototype as the starting point for the next development.
Oh this is already nice, I am going to add this to my OP into the TODO section for now. These are all good points we should definitely implement.
Isnāt secret token part of the URL which should be configurable? I expect both payload (body) and URL to be fully configurable on a per hook basis using ERB, so any variable or static text (a token) could be entered.
@tbrisker: If you temporarily grant me permission, Iāll move the repo. Looks like itās not possible to grant you admin permissions on a personal repo.
This makes a lot of sense technically, but I need to ask: what is the business purpose?
I mean, webhooks is the defacto standard, every company or most individual know how to write and deploy a web service, thatās why webhooks are popular. In fact, microservices is a big thing and itās literally based on web services.
The current hooks plugin is based on spawning a process for each individual event, thatās quite an operation even for UNIX. In the current proposal, webhooks should be handled by ActiveJob which is much lighter resource than a full UNIX process. At least thatās my impression today.
I am not big fan or making a code āready to be extended laterā. I always push for simplest code as possible, less code means less bugs. Then, the time that was saved not doing any crazy design patterns can be invested in refactoring later, if needed. Often times, there is no refactoring at all
Oh, I didnāt mean that it needs to be an actual āserviceā. I wanted to write āservice/job/whateverā but used service for some reason, ActiveJob is reasonable imo.
Same for the āWEB_SOCKETS_SERVICEā that can just be a route in the current API
Web sockets are really nice, itās actually a messaging platform ideal for this. I suggest we would write a new plugin that would expose all internal events over websockets if UI needs this. Or it can be implemented in core as well.
I just donāt see any use case yet other than user would be able to subscribe to arbitrary events as notifications (?). Thatās indeed interesting.
For the record, we are about to merge a change in core subscription API. Instead of having stripped-down payload (just ID, name in case od delete) we are going to pass whole ActiveRecord object. The reason for that is plugin authors would do Model.find(id) anyway so there is no added value.
Another big reason for this is we would like to refactor foreman_webhooks plugin and enhance it with templates, so users would be able to define their own payloads. We will ship a default template that will have basic details for models like host, hostgroups and other important ones and this will be our API, but more flexible and testable (we will have a test which will render all templates to ensure they work).
To ensure the current design fits real-world scenarios, I am going to do a research of publicly avaialble API documentation for services our users would like to integrate with. Feel free to add your own services or do research for other technologies. This entry is a wiki.
Typically OAuth2 is a two-step process when client id + secret is used to acquire a temporary access token (and refresh token). However many services have configurable time for both access and refresh token expiration. It can be set to longer timeframe (month, year) so the token can be then hardcoded into HTTP header. This is the only way webhooks plugin can work, otherwise webhook must be implemented as a 3rd party service.
It looks like Netbox has quite similar design we target for - their webhooks are highly configurable (URL, actions, headers, body) with jinja2 templating engine:
Name - A unique name for the webhook. The name is not included with outbound messages.
Object type(s) - The type or types of NetBox object that will trigger the webhook.
Enabled - If unchecked, the webhook will be inactive.
Events - A webhook may trigger on any combination of create, update, and delete events. At least one event type must be selected.
HTTP method - The type of HTTP request to send. Options include GET, POST, PUT, PATCH, and DELETE.
URL - The fuly-qualified URL of the request to be sent. This may specify a destination port number if needed.
HTTP content type - The value of the requestās Content-Type header. (Defaults to application/json )
Additional headers - Any additional headers to include with the request (optional). Add one header per line in the format Name: Value . Jinja2 templating is supported for this field (see below).
Body template - The content of the request being sent (optional). Jinja2 templating is supported for this field (see below). If blank, NetBox will populate the request body with a raw dump of the webhook context. (If the HTTP cotent type is set to application/json , this will be formatted as a JSON object.)
Secret - A secret string used to prove authenticity of the request (optional). This will append a X-Hook-Signature header to the request, consisting of a HMAC (SHA-512) hex digest of the request body using the secret as the key.
SSL verification - Uncheck this option to disable validation of the receiverās SSL certificate. (Disable with caution!)
CA file path - The file path to a particular certificate authority (CA) file to use when validating the receiverās SSL certificate (optional).
From what I can see, it looks like the most common approach is ability to generate an oauth2 token/secret with configurable timeout which is then used to get session token. Some parties allows temporary tokens and some plain username/password authentication. With templating, we should be able to cover some integration, definitely not all.
When an app supports plain HTTP or allows generation of permanent token or session token with infinite/long timeout a single call can be used. How feasible this is gonna be in practice is hard to tell, Iād have to integrate every single one of them to find out what session API endpoint returns.
Overall I am impressed how much security by obscurity is this, since all calls should be HTTPS with CA check by default, it should not matter whatās transferred over the secure line. Even plain HTTP auth should be okay as long as TLS/SSL is not flawed. You either leak username/password, or oauth2 initial token/secret or predefined permanent token. Results are the same. I do not buy this explanation.
Anyway, there always be a chance to integrate against man-in-the-middle service which can provide the access via oauth2 short-lived tokens. So I suggest to go with our proposal, unless there is something I donā t see. What you think? @TimoGoebel@Marek_Hulan@ofedoren and others?
