Monitoring smart proxy

Development
1

Hi all,

We use foreman to deploy and manage our GNU/Linux servers and we are happy with it (thanks for your work !). I made some (very) little contributions in the last months and now i wanted to automate the creation/deletion of my hosts in the monitoring software, like it's done for example in the freeipa by the realm smart proxy. I know that i can do this with foreman_hooks, but i wanted to know if it make sense (and other persons are interested in) to make a monitoring smart proxy with maybe deeper integration in foreman in a second time.

My point of view on this will be, like DHCP smart proxy, create a monitoring smart proxy with some "monitoring providers" (like nagios, zabbix, centreon…). We use centreon monitoring tool, so i can get it work for it, and maybe one more implementation if it ease adoption (zabbix ? i don't know what monitoring tool is the most used by the foreman users), but i didn't take a look at others monitoring APIs at the moment.

step 1:

  • Find and test APIs of monitoring tools
  • Define the monitoring API v0.1 (which args the foreman pass to the monitoring smart proxy ? which information smart proxy get from monitoring tool to the foreman ?)
  • Implement 1 (or 2) provider and two actions (create/destroy host in monitoring tool)

step 2:

  • create a foreman plugin to add monitoring information to foreman host page ?

Any advise/comment or person interested ?

If it don't make sense or nobody is interested, no problem, i will make this with foreman_hook :wink:

I will be at the centos dojo and fosdem at the end of the week, so maybe we can talk about it if someone is interested.

Have a nice day.

Regards.

··· -- Baptiste AGASSE
2

> Any advise/comment or person interested ?

Hello,

first and foremost, we are adding a simple log buffer into Smart Proxy
and UI in Foreman that gives you Smart Proxy logs in comfortable format.
This is rather troubleshooting than monitoring, but it's part of "Proxy
Visibility" feature we are working on which is rather close to this.
The Smart Proxy API was merged already, the UI is emerging [1].

Second, I created a Smart Proxy Plugin [2] that gets some real-time
information from inside of the proxy process like initialized modules,
number of HTTP requests, avg request time as well as some notifications
and it integrates with collectd for transport. The plugin is rather a
prototype, but it works [2].

On the other side, there is a Foreman Plugin that presents the real time
data in Foreman. I currently implemented a table on Host detail page, a
plot widget that refreshes every second and I am gonna use pagelets to
extend Smart Proxy page with usable plots like CPU usage or memory.
Again, this is also a fully functional prototype [3].

Take a look on these, maybe you find them useful. Collectd is not a
monitoring software, it's rather a framework to read, transport and
write values and notifications. It has good integration points to
monitoring solutions, maybe you could use it as a reliable and efficient
transport.

[1] https://github.com/theforeman/foreman/pull/3054
[2] https://github.com/lzap/smart_proxy_colly
[3] https://github.com/lzap/foreman_colly

··· -- Later, Lukas #lzap Zapletal
3

I think Baptiste is suggesting an integration with existing monitoring
software such as Nagios or Zabbix, rather than having the proxy do the
monitoring itself - in much the same way as we use the proxy to add DNS
records to Bind, not run our own DNS implementation.

Baptiste, I think that's a great idea - I'd been having similar thoughts
myself. On the proxy side, appropriate API calls to add/remove hosts should
be easy enough, but how would you define the services that need monitoring?

On the UI side, take a look at foreman_cockpit - it replaces the default
graphs on the Host page with data from Cockpit. It seems to me the same
idea could be used to fetch data from the monitoring system API and display
basic health data on the Host page.

Good luck - let us know if you need any help!

Greg

4

There is some support already in the smart-proxy to allow for a plugin
such as you describe: getting ahold of events, including host-creation
related ones should is possible. What’s missing at the moment is
ability to create and execute plugins that do not expose http api,
which is coming.

Cheers,
-d

··· On Tue, Jan 26, 2016 at 9:45 AM, Greg Sutcliffe wrote: > I think Baptiste is suggesting an integration with existing monitoring > software such as Nagios or Zabbix, rather than having the proxy do the > monitoring itself - in much the same way as we use the proxy to add DNS > records to Bind, not run our own DNS implementation. > > Baptiste, I think that's a great idea - I'd been having similar thoughts > myself. On the proxy side, appropriate API calls to add/remove hosts should > be easy enough, but how would you define the services that need monitoring? > > On the UI side, take a look at foreman_cockpit - it replaces the default > graphs on the Host page with data from Cockpit. It seems to me the same idea > could be used to fetch data from the monitoring system API and display basic > health data on the Host page. > > Good luck - let us know if you need any help! > > Greg > > -- > You received this message because you are subscribed to the Google Groups > "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to foreman-dev+unsubscribe@googlegroups.com. > For more options, visit https://groups.google.com/d/optout.
5

> There is some support already in the smart-proxy to allow for a plugin
> such as you describe: getting ahold of events, including host-creation
> related ones should is possible. What’s missing at the moment is
> ability to create and execute plugins that do not expose http api,
> which is coming.

I will take a look at it, but if i have to pass parameters from foreman to monitoring tools on create action, i think maybe it should have an http api.

>
> Cheers,
> -d
>
>> I think Baptiste is suggesting an integration with existing monitoring
>> software such as Nagios or Zabbix, rather than having the proxy do the
>> monitoring itself - in much the same way as we use the proxy to add DNS
>> records to Bind, not run our own DNS implementation.

Yes, that's it.

>>
>> Baptiste, I think that's a great idea - I'd been having similar thoughts
>> myself. On the proxy side, appropriate API calls to add/remove hosts should
>> be easy enough, but how would you define the services that need monitoring?

I know that some monitoring tools have template functionality. So we can maybe pass hostgroups or maybe some pre-defined host param(s) to the monitoring smart proxy that will be mapped to monitoring templates or that provide the list of services to monitor. I don't know how to handle this now, i have to take a look at most common monitoring tools to see what is possible to do.

>>
>> On the UI side, take a look at foreman_cockpit - it replaces the default
>> graphs on the Host page with data from Cockpit. It seems to me the same idea
>> could be used to fetch data from the monitoring system API and display basic
>> health data on the Host page.

This part is for the step2, i will try to implement create/delete host in the monitoring tool and then

>>
>> Good luck - let us know if you need any help!
>>
>> Greg
>>
>> –
>> You received this message because you are subscribed to the Google Groups
>> "foreman-dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to foreman-dev+unsubscribe@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
> –
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to foreman-dev+unsubscribe@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

Have a nice day.

··· ----- Le 26 Jan 16, à 10:50, Dmitri Dolguikh witlessbird@gmail.com a écrit : > On Tue, Jan 26, 2016 at 9:45 AM, Greg Sutcliffe > wrote:

–
Baptiste