Hi,
here's what we talked about today in Brno. The goal is for the Foreman
to be able to display the information that program X crashed on host Y
at time Z. The workflow(bugflow?) will be as follows:
Whenever a crash happens on host, it sends the report to the smart
proxy. The report is JSON formatted [1] and is sent through https
connection which uses the Puppet certificate for client authentication.
The smart proxy receives the report, verifies the https connection
against Puppet CA certificate and remembers the common name from the
certificate in order to identify the host.
The smart proxy optionaly forwards the report to the ABRT server
(a.k.a. FAF).
The smart proxy formats the report to the same format as Puppet uses
and sends it to Foreman. It probably always shouldn’t do so immediatelly
as that could DOS the Foreman in case a lot of hosts will be sending
crash reports very often. It should store the reports in memory and
aggregate them according to the their identifier (a.k.a. hash). The
hash can be computed by the satyr library [2].
Foreman receives the report and displays it in some list, associated
with the host it came from.
TODO:
ABRT people will provide Ruby bindings for satyr
Foreman people will start working on the smart proxy support once
their current sprint is over
Please correct me if I got anything wrong or forgot something.
I think you described it pretty well. Only thing that I'm not sure we agreed
on is that we'll work on smart proxy part. We'd be glad to help with any
questions but I'm not sure we have manpower to write it. We could possibly
create some scaffold with empty methods. Maybe some input from Ohad would help.
Anyway I sent you mail with some tips where to start. I think the "foreman"
part will be very similar to what exists for chef reports, for other parts
like aggregation I'd say you know better how to do it.
···
--
Marek
On Tuesday 21 of January 2014 17:08:57 Martin Milata wrote:
Hi,
here’s what we talked about today in Brno. The goal is for the Foreman
to be able to display the information that program X crashed on host Y
at time Z. The workflow(bugflow?) will be as follows:
Whenever a crash happens on host, it sends the report to the smart
proxy. The report is JSON formatted [1] and is sent through https
connection which uses the Puppet certificate for client authentication.
The smart proxy receives the report, verifies the https connection
against Puppet CA certificate and remembers the common name from the
certificate in order to identify the host.
The smart proxy optionaly forwards the report to the ABRT server
(a.k.a. FAF).
The smart proxy formats the report to the same format as Puppet uses
and sends it to Foreman. It probably always shouldn’t do so immediatelly
as that could DOS the Foreman in case a lot of hosts will be sending
crash reports very often. It should store the reports in memory and
aggregate them according to the their identifier (a.k.a. hash). The
hash can be computed by the satyr library [2].
Foreman receives the report and displays it in some list, associated
with the host it came from.
TODO:
ABRT people will provide Ruby bindings for satyr
Foreman people will start working on the smart proxy support once
their current sprint is over
Please correct me if I got anything wrong or forgot something.
> Hi,
> here's what we talked about today in Brno. The goal is for the Foreman
> to be able to display the information that program X crashed on host Y
> at time Z. The workflow(bugflow?) will be as follows:
>
> ±--------+ ±----+
> > foreman | | FAF |
> ±--------+ ±----+
> ^ ^
> \ /
> \ /
> \ /
> \ /
> ±------------+
> > smart proxy |
> ±------------+
> ^
> >
> >
> ±-----+
> > host |
> ±-----+
>
> 1) Whenever a crash happens on host, it sends the report to the smart
> proxy. The report is JSON formatted [1] and is sent through https
> connection which uses the Puppet certificate for client authentication.
>
> 2) The smart proxy receives the report, verifies the https connection
> against Puppet CA certificate and remembers the common name from the
> certificate in order to identify the host.
>
> 3) The smart proxy optionaly forwards the report to the ABRT server
> (a.k.a. FAF).
>
> 4) The smart proxy formats the report to the same format as Puppet uses
> and sends it to Foreman. It probably always shouldn't do so immediatelly
> as that could DOS the Foreman in case a lot of hosts will be sending
> crash reports very often. It should store the reports in memory and
> aggregate them according to the their identifier (a.k.a. hash). The
> hash can be computed by the satyr library [2].
>
We could reuse the foreman-background plugin [1], to async handle reports,
but accept them all on foreman side, this will reduce the amount of logic
required on the proxy side.
> 5) Foreman receives the report and displays it in some list, associated
> with the host it came from.
>
> TODO:
> - ABRT people will provide Ruby bindings for satyr
> - Foreman people will start working on the smart proxy support once
> their current sprint is over
>
I would like to finalize the design first before we start defining dates
I havent attended the discussion, but why the host cannot talk to the
FAF directly?
the host —> FAF <----> smart proxy <-----> foreman
I assume that would be scenario A) in your former proposal. We kinda
discussed both A and B options and I have to admit I leaned towards B
option (the picture above), after reading this:
> 4) The smart proxy formats the report to the same format as Puppet uses
> and sends it to Foreman. It probably always shouldn't do so immediatelly
> as that could DOS the Foreman in case a lot of hosts will be sending
> crash reports very often. It should store the reports in memory and
> aggregate them according to the their identifier (a.k.a. hash). The
> hash can be computed by the satyr library [2].
I am not so sure now. I am late on emails today, but I haven't seen any
pros/cons or wider discussion why we should integrate like that.
Sure, just the scaffolding would greatly help if you have time for that.
I'll try to look at the stuff you sent and see if we can reuse some of
that in the meantime.
Thanks!
Martin
···
On Tue, Jan 21, 2014 at 17:54:58 +0100, Marek Hulan wrote:
> Hello,
>
> I think you described it pretty well. Only thing that I'm not sure we agreed
> on is that we'll work on smart proxy part. We'd be glad to help with any
> questions but I'm not sure we have manpower to write it. We could possibly
> create some scaffold with empty methods. Maybe some input from Ohad would help.
>
> Anyway I sent you mail with some tips where to start. I think the "foreman"
> part will be very similar to what exists for chef reports, for other parts
> like aggregation I'd say you know better how to do it.
>
> --
> Marek
>
> On Tuesday 21 of January 2014 17:08:57 Martin Milata wrote:
> > Hi,
> > here's what we talked about today in Brno. The goal is for the Foreman
> > to be able to display the information that program X crashed on host Y
> > at time Z. The workflow(bugflow?) will be as follows:
> >
> > +---------+ +-----+
> >
> > > foreman | | FAF |
> >
> > +---------+ +-----+
> > ^ ^
> > \ /
> > \ /
> > \ /
> > \ /
> > +-------------+
> >
> > > smart proxy |
> >
> > +-------------+
> > ^
> >
> >
> > +------+
> >
> > > host |
> >
> > +------+
> >
> > 1) Whenever a crash happens on host, it sends the report to the smart
> > proxy. The report is JSON formatted [1] and is sent through https
> > connection which uses the Puppet certificate for client authentication.
> >
> > 2) The smart proxy receives the report, verifies the https connection
> > against Puppet CA certificate and remembers the common name from the
> > certificate in order to identify the host.
> >
> > 3) The smart proxy optionaly forwards the report to the ABRT server
> > (a.k.a. FAF).
> >
> > 4) The smart proxy formats the report to the same format as Puppet uses
> > and sends it to Foreman. It probably always shouldn't do so immediatelly
> > as that could DOS the Foreman in case a lot of hosts will be sending
> > crash reports very often. It should store the reports in memory and
> > aggregate them according to the their identifier (a.k.a. hash). The
> > hash can be computed by the satyr library [2].
> >
> > 5) Foreman receives the report and displays it in some list, associated
> > with the host it came from.
> >
> > TODO:
> > - ABRT people will provide Ruby bindings for satyr
> > - Foreman people will start working on the smart proxy support once
> > their current sprint is over
> >
> > Please correct me if I got anything wrong or forgot something.
> >
> > Thanks!
> > Martin
> >
> > [1] https://github.com/abrt/faf/wiki/uReport#ureport2
> > [2] https://github.com/abrt/satyr
>
>
> > Hi,
> > here's what we talked about today in Brno. The goal is for the Foreman
> > to be able to display the information that program X crashed on host Y
> > at time Z. The workflow(bugflow?) will be as follows:
> >
> > ±--------+ ±----+
> > > foreman | | FAF |
> > ±--------+ ±----+
> > ^ ^
> > \ /
> > \ /
> > \ /
> > \ /
> > ±------------+
> > > smart proxy |
> > ±------------+
> > ^
> > >
> > >
> > ±-----+
> > > host |
> > ±-----+
> >
> > 1) Whenever a crash happens on host, it sends the report to the smart
> > proxy. The report is JSON formatted [1] and is sent through https
> > connection which uses the Puppet certificate for client authentication.
> >
> > 2) The smart proxy receives the report, verifies the https connection
> > against Puppet CA certificate and remembers the common name from the
> > certificate in order to identify the host.
> >
> > 3) The smart proxy optionaly forwards the report to the ABRT server
> > (a.k.a. FAF).
> >
> > 4) The smart proxy formats the report to the same format as Puppet uses
> > and sends it to Foreman. It probably always shouldn't do so immediatelly
> > as that could DOS the Foreman in case a lot of hosts will be sending
> > crash reports very often. It should store the reports in memory and
> > aggregate them according to the their identifier (a.k.a. hash). The
> > hash can be computed by the satyr library [2].
>
> We could reuse the foreman-background plugin [1], to async handle reports,
> but accept them all on foreman side, this will reduce the amount of logic
> required on the proxy side.
Thanks for the tip, I'll take a look at it.
> > 5) Foreman receives the report and displays it in some list, associated
> > with the host it came from.
> >
> > TODO:
> > - ABRT people will provide Ruby bindings for satyr
> > - Foreman people will start working on the smart proxy support once
> > their current sprint is over
> >
>
> I would like to finalize the design first before we start defining dates
>
Sorry, I didn't mean to define a date, just wanted to note that it was
mentioned you cannot help us before the current sprint is over.
On our side, we have to understand the foreman design first, perhaps
writing some proof-of-concept code in the progress. If you have any
suggestions or questions regarding the ABRT support, feel free to let me
know.
Martin
···
On Wed, Jan 22, 2014 at 11:46:23 +0200, Ohad Levy wrote:
> On Tue, Jan 21, 2014 at 6:08 PM, Martin Milata wrote:
>> ±--------+ ±----+
>> > foreman | | FAF |
>> ±--------+ ±----+
>> ^ ^
>> \ /
>> \ /
>> \ /
>> \ /
>> ±------------+
>> > smart proxy |
>> ±------------+
>> ^
>> >
>> >
>> ±-----+
>> > host |
>> ±-----+
>
> I havent attended the discussion, but why the host cannot talk to the
> FAF directly?
>
> the host —> FAF <----> smart proxy <-----> foreman
>
>
> I assume that would be scenario A) in your former proposal. We kinda
> discussed both A and B options and I have to admit I leaned towards B
> option (the picture above), after reading this:
>
>> 4) The smart proxy formats the report to the same format as Puppet uses
>> and sends it to Foreman. It probably always shouldn't do so immediatelly
>> as that could DOS the Foreman in case a lot of hosts will be sending
>> crash reports very often. It should store the reports in memory and
>> aggregate them according to the their identifier (a.k.a. hash). The
>> hash can be computed by the satyr library [2].
>
> I am not so sure now. I am late on emails today, but I haven't seen any
> pros/cons or wider discussion why we should integrate like that.
the main idea is that managed hosts dont need to have direct
connection to a secure foreman server.
Ohad
···
On Thu, Jan 23, 2014 at 4:35 PM, Lukas Zapletal wrote:
>
> --
> Later,
>
> Lukas "lzap" Zapletal
> irc: lzap #theforeman
>
> --
> You received this message because you are subscribed to the Google Groups "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscribe@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
The reason is that we'd like to make use of the reports without the need
to install FAF, as:
FAF is rather heavyweight piece of software that has lots of
dependencies, needs SQL database, etc.
Its features are mostly targeted at developers of the software for
which the reports are generated. I assume that it wouldn't be actually
that useful for typical foreman user.
The architecture in the picture allows you to be informed about crashes
on your managed hosts. Optionally, you can deploy your own FAF instance
for better crash statistics and more crash data, or send the reports to
FAF instance of your software supplier (e.g. Red Hat).
···
On Thu, Jan 23, 2014 at 15:35:41 +0100, Lukas Zapletal wrote:
> > +---------+ +-----+
> > > foreman | | FAF |
> > +---------+ +-----+
> > ^ ^
> > \ /
> > \ /
> > \ /
> > \ /
> > +-------------+
> > > smart proxy |
> > +-------------+
> > ^
> > >
> > >
> > +------+
> > > host |
> > +------+
>
> I havent attended the discussion, but why the host cannot talk to the
> FAF directly?
>
> the host ---> FAF <----> smart proxy <-----> foreman
>
> I assume that would be scenario A) in your former proposal. We kinda
> discussed both A and B options and I have to admit I leaned towards B
> option (the picture above), after reading this:
>
> > 4) The smart proxy formats the report to the same format as Puppet uses
> > and sends it to Foreman. It probably always shouldn't do so immediatelly
> > as that could DOS the Foreman in case a lot of hosts will be sending
> > crash reports very often. It should store the reports in memory and
> > aggregate them according to the their identifier (a.k.a. hash). The
> > hash can be computed by the satyr library [2].
>
> I am not so sure now. I am late on emails today, but I haven't seen any
> pros/cons or wider discussion why we should integrate like that.
> the main idea is that managed hosts dont need to have direct
> connection to a secure foreman server.
I see, so proxy just mediates the upload to FAF, which is the right
place to process the stuff and hand over statistics to the Foreman
instance. Looks sane to me.
> The reason is that we'd like to make use of the reports without the need
> to install FAF, as:
> - FAF is rather heavyweight piece of software that has lots of
> dependencies, needs SQL database, etc.
> - Its features are mostly targeted at developers of the software for
> which the reports are generated. I assume that it wouldn't be actually
> that useful for typical foreman user.
>
> The architecture in the picture allows you to be informed about crashes
> on your managed hosts. Optionally, you can deploy your own FAF instance
> for better crash statistics and more crash data, or send the reports to
> FAF instance of your software supplier (e.g. Red Hat).
