E-mail in Foreman 1.7 / Katello 2.1

Development
#1

Hi all,

One of the issues I'm working on this sprint is designing #7129 "As a user, I
should get errata notifications via email that indicate when new errata need to
be applied and to what hosts."

In order to accomplish that, it would be nice to have a framework for e-mail in
Foreman. Partha and I talked a bit about it yesterday. Eric also has some notes
on the Katello issue: Feature #7129: As a user, I should get errata notifications via email that indicate when new errata need to be applied and to what hosts. - Katello - Foreman

Here's a short summary of the ideas/goals. I'll send out an invite for a quick
30 minute discussion next week - would google hangouts be good for that?

I'm thinking of two types of e-mails to send:

- Summaries (recurring)
- Notifications (ad hoc)

Foreman mail notifications today:
Mail Notifications - Foreman

Currently, Foreman sends it's emails via a unix cron job that launches a rake
task. We can take that and make it more generic - run every 30 or 60 minutes to
poll for any mail it needs to send. Notifications would be live, of course.

For Katello, we could use the same strategy, but have our own rake task to hand
off the summary generation to an async task instead. AFAIK, there's no built-in
scheduling ability in foreman-tasks today, right? I know there's some gems out
there that support various scheduling activites, but I do prefer cron.

As part of this, we would introduce a mail preferences tab to the My Account
page where a user can decide what they want to receive. Plugins, of course,
would be able to register their own items easily.

These would all be confined by the Foreman permissions model, so you can limit
which notifications/summaries a user can view, and the content of them as well
would be limited to what the user can see.

For summaries, they should be generic - e-mailable, viewable with hammer, and
maybe eventually in the UI under Monitor.

Hammer would look something like:
hammer summary show --name katello-daily-errata

To start out with, I'm thinking:
- Daily Errata Report (what hosts have what errata needs)
- Puppet summary (existing summary in Foremna)
- Subscription Usage? (Feature #7536: As a user I would like a way to view reports on Subscription Consumption in Katello - Katello - Foreman)

For notifications, I'm thinking of being able to subscribe to events like:

  • Bad puppet report (already there today in the rake task)
  • New errata (Option to receive only security, or the mail bomb of everything
    like Spacewalk)

Thoughts?

Thanks!

··· -- Stephen Benjamin

Red Hat GmbH | http://de.redhat.com/ | Sitz: Grasbrunn
Handelsregister: Amtsgericht München, HRB 153243
Geschäftsführer: Charles Cachera, Michael Cunningham,
Michael O’Neill, Charles Peters

#2

> Hi all,
>
> One of the issues I'm working on this sprint is designing #7129 "As a user, I
> should get errata notifications via email that indicate when new errata need to
> be applied and to what hosts."
>
> In order to accomplish that, it would be nice to have a framework for e-mail in
> Foreman. Partha and I talked a bit about it yesterday. Eric also has some notes
> on the Katello issue: Feature #7129: As a user, I should get errata notifications via email that indicate when new errata need to be applied and to what hosts. - Katello - Foreman
>
> Here's a short summary of the ideas/goals. I'll send out an invite for a quick
> 30 minute discussion next week - would google hangouts be good for that?
>
> I'm thinking of two types of e-mails to send:
>
> - Summaries (recurring)
> - Notifications (ad hoc)

How does this relate to audits? Can we make the trigger be audits?

>
> Foreman mail notifications today:
> Mail Notifications - Foreman

Some things I would ask:

  • Where does the text of the email come from? Is this in an erb template
    like other text items? Ideally, I can edit the templates in the webapp.
  • How do the templates and configuration work in a multi-org
    environment? Can the mail server be unique? Can the from be unique? Can
    the text be unique?
  • Localization: Can you and I get the same notification, but in
    different languages?

>
> Currently, Foreman sends it's emails via a unix cron job that launches a rake
> task. We can take that and make it more generic - run every 30 or 60 minutes to
> poll for any mail it needs to send. Notifications would be live, of course.
>
> For Katello, we could use the same strategy, but have our own rake task to hand
> off the summary generation to an async task instead. AFAIK, there's no built-in
> scheduling ability in foreman-tasks today, right? I know there's some gems out
> there that support various scheduling activites, but I do prefer cron.

Are you saying katello has its own email logic? This seems like
something we would want in the core that any plugin can use.

>
> As part of this, we would introduce a mail preferences tab to the My Account
> page where a user can decide what they want to receive. Plugins, of course,
> would be able to register their own items easily.
>
> These would all be confined by the Foreman permissions model, so you can limit
> which notifications/summaries a user can view, and the content of them as well
> would be limited to what the user can see.
>
> For summaries, they should be generic - e-mailable, viewable with hammer, and
> maybe eventually in the UI under Monitor.
>
> Hammer would look something like:
> hammer summary show --name katello-daily-errata
>
> To start out with, I'm thinking:
> - Daily Errata Report (what hosts have what errata needs)
> - Puppet summary (existing summary in Foremna)
> - Subscription Usage? (Feature #7536: As a user I would like a way to view reports on Subscription Consumption in Katello - Katello - Foreman)
>
> For notifications, I'm thinking of being able to subscribe to events like:
>
> - Bad puppet report (already there today in the rake task)
> - New errata (Option to receive only security, or the mail bomb of everything
> like Spacewalk)
>

– bk

··· On 09/19/2014 07:50 AM, Stephen Benjamin wrote:
#3

> Hi all,
>
> One of the issues I'm working on this sprint is designing #7129 "As a user, I
> should get errata notifications via email that indicate when new errata need to
> be applied and to what hosts."
>
> In order to accomplish that, it would be nice to have a framework for e-mail in
> Foreman. Partha and I talked a bit about it yesterday. Eric also has some notes
> on the Katello issue: Feature #7129: As a user, I should get errata notifications via email that indicate when new errata need to be applied and to what hosts. - Katello - Foreman
>
> Here's a short summary of the ideas/goals. I'll send out an invite for a quick
> 30 minute discussion next week - would google hangouts be good for that?
>
> I'm thinking of two types of e-mails to send:
>
> - Summaries (recurring)
> - Notifications (ad hoc)
>
> Foreman mail notifications today:
> Mail Notifications - Foreman
>
> Currently, Foreman sends it's emails via a unix cron job that launches a rake
> task. We can take that and make it more generic - run every 30 or 60 minutes to
> poll for any mail it needs to send. Notifications would be live, of course.
>
> For Katello, we could use the same strategy, but have our own rake task to hand
> off the summary generation to an async task instead. AFAIK, there's no built-in
> scheduling ability in foreman-tasks today, right? I know there's some gems out
> there that support various scheduling activites, but I do prefer cron.
I wasn't 100% sure here what the full goal was. If the goal is to allow
the user to set their own schedule I'd prefer we use an existing gem to
do the scheduling.

If the goal is for schedules that are hard coded per task (i.e. daily
errata notification) I like this approach and i think its a great first
step.

-Justin

··· On 09/19/2014 07:50 AM, Stephen Benjamin wrote:

As part of this, we would introduce a mail preferences tab to the My Account
page where a user can decide what they want to receive. Plugins, of course,
would be able to register their own items easily.

These would all be confined by the Foreman permissions model, so you can limit
which notifications/summaries a user can view, and the content of them as well
would be limited to what the user can see.

For summaries, they should be generic - e-mailable, viewable with hammer, and
maybe eventually in the UI under Monitor.

Hammer would look something like:
hammer summary show --name katello-daily-errata

To start out with, I’m thinking:
- Daily Errata Report (what hosts have what errata needs)
- Puppet summary (existing summary in Foremna)
- Subscription Usage? (Feature #7536: As a user I would like a way to view reports on Subscription Consumption in Katello - Katello - Foreman)

For notifications, I’m thinking of being able to subscribe to events like:

  • Bad puppet report (already there today in the rake task)
  • New errata (Option to receive only security, or the mail bomb of everything
    like Spacewalk)

Thoughts?

Thanks!

–
Stephen Benjamin

Red Hat GmbH | http://de.redhat.com/ | Sitz: Grasbrunn
Handelsregister: Amtsgericht München, HRB 153243
Geschäftsführer: Charles Cachera, Michael Cunningham,
Michael O’Neill, Charles Peters

#4

>
>
> >Hi all,
> >
> >One of the issues I'm working on this sprint is designing #7129 "As a user, I
> >should get errata notifications via email that indicate when new errata need to
> >be applied and to what hosts."
> >
> >In order to accomplish that, it would be nice to have a framework for e-mail in
> >Foreman. Partha and I talked a bit about it yesterday. Eric also has some notes
> >on the Katello issue: Feature #7129: As a user, I should get errata notifications via email that indicate when new errata need to be applied and to what hosts. - Katello - Foreman
> >
> >Here's a short summary of the ideas/goals. I'll send out an invite for a quick
> >30 minute discussion next week - would google hangouts be good for that?
> >
> >I'm thinking of two types of e-mails to send:
> >
> > - Summaries (recurring)
> > - Notifications (ad hoc)
>
> How does this relate to audits? Can we make the trigger be audits?
>
> >
> >Foreman mail notifications today:
> > Mail Notifications - Foreman
>
> Some things I would ask:
>
> * Where does the text of the email come from? Is this in an erb template
> like other text items? Ideally, I can edit the templates in the webapp.

Ultimately an ERB. What reports would user want to write by hand?

This would be difficult, though.

The logic for the reports belongs in a helper or something, it will get
messy to have a fully ERB only template. Provisioning templates are
simpler.

We'd also need to really way open the Jail to allow a lot more than it
does, I think that'd be alot of work to carefully expose the interesting
reporting objects to a user without giving them the ability to do bad
things.

If a user really wants custom reports, they'd probably be better off
writing foreman extensions with a plug-in, although that's not
sysadmin-friendly.

> * How do the templates and configuration work in a multi-org environment?

I don't see that being an issue at first, we'll ship default summaries,
and send them to the user with only the objects they can see.

We could eventually add the ability to filter by organzations and such
if it's really needed.

> Can the mail server be unique?
> Can the from be unique? Can the text be
> unique?

These should probably be Foreman settings, and at the moment, these are
global, there's no option for per-taxonomy settings. That's a whole
different can of worms.

> * Localization: Can you and I get the same notification, but in different
> languages?

Yes, I assume that will just work by making sure the strings are marked
correctly.

> >
> >Currently, Foreman sends it's emails via a unix cron job that launches a rake
> >task. We can take that and make it more generic - run every 30 or 60 minutes to
> >poll for any mail it needs to send. Notifications would be live, of course.
> >
> >For Katello, we could use the same strategy, but have our own rake task to hand
> >off the summary generation to an async task instead. AFAIK, there's no built-in
> >scheduling ability in foreman-tasks today, right? I know there's some gems out
> >there that support various scheduling activites, but I do prefer cron.
>
> Are you saying katello has its own email logic? This seems like something we
> would want in the core that any plugin can use.

No, the only difference is how the jobs get launched.

Foreman core, if it continues to ship without foreman-tasks, will need
to have a rake task that kicks off the email generation. Katello can do
the same, but instead of doing the e-mails in the rake task itself,
would push it off as an async foreman-tasks job.

Unless Foreman team wants to ship foreman-tasks with 1.7 by default. I'd
love to see that.

··· On Fri, Sep 19, 2014 at 09:36:49AM -0400, Bryan Kearney wrote: > On 09/19/2014 07:50 AM, Stephen Benjamin wrote:

As part of this, we would introduce a mail preferences tab to the My Account
page where a user can decide what they want to receive. Plugins, of course,
would be able to register their own items easily.

These would all be confined by the Foreman permissions model, so you can limit
which notifications/summaries a user can view, and the content of them as well
would be limited to what the user can see.

For summaries, they should be generic - e-mailable, viewable with hammer, and
maybe eventually in the UI under Monitor.

Hammer would look something like:
hammer summary show --name katello-daily-errata

To start out with, I’m thinking:
- Daily Errata Report (what hosts have what errata needs)
- Puppet summary (existing summary in Foremna)
- Subscription Usage? (Feature #7536: As a user I would like a way to view reports on Subscription Consumption in Katello - Katello - Foreman)

For notifications, I’m thinking of being able to subscribe to events like:

  • Bad puppet report (already there today in the rake task)
  • New errata (Option to receive only security, or the mail bomb of everything
    like Spacewalk)

– bk

–
You received this message because you are subscribed to the Google Groups “foreman-dev” group.
To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

–
Stephen Benjamin

Red Hat GmbH | http://de.redhat.com/ | Sitz: Grasbrunn
Handelsregister: Amtsgericht München, HRB 153243
Geschäftsführer: Charles Cachera, Michael Cunningham,
Michael O’Neill, Charles Peters

#5

Yea, that's what I was thinking, it would be easy enough to just send
reports at specific times, and then add user-selected delivery time
later. Maybe foreman-tasks could eventually support scheduling?

··· On Fri, Sep 19, 2014 at 10:56:14AM -0400, Justin Sherrill wrote: > On 09/19/2014 07:50 AM, Stephen Benjamin wrote: > >Hi all, > > > >One of the issues I'm working on this sprint is designing #7129 "As a user, I > >should get errata notifications via email that indicate when new errata need to > >be applied and to what hosts." > > > >In order to accomplish that, it would be nice to have a framework for e-mail in > >Foreman. Partha and I talked a bit about it yesterday. Eric also has some notes > >on the Katello issue: http://projects.theforeman.org/issues/7129 > > > >Here's a short summary of the ideas/goals. I'll send out an invite for a quick > >30 minute discussion next week - would google hangouts be good for that? > > > >I'm thinking of two types of e-mails to send: > > > > - Summaries (recurring) > > - Notifications (ad hoc) > > > >Foreman mail notifications today: > > http://projects.theforeman.org/projects/foreman/wiki/Mail_Notifications > > > >Currently, Foreman sends it's emails via a unix cron job that launches a rake > >task. We can take that and make it more generic - run every 30 or 60 minutes to > >poll for any mail it needs to send. Notifications would be live, of course. > > > >For Katello, we could use the same strategy, but have our own rake task to hand > >off the summary generation to an async task instead. AFAIK, there's no built-in > >scheduling ability in foreman-tasks today, right? I know there's some gems out > >there that support various scheduling activites, but I do prefer cron. > I wasn't 100% sure here what the full goal was. If the goal is to allow the > user to set their own schedule I'd prefer we use an existing gem to do the > scheduling. > > If the goal is for schedules that are hard coded per task (i.e. daily errata > notification) I like this approach and i think its a great first step.
#6

>>
>>
>>> Hi all,
>>>
>>> One of the issues I'm working on this sprint is designing #7129 "As a user, I
>>> should get errata notifications via email that indicate when new errata need to
>>> be applied and to what hosts."
>>>
>>> In order to accomplish that, it would be nice to have a framework for e-mail in
>>> Foreman. Partha and I talked a bit about it yesterday. Eric also has some notes
>>> on the Katello issue: Feature #7129: As a user, I should get errata notifications via email that indicate when new errata need to be applied and to what hosts. - Katello - Foreman
>>>
>>> Here's a short summary of the ideas/goals. I'll send out an invite for a quick
>>> 30 minute discussion next week - would google hangouts be good for that?
>>>
>>> I'm thinking of two types of e-mails to send:
>>>
>>> - Summaries (recurring)
>>> - Notifications (ad hoc)
>>
>> How does this relate to audits? Can we make the trigger be audits?
>>
>>>
>>> Foreman mail notifications today:
>>> Mail Notifications - Foreman
>>
>> Some things I would ask:
>>
>> * Where does the text of the email come from? Is this in an erb template
>> like other text items? Ideally, I can edit the templates in the webapp.
>
> Ultimately an ERB. What reports would user want to write by hand?
>
> This would be difficult, though.
>
> The logic for the reports belongs in a helper or something, it will get
> messy to have a fully ERB only template. Provisioning templates are
> simpler.
>
> We'd also need to really way open the Jail to allow a lot more than it
> does, I think that'd be alot of work to carefully expose the interesting
> reporting objects to a user without giving them the ability to do bad
> things.
>
> If a user really wants custom reports, they'd probably be better off
> writing foreman extensions with a plug-in, although that's not
> sysadmin-friendly.
>

user-editable email templates seems way beyond the scope of the initial
feature. Lets get basic emailing going and if later on we want to be
able to offer users the ability to edit the templates, as you can with
say, Provisioning Template we could go there.

>> * How do the templates and configuration work in a multi-org environment?
>
> I don't see that being an issue at first, we'll ship default summaries,
> and send them to the user with only the objects they can see.
>
> We could eventually add the ability to filter by organzations and such
> if it's really needed.
>
>> Can the mail server be unique?
>> Can the from be unique? Can the text be
>> unique?
>
> These should probably be Foreman settings, and at the moment, these are
> global, there's no option for per-taxonomy settings. That's a whole
> different can of worms.

we really shouldn't need this at the start, IMO

>
>> * Localization: Can you and I get the same notification, but in different
>> languages?
>
> Yes, I assume that will just work by making sure the strings are marked
> correctly.

the user would be required to have a Locale set on their preferences, if
it was marked as 'Browser Locale' they would be stuck on en_US unless we
did some fancy auto-updating based on their last login and whatever the
Accept-Language was sent down.

>
>>>
>>> Currently, Foreman sends it's emails via a unix cron job that launches a rake
>>> task. We can take that and make it more generic - run every 30 or 60 minutes to
>>> poll for any mail it needs to send. Notifications would be live, of course.
>>>
>>> For Katello, we could use the same strategy, but have our own rake task to hand
>>> off the summary generation to an async task instead. AFAIK, there's no built-in
>>> scheduling ability in foreman-tasks today, right? I know there's some gems out
>>> there that support various scheduling activites, but I do prefer cron.
>>
>> Are you saying katello has its own email logic? This seems like something we
>> would want in the core that any plugin can use.
>
> No, the only difference is how the jobs get launched.
>
> Foreman core, if it continues to ship without foreman-tasks, will need
> to have a rake task that kicks off the email generation. Katello can do
> the same, but instead of doing the e-mails in the rake task itself,
> would push it off as an async foreman-tasks job.
>
> Unless Foreman team wants to ship foreman-tasks with 1.7 by default. I'd
> love to see that.
>
>>>
>>> As part of this, we would introduce a mail preferences tab to the My Account
>>> page where a user can decide what they want to receive. Plugins, of course,
>>> would be able to register their own items easily.
>>>
>>> These would all be confined by the Foreman permissions model, so you can limit
>>> which notifications/summaries a user can view, and the content of them as well
>>> would be limited to what the user can see.
>>>
>>> For summaries, they should be generic - e-mailable, viewable with hammer, and
>>> maybe eventually in the UI under Monitor.
>>>
>>> Hammer would look something like:
>>> hammer summary show --name katello-daily-errata
>>>
>>> To start out with, I'm thinking:
>>> - Daily Errata Report (what hosts have what errata needs)
>>> - Puppet summary (existing summary in Foremna)
>>> - Subscription Usage? (Feature #7536: As a user I would like a way to view reports on Subscription Consumption in Katello - Katello - Foreman)
>>>
>>> For notifications, I'm thinking of being able to subscribe to events like:
>>>
>>> - Bad puppet report (already there today in the rake task)
>>> - New errata (Option to receive only security, or the mail bomb of everything
>>> like Spacewalk)
>>>
>>

we can start off with 'on/off' preferences for each type of notification
and later on can add a frequency as well if users desire these things in
a higher volume.

Mike

··· On 09/19/2014 07:46 AM, Stephen Benjamin wrote: > On Fri, Sep 19, 2014 at 09:36:49AM -0400, Bryan Kearney wrote: >> On 09/19/2014 07:50 AM, Stephen Benjamin wrote:

–
Mike McCune
mmccune AT redhat.com
Red Hat Engineering | Portland, OR
Systems Management | 650-254-4248