Regular Foreman-core issue triage?

So on IRC the idea of a regular issue triage for core issues in redmine
came up, and I think it's a pretty good idea.

I think we'd want to do this on a public stream (but recorded, I think),
and then anyone interested can join. We'd need a minimum number of
people involved to make it work, I guess we're looking at anyone with
commit / interest in foreman core itself. Proxy, plugins (and possibly
the installer) are out of scope, at least to start with.

Is anyone interested in participating in this? When should we hold it?
I'm guessing around lunch-time GMT if we want maximum coverage, or we
could alternate EMEA-friendly/US-East-friendly time slots? Avoid
Thursday if you can as demos and other recorded content tend to happen
then :wink:

Greg

I partially suggested in light of the concerns I heard on IRC and how the
Katello plugin has dealt with similar issues by having a weekly triage.
We've been doing this for quite a while and I think it works quite well to
address every issue. What follows is a run down of how we manage that
process. I'll start with some background information that will help the
process explanation make more sense. The Pulp team runs a community triage
via IRC using a bot to help out which is another avenue that could be
pursued.

If any of the Katello triage runners see something I missed or mis-quote
here please correct me. Also, as I finished writing this I realized I
should transfer this to our official docs.

Triage Runner

We have a few standing developers that switch off running triage. Typically
it is myself or Justin but we try to get the current release owner to jump
in and run it as well as encouraging others to take it on. Takes one or two
to get the hang of it, but it is a good experience.

Frequency

Triage is done once a week for an hour. As long as a week is not skipped,
this has been enough time to get through every issue and leave 5-15 minutes
extra. When a release is pending, or a new release has been sent out the
volume does spike.

Issue States

There are a few important states that issues can be in that are worth
knowing about before going into the process:

  • New: Unassigned issue
  • Assigned: Developer has taken responsibility
  • Needs more information: User who filed the issue has been asked for
    further information to help triage or debug the issue
  • Needs design: The issue requires investigation by a developer
  • Ready for Testing: An issue has an open PR
  • Closed: An issue has been fixed

Release States

There are a few important states the issue can be in with respect to the
'Release' field within Redmine. These release states have an impact not
only a release itself but the triage process as a whole. A quick definition
of each that will make more sense in the process section:

  • Empty: No release has been set, this is the "untriaged" bucket
  • Katello Backlog: Issues that have been put on the backlog and not
    scheduled for a release due to any number of reasons
  • Katello Recycle Bin: Issues that have been rejected, or are a
    duplicate, or have been in 'Needs more information' for longer than a week
  • Katello X.Y.Z: Standard releases for issues to be targeted at

Process

The triage runner starts the meeting by going through sets of issues broken
down into different groups by priority of the grouping. This is so that if
time were to run out, the most critical categories of issues are triaged.
This list (with easy links for the runner) are at [1]. The runner is in
charge of ensuring the flow of the meeting, and updating issues or ensuring
that someone on the call takes responsibility for updating an issue. This
latter part sometimes occurs if a duplicate needs to be looked up, or
another developer on the call can explain/ask the user for more information
more clearly than the runner.

Recycle Bin

The first on the list is our 'recycle bin'. This the place to put issues
that don't fit into a release, or were filed by a user who has since gone
dark on the issue. Another way to look at it is a place for unresolvable
issues. Issues are moved onto the recycle bin if there has been no activity
by a user for a week. And they are closed if a week on the recycle bin goes
by without any further updates.

General Flow for New or Needs More Information

The general flow is to open an issue and the runner to exam the issue,
typically reading the general idea outloud. This allows for any developers
with knowledge to chime in with any relevant info. The runner attempts to
answers the following questions:

  • Does this issue need more information from the user?
    • If yes, leave a comment asking for that information
  • Does this issue need a developer to investigate it further to properly
    triage it?
  • What release should the issue be targeted for? (e.g. next z-stream,
    current release, next release, backlog)
  • Does any developer want to own it?
    • If yes, assign it
  • Issue category?

General Flow for Ready for Testing

  • Will this PR get completed by the next release?
  • What release should the issue be targeted for? (e.g. next z-stream,
    current release, next release, backlog)

General Flow for Closed

  • Should this issue be targeted at the next z-stream? Current RC phase?
    Or just throw it in the "next" release (aka develop or master)

Notes

A few notes about some automatic operations that are performed by the
prprocessor that help facilitate this workflow.

  • All issues are set to an empty release, which again represents
    "untriaged"
  • Any issues on the backlog that a PR is opened for has its release set
    to 'empty' to ensure that it flows through the triage process
  • Developers that believe an issue should be considered for a release,
    should set the release to 'empty' and show up to triage to argue their case

[1] Foreman :: Plugin Manuals

··· On Wed, Nov 8, 2017 at 10:28 AM, Greg Sutcliffe wrote:

So on IRC the idea of a regular issue triage for core issues in redmine
came up, and I think it’s a pretty good idea.

I think we’d want to do this on a public stream (but recorded, I think),
and then anyone interested can join. We’d need a minimum number of
people involved to make it work, I guess we’re looking at anyone with
commit / interest in foreman core itself. Proxy, plugins (and possibly
the installer) are out of scope, at least to start with.

Is anyone interested in participating in this? When should we hold it?
I’m guessing around lunch-time GMT if we want maximum coverage, or we
could alternate EMEA-friendly/US-East-friendly time slots? Avoid
Thursday if you can as demos and other recorded content tend to happen
then :wink:

Greg


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Eric D. Helms
Red Hat Engineering

I'd join regularly, after few years for which I receive all notifications
from redmine, I can confirm there are bugs without much attention.

If we won't have representatives from all areas, we might need some tooling
to ping people in redmine tickets. Again, after few years, I can tell that
mentioning name in comment does not always work. There used to be a
question plugin which works similarly as needinfo in BZ. Perhaps we could
install it?

Thanks Greg for bringing this up

··· -- Marek

On November 8, 2017 16:28:59 Greg Sutcliffe greg@emeraldreverie.org wrote:

So on IRC the idea of a regular issue triage for core issues in redmine
came up, and I think it’s a pretty good idea.

I think we’d want to do this on a public stream (but recorded, I think),
and then anyone interested can join. We’d need a minimum number of
people involved to make it work, I guess we’re looking at anyone with
commit / interest in foreman core itself. Proxy, plugins (and possibly
the installer) are out of scope, at least to start with.

Is anyone interested in participating in this? When should we hold it?
I’m guessing around lunch-time GMT if we want maximum coverage, or we
could alternate EMEA-friendly/US-East-friendly time slots? Avoid
Thursday if you can as demos and other recorded content tend to happen
then :wink:

Greg


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[tons of useful stuff]

Thanks Eric! I think that format will work for us too, might take a
little practice. We'll need volunteers to be the runner, ofc :wink:

> I'd join regularly, after few years for which I receive all
> notifications from redmine, I can confirm there are bugs without much
> attention.
>
> If we won't have representatives from all areas, we might need some
> tooling to ping people in redmine tickets. Again, after few years, I can
> tell that mentioning name in comment does not always work. There used to
> be a question plugin which works similarly as needinfo in BZ. Perhaps we
> could install it?

http://www.redmine.org/projects/redmine/wiki/PluginQuestion is what
you're referring to right? Looks nice, I can look into adding it - some
Redmine work is definitely on my short-term todo list anyway.

> Thanks Greg for bringing this up

Still looking for suggested time slots. Perhaps I should create a poll?

Greg

··· On 08/11/17 16:47, Eric D Helms wrote: On 09/11/17 07:03, Marek Hulan wrote:

Writing this up inspired me to capture it for the long term [1]. I'd be
happy to run the first one or two given my experience with it (and assuming
the timeslot works) just to get into the groove. Note that our process for
triaging does require some overall Redmine process change with the way we
uses some of the empty and Recycle Bin releases.

[1] https://github.com/theforeman/theforeman.org/pull/970

··· On Thu, Nov 9, 2017 at 7:54 AM, Greg Sutcliffe wrote:

On 08/11/17 16:47, Eric D Helms wrote:
[tons of useful stuff]

Thanks Eric! I think that format will work for us too, might take a
little practice. We’ll need volunteers to be the runner, ofc :wink:

On 09/11/17 07:03, Marek Hulan wrote:

I’d join regularly, after few years for which I receive all
notifications from redmine, I can confirm there are bugs without much
attention.

If we won’t have representatives from all areas, we might need some
tooling to ping people in redmine tickets. Again, after few years, I can
tell that mentioning name in comment does not always work. There used to
be a question plugin which works similarly as needinfo in BZ. Perhaps we
could install it?

http://www.redmine.org/projects/redmine/wiki/PluginQuestion is what
you’re referring to right? Looks nice, I can look into adding it - some
Redmine work is definitely on my short-term todo list anyway.

Thanks Greg for bringing this up

Still looking for suggested time slots. Perhaps I should create a poll?

Greg


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Eric D. Helms
Red Hat Engineering

Bump - any thoughts here from the -dev community? Simple +1 or -1 or even a
+0 for indifference would be great to just know where folks stand and
whether this would be worth while.

··· On Thu, Nov 9, 2017 at 8:28 AM, Eric D Helms wrote:

Writing this up inspired me to capture it for the long term [1]. I’d be
happy to run the first one or two given my experience with it (and assuming
the timeslot works) just to get into the groove. Note that our process for
triaging does require some overall Redmine process change with the way we
uses some of the empty and Recycle Bin releases.

[1] https://github.com/theforeman/theforeman.org/pull/970

On Thu, Nov 9, 2017 at 7:54 AM, Greg Sutcliffe greg@emeraldreverie.org > wrote:

On 08/11/17 16:47, Eric D Helms wrote:
[tons of useful stuff]

Thanks Eric! I think that format will work for us too, might take a
little practice. We’ll need volunteers to be the runner, ofc :wink:

On 09/11/17 07:03, Marek Hulan wrote:

I’d join regularly, after few years for which I receive all
notifications from redmine, I can confirm there are bugs without much
attention.

If we won’t have representatives from all areas, we might need some
tooling to ping people in redmine tickets. Again, after few years, I can
tell that mentioning name in comment does not always work. There used to
be a question plugin which works similarly as needinfo in BZ. Perhaps we
could install it?

http://www.redmine.org/projects/redmine/wiki/PluginQuestion is what
you’re referring to right? Looks nice, I can look into adding it - some
Redmine work is definitely on my short-term todo list anyway.

Thanks Greg for bringing this up

Still looking for suggested time slots. Perhaps I should create a poll?

Greg


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Eric D. Helms
Red Hat Engineering


Eric D. Helms
Red Hat Engineering

It would be interesting to try giving this a try, I think right now we
don't much insight into new issues raised and no prioritization of
them.
Previously Dominic used to do at least a minimal triage on all
incoming issues, but right now I don't think anyone does that.
One concern though is the amount of time it would take - the volume of
issues in Foreman is much larger then in Katello, and if it takes the
Katello team an hour per week to do, i assume it would likely take
double that for Foreman. Multiply that by the number of developers
taking part in the triage meeting and you lose about a developer day
every week.
So, to sum it up - I wouldn't mind trying for a few weeks but we
should be mindful of the time spent and whether the value gained from
it is worth it.

··· On Wed, Nov 15, 2017 at 12:12 AM, Eric D Helms wrote: > Bump - any thoughts here from the -dev community? Simple +1 or -1 or even a > +0 for indifference would be great to just know where folks stand and > whether this would be worth while. > > On Thu, Nov 9, 2017 at 8:28 AM, Eric D Helms wrote: >> >> Writing this up inspired me to capture it for the long term [1]. I'd be >> happy to run the first one or two given my experience with it (and assuming >> the timeslot works) just to get into the groove. Note that our process for >> triaging does require some overall Redmine process change with the way we >> uses some of the empty and Recycle Bin releases. >> >> >> [1] https://github.com/theforeman/theforeman.org/pull/970 >> >> On Thu, Nov 9, 2017 at 7:54 AM, Greg Sutcliffe >> wrote: >>> >>> On 08/11/17 16:47, Eric D Helms wrote: >>> [tons of useful stuff] >>> >>> Thanks Eric! I think that format will work for us too, might take a >>> little practice. We'll need volunteers to be the runner, ofc ;) >>> >>> On 09/11/17 07:03, Marek Hulan wrote: >>> > I'd join regularly, after few years for which I receive all >>> > notifications from redmine, I can confirm there are bugs without much >>> > attention. >>> > >>> > If we won't have representatives from all areas, we might need some >>> > tooling to ping people in redmine tickets. Again, after few years, I >>> > can >>> > tell that mentioning name in comment does not always work. There used >>> > to >>> > be a question plugin which works similarly as needinfo in BZ. Perhaps >>> > we >>> > could install it? >>> >>> http://www.redmine.org/projects/redmine/wiki/PluginQuestion is what >>> you're referring to right? Looks nice, I can look into adding it - some >>> Redmine work is definitely on my short-term todo list anyway. >>> >>> > Thanks Greg for bringing this up >>> >>> Still looking for suggested time slots. Perhaps I should create a poll? >>> >>> Greg >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "foreman-dev" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to foreman-dev+unsubscribe@googlegroups.com. >>> For more options, visit https://groups.google.com/d/optout. >> >> >> >> >> -- >> Eric D. Helms >> Red Hat Engineering > > > > > -- > Eric D. Helms > Red Hat Engineering > > -- > You received this message because you are subscribed to the Google Groups > "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to foreman-dev+unsubscribe@googlegroups.com. > For more options, visit https://groups.google.com/d/optout.


Have a nice day,
Tomer Brisker
Red Hat Engineering

We could time-limit it to one hour, and start with "New" issues each
time, sorted by oldest first. That way anything we don't get to one week
would be present the next week. That's still better than we have today,
and as we improve we might get on top of it.

Greg

··· On 15/11/17 07:42, Tomer Brisker wrote: > One concern though is the amount of time it would take
I do check Redmine issues everyday (http://projects.theforeman.org/projects/foreman/activity) If there are many, I restrict the search to just Foreman (with subprojects).

Usually it doesn't take more than 30 minutes. I basically look for:

- New issues: if they look critical or very easy I might work on a fix right away. There are few critical issues like that.

- Ready for testing: if I'm interested I look at the PR. but at least I know about what was fixed

- Closed: same, and also set the Release flag if it's not set

I like the flow of visiting this Activity page every day to keep track of things, for me it's not a significant time investment & ROI is good, so -1


··· On 11/15, Greg Sutcliffe wrote:

On 15/11/17 07:42, Tomer Brisker wrote:
One concern though is the amount of time it would take

We could time-limit it to one hour, and start with "New" issues each
time, sorted by oldest first. That way anything we don't get to one week
would be present the next week. That's still better than we have today,
and as we improve we might get on top of it.

Greg

--
You received this message because you are subscribed to the Google Groups "foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Daniel Lobato Garcia

@dLobatog
blog.daniellobato.me
daniellobato.me

GPG: http://keys.gnupg.net/pks/lookup?op=get&search=0x7A92D6DD38D6DE30 Keybase: https://keybase.io/elobato


+1 the open issues need to be reviewed. I don't know if this is the best solution but can't hurt to get more eyes on them.

What I don't see is something to go over old issues that have been triaged at some point. I did that a while back for the installer and could close a few issues that had been fixed already or could be closed with WONTFIX.


··· On Tue, Nov 14, 2017 at 05:12:36PM -0500, Eric D Helms wrote:
Bump - any thoughts here from the -dev community? Simple +1 or -1 or even a
+0 for indifference would be great to just know where folks stand and
whether this would be worth while.

On Thu, Nov 9, 2017 at 8:28 AM, Eric D Helms <ericdhelms@gmail.com> wrote:

Writing this up inspired me to capture it for the long term [1]. I'd be
happy to run the first one or two given my experience with it (and assuming
the timeslot works) just to get into the groove. Note that our process for
triaging does require some overall Redmine process change with the way we
uses some of the empty and Recycle Bin releases.

[1] https://github.com/theforeman/theforeman.org/pull/970

On Thu, Nov 9, 2017 at 7:54 AM, Greg Sutcliffe <greg@emeraldreverie.org> >> wrote:

On 08/11/17 16:47, Eric D Helms wrote:
[tons of useful stuff]

Thanks Eric! I think that format will work for us too, might take a
little practice. We'll need volunteers to be the runner, ofc ;)

On 09/11/17 07:03, Marek Hulan wrote:
I'd join regularly, after few years for which I receive all
notifications from redmine, I can confirm there are bugs without much
attention.

If we won't have representatives from all areas, we might need some
tooling to ping people in redmine tickets. Again, after few years, I can
tell that mentioning name in comment does not always work. There used to
be a question plugin which works similarly as needinfo in BZ. Perhaps we
could install it?

http://www.redmine.org/projects/redmine/wiki/PluginQuestion is what
you're referring to right? Looks nice, I can look into adding it - some
Redmine work is definitely on my short-term todo list anyway.

Thanks Greg for bringing this up

Still looking for suggested time slots. Perhaps I should create a poll?

Greg
+1 the open issues need to be reviewed. I don't know if this is the best
solution but can't hurt to get more eyes on them.

What I don't see is something to go over old issues that have been
triaged at some point. I did that a while back for the installer and
could close a few issues that had been fixed already or could be closed
with WONTFIX.

Bump - any thoughts here from the -dev community? Simple +1 or -1 or even
a
+0 for indifference would be great to just know where folks stand and
whether this would be worth while.


+0, or at least not relying on it as the only way the triage happens.

+1 for resolving the needinfo feature in redmine: the triage mtg has bigger potential with it

-- Ivan


··· On Thu, 16 Nov 2017 at 12:13, Ewoud Kohl van Wijngaarden < ewoud@kohlvanwijngaarden.nl> wrote:
On Tue, Nov 14, 2017 at 05:12:36PM -0500, Eric D Helms wrote:



On Thu, Nov 9, 2017 at 8:28 AM, Eric D Helms <ericdhelms@gmail.com> > wrote:

Writing this up inspired me to capture it for the long term [1]. I'd be
happy to run the first one or two given my experience with it (and
assuming
the timeslot works) just to get into the groove. Note that our process
for
triaging does require some overall Redmine process change with the way
we
uses some of the empty and Recycle Bin releases.

[1] https://github.com/theforeman/theforeman.org/pull/970

On Thu, Nov 9, 2017 at 7:54 AM, Greg Sutcliffe <greg@emeraldreverie.org > > > >> wrote:

On 08/11/17 16:47, Eric D Helms wrote:
[tons of useful stuff]

Thanks Eric! I think that format will work for us too, might take a
little practice. We'll need volunteers to be the runner, ofc ;)

On 09/11/17 07:03, Marek Hulan wrote:
I'd join regularly, after few years for which I receive all
notifications from redmine, I can confirm there are bugs without much
attention.

If we won't have representatives from all areas, we might need some
tooling to ping people in redmine tickets. Again, after few years, I
can
tell that mentioning name in comment does not always work. There
used to
be a question plugin which works similarly as needinfo in BZ.
Perhaps we
could install it?

http://www.redmine.org/projects/redmine/wiki/PluginQuestion is what
you're referring to right? Looks nice, I can look into adding it - some
Redmine work is definitely on my short-term todo list anyway.

Thanks Greg for bringing this up

Still looking for suggested time slots. Perhaps I should create a poll?

Greg

--
You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

I've just finished rebasing our Redmine patches onto each of the stable branches in the upstream repo (see [1]). Next up is testing the DB migrates correctly, and then I can add the question plugin Marek mentioned as part of the upgrade.

Greg


··· On 16/11/17 11:23, Ivan Necas wrote:

+0, or at least not relying on it as the only way the triage happens.

+1 for resolving the needinfo feature in redmine: the triage mtg has bigger
potential with it
Links help :facepalm:

[1] https://github.com/GregSutcliffe/redmine/branches/yours

There's more to say on this but it's derailing the triage discussion, I'll start a new thread for that.

Greg


··· On 16/11/17 11:51, Greg Sutcliffe wrote:

I've just finished rebasing our Redmine patches onto each of the stable
branches in the upstream repo (see [1]). Next up is testing the DB
migrates correctly, and then I can add the question plugin Marek
mentioned as part of the upgrade.