Discussion on Future of Issue tracking in Foreman

Redmine vs Github

Inspired by a few conversations, I wanted to raise a discussion around the use of Redmine versus Github for issue tracking for the project as a whole. I think it worth having a discussion to think about and decide:

Do we want to make a change to adopt Github for issue and release management or lean into Redmine and prioritize outstanding actions and usage of it.

There are two perspectives by which to consider this:

  • What would developers like the workflow to be for handling user reported issues and handling of releases
  • What would be best for our users filing issues, tracking them, contributing and communicating with developers

My ask is that both developers and users share their opinions, usage requirements or restrictions and general feedback.

Current State of Issue Tracking

The current state of issue tracking within Foreman is a mixture of Redmine and Github issues. The primary projects continue to use and (mostly) enforce a Redmine issue for any pull requests. While a lot of secondary projects and some plugins have opted for using Github issues.

Examples using Redmine strictly:

  • Foreman / smart-proxy / installer
  • Katello
  • Remote Execution
  • Tasks
  • Discovery
  • Webhooks

Examples using Redmine sometimes, Github issues sometimes, just PRs sometimes:

  • foreman-installer
  • puppet modules
  • foreman_rh_cloud

Examples using Github issues and/or just pull requests:

  • foreman-ansible-modules
  • foreman-operation-collection
  • foreman-documentation
  • foreman_ygg_worker
  • foreman_puppet
  • foreman-js
  • forklift / foreman-infra / foreman-packaging

Pros and Cons

Redmine

Pros

  • Redmine as the place for issues has existed for 10+ years within the project
  • Our tooling and scripts know about Redmine
  • Can create relationships between issues across different projects in Redmine

Cons

  • A piece of infrastructure we have to manage, monitor and keep up to date
  • Requires users who want to file issues to create yet another account for our Redmine
  • Separated from the place the work is done - Github pull requests

Github

Pros

  • Github issues are a tab away from pull requests and can reference each other
  • Github accounts are more common and extract away the projects need to maintain separate logins for users
  • Lower overhead to fix and report an issue
  • Would mostly eliminate the need to run https://github.com/theforeman/prprocessor reducing infrastructure further
  • Github provides Projects feature to coordinate multiple repositories issues together around some theme

Cons

  • Scripts and release tooling would need to be updated for the major projects
  • Redmine issues in the backlog that would need to migrate
  • Redmine would need to run for some period of time as an archive for ability to reference older, closed issues

Infrastructure Needs Impacting Decision

Redmine is an application that we as the community run and manage. There are pending changes that need action taken on them that are elevated or negated by this conversation. Those needs::

  • Redmine needs to be migrated off Scaleways
  • Redmine needs to be migrated onto an EL8+ host
  • Redmine needs to be upgraded from Redmine 3 to 4 (and ensure plugins continue to work)

As mentioned previously, there would need to be some element of keeping Redmine around in a read-only style mode somewhere for archive reasons.

Adding a poll:

  • Switch to Github for all projects
  • Continuing using Redmine and Github in hybrid mode
  • Invest in Redmine and use it in more projects

0 voters

1 Like

Count my perspective more as a user one. I would prefer Github only as everything would be in one place then and no need to know how this part of the project is handling the workflow. I would also think with good issue templates, the bot usage (like we already have for packaging) making labeling and other features very easy and perhaps some other features could lead to a better issue quality, too.

I remember I was told Redmine is also used because of the integration into Red Hat workflows like bugzilla, not sure if this is still a thing to consider.

Pure user perspective here:
I find the usage of redmine rather annoying and would like to get rid of it.
Redmine is a system most users are not very familiar with, and so find hard to navigate and use, in addition to the need to create another account. Github issues on the other hand are something most users should be familiar with from other projects with and know how to work with them already.
Personally I think the choice should be made manly based on the question which tool is better for the future development of the project, but migrating to Gitlab issues might potentially have the benefit of more exposure to and interactions from the community.

I think we ended up using Redmine in a way, it’s rather one unnecessary additional step in the process of reporting/fixing the issue. Meanwhile Github introduced quite a lot of features to the issue tracking.

The previous conversations on this topic was in favor of Redmine. I changed my mind since 2013 and I’d vote for transition to Github, but it would be good to have an answers on howto do the

  1. mapping and finding the history from commit messages easily (I guess that read only instance)
  2. how much effort it is to change all existing tooling
  3. identify the person who will take the responsibility to do it

Fun fact, Katello has moved from Github to Redmine in 2014

Perhaps this could be helpful for answering howto: https://github.com/Icinga/ruby-redmine-github
A colleague has written this when Icinga did the same migration in 2017. If I remember correctly there was also some contact with Github support involved as some rate limits had to be temporarily increased.

1 Like

Frankly, I think such a huge portion of people in the open source community already have github accounts and are users of it already that its almost a no brainer.

It may be a painful process initially, but I think migrating to Github would make a lot of sense. It keeps everything in one place, and there have been a lot of improvements on the Github Issue side. Task Lists, Project Tables and Issue Templates amongst other things. All of which should improve the process and developer experience over time.

Short term pain for long term gain.

I’ll possibly swim against the current and vote for redmine here.

My stance is probably somewhat different than what others might have. I maintain a bunch of plugins, the only person making changes to them regularly is me. Every now and then I get a PR from someone else, but that’s somewhat rare. Most of the changes consist of making a single patch and then cherry-picking it to one or two stable branches.

Yes, redmine is clunky and rigid, but to me that (the rigidness, not clunky-ness) is a benefit over the lack of structure that github issues have. A single issue in redmine can be used to track the same change across all the versions into which it should go, but more importantly, I can later search by the version. Same goes for linking against other issue trackers.

You can log in into our redmine with your github account, can’t you? Also it feels like since we introduced discourse, people outside the project don’t really open issues in redmine all that much anymore.

Personally I don’t really see the improvement in process and experience that we should gain later by ditching redmine.

1 Like

To understand this, I think it’s also important to understand the role Redmine plays today.

There are roughly 3 personas we can identify:

  • User
  • Developer
  • Release Manager

Each one of these uses Redmine in a different way.

There is also the Plugin Author, which is somewhere between a Developer and Release Manager. I don’t explicitly mention this because it’s covered via the other two, but keep this in mind.

How we use Redmine today

Release Manager

Today Redmine is used to do release management. It allows for correlation between projects. For example, we can see what’s needed to finish Foreman 3.4.0 via 3.4.0 release TODO - Foreman. Note this includes the 4 main projects that make up a Foreman release:

  • Foreman
  • Foreman Proxy/Smart Proxy
  • Foreman Installer
  • Foreman SELinux

What this means is that as long as you set a proper target version, it will show up and allows for a central release manager to maintain overview.

Once the release has been built, release notes are generated.

Additionally, it allows for proper tracking of which release contains which fixes (again, used in the changelog generation).

Moving to GitHub means we lose the cross-project tacking. GitHub milestones are project specific and you increase the burden on the release manager.

This is the most difficult part to replace.

Developer

Developers are, depending on the project, required to file a Redmine issue. The prprocessor enforces this.

Additionally, they can see reported issues and pick them up. There is additional metadata, like affected versions.

I get the impression that most consider this an overhead and there are sufficient alternatives in GitHub to replace this use.

User

The user can find existing issues, their discussions, relations and when it’s been fixed. Additionally new issues can be filed. Indirectly they consume the changelogs which refer to Redmine issues.

This could easily change to GitHub, as long as suitable replacements are implemented. Most of this already exists.

Today you can make the comment that we don’t properly triage the Redmine issues and that’s a fair assessment but keep in mind that you can also easily forget to triage GitHub issues.

Red Hat

Red Hat also uses Redmine as a way to correlate issues. You may see a Bugzilla link on each issue. Bug filed by customers end up being cloned to Redmine when it’s determined it needs to be fixed upstream. The automation then monitors progress, moving the Bugzilla state when it’s been merged upstream.

This could be rewritten to GitHub.

Possible future

From the various personas, only the release manager is difficult to change and significant efforts needs to be put in to make sure we can continue releasing on time without sacrificing quality. Over the past years we’ve worked hard on achieving a reliable release cycle and a migration can seriously endanger that. So let’s focus on that role.

Historically the release manager has always had knowledge of all 4 projects. Today that’s no longer true, meaning the release manager already relies on others to do the actual project-specific work.

One can think of changing the relation to make a project-specific team (like core developers for Foreman, platform team for the installer, etc) responsible to deliver a release on time. They are also responsible to write up the release notes for each project. From there on they are free to organize their project the way they want.

The responsibility of the release manager changes to making sure all teams deliver their deliverables on time, is aware of potential issues that may delay/block the release etc.

In practice this can mean that each project needs to send a delegate to the release meeting, or at least make sure they report their status around releases. They also become responsible for handling cherry picks.

Longer term you could even look at decoupling versions. The Foreman Proxy doesn’t change at nearly the same pace as Foreman. Today the version numbers are kept in sync, but most of the time the only diff between x.y.n and x.y.(n+1) is the VERSION file. This has support implications: users need to understand if their setup is compatible. Today the Foreman UI checks versions numbers, which would obviously break. I have plenty of ideas of how to deal with that (short version: capabilities and extending the Foreman plugin DSL with ways to declare the required capabilities along with proper reporting), but that’s a large topic.

Special attention here must be given to projects that are inherently cross-project:

  • Installer
  • Packaging
  • SELinux

Keeping track of what’s in certain versions becomes even more important for these.

Responses to the initial post

Instead, we would need to write a lot of automation using GitHub actions that’s also replicated across repositories. This means that instead of a single central service we place a burden on every repository to maintain files.

Additionally, we would probably look at GitHub Apps - Patchback · GitHub (used by Ansible and Pulp) which is in fact also a webservice. So really, we’d trade one service for the other.

In my experience, GH projects have been horrible to use. Unless you write automation, there’s a lot of duplicated state: closing an issue doesn’t close it on the board and vice versa. Also, the cross project side is painful. I have tried to create a board to track modulesyncs across our Puppet modules, but in the end I found it easier to open an issue with a list of PRs than to manage the project. Note how I couldn’t find a cross project milestone, which would have probably also worked.

This isn’t just some period of time, IMHO while the project exists it should remain available. Longer term I’d investigate some way to convert the issues to a static website which means you lose search, but at least have individual pages.

Short term you could use permissions to make a project read only.

Possible path forward

I’d suggest to look at plugins. They are always a good place to experiment and have their own release cycle anyway, meaning they don’t affect the established release management.

This means taking a plugin (like foreman_ansible) and figuring out how to do release management:

Given these are configured using the .github directory inside the project, we’d also need an easy way to sync it. In the Puppet modules we use modulesync but there may be better alternatives.

3 Likes

I looked into the changelog aspect.

I played with release-drafter in https://github.com/ekohl/release-drafter-example but I don’t think it useful to us. It doesn’t understand stable branches and very much focuses on a single branch.

My test was:

  • Release 0.1.0
  • Create a 0.1-stable branch
  • Release 0.2.0 from master
  • Merge a bugfix PR to 0.1-stable

I had hoped it’d draft 0.1.1 but it drafted 0.2.1. This is extremely common in tools that read GH PRs.

For our Puppet modules we now use https://github.com/github-changelog-generator/github-changelog-generator but that can’t deal with it either. Since we barely have stable branches there, it’s good enough to deal with it manually. For serious projects it probably wouldn’t be.

I know Pulp uses towncrier. It uses a concept where every commit includes a new file in the CHANGES directory. Then on release you squash all files in CHANGES together into the release notes. Since a cherry pick includes the change documentation, you automatically get all the release notes.

The downside is that you do require contributors to write up the CHANGES, but you can write a CI check to require this.