Reducing "time to package" for Debian packaging

As noted in Future of debian packaging, @baboon and I would like to make adding new (and updating existing) Debian packages as easy and straight forward as it is for RPM.

To tackle this, I would like first to gather the current problems, and welcome everyone who had to deal with Debian packaging recently to chime in with their problems (hi @lzap and @rabajaj !).

Problem 1: building dependencies

dependencies are our most Debian-style Debian packages: we have one folder per package, that essentially contains all of the debian/ part of the packaging and we merge that with the unpacked gem as part of our Jenkins build jobs (this is the non-standard part, Debian by default stores the tarballs next to the packaging recipe).
Nevertheless the addition of new packages is a multi-step process that should be automated away, as the current process is prone to subtle errors, especially around folder naming and dependency handling.

Problem 2: building plugins

plugins is what the average developer touches most, when they release new versions and thus bump packaging. Pure bumping is scripted and thus easy. However, every now and then a plugin adds webpack/nodejs/api/you name it and package builds need adjustment as we don’t use tempaltes/macros as heavily as in RPM.

See https://github.com/theforeman/foreman-packaging/commit/f759b4a07302c45787cbad05680b7cb46a057c22 for an example of changes that were needed “just because discovery now uses nodejs”

Problem X

I’ve deliberately left out Foreman core here, as I think while it could use some improvement, this is something that not everyone needs to touch (today) and thus the benefit of investing time here is smaller.

Please, everyone, speak up where you did spend most time/frustration on in Debian land, so we can tackle those portions.

5 Likes

I think there is another problem which is the post-merge work needed. For most RPMs (though not all) once the packaging PR is merged, pipelines are triggered that automatically build the needed package and release them to the relevant repo (either directly or daily).
For debian builds, there is an additional manual step needed on Jenkins to actually build the package and start the pipeline. This can only be done by people who have the needed permissions on jenkins and know how to do it, limiting the number of people capable of actually releasing packages. If the part was automated, anyone with commit access to the packaging repo could see that a debian build was green and merge PRs.

1 Like

While I agree this is a needed improvement, it more belongs to the CI part (and I forgot to mention it there).

What comes to my mind immediately is that we keep packaging metadata in three separate directories for three different OS versions and I was wondering if things like symlinks or a different solution could help.

Thanks for doing this.