Rebuilding Koji

Development Infra SIG
1

All I wish to tackle the build / rebuild of the Koji RPM building and tracking system.

I have done some research and i believe the best course going forward is we (The project) are going to need about 3 systems to handle the various parts of Koji.

Systems Needed

  • Koji
    • The system that’s going to house the web frontend and tools.
  • PostgreSQL Server
    • Database server for the Koji system
  • Storage (Does not need to be its own system, but we will need a good amount of storage per documentation)
    • Storing all the Koji data

Proposal

I propose that this round of Koji builds are via code. This will ensure a standard when building the Koji environment.

The question now is to we want to standardize the code via Puppet or Ansible? I am all for developing the Koji endowment in Ansible but I would love to get some feedback and guidance from the community.

2

Thanks for jumping in to the fray @aheath1992 !

Pinging @pcreech directly as I know he has had thoughts in the past on the architecture and would like him to share those with respect to your proposal

3

About a year ago @bender1215 was also interested in adding client builds for s390 systems which hit a roadblock with Koji capabilities, I wonder if that’s still desired and if we could figure a way forward for it with this rebuild?

4

So @tbrisker & @bender1215, if I am reading the documentation correctly Koji uses RPM build and mock under the hood to build the source RPM’s. Looking at my RPM build system (not a Koji system) mock does support s390 and s390x architecture. So with the rebuild that might be something that is possible.

[anheath@localhost ~]$ ll /etc/mock/*-s390.cfg
-rw-r--r--. 1 root mock 172 Sep  2 19:33 /etc/mock/custom-1-s390.cfg
[anheath@localhost ~]$ ll /etc/mock/*-s390x.cfg
-rw-r--r--. 1 root mock 167 Sep  2 19:33 /etc/mock/custom-1-s390x.cfg
-rw-r--r--. 1 root mock 158 Sep  2 19:33 /etc/mock/fedora-31-s390x.cfg
-rw-r--r--. 1 root mock 158 Sep  2 19:33 /etc/mock/fedora-32-s390x.cfg
-rw-r--r--. 1 root mock 158 Sep  2 19:33 /etc/mock/fedora-33-s390x.cfg
lrwxrwxrwx. 1 root mock  24 Sep  2 19:33 /etc/mock/fedora-34-s390x.cfg -> fedora-rawhide-s390x.cfg
-rw-r--r--. 1 root mock 120 Sep  2 19:33 /etc/mock/fedora-eln-s390x.cfg
-rw-r--r--. 1 root mock 124 Sep  2 19:33 /etc/mock/fedora-rawhide-s390x.cfg
-rw-r--r--. 1 root mock 158 Sep  2 19:33 /etc/mock/rhel-7-s390x.cfg
-rw-r--r--. 1 root mock 116 Sep  2 19:33 /etc/mock/rhel-8-s390x.cfg
5

Mock provides config for s390s, but still requires a s390x host for it. You can see this at the config option legal_host_arches in the s390x mock configs.

6

This is mostly what’s in line with what my intents are when cycles became available. The main thing is I was intending on having “koji-hub” house the database and storage as well, since hub isn’t that resource intensive and one machine in aws would be able to handle it. And remove the builder role from hub.

The builders just need read-only access to what is referred to in the docs as “/mnt/koji”. This can be shared as a simple nfs share to the builders in the same environment.

Utilizing ansible to manage the configuration/deployment of koji is in line with my goals/intents with the koji re-architecture. I’d also like to migrate managing the configuration of build tags/targets/etc… but this could be tackled later.

There are some certain usage patterns that help dictate some of these decisions. I’d be happy to chat with you in detail about them if you’d like.

Just for reference for anyone interested in this thread:
https://docs.pagure.org/koji/server_howto/

7

Hey, I know there has not been alot of traction on this project. I have created a git for the this effort. Once it’s in a usable state I’ll create a PR to add it to the main Foreman repo.