Infra SIG meeting 2021-10-07
These are the notes from today’s meeting. Our living SIG document for all the background.
People present: @evgeni (chair), @ekohl, @Odilhao, Kenny Tordeur
Various updates
Since our last meeting a few updates have been posted as logs:
- Jenkins updated to 2.303.1
- Updated Discourse mail receiver setup
- New Jenkins node, removed fast label from OSUOSL nodes
- Our Github organization is now sponsored on the teams plan
Initiatives
Redmine migration
No update
Migrate ci.centos.org setup to OpenShift
@evgeni has made some progress. By giving the anonymous user permissions it’s now publicly visible as our current CI but we’re blocked on a missing Ansible installation in the nodes. Our pipelines heavily rely on this. The issue has been open for a while.
Action Item: @evgeni will ping the issue
ci.centos.org limits with testing matrix
No update
Archiving Old Debian Releases
Still needs documentation.
Action Item: @evgeni to document procedure to archive
Auto-building Debian on PR merge
Various code has been written but it has not been tested yet. Right now @ekohl and @evgeni don’t have time to monitor it. We’ll wait for @ehelms to return to decide when to enable it.
Rackspace migration
We’d like to investigate if the OpenShift hosted Jenkins instance that’s intended to replace ci.centos.org can serve as a replacement instead. For this we would need to see if we can add our own nodes to that Jenkins instance.
For Foreman/Puppet: we can possibly decommission Jenkins nodes on OSUOSL to free up capacity. This would give us space for the new machines. Decomissioning those is made possible by the new machine as Conova. More on that later.
Rebuilding Koji
No update
Use of Jenkinsfiles
No update
New sponsor
Conova sponsors a dedicated physical machine (node01.conova.t.o). @evgeni has installed CentOS 8 Stream on it together with libvirt. Two new Jenkins nodes (deb-node01.jenkins.conova.t.o and node01.jenkins.conova.t.o) are now running on it and working well. These are much faster than the old ones. We’ve seen about 2x to 3x faster builds on average.
The name node01 is confusing since we use the same name for our Jenkins nodes. The name virt01 is clearer.
Action item: Rename node01.conova.t.o to virt01.conova.t.o
deb-node01.jenkins.conova.t.o is now Debian 11 while deb-node01.jenkins.osuosl.t.o was running Debian 10. Right now the node at OSUOSL is still running, but doesn’t have a label so it’s not used. Decomissioning is needed now that the new node has proven stable and reliable.
node01.jenkins.conova.t.o has the label rpmbuild, but this eats up an executioner slot with basically just waiting for Koji to finish. Our slow nodes can take care of this.
Action item: @ekohl remove the rpmbuild label (done)
@ekohl has also installed and configured foreman-libvirt on our Foreman instance. This allows restarting nodes but provisioning has not yet been set up. We can also set up foreman-openstack for our accounts at Netways and OSUOSL. There is no objection to this, but at least for OSUOSL we’ll need a service accounts so it’s not tied to a personal account.
Action item: Set up compute resources for Netways and OSUOSL
CDN for the Website
No update
RHEL for Open Source Infrastructure
@pcreech sent an email but there hasn’t been a reply yet.
Action Item: @evgeni / @pcreech: ping to get an update
OSCI.io
No update
Move prprocessor to shared infra
No update
Testing multiple plugins together
There was a posssible solution but it’s unclear if that has been tested. @Ondrej_Prazak has been pinged if that worked (Testing multiple plugins together - #3 by evgeni).
Space on web01
The existing cleanup for Debian builds has some issues, need to dive into this. For example, foreman-dbgsym is no longer generated but remains in the repository. This needs some attention but right now it’s unclear where to exactly focus. There are no immediate problems now.
CentOS Stream 9
No packaging progress. Puppet modules are blocked on a missing puppet-agent RPM for EL9 which makes continuous integration hard.
Azure credits
Azure gives credits for FOSS projects. This may be interesting to us.
Action Item: @Odilhao look into the possibilities