Infrastructure SIG Meeting Notes 2022-01-14

ekohl · January 14, 2022, 5:31pm

Infrastructure SIG Meeting 2021-01-13

It’s been just over a month since the last Last Infra SIG meeting and the first meeting in the new year. Here are the notes for today’s meeting.

As always, the agenda is first some general topic and then go through the foreman-infra project board.

People present: @evgeni (chair), @ekohl (keeping notes), @mcorr, @ezr-ondrej, @Odilhao, @Zhunting.

Updates since last time

A short summary of Discouse posts since the last meeting:

HackMD retired

As decided last time, HackMD is replaced by GH issues. No new issues on HackMD so it’s officially retired.

In Progress column

Migrate to new ci.centos.org infra on OpenShift

The bot user is present and has the right permissions.
Next steps: automate deployment of our current jobs on the new Jenkins.

Investigate Azure’s FOSS credits

Odilion gathered all information
Next steps: send a request with this info to Azure.

Automation of package bumping plugins and dependencies

@ezr-ondrej opened a PR and @ekohl took it further with some additional help by @evgeni. All automation should be ready and it is merged, but hasn’t ran yet. For now needs to be called manually so we can see how well it works.
Next steps: kick it off and see how it goes.
Future step:

Change manual invocation to a schedule
Add updating of Foreman & Katello spec file dependencies

Update Fastly TLS validation

Fastly is changing the verification mode of domains. They now use the ACME protocol as established by Let’s Encrypt. Evgeni has modified the DNS zone to allow creating wildcards. The only downside is that we can no longer do DNS based validation anymore, but on our servers we use HTTP validation. If we would want DNS validation in the future, we’d need to fall back to mail based for Fastly.

Discourse maintenance and management

Calendar ICS was fixed by updating:

Blog syncing also works again: Nobody changed anything, but may have been fixed by a restart. Another possibly also that one blog was broken and that’s now out of the list of recent posts.

This still needs an update of the server itself and a structural solution.

Figure out if we can build on RHEL in our Koji setup

@ehelms posted an RFC which some discusion. There has also been a separate meeting to discuss the results and look at technical solutions.

The conclusion is that we want to build on RHEL. @pcreech has been in contact with the relevant Red Hat people and the short summary is that a local private mirror in Koji will be set up. This is a dedicated machine (VM really) with a RHEL8 sync on it that’s exported via NFS to the builders. The VM has been created but no further progress.

Today we already have a local copy of CentOS 8 so if that disappears from the mirrors, we don’t be affected. It just gets stale since it’s EOL.

Untriaged Column

Update Foreman 3.0

Updated card to 3.1 and moved it to the To Do column.

Migrate old Debian packaging docs

Updated the issue to reflect the current state and moved it to the To Do column.

Logging for Fastly

The logging was disabled as part of the migration, but we haven’t missed this. Not having logs is better for users’ privacy. It would only be an issue for the RSS feed on the website which we do use. Until we move the website behind the CDN, this isn’t an issue.

Close issued the issue and opened Move the website behind the Fastly CDN · Issue #1701 · theforeman/foreman-infra · GitHub.

There was a discussion about which system to use. Zabbix was mentioned by @Odilhao while @ekohl mentioned Icinga. The former has the benefit that Odilon has experience with it while the latter has a foreman_monitoring plugin (though a Zabbix backend could be written for smart_proxy_monitoring. It would also run on Netways infrastructure and Netways maintains Icinga.

Agreed on the requirements:

It needs to implement disk checks since this is what we mostly run into. While every monitoring solution implements this, it should be seen as the focus point to start with.
Puppetized since Foreman’s infrastructure is maintained with Puppet and we don’t want to manually deploy anything.

Moved to the To Do column and assigned to @Odilhao.

Migrate Puppetserver to standalone instance

Moved to the To Do column

Done

All cards in this column were archived. The column is now empty. Next time I’ll add a list of resolved cards.