Path to Ruby 3.0, 3.1, EL9 and Ubuntu 22.04

Today Foreman (3.5-develop) is compatible with Ruby 2.7 (older was dropped with Foreman 3.4). It is not yet compatible with Ruby 3, but both EL9 and Ubuntu 22.04 both require that compatibility. This is a rough roadmap with the major milestones to run Foreman and Foreman Proxy on the latest EL and Ubuntu versions. It is likely incomplete and we’ll figure out more things along the way.

How to bootstrap a new platform

This is a small reminder of the general steps to get Foreman running on a new platform.

First of all, there are two major projects that can each independently move. These are Foreman and Foreman Proxy. For both, the steps are largely the same.

  • Get the core project’s test suite runing on the Ruby version
  • Get all plugin’s test suites running on the Ruby version
  • Package them

Once they’re packaged, the installer side can be updated. Additionally, on EL there is an SELinux policy.

In reality it’s not so linear. For example, if it’s a previously unpackaged platform, packaging can start even if plugins aren’t ready. Additionally, some installer modules work with core platform components such as dhcp, dns and tftp.

It should also be noted that a new platform doesn’t mean old platforms are dropped. There is always an overlap which allows users to update.

How to apply this into practice now

Ruby 3 is a big change, but many small steps can be taken. I’ll go over them for each project

Foreman

Today Foreman uses Ruby 2.7 and Rails 6.1. Long term we should aim for for Ruby 3.1, but Rails 7.0 is the first version to support it. Previously I played around with this and created https://github.com/theforeman/foreman/pull/9328. A lot of smaller tasks have already been split off and merged, but it’s a large task that means we should break this up into smaller pieces. Many of these pieces can move in parallel.

A critical set of plugins should be selected. These plugins should all be made compatible with new versions before changing defaults.

Ruby 3.0

A major step is updating to Ruby 3.0. In Ruby 3.0 some gems changed. The Ruby 3.0.0 release notes mention both rss and rexml were changed from default to bundled gems. This means they must now be specified as a dependency. Additionally some stdlib modules are now gems.

More significant is kwargs. In Ruby 2 the last argument could be a hash and keywords were stored there. In Ruby 2.7 it became possible to have a separate keyword argument parameter. Ruby 3 removes the old behavior. See Separation of positional and keyword arguments in Ruby 3.0 for more information. This is tracked in #35300. Because of this a lot of gems have dropped Ruby older than 2.7, because it’s very hard to support more. Because of this Foreman 3.4 has set the minimum Ruby version to 2.7. Because of this there are gems which even today raise deprecation messages, such as safemode and graphql. Updating gems is a large part of this task.

This is an incomplete list, but some concrete work that must happen:

There are also many more dependencies that could be updated. All of them should be checked.

Once it passes, CI should be modified to also run on Ruby 3.0.

At this point all plugin authors should be notified to perform the same steps and packaging can commence.

Zeitwerk

First of all, there is still some remaining work on Rails 6.1. Updating to Rails 6.1 defaults doesn’t mean we need to have all Rails 6.1 defaults, but at least call load_defaults 6.1. Not all values must match the defaults, but the test must pass. It’s also fine to do this incrementally (first 5.0, then 5.1, etc). Zeitwerk is considered out of scope for this part.

The next major step is Zeitwerk. The new default loader in Rails 6.1 and only loader in Rails 7. This is a large effort. The first goal should be to use the classic loader, but support using Zeitwerk. Once Foreman itself is compatible, we should give plugin developers some time to update before we switch the default.

Rails 7.0

Once Ruby 3.0 and Zeitwerk are in place, Rails 7 can be the next focus. The Rails 7.0 release notes and upgrading guide should be helpful.

The previous mechanism to support multiple Rails versions stopped working, so it was removed. A new way should be found so Foreman can default to Rails 6.1 but run with Rails 7.0 too. This helps plugin authors migrate.

This also involves packaging work.

Ruby 3.1

This depends on both Ruby 3.0 and Rails. It is largely the same as Ruby 3.0. The Ruby 3.1.0 release notes mentions more gem changes. These are likely all rather trivial. It is also not required since both EL9 and Ubuntu 22.04 have Ruby 3.0, though EL9 also has a Ruby 3.1 module. Given this is likely a small task, it’d be good to support.

Once it passes, CI should be modified to also run on Ruby 3.1.

Again, plugins need to be verified too.

Foreman Proxy

Foreman Proxy is a lot smaller than Foreman, both in the size of the code base and it has fewer dependencies. There are no frameworks to update, so it’s limited to the Ruby updates. Today the test suite already passes, but the dependencies do need to be fixed.

Similar to Foreman, there are also plugins to consider. These are usually small, but still need to be verified.

Packaging can start fairly soon.

Packaging

The initial goal is to only create new platforms, not switch over existing platforms. On Debian/Ubuntu there’s only a single Ruby interpreter so that’s certainly out of scope. On EL there are modules, but that requires the whole package set to be compatible. Instead, the work will start with building the new platforms (EL9 and Ubuntu 22.04) with components that are compatible. This means initializing empty repositories and getting foreman-release out. Then once components are ready, they are packaged.

It should be noted that today we have gating by pipelines, but that requires more work so initially they will be pushed to nightly without any verification.

Installer

First all individual modules must be made compatible. Many components describe parts that are delivered as part of the OS, such as Apache, PostgreSQL, ISC DHCP, ISC BIND and TFTP. That work can start and in fact already has. Other modules, such as foreman and foreman_proxy need a packaged version. This means it depends on the previous steps.

Once a complete scenario is available, the foreman-installer package itself can be packaged as well. The foreman-proxy-content scenario is the most likely candidate since the underlying pieces are furthest along (Pulp already runs on EL9).

Timeline

All of this work can happen in parallel, but all projects need to be available before it’s officially released. Having looked at the work needed, it’s very likely work won’t be finished before we branch Foreman 3.5 but are not yet ready. We’ll have to decide if we then have an incomplete repository or only have the changes in nightly.

3 Likes

I’m not sure I understand this part. Incomplete repository means some parts Ruby 3.0 compatible but some not? And if that’s the case, parts means Foreman Proxy and Foreman or even e.g. Foreman dependencies?

Otherwise a great write up, it’s good to see what all is ahead of us. That feels like a lot of work and more hands would be… handy :slight_smile:

Indeed. This is because today you can’t even get Foreman to run on Ruby 3, so you can’t even build a package. With EL9 we will start with an empty repository (no packages). Then we’ll add foreman-release. Once it’s ready, we can add foreman-proxy. However, until foreman is added I consider the repository incomplete. There are probably more packages that must be present and I didn’t dive into it now, but we will find that out along the way.

1 Like

Something I completely forgot, but there’s also NodeJS to consider.

Today we build with NodeJS 12, but support 14. EL9 has version 16 by default and 18 in a module. Ubuntu 22.04 has version 12, but there we’ve used the nodesource repositories to update. I’d suggest to aim for a single version for our packaging environment, but have to rely on people more experienced with it to determine if it should be 16 or 18.

I think our current Webpack 3 (way outdated) is incompatible with NodeJS 16, so that also requires a serious effort. There’s already a plan written for that:

2 Likes

Got it, that makes sense.

Big + for the summary what needs to be done, suddenly it doesn’t seem to be so easy :smiley:

My addition to this is about the plugin extensions, due to Zeitwerk we will have to drop all the extensions that are monkey patching Foreman internals and replace them with our plugin DSL, like in foreman_ansible for example.
It’s bad practice anyway so even if we won’t needed it will be great improvement.

In some cases DSL is already there, but for many cases we don’t have it. I’ll gladly take responsibility for this part, I will probably start with Katello as there will be a lot of work and then go over other smaller plugins.

2 Likes