Best approach for scheduled package upgrades across multiple hosts (Katello vs Ansible/Puppet)?

Hi everyone,

I’m fairly new to Foreman and currently setting up a test environment (Foreman 3.19, Oracle Linux 9, Puppet/OpenVox + Ansible) as part of evaluating it for managing a larger fleet of Linux servers in the future.

My current goal is to manage package versions (like PHP via Remi repository) across groups of hosts with the ability to:

  1. Control which version of a package is installed on which group of hosts

  2. Schedule upgrades for a specific date/time (for example “upgrade PHP from 7.4 to 8.4 on all organization hosts next Tuesday at 12:00”)

  3. Ensure the desired version is maintained continuously after the upgrade

So far I’ve tested three approaches without Katello:

  • Puppet module with Smart Class Parameters - works great for maintaining desired state, but can’t truly schedule “upgrade at date X” since changing the parameter triggers the upgrade on the next Puppet run (within 30 min), not at a planned time

  • Scheduled Remote Execution with dnf command - simple one-off upgrade at a planned time, but no continuous state management afterwards, need to manually update Puppet parameter after. And its not real automation, for me tbh its useless.

  • Scheduled Ansible playbook/roles - reusable with before/after version logging, but has two issues: running a standalone playbook via Remote Execution requires targeting the Foreman server rather than the host (awkward), and using “Run Ansible Roles” via Schedule executes all assigned roles on the host, not just the upgrade role. There seems to be no way to schedule a single Ansible role - it’s all or nothing, which means unrelated roles run unnecessarily every time I just want to upgrade PHP.

My questions:

  1. Is Katello the right solution here? From what I understand, Content Views + Lifecycle Environments would let me control what packages hosts see, and I could schedule promotions via Hammer CLI + cron. Is that the recommended approach?

  2. For those managing package versions without Katello - what’s your preferred workflow? Is there a cleaner way to combine Puppet/Ansible with scheduled Remote Execution that I’m missing?

  3. Any tips for someone planning to scale this to 100+ servers?

Thanks in advance for any guidance!

Yes, definitely it is. I have done many projects at customers which are happily using Katello in different ways. Some stage the packages based on content, some on projects and quite a big number based on time. Controlling what a host sees is in my opinion the only clean way to do management of the software.

While hammer + cron is fine, I would recommend nowadays using the Ansible collection theforman.foreman to configure all the things in Foreman. Scheduler is still needed than of course, this could be Foreman itself, a systemd timer or good old cron.

I think there is no clean way to do so, only if you manage the mirror server with this tools and then why not using an existing solution like Katello.

Default Scaling of Foreman is for a much bigger load, so do not worry. Only consideration would be how is network structure affecting the environment as in are there network segments which require there own Smart Proxy to keep things local.

2 Likes

Thanks Dirk, that’s really helpful!

Two follow-up questions:

  1. You mentioned the theforeman.foreman Ansible collection - does it also cover Katello operations like creating Content Views, syncing repos, and scheduling promotions? Any example playbooks or docs you could point me to?

  2. Is it common to use Foreman with Compute Resources (e.g. oVirt) to automate the full provisioning flow? I’m thinking about a scenario where instead of manually creating VMs and then PXE booting them, Foreman would handle the entire lifecycle - spin up a new VM with the right specs (CPU, RAM, disk, network), automatically PXE boot it, run the Kickstart installation, and then apply Puppet modules and Ansible roles for post-install configuration. Basically going from “I need a new server with role X” to a fully configured machine with one click. Is this realistic and worth the setup effort, or do most people just handle VM creation separately and let Foreman take over from the PXE boot stage?

Thanks again!

Hello and welcome!

Some do use Copute Resources, some don’t. It depends, our implementation of Compute Resources will always be limited at some point. There is so much fields we can support, but these days EC2 launch form has like 200 fields so there are limitations.

My advice is try it and if that works for you, do it. Otherwise files a bug for a missing feature.

1 Like

Thanks lzap! I’ll definitely try Compute Resources with oVirt then.

One more question - is there currently any way to run a single Ansible role on a host through the Foreman UI? From what I’ve tested, “Run Ansible Roles” always executes all roles assigned to the host. If I only want to run one specific role (e.g. a PHP upgrade role) without triggering all the others, the only workaround I found is to unassign the other roles first, run, then reassign them - which isn’t practical. Is this a known limitation, or am I missing something? Or maybe I have to use playbooks instead of roles?

Yes, it covers not all things but the most common ones of Foreman, Katello and other plugins are covered for sure and it is still growing.

None of our examples is public I think, but I have one which was shown at a Foreman Birthday event: codeberg.org/stdevel

I typically recommend it, especially for local solutions. But as lzap said there are limitations and other reasons not to do which you will perhaps find in your environment when trying. For example I had one environment where capacity manager did simply not allow that someone else creates VMs or in a more recent example we needed virtual TPM which is not supported bei the Compute Resource Proxmox yet. Open issues if you hit a limitation only by doing so the project can improve.

No job like this is provided, but you can simply create your own job.

2 Likes

Thanks for your advice. I created a job template and it’s working like I wanted. I would never have known about this option on my own - it’s weird to me why I can’t just run one specific Ansible role instead of all of them :sweat_smile:

This is because the “Run all Ansible roles” is more a configuration management approach which is where Foreman comes from, while running single playbooks, roles, and I would also say shell scripts is more the orchestration approach which is what the remote execution plugin adds.

With all other solutions those two are more separated than with Ansible. Puppet/OpenVox has the puppet manifest for configuration management and bolt for orchestration. Salt has an option for both which differ in the way they are executed (I am not very deep into Salt). Ansible is by default for me more orchestration but with a scheduler added on top of the same mechanism it can handle the configuration management (regularly removing a drift between current and desired state) in a similar fashion.

2 Likes