Creating Operating Systems and assigning templates to them is right now a little bit confusing and not user friendly, I must say.
This is the typical scenario user have to go through:
Go to Operating systems
Create new OS
Go to Provisioning templates
Find the template(s)
Associate template(s) with OS
Go back to OS
Edit the OS, select the templates, save.
That’s a lot of steps. And it can be confusing (and kinda annoying) for many users including me, as you can see: [0], [1], [2], [3] and [4].
I’ve been thinking about current situation, what to do with it and how to make it better and simpler for users, and so far I came up with the following ideas.
#1 Remove template association to OS
By default all templates will be available to all OS.
Pros:
Easier to use, from 7 steps to 2 steps
Cons:
Instances with hundreds or thousands templates will have too many options in the selects.
#2 Allow to assign templates when creating OS
When creating OS, display all templates (sorted by kind) without limitation to OS.
Pros:
Easy to implement
Easier to use, from 7 steps to 2 steps
Cons:
User can make mistake when selecting template
Editing OS will still require 7 steps scenario, can be confusing for users
#3 Associate template to OS automatically
Katello is already doing it [5], but only for Red Hat OS family.
There was a PR for Foreman [6], but it was closed & not merged.
Cons:
Difficult to implement (choosing what OS to what template can be pretty
complex)
Do we want to do it behalf the users?
#4 Add flag “for all os” to templates
This leaves the final decision on the users. If they want to select the OS for each template, as they do now, they can.
If they want to ignore the scoping, they can set flag to true
And that’s all of my ideas. I would like to hear opinion from the community, what do you think and what pros and cons would be if we change the behavior.
I’m looking forward to your thoughts and opinions.
Thanks for bringing this up! I think whatever we’ll do in this area will lead to an improvement.
I like option one (Remove template association to OS). If I understand that correctly, we’d drop association on the template side to specific OS, instead we’d only select on the OS side the specific template (not just from the list of template that were assigned to that OS). I like it because our templates usually supports multiple version of OS, the risk is that I’d select preseed for rpm-based distro. But since we have select2, it’s easy to quickly find the desired template with low probability of misclick.
I’m not entirely sure if I fully understand the option two. Does that mean we’d allow user to select any template and then performed the steps 3-5 as part of it automatically as part of step 7? That would be quick and easy win. At the same time, it renders the template -> OS assocition quite useless and we should get rid of it later.
Third option (Associate template to OS automatically) - I like this too, it seems the original PR wasn’t rejected, there was just lack of interest back then. I think we could revisit that. I’d prefer if OS had the templates preselected so users don’t have to think, which template to use when they create a new minor version of OS. I see this as a complement to previous options, I think we can find a solution that’s not overly complicated to implement.
The last option is good if we see a strong push back from users (or anyone in this thread). If our default templates are considered flagged by default, that is also an enhancement.
I’d be in favor of 1 (Remove template association to OS) followed by 4 (Associate template to OS automatically). If 1 is too much work we can’t handle soon (e.g. API changes), I’d vote for 2 followed by 4.
Could you add numbers to suggestion for easier reference please?
I believe our model (e.g. all the associations) is quite good, however, we lack some important features.
a) is described in #3. We know, at least for the default shipped templates, what OSes they’re valid for in the metadata. We should use that information to set things up properly.
b) when an operating system is automatically created, e.g. by a fact import, we should clone the “older” version and preserve all associations rather than creating a blank object. In foreman_omaha we have implemented something like this (a bit hacky): https://github.com/theforeman/foreman_omaha/blob/3ea64913dfffa827dcef0acdf15bae70825553ad/app/services/foreman_omaha/fact_parser.rb#L8-L10
Regarding creating new OSes I actually think we should change the OS model. For many distros the minor version isn’t really that relevant. For example, with Debian only version 10 is supported. That 10.6 is also what you get when you install 10.0 and run apt upgrade. You can’t install 10.5 after 10.6 was released.
Not sure how it should be modeled, but making minor versions optional would already be a step forward. One snag is that RHEL does have minor versions that it supports.
So while this RFC is aimed at making the relations easier, this suggestion is aimed at reducing the number of times you need to make those relations so it may be in scope.
I like our current model, it’s very flexible and I do believe auto-selecting templates (the PR that was not merged) is the smoothest way of fixing the problem. We also already have the metadata feature, it should be probably improved if needed.
Bigger changes will hurt a lot, these will be breaking changes to all provisioning workflows and I don’t think it is worth the effort since there is an easier way. Let’s rather save our resources for other tasks.
Thanks for looking into this, the way newcomers need to figure out is horrible and this should have been fixed years ago. I think the reason why I personally did not start working on this was that we are building the new UI and once we get to the Operating System page, there should be a wizard that would guide user through the process and it would do this explicitly asking which templates should be associated and providing some best options. But simply auto-selecting them is a good enough solution too.
Both things could be improved how it is done and how often it is done.
My biggest problem here is that it has to be repeated when a new operating system minor version is created via a config management report, so why not copy over the information from the last minor of the same major. If changes are required this could be done afterwards.
When we automatically create an OS, I think we should automatically assign the correct templates. This particular use case doesn’t make sense to require manual intervention.
Does that mean we’d allow the user to select any template and then perform steps 3-5 as part of it automatically as part of step 7?
Yes, on the new OS form show tab Templates, where we will list all templates, sorted by category. I have a proof-of-concept branch [0] for it, you can try it by yourself.
I’d prefer if OS had the templates pre-selected so users don’t have to think, which template to use when they create a new minor version of OS
This is is good idea, I could do it as a part of that PR I mentioned above
when an operating system is automatically created, e.g. by a fact import
To be honest this didn’t came up to my mind until you mentioned,
but it’s definitely good idea
Not sure how it should be modeled, but making minor versions optional would already be a step forward.
Minor version is already not required field, only required fields are name & major version.
Or are you talking about something else?
… this suggestion is aimed at reducing the number of times you need to make those relations so it may be in scope.
Good point, do you have any other ideas (with the idea about ignoring minor versions) that could make the WF easier? Including the idea to ignore minor versions for certain OS.
I think the reason why I personally did not start working on this was that we are building the new UI and once we get to the Operating System page, there should be a wizard that would guide user through the process and it would do this explicitly asking which templates should be associated and providing some best options.
I didn’t know about the new UI. Where can I get more information, or can I get involved? Sounds interesting to me.
So far it looks that most people are for the option of auto-selecting templates when creating a new OS.
Yes and no.
If the operating system is new, assign the default template specified in the template metadata.
If there is an operating system already (same name and family) present, copy the existing associations. I believe we can be a little opinionated here and assume that our templates work with a new version. In my opinion, we should encourage users to use the same templates for different operating system versions.
We currently have ansible playbooks that setup Foreman to work with a specific os release. So when e.g. RHEL 7.6 came out, we just added that so it works the same as 7.5.
Honestly, I don’t like a machine to be performing a heuristic find on how to setup provisioning templates, this can go bad quickly leading to some incorrectly set templates. I’d rather see a UI feature that would set templates via a button press so a human could review it.
What what you’re saying can be achieved by picking:
templates from the same family OS which was created_at recently
or using default templates if none was found
Nice, I would not be against “learning from the past” approach Timo described above.
UI feature here is worthless for most users. The most common way for a new OS to be created is when facts are updated due to a new release. At least for me, the common workflow is that CentOS/Debian x.y is running, then x.(y+1) is released. yum-cron/unattended-upgrades updates my OS and now I have new OS in Foreman. Now I have to complete the info and clear out the old distros.
However, both CentOS and Debian don’t really care about .y versions. That’s why I argued that we should make it possible to not differentiate based on that.
@TimoGoebel’s proposal is to at least ease the pain by automatically filling in the missing data so you don’t have to go to the UI/API to fix things.
Those 2 solutions can both be implemented. Both because some distros (RHEL) do care about minor versions.
Well at the moment there is a bug in Anaconda RHEL 8.3 which requires us to do a kernel command line workaround only for this particular version (8.2 is not affected). We need minor release numbers. It also has an inventory purposes (how many servers are still running on 7.0 etc).
I wonder why we actually need the OS association to by updated by puppet? It should have been either a different association or simply just a fact you can search. Because we will always the “fight” between what users set and what puppet reports back. The same for domain and subnet. We even have an option to stop updating them which indicate to me something is not right with this design.
Also, the auto-create feature is not very useful for provisioning because you still need to create a hostgroup for the new version. Don’t tell me you want Puppet to also copy-paste a hostgroup.
I don’t know what’s the best thing to do and I am open to listening to our users. If we implement both UI improvement and puppet auto-creation I won’t block this. I just wonder if there is a bigger problem we are trying to mask here.
Right now you can’t provision CentOS 8.2. That’s why I said that for CentOS the minor version is irrelevant while for RHEL it is. There may be an exception for Katello where you can have content views.
So if you want to make my suggestion more concrete, I’d suggest a has_minor_version option on the Operating System model. We can suggest a default value. If it’s set to false, no minor version is tracked and the field is disabled (hidden?) in forms. In case you do have content views, you can set it to true, even for CentOS. In that case the old behavior would apply again.
All the same can be applied to Debian. There supported minor versions were dropped in Debian 4 but there can be content views.
For Ubuntu there are supported minor releases according to Releases - Ubuntu Wiki. You do have 18.04.5. In this case 5 is the minor version.
That is indeed the point I’m trying to make. We treat minor versions as something that exists everywhere, but it doesn’t.
Isn’t just allowing minor version to be blank a better option than introducing a new flag? We can define this for each individual family then, e.g. for Debian we would not require it (actually we could enforce it’s empty) and for Red Hat we would require it.
Then it templates we can decide what to do depending on the situation.
Also, with CentOS 8 Stream, things might actually change. I am not sure yet how kickstart are published and ho woften, I need to try. Let’s actually find out, it’s December 15th 20:50 CET at the moment and this is the listing of CentOS Mirror
Looks like initramdisk has been refreshed today. Let’s revisit the link after few days, this would actually mean PXE files do have updates quite often, which is a big problem for us. This will break our wget -c download policy (continue = just append to existing file → breaks). We need to fix this.
At the moment it would not be enough to allow empty as you can do so already, but then the configuration management report will come and create a new version including the minor and switch the host to this os.
I wouldn’t go that far. I think what @lstejska is trying to solve is another issue than you’re talking about. Issue 1) new user is lost when she tries to setup provisioning, Issue 2) existing user already has provisioning configured, but in combination with cfgmgmt, it creates new and misconfigured OS instances. While I agree 2) hits most of existing users, 1) hits every new user of Foreman. While I think we should also improve 2) this RFC tried to address 1) and imho it is absolutely worth improving.
I think we absolutely need to keep this feature and behavior. In my world, a user creates a new host, defines the desired os level (usually the latest available version from a specific major version) and then the host comes to life. During the lifespan of the host the version changes and you want Foreman und the UI to keep an up to date version for inventory, reporting and much more purposes. What I could imagine is that we prohibit a user to change the version in the UI while a host is not in build mode.
Why would a user want to change the version in Foreman and create a conflict?
Ok. So this is RFC is about a fresh Foreman setup. Imho the main question is: Why isn’t everything already setup for provisioning? Why does a user need to make any changes at all? What is missing?
