Well what’s I think is not relevant, it needs lot of opinion of others. 
Overall we should aim for:
- Easy to understand UX
- Easy extensibility by plugins
- Paths that make sense
2 Likes
I believe we’re doing the second step before the first one. I think the aim should be to identify the simplest workflow we currently have and find a solution for that. Without any extension points, edge cases, whatever. Or we can just keep the current workflow.
My suggestion is to use the discovered host workflow because a lot of things don’t need to happen there. Network and HW customization for example is not required for a discovered host.
The way I imagine is the following:
- A user gets a list of newly discovered hosts
- The user picks a host from that list.
- The user picks the desired organization (if in any org)
- The user picks the desired location (if any in any org, we might be able to guess a sane default from the host’s subnet)
- The user picks the desired OS and a predefined partition layout
- The user picks and a hostname
That’s it. This should be enough to install the host. We have to think about the media selection, but I’d love to hide this from the user. The current (Katello) scenario with Content Source etc. is just horrible. No sane person would want to keep this if the goal is a simple host wizard. Without Katello I’d just choose the first match installation media (for the org, location and os). If we could link the installation media to the subnet would also be great.
I don’t want a user to select the provisioning method (discovery just supports PXE anyway) or the pxe loader (if the discovery os was booted via bios, use that - uefi otherwise). Also the architecture, model, … e.g. is clear beforehand.
In the long term we should differentiate between two personas:
- The admin who defined how and where a host should be installed.
- The user who installs systems (and can also be an admin). He does not want to answer any questions about how the host should be installed, but what kind of a host should be installed.
I know puppet, ansible and networking is not solved in the first iteration. But we can add these steps later (and make 'em optional).
We can than iterate on this. But let’s start with an easy solution. If we try to support every possible scenario in the first iteration we’ll never make any progress here. The discussion about this has been going on for ages and I’m starting to get really fed up with this. This is not the place for over-achieving or over-engineering.
Foreman’s differentiator to cloud service providers is it’s bare metal capabilities. We should invest on this further imho, that’s why a discovery only use case sounds like a good investment in my opinion.
4 Likes
I don’t agree, we should think though at least most obvious extensions because host form was THE major painpoint for plugins to integrate with. I am not saying we should not start coding the easiest or cleanest path - sure that’s logical step to do. However if we know that for example installation media selection must be done in a way that a plugin can contribute (and hide) some irrelevant screens let’s identify it early.
In my proposal these screens are optional, I thought it’s obvious but when you select “Direct media” you only get Installation media picker as the next step.
I think this should be out of the table now, remember we want to develop new host form side by side with the old one. If we want to do such a big refactoring sure let’s do it as a separate task instead and only after that we can think dropping this from both host forms.
What do you mean by solved? I am saying let’s have a clean plan what the screen would look like and then sure let’s add them as last pages. But not thinking about this is not a good way of doing things.
Then let’s do a strong opinion and remove most flags from hosts and make host groups a strong dependency. I am all in, the host-hostgroup relationship has always been problematic. Honestly I’d also like the hostgroup nesting to go away because it’s easily possible with just picking some naming convention in the flat structure like “RHEL7-base-puppet-sql”. Also upgrade path is pretty clear here.
Let’s define most of the pages first (on paper) and then let’s iterate development. We cannot end up with a nice and clean workflow if we start doing things without thinking about other workflows which are also important.
I agree that we should put less effort into cloud providers which literally everybody can do and do it better than us and invest much more into Discovery and Bootdisk. That does not necessary mean we are going to throw away everything else.
Honestly I am not following you attitude, what you are saying is “stop drawing, just dive in”. While I am big fan of early prototypes and iterating, in this case it’s very much twin-bladed.
I vote to drop most of the flags from host and to require hostgroup always to be set. I want this not only because it make sense for most use cases, but because our code is not clean and buggy (inheritance). Also nesting does not work well and if you add Compute Profile into the mix it’s a buggy disaster.
However this does not mean we don’t need to refactor Hostgroup form itself - we are just moving the problem of spagetti UI code from host wizard to hostgroup 
I believe we’re not very far off on this topic. I do agree with your proposal, but
I’m just saying that we should also iterate on the page definition on paper. I believe we’re not at a point where we need to write a single line of code. I’m all for plugin extension points etc., but not now. The host form is complex. I want to get the complexity out of everybody’s minds so we can start being creative and productive.
So: Yes, we need a pretty complete plan, before we start coding. But we also can’t start drawing and will have the perfect solution right from the start. That’s why I can only strongly advice to think through the actual workflows and start with an easy one before we try to solve the rocket-science problems and edge cases.
FWIW, we don’t use hostgroups at all. I never got the point of them. But I think this shows, that we can’t just create a new form. We have to think about the big picture.
I’m probably not making any friends with this, but from my real-world experience, compute profiles don’t work in the real world. That’s why we don’t use them as well.
I have to agree, because I do not use them at any customer, only use case I have is setting sane defaults for my demo vms like use QCOW for snapshot capability and 2 GB RAM so CentOS installer runs fine.
But nearly the same applies for Hostgroups, near to no customer uses them, those using them have created weird structures.
I like the idea of templating things, especially as it allows to use the current host form with just naming the host and picking a hostgroup which also sets a compute profile and both setting all defaults. So perhaps it is really time to re-define workflow and not only host form. I remember a discussion @tbrisker started about re-defining hostgroups and possibly splitting it into hostgroups for grouping and templating for defaults.
1 Like
I’d like to get rid of inheritance for both hostgroups and compute profiles and I very much like the idea of having a “template” or “blueprint”. Because having two models (host and hostgroup) created a lot of duplicated code and effort. When I was thinking about screens yesterday, I was very tempted to add an entry next to Discovered and Unmanaged host called Existing host. When I think about it, what would be the best blueprint for a new host? An existing host. The very same model! All we needed was to have some flag called “abscract” which would differentiate those hosts from real hosts so edits would trigger no orchestration or real actions.
Then we could simply ditch the hostgroups completely in the new host form while still we would keep them until the old host form is removed. Hosts with and without hostgroups can coexist pretty easily, abstract hosts would be simply just hosts as well (you could filter them in or out easily in the UI).
This could be a nice start towards removing hostgroups and possibly compute profiles because blueprints would work with compute resources as well. It also gives a nice overall workflow:
- Pick a source (Discovered, Unmanaged, Existing host)
- Pick a taxonomy (skipped if it is already selected in the top-bar).
- Review and modify only what’s needed
- Or simply just Submit
The very basic workflow could be as easy as two screens (pick a source, review & submit).
To elaborate a little on these blueprints, my idea is:
- A blueprint host is simply a host with blueprint flag set to false.
- A host from a blueprint is simply new record with all its fields copied and a reference
blueprint_idin the database. - When a blueprint is created or modified, no orchestration happens.
- But when a blueprint is created or modified, users is asked which of the referenced hosts which were created from the blueprint should be modified too. User could interactively pick which hosts to update and which not.
1 Like
You are right, in many environments (I would say all if all have the same amount of standardization) an existing host is the best blueprint. Having an abstract one would perhaps the smallest change required. 
What is the different between an abstract host and the host we currently have? I don’t get why we would need a new concept.
It would only be needed if we want to have a dedicated template (as abstract host) instead of a host. It could be handy in environements were hosts get more costumized and cloning the host would mean more changes than starting with a template.
Yeah. It’s technically not necessary, however the flag is good for being able to filter hosts which are more likely to be used as blueprints - I am sure customers with 80k hosts would eventually ask for this because the dropdown has “too many hosts”. You could workaround that by naming your blueprints like “template-hostXYZ”, sure.
The association between host and its blueprint is then good for ability to perform mass changes on hosts. Again, something you can probably script via CLI/API but since it’s just a simple column I don’t think it necessary make things more complex compared to hostgroups.
I must say I’m surprised the hostgroups aren’t used by you in practice. I’d say the impression among many developers is that this is a heavily used concept.
Do you not use them at all? If so, how do you handle Puppet? Assign classes to each host or fully on hiera?
I would say 90-95% do not use Foreman’s capabilites for Puppet, 1% uses Configgroups for Profiles, Hostgroups as Roles and Parameter in Foreman, the others parameterize in Hiera and assign Classes (Roles/Roles or component modules) in Foreman or combine ENC and Puppetcode.
I like the capabilities and also many users, but they prefer having things under git control.
I guess that makes sense. Using hostgroups to assign classes is also how I was thinking about it.
What are arguments for preferring editing files + git? Is it for review, history, easier to modify and/or another reason?
In most cases it is the git workflow with branch, merging, …, which is more flexible than the audit which only show changes and do not allow revert. In more and more environments there is als some usage of CI/CD to test, review and deploy which allows also for complexer environments like multi-master or different masters per department or similar.
That’s good to know. It’s probably out of scope for this discussion but soon we should have one about the use of Puppet and future integration.
I guess we’ve already entered post-Puppet era.
Many interesting poins were mentioned here. I do not think we should use hostgroup in the new host wizard at all. There are too many unrelated things attached to it which does not help us in simplifying the workflow but rather leads us in direction of ‘how to turn existing host form into wizard’.
I do like the idea of host template/blueprint as @lzap mentioned but I do not agree template/blueprint should be an existing host for the reasons already mentioned. Moreover, we have that feature already - just click ‘clone’ button on host show page ;-). This also ties to what @tbrisker proposed for hostgroups - separating various parts and distinguishing between configuration in Foreman and what is reported in facts. In this regard, host template/blueprint is a bit misleading name because it would deal only with provisioning and would be one of the hosts aspects (I intentionaly avoid word ‘facet’ here, though existing facet framework looks like a good fit but I leave it to @Shimon_Shtein to confirm). Creating a host from a template/blueprint (just one type for start, regardless of whether it is virtual or bare metal) seems like a good candidate for the first version of hosts wizard to me.
<offtopic>It would be nice to create a template/blueprint from an existing host though.</offtopic>
I see a host as a composition of ‘funcionality/feature groups’ that may or may not be present regardless of implementation. This is useful for extensiblity, because on one of the first steps in wizard could be a set of questions like the following:
- Would you like to use Puppet with this host?
- Would you like to use Ansible with this host?
- Would you like to use Openscap with this host?
- Would you like to subscribe this host to consume content?
Based on the answers, steps could be ommited, which would satisfy the requirements for a short wizard. This is in line with previous suggestions. I just wonder how well this plays with @TimoGoebel’s suggestion about admin and user who installs systems.
I see the suggestions for bare metal capabilities as the first worflow that we should go for. Any other thoughts?
1 Like
I think we’re making progress here. My point is, that the questions you listed above are usually questions you ask once. And not for very host.
Can you think of a scenario where a Foreman user would answer these questions differently for each host? I like them, but I think they make more sense in a setup wizard (or to create a blueprint that you can create a bunch of hosts off of).
1 Like
I can’t think of a scenario where these questions would be answered differently for each host. I think they would be answered differently for various groups of hosts.
It ultimately comes down to how much customization the user creating hosts wants/needs to do. If the workflow could be summarized as “I want Foreman to create a host that has X and Y using as few clicks as possible and I do not want to configure details myself”, then blueprint is a good place to put those questions.
With a blueprint, we could have 2 wizards that would be optionally chained together:
One possible concern with having a highly configurable blueprint and few options to configure in host wizard is having a state explosion in blueprint. I do not know how many different groups of hosts users usually have. I suppose if someone wants a host that is just a little different, it is convenient to clone and modify in hosts form, but that is just my guess.
2 Likes