We all know that creating an host is one of our app’s pain points,
over the years the ‘create host’ form have grown to support
multiple edge cases and became too complex.
We decided to create an easier wizard for the users which will not replace the existing form for now,
but will give an easier experience for users.
One of the suggested flows that we came with for the first “happy path” version is:
Choose what type of an Host Group which will contain all of the rules *and the OS.
Choose where to create the host - Discoverd Host / Provider
Host settings - give it a name, size and other properties.
The wizard will be written in React & Redux
with API calls to the server which I will need some help with, who wants to help?
I just started to create the basic React component that will be used,
and right now mostly waiting for the feedback from you.
Thanks, Ron
*Still thinking whether the OS should be predefined in the host group or should it be selected later,
in this case we should think how to make it the most simple to users.
I believe that we can limit the host group selection only to those, which have all the attributes defined, so the backend changes would be only to provide the preview based on the host group selected.
That’s IMO a lot of additional value to see what’s going to be provisioned.
I have a generic suggestion - can we start building the new wizard with normal UI components like a table, a drop down and a button? These cards worries me a bit, it might look fine on a prototype but with 100k hosts that could be a pain to work with. Also with components we all know it will be much faster to develop, test and iterate those screens. We can consider changes later on, but expect me to vote vote against crazy things anyway.
Now to the proposal. I am missing taxonomy, I believe it should be either presented as the first or the second screen or ideally to be on the first screen together with hostgroup selection. That creates one dependency on a single wizard page which we want to avoid, however having wizard with 15 screens is something we likely want to avoid. Remember, some workflows/plugins require a host to be created in an organization, so the moment user is switched to Any Organization the wizard must ask first. So I think the best is either the very first screen or the first screen together with hostgroup (pick or change existing organization/location, then load available hostgroups from there, go next or allow continuing without selecting any hostgroup).
Second screen is bad UX IMHO, we should present two dropdowns with discovered hosts and compute resources so user would save one extra screen. Again, we should aim to have less wizard pages. Note this page needs to have an extension point - any plugin must be able to contribute to this page (discovery, network discovery plugin that Ondra wrote). Also we should consider putting also unmanaged hosts there.
The rest is of the path is not correct, I believe it should be pretty different. Let me draw my idea and make a photo of it. It takes Katello and Discovery workflows into account, however I did not detailed all screens - it is missing Interfaces, Ansible and Puppet as the most obvious ones (these would be probably just extra screens at the end of the workflow).
I can explain more details when asked. If there is some good UX tool we could use to share and recycle our ideas that would be great, I find drawing on paper not very convenient.
In any case, it is hugely important that the whole wizard has a fully extensible data pipeline so plugins can easily change the path through the screens completely (Katello) or manipulate data (Discovery). This should be taken into account in the very early stage of development.
@lzap, you have touched a lot of important things,
And thank you for putting an effort sketching the flow…
My slide was just presenting the idea, it was hard to create dropdowns,
I don’t mind using tables too.
and I agree using existing components would make everything much faster,
about the actual flow that you created,
I will wait for people with more experience with creating hosts to comment about it.
But do you think there is an initial version which is simpler to start from?
Looking at the wizard it appears to be much more of a HostGroup Host Wizard. That can be good, but then you can actually go further.
A hostgroup can already select whether it’s bare metal or a compute resource. That becomes redundant so you may want to change it into an option ‘override deployment’ which then shows the compute/bare metal.
The same goes for the OS: already part of the hostgroup. When you do override it, you probably also need to select provisioning templates and you’re pulling in a lot of the complexity you’re trying to avoid.
Discovered hosts already have a wizard to select the hostgroup. That already has most of the flows you’re trying to achieve here.
I’d like to ask what the goal is rather than focus on the implementation. If it’s to deploy hosts into preconfigured groups then you can drop a few steps. If it’s to deploy hosts based on preconfigured groups then you’re missing a lot of customization. In short: where on the scale of configurations do you want to be. My guess is that it’s somewhere like this now:
I believe we’re doing the second step before the first one. I think the aim should be to identify the simplest workflow we currently have and find a solution for that. Without any extension points, edge cases, whatever. Or we can just keep the current workflow.
My suggestion is to use the discovered host workflow because a lot of things don’t need to happen there. Network and HW customization for example is not required for a discovered host.
The way I imagine is the following:
A user gets a list of newly discovered hosts
The user picks a host from that list.
The user picks the desired organization (if in any org)
The user picks the desired location (if any in any org, we might be able to guess a sane default from the host’s subnet)
The user picks the desired OS and a predefined partition layout
The user picks and a hostname
That’s it. This should be enough to install the host. We have to think about the media selection, but I’d love to hide this from the user. The current (Katello) scenario with Content Source etc. is just horrible. No sane person would want to keep this if the goal is a simple host wizard. Without Katello I’d just choose the first match installation media (for the org, location and os). If we could link the installation media to the subnet would also be great.
I don’t want a user to select the provisioning method (discovery just supports PXE anyway) or the pxe loader (if the discovery os was booted via bios, use that - uefi otherwise). Also the architecture, model, … e.g. is clear beforehand.
In the long term we should differentiate between two personas:
The admin who defined how and where a host should be installed.
The user who installs systems (and can also be an admin). He does not want to answer any questions about how the host should be installed, but what kind of a host should be installed.
I know puppet, ansible and networking is not solved in the first iteration. But we can add these steps later (and make 'em optional).
We can than iterate on this. But let’s start with an easy solution. If we try to support every possible scenario in the first iteration we’ll never make any progress here. The discussion about this has been going on for ages and I’m starting to get really fed up with this. This is not the place for over-achieving or over-engineering.
Foreman’s differentiator to cloud service providers is it’s bare metal capabilities. We should invest on this further imho, that’s why a discovery only use case sounds like a good investment in my opinion.
I don’t agree, we should think though at least most obvious extensions because host form was THE major painpoint for plugins to integrate with. I am not saying we should not start coding the easiest or cleanest path - sure that’s logical step to do. However if we know that for example installation media selection must be done in a way that a plugin can contribute (and hide) some irrelevant screens let’s identify it early.
In my proposal these screens are optional, I thought it’s obvious but when you select “Direct media” you only get Installation media picker as the next step.
I think this should be out of the table now, remember we want to develop new host form side by side with the old one. If we want to do such a big refactoring sure let’s do it as a separate task instead and only after that we can think dropping this from both host forms.
What do you mean by solved? I am saying let’s have a clean plan what the screen would look like and then sure let’s add them as last pages. But not thinking about this is not a good way of doing things.
Then let’s do a strong opinion and remove most flags from hosts and make host groups a strong dependency. I am all in, the host-hostgroup relationship has always been problematic. Honestly I’d also like the hostgroup nesting to go away because it’s easily possible with just picking some naming convention in the flat structure like “RHEL7-base-puppet-sql”. Also upgrade path is pretty clear here.
Let’s define most of the pages first (on paper) and then let’s iterate development. We cannot end up with a nice and clean workflow if we start doing things without thinking about other workflows which are also important.
I agree that we should put less effort into cloud providers which literally everybody can do and do it better than us and invest much more into Discovery and Bootdisk. That does not necessary mean we are going to throw away everything else.
Honestly I am not following you attitude, what you are saying is “stop drawing, just dive in”. While I am big fan of early prototypes and iterating, in this case it’s very much twin-bladed.
I vote to drop most of the flags from host and to require hostgroup always to be set. I want this not only because it make sense for most use cases, but because our code is not clean and buggy (inheritance). Also nesting does not work well and if you add Compute Profile into the mix it’s a buggy disaster.
However this does not mean we don’t need to refactor Hostgroup form itself - we are just moving the problem of spagetti UI code from host wizard to hostgroup
I believe we’re not very far off on this topic. I do agree with your proposal, but
I’m just saying that we should also iterate on the page definition on paper. I believe we’re not at a point where we need to write a single line of code. I’m all for plugin extension points etc., but not now. The host form is complex. I want to get the complexity out of everybody’s minds so we can start being creative and productive.
So: Yes, we need a pretty complete plan, before we start coding. But we also can’t start drawing and will have the perfect solution right from the start. That’s why I can only strongly advice to think through the actual workflows and start with an easy one before we try to solve the rocket-science problems and edge cases.
FWIW, we don’t use hostgroups at all. I never got the point of them. But I think this shows, that we can’t just create a new form. We have to think about the big picture.
I’m probably not making any friends with this, but from my real-world experience, compute profiles don’t work in the real world. That’s why we don’t use them as well.
I have to agree, because I do not use them at any customer, only use case I have is setting sane defaults for my demo vms like use QCOW for snapshot capability and 2 GB RAM so CentOS installer runs fine.
But nearly the same applies for Hostgroups, near to no customer uses them, those using them have created weird structures.
I like the idea of templating things, especially as it allows to use the current host form with just naming the host and picking a hostgroup which also sets a compute profile and both setting all defaults. So perhaps it is really time to re-define workflow and not only host form. I remember a discussion @tbrisker started about re-defining hostgroups and possibly splitting it into hostgroups for grouping and templating for defaults.
I’d like to get rid of inheritance for both hostgroups and compute profiles and I very much like the idea of having a “template” or “blueprint”. Because having two models (host and hostgroup) created a lot of duplicated code and effort. When I was thinking about screens yesterday, I was very tempted to add an entry next to Discovered and Unmanaged host called Existing host. When I think about it, what would be the best blueprint for a new host? An existing host. The very same model! All we needed was to have some flag called “abscract” which would differentiate those hosts from real hosts so edits would trigger no orchestration or real actions.
Then we could simply ditch the hostgroups completely in the new host form while still we would keep them until the old host form is removed. Hosts with and without hostgroups can coexist pretty easily, abstract hosts would be simply just hosts as well (you could filter them in or out easily in the UI).
This could be a nice start towards removing hostgroups and possibly compute profiles because blueprints would work with compute resources as well. It also gives a nice overall workflow:
Pick a source (Discovered, Unmanaged, Existing host)
Pick a taxonomy (skipped if it is already selected in the top-bar).
Review and modify only what’s needed
Or simply just Submit
The very basic workflow could be as easy as two screens (pick a source, review & submit).
To elaborate a little on these blueprints, my idea is:
A blueprint host is simply a host with blueprint flag set to false.
A host from a blueprint is simply new record with all its fields copied and a reference blueprint_id in the database.
When a blueprint is created or modified, no orchestration happens.
But when a blueprint is created or modified, users is asked which of the referenced hosts which were created from the blueprint should be modified too. User could interactively pick which hosts to update and which not.
You are right, in many environments (I would say all if all have the same amount of standardization) an existing host is the best blueprint. Having an abstract one would perhaps the smallest change required.
It would only be needed if we want to have a dedicated template (as abstract host) instead of a host. It could be handy in environements were hosts get more costumized and cloning the host would mean more changes than starting with a template.
Yeah. It’s technically not necessary, however the flag is good for being able to filter hosts which are more likely to be used as blueprints - I am sure customers with 80k hosts would eventually ask for this because the dropdown has “too many hosts”. You could workaround that by naming your blueprints like “template-hostXYZ”, sure.
The association between host and its blueprint is then good for ability to perform mass changes on hosts. Again, something you can probably script via CLI/API but since it’s just a simple column I don’t think it necessary make things more complex compared to hostgroups.
I must say I’m surprised the hostgroups aren’t used by you in practice. I’d say the impression among many developers is that this is a heavily used concept.
Do you not use them at all? If so, how do you handle Puppet? Assign classes to each host or fully on hiera?
I would say 90-95% do not use Foreman’s capabilites for Puppet, 1% uses Configgroups for Profiles, Hostgroups as Roles and Parameter in Foreman, the others parameterize in Hiera and assign Classes (Roles/Roles or component modules) in Foreman or combine ENC and Puppetcode.
I like the capabilities and also many users, but they prefer having things under git control.
