Hey,
I’ve stumbled upon this thread:
https://www.reddit.com/r/linuxadmin/comments/e5cqx8/anyone_using_foreman/
Is this something we can learn from?
3 Likes
Thanks for bringing that up 
IMO we should follow this discussion and learn from it about the user experience, as we know there are many things we can do better.
Wondering if we should be a part of this discussion as a community.
3 Likes
I was very curious if there were any new points mentioned in thread. If I ignore all the swearing, there aren’t any new arguments. In my opinion, most criticism is regarding the content workflow. I believe we’ve talked about that enough already.
With puppet and rpm content, Foreman isn’t a batteries-included approach. It’s more of a (complex) framework that you need to know and build tooling around to make it work. From my experience, the Framework helps a lot, though. If you try to build all that from scratch, you’ll never get the job done.
3 Likes
I’m struck but not surprised by how often the “terrible UI” is mentioned:
“struggle through the horrid UI to deploy Puppet packages”
“The UI is garbage for managing them for groups of hosts, the abstractions are very unfriendly (content views, life cycle environments, repository sets and overrides, ugh)”
“Provisioning is also a massive pain. Nothing ever works and again the UI is awful.”
“I’m glad it wasn’t just me thinking the UI was fucking horrible and a lot of the terminology confusing”
I agree that a lot of the UI is confusing, and as Walden says all the time, it’s designed not around real-life workflows but around visualizing database tables. If we changed our design thinking to focus more on getting tasks done rather than being a 1:1 reflection of the backend data, it could help a lot.
I also agree that a lot of the terminology is confusing, and even after several months here I’m not confident I could explain all of it to someone new. I think a lot of improvement could be made to what we call things, but it would have to be a holistic initiative. Also I think more visuals would help, especially in understanding terms around lifecycle management, products, environments, etc.
As far as reliability and things not crashing, that’s one thing that we as developers should be able to help with.
I think we all mean well, but at the end of the day we’re software developers and sysadmins, not product designers. We could really use some more focus and driving force around customer experience.
8 Likes
I experienced this first hand trying to port Foreman to s390x manually… two months later and I was ripping my hair out.
One the one hand that’s true, but being “database table driven” has made it a flexible tool for users who do want to define their own workflows. This will be hard to unify. I’m looking forward to a proposal for this that’s flexible enough to support multiple workflows but not present 5 different wizards for the same workflow.
IMHO the UI has been driven by flexibility in workflows but that’s a trade off against beginners.
I’d also say there’s quite a big difference in how Katello views workflows compared to Foreman. Foreman is much more flexible where Katello is much more opinionated. This leads to statements like:
struggle through the horrid UI to deploy Puppet packages
Where Foreman doesn’t try to manage this and leaves it up to the user.
1 Like
Maybe we should build a “Construction Manager” app that’s an opinionated alternative UI to Foreman 
IMHO a separate app is certainly a good idea, even if it’s to experiment with new ideas. In Foreman we need ideas to be supportable for some time but separate applications give more freedom there. If it works well, we can see how to proceed. Whether that’s merging it into Foreman, moving Foreman over or something else entirely.
I wasn’t going to bring this up again, but since you did it, I want to weigh in.
In my opinion, Foreman is a rock-solid CMDB. The UI is designed around the database structure. But that’s a good thing. A database with a nice UI is what a sysadmin needs. The data in Foreman is always correct by displaying and processing the facts it gathers from various systems. And Foreman can create a desired state for you (create a host). In general, all of that is pretty trivial and I believe most users don’t have a problem here.
When they want to integrate puppet, Foreman does not take care of code deployments. So as a user you usually need gitlab, ci, r10k and friends to build a system that works for you and finally import the data into Foreman so you can assign classes to hosts. This is hard to set up, messy and that is something we could help with. How nice would it be if you could issue and API request and ask Foreman to deploy a tar archive with a compiled puppet environment to the master, refresh the cache of the puppetmaster, create the environment in Foreman and import the classes into Foreman. Batteries included.
The host form is complicated to use, we could offer some guidance to users here. That’s why we created LISA btw.
The current content workflows don’t make any sense in my opinion. My impression is, nobody really wants to use them. But are there any alternatives? In a greenfield deployment, I’d always choose something else.
2 Likes
This reddit thread definitely touches on some areas I have been thinking about a lot. I’ve been on the team for over 4 years and I think we have progressed quite a bit in terms of stability. This is great, but what I find frustrating is that we don’t acknowledge there is a huge issue of poor user experience in Foreman and Katello. Our software is difficult to set up an difficult to use, and we make it very easy for users to paint themselves in a corner and shoot themselves in the foot. We’ve seen threads like this before and we’ve gotten feedback like this before, but we don’t make any significant changes, which I find frustrating.
I think the first thing we can do is acknowledge there is a problem: Our application is not user-friendly. We’ve heard it loud and clear and we have to take this feedback on the chin and improve. It pains me that we don’t acknowledge this fact that the rest of the world has agreed on.
The good news is the general sentiment (at least from my interpretation) seems to be “Foreman (and Katello) is a pain to set up, but once it works and you have fought to get it working, it is very helpful”. So people do have good experiences with Foreman and Satellite, but have to fight to get to that point.
I don’t think improvement will be a quick fix, there is not magic bullet or hot new technology we can introduce to fix a bad user experience. It starts from the moment they go to theforeman.org to install foreman.
Some thoughts of areas that we could see the most benefit improving:
- Documentation - this is actually the easiest one. If Foreman and plugins are hard to set up and configure, we could significantly improve this without touching a line of code by improving our documentation. There is not much workflow-based documentation out there that takes you from a blank box to a fully installed Foreman instance with plugins that is set up to achieve certain goals. Users are left to figure this out themselves, and they often go down the wrong routes. If we do want to keep a “configuration over convention” approach, we need to improve our documentation to reflect this and show users the “happy paths” to setting up a functional foreman instance. I know @lzap has an ongoing effort in this area and I think we can continue there and look to other areas to improve documentation.
- Installer - The installer prevents the users with a ton of options. It is great to have so much configuration as users tend to want different things, but most users just want to get up and running when they are getting started. Also, what are our defaults? As DHH puts it “defaults reveal the soul of an application”. Users do not want to make decisions, and yes this even means system administrators. Having more opinionated defaults will get users up and running and leave configuration for after the fact. Just spitballing - with the success of foreman-maintain in upgrades, I can’t help but thing extending it (or creating a new tool) to help with the initial install would greatly improve the user experience. I know there have been similar threads, but again, no action taken.
- UI - @jeremylenz detailed out the issues here and I think it will take effort to improve things. I think wizards could be helpful, but not only this, our goal should be a UI that “forcibly guides” the user to do things in a proven, tested way that we know works, but also is flexible enough to accommodate the various types of users we see.
- Testing - Even if our app stays as complicated, hard to configure software (which it should not IMO) we should at least make sure it works and regressions are not introduced. I’m collaborating with a few other devs to meet and improve our testing of user workflows and adding tests that reflect real-world scenarios, hopefully this improves our overall stability.
- Scope - This is one that isn’t talked about much, but Foreman and Katello try to do a lot as an application. Much like when I walk into a restaurant with a huge menu and question the quality and freshness of every dish, software that tries to do everything can often not do everything well. Maybe we can look at narrowing the scope of our application, as counter-intuitive as it seems, this allows us to have clearly-defined goals to test and document.
This is just a brain dump of my thoughts and I’m sure everyone has ideas as well. I know that we can justify the current choices we have made and defend the current user-experience, but if we don’t listen to very verbose feedback from our users, how can we improve as an application and compete with applications that do similar things? I think we need to let go of some of our assumptions and try new approaches.
4 Likes
There is a lot of glossing over foreman issues pointed out in this reddit thread. Let’s not act like there aren’t plenty of problems with the foreman UI:
I struggled with Foreman’s Puppet Master implementation for months which drove me to hate Puppet. Do not use it.
Provisioning is also a massive pain. Nothing ever works and again the UI is awful.
Honestly I would advise against ever implementing foreman or satellite unless I absolutely had to due to Red Hat support, and even then only use it for Yum repos and subscription management stuff.
We had a fully functioning Foreman installation… but holy cow it’s rough to get up and going, and it seemed like I was constantly tweaking it to resolve issues.
We’re using it. It’s rough and I wish we used something else…We use it as external node classifier for puppet
Even the thought of Foreman makes me come out in a rash and feel nauseous. Used it in my last job and even our best, total ninja sysadmin struggled with it and Puppet integration.
etc.
That’s what CLIs or APIs are for, or the ability to “show advanced fields” in the UI. We confuse 99% in order to provide every possible thing for the 1%.
I personally think that if you can capture > 60% of the workflows in the UI then it’s sufficient as long as part of that 60% is the most common use case(s). Usually I’d be a proponent for capturing 100% but in such a complex application with technical users I’m inclined to aim lower.
A lot of the UI (especially in foreman) has been created without any user testing or research. This results in a "let’s capture everything humanly possible"™ approach where you have to spend hours learning terminology and reading the documentation to get anything done. I still don’t know how to provision a host on a new install without running into several errors about what prerequisites are needed first.
Hopefully we have the APIs you’d need to build that.
Do you have any examples of this? I’d honestly argue that katello isn’t opinionated enough.
You only say this because you have used foreman for a number of years. It is very difficult to create a host without spending hours researching how to or without someone hand-holding you through it. The same is true for anything in katello as well. There are several abstract concepts that we require the user to learn in order to do anything useful.
Agreed. You must learn what content views and environments are before you can do anything useful, for example, which is a failing of our UI to guide the user.
What are you basing this impression that no one wants to use them off of?
Absolutely, 100%. Instead of adding new features we should revisit the broken UI/UX of our application.
Yeah, exactly. We shouldn’t present first time users with the entire schema of the foreman/katello databases with only “read the docs” as help. I would imagine most admins would rather do other things with their time than being forced to figure out the inner workings of foreman/katello in order get anything accomplished.
We’ve been there.
I also believe that we only need some UX around host form, everything else should stay “the Rails way” because most IT people expect that to be this experience.
They were designed for the TOP 500 Fortune companies and then simplified a bit, it’s still not enough. I think we need a super-simple and fast content workflow for beginners to start with. Lemme say “two gears” Katello. This could go hand in hand with splitting Candlepin and Pulp backends into two plugins (or at least hiding one or the other from the UI).
Please do not take this as UI unfriendly, while we have some UX issues this you need to take a look at the whole picture. We have huge gaps across everything, not just UI-UX.
Let me say this as an engineer who is given credit for the best cooperation with the Red Hat documentation team: This is easy to say, hard to execute. If you think you can help, please get in touch with me, @mcorr or @bbuckingham. We need more people to be involved in Foreman New Generation Documentation effort and remember - this is not about writing docs, this is about turning Red Hat Satellite 6 docs into Foreman/Katello upstream documentation.
The problem here is puppet and the way it works. Puppet is great for defining your own infra from scratch, it’s really bad for writing a generic installer that should cover everyone needs. We have ended up with 500 options because once puppet manages a configuration file, it manages it all (service, configuration file, cron job, logrotate and the rest). So we were adding more and more to the extreme and we still miss a lot of options to cover them all.
Imagine this: I have assigned you a task of redoing the whole installation and deployment in January 2020 with current tools and technologies. Limitation: no bare metal support, only VM, cloud instances or containers preferably multiple hosts per a single deployment. What would you do?
I don’t believe this is a complete answer, less confusing installation - sure. More documentation for edge cases (thus various detailed settings) - sure.
I’d say NO to upgrades for the next generation Foreman. If deployment would be as seamless as possible, upgrade would be as simple as deploying new version next to the current and migrating all the data. No database sharing, always start with a clean sheet.
I strongly believe that we don’t have time to redo Foreman and the only good option to improve is drastically narrowing down the scope of our application. Let’s not fight with other open source projects who do things better!
3 Likes
We had a chat with Timo at OSAD in Munich this year and we’ve touched one thing that we both agreed in and which resonates in my head for months now. The gist of it is:
If you launch VMs or instances via Foreman, you are probably doing it wrong because there are better tools to do the job.
I believe we have several parts of Foreman which can be considered either legacy or hard to operate or understand.
Narrowing area: Cloud
To elaborate the initial idea, there are better tools to launch and deploy OS and software in the cloud and virtualization. Foreman capabilities shrinks to pretty limited New VM form, ssh or cloud template with “call home” request. Bootstraping configuration management is also not huge value. We could talk about removing all cloud compute resources from Foreman.
Virtualization is still relevant as people can kickstart systems, so that would be a good candidate to keep. The reddit thread mentions “ancient kickstarting”, this is a point where we can differentiate. Kickstarting is still very much relevant on bare-metal, although even Anaconda these days supports image-based provisioning which we already have a template for.
Expanding area: Bare-metal provisioning
This is where Foreman is strong and I believe it’s one of the drivers for newcomers to think about deploying Foreman. They want to build datacenters, clouds, container platforms or simply workstations. They come, struggle to understand our documentation (provisioning coverage in upstream docs is tragically low), struggle to understand deployment (do they need Katello or not?) and let’s be honest provisioning is also quite complex (DHCP, TFTP, PXE loader, iPXE, UEFI, DNS, smart-proxies, subnets, IPv4 vs IPv6). It’s overhelming, we should invest more time to streamline the experience for these people.
Narrowing area: Exotic provisioning workflows
On the other hand, we offer too many provisioning workflows and scenarios that it’s almost impossible to keep up with all the problems, questions, bugs, threads and cases. I’d like to put more emphasis on discovery to make bare-metal provisioning even stronger feature of Foreman. Any other bare-metal provisioning workflows should probably be more of a documentation effort on how to achieve things needed. To create a bootdisk, there is no need of a plugin - it can be simply a script that downloads a template from Foreman.
Narrowing area: Configuration management
I think Foreman is no longer the best ENC for Puppet, at lest the thread talks a bunch about it. We used to be the best one in the world back in the days, things changed now. I know very little about this area, however it’s been discussed multiple times that we must drop some complex things like smart parameters. I’d not hesitate to take this further and simply define Foreman as an inventory and facts and reports storage at least in the documentation.
Narrowing area: Content and subscription management
I think it’s fair to say this has been overengineered. While the workflow Katello recommends would fit for big parties with tens of thousands of hosts, it’s so painful for smaller ones. The next major redesign should allow just fast sync and super-easy consumption of content. Many fields should be optional to simplify the host form, probably wizard can help here to hide all the complex things from the user who don’t need it.
We should also consider making Pulp and Candlepin optional and merging Katello into Foreman to speed up development and increase robustness overall. Also this would erase the confusion around Foreman vs Katello deployments.
Expanding area: Remote execution
I believe we need to differentiate from the pack and since I am talking about Foreman as “the inventory” remote execution is a great complementary feature for ad-hoc operations. We should keep the feature, make it more robust, stable and integrate it as much as possible. I believe this should be integral part of Foreman, not a plugin anymore.
Relevant: Taxonomy
As much as painful our implementation is, this is a valid part of the system and in order to see Foreman to succeed at big parties, we need this. We need to find a way how to make this more robust and allow users to quickly identify what resources are missing so error reporting is better.
Expanding area: Container deployment
If we succeed defining what the new Foreman really is and merge all important plugins into core, it could actually enable us moving forward with containers. I believe that we could hide most of the installation difficulties with containers. This alternative and opinionated (and maybe stripped down) version of Foreman could be the one for beginners or smaller parties. I would discourage from shipping multiple versions of containerized Foreman, with/without plugins etc. I think if there is a lesson to learn, it’s to avoid choice in the next era of deployment. The dream scenario is an Ansible play that would provision multiple VMs/instances with containerized and opinionated Foreman.
I know there’s more, this just randomly jumped out of my head.
2 Likes
The user didn’t specify whether it was vanilla Foreman or Katello. Katello applies a Puppet workflow on top that’s very restrictive. You manage Puppet environments via life cycle environments. Because it’s using Pulp to deploy it, there’s no way to customize it. You can’t store Hiera data in environments. I can imagine why the Katello view of using Puppet makes people hate it.
I disagree here. Foreman was developed by a community of users who needed functionality because they had real world problems to solve. Granted, most of those users weren’t UI people and honestly, for a lot of it the UI was too hard. For many things I personally would have preferred config files over a UI and still do. Which I think why foreman-ansible-modules might become the preferred UI for many users.
I’ll admit I never really used content views. The first time I tried it, I wanted a local copy of a yum repository for Icinga. The library would probably have been enough. However, it meant that I also needed to create a product. Maybe this isn’t opinionated but just a complex model optimized for a world where you care about subscriptions.
I strongly disagree with the statement that Puppet is unfit for it.
Currently we do expose the user to too many options and the Katello installer scenario is suffering from a bad design and a very complex platform. Right now I’m working on an experiment to redesign it. It’ll also be much more straight forward once Pulp 2 is dropped. We can also hide many more options.
Another thing that will improve is the applications themselves. Currently there’s a very static configuration of Pulp in Katello. with Pulp 3 we’ve taken lessons and made it more dynamic and loosely coupled. This means we no longer configure available content types in both Pulp and katello.yaml. Katello will detect the available content types. This makes the installation easier, including potential container deployments.
I’d say this is fine and already how a lot of users use Foreman and Puppet. They use Hiera to set class parameters. However, Katello has a much more opinionated view that makes this much harder and guides users in to fully using the ENC. Since there’s no Puppet plugin planned for Pulp 3, the Pulp 2 removal will free users from this burden. However, I’m still not entirely convinced we should stop supporting this entirely.
The installer has had an option to manage a git repo with a hook to deploy repositories based on branches. This is older than r10k and based on pure git envs (like git submodules). Obviously this is dated by now and probably not used by people. Still, I like the idea of a recommended workflow that’s up to date with the best practices. Puppet enterprise has Code Manager for this and puppet_webhook also exists. I’d also consider an extension to the smart-proxy Puppet module. That already knows how to talk to Puppetserver (to purge caches) and Foreman (to import classes). Might mean you need to depend on smart proxy dynflow.
1 Like
Fair point, but…
I agree, that the host create form isn’t good. I wanted to say, that most companies struggle with implementing a CMDB and keeping the data correct. Foreman can be your single source of truth about what is going on in your data center and that’s why I believe it’s a good tool to have.
From my observation, sysadmins find the foreman part of the ecosystem simpler to use because objects in Foreman represent what they know: A host, a subnet, an ip address, a domain, a puppet class. That doesn’t need any explanation. The template association with operating systems is a bit rough, that is something we should improve.
From my observation, sysadmins like to have control and choice. As stated above, imho the Foreman terminology is already common knowledge. Even with several years of Foreman experience I still struggle to understand what a Composite Content View is.
Mostly from chats with community members (e.g. at conferences) or by mentoring sysadmins who are new to the topic.
Most people want a simple host-centric workflow. Let’s test patches in dev, a week later in stage and then deploy them a week later (or e.g. every third Sunday each month) in production.
1 Like
UX will often get conflated with UI, but it includes the documentation, getting the repos set up, the installer, and UI/CLI/FAModules. I did touch on many areas in my post. Overall it is not a user-friendly experience.
I can understand that, it does seem like a larger effort when I think about it more. Do you think we could do things outside of this effort too? It seems like we could do well creating docs around our expected workflows too. But of course, it takes developer’s time, and that is hard to come by.
I don’t think a full rewrite of the installer is in the cards, so its probably safe to assume it will stay in puppet. So with this assumption, it seems like we need to have a way to make this hundreds-of-options system more approachable and user-friendly. Users just want to get Foreman installed and running and we can talk about puppet’s limitations all day, but why would a user care? It should just work. It was a pretty embarrassing moment for me at a local meetup when someone said they tried to install and setup Foreman, but just gave up trying to do so because it was so complicated. Similar comments were made in the reddit thread too. So we have a general consensus, but we aren’t doing anything about it.
I came up with (but I think it has been suggested before) the idea of extending foreman-maintain to help with the initial install or creating a new tool to do so. I don’t know if this is the best way, I would love to hear more ideas from other devs on how we can improve the initial install and setup.
This is really the big one. The overall impression I get is we try to do many things and serve many users and we don’t do all of it well. Most of us are spread thin and don’t have time to do all the maintenance and refactorings we would like, leading to technical debt. I’m not an expert in all of the offerings out there for sysadmins, but it seems like the landscape has changed quite a bit and there are many more tools out there in the areas we manage. So it does seem like refining Foreman/Katello down to a collection of sharp tools that execute well is a really great way to pivot into an application that sysadmins find helpful.
It would be great to get some feedback on what parts of Foreman/Katello users find work well and what parts they are actually using. Maybe we can get feedback from users in our yearly survey or similar. As a developer, I often feel like I don’t understand the users needs well. I think this leads to decisions to placate all users rather than knowing what users actually need. There is a Steve Jobs quote that goes something along the lines of “configuration is for those who don’t understand their users”. Now that is pretty extreme and probably should be taken with a huge grain of salt, but I think there is some nugget of truth for us in there, we could understand our user’s needs better.
We can also take a step back and define our goals as an application. What problems are we trying to solve for sysadmins in 2020? If we have defined goals that makes making decisions easier and tell us what parts of the application we should focus on and how we can structure our application.
I think a turning point is needed. It would really be a shame if we didn’t take this feedback (which we have seen before) and make significant changes to improve. I think there will have to be efforts in many areas and large changes, but its definitely possible to do so over the next year or so.
2 Likes
I don’t share the opinion similar to this one: “If you need a setting for something, you’ve failed.”
I think the key for success is both in narrowing down the scope and making the default installation working for the target group. We need to change Foreman in a way that new users would not search for those settings and could use it for year until they want more from it.
I think newcomers are overwhelmed with too many features. Thus the confusion, anger and these reddit.com posts.
I’d like to take a moment to discuss this, as the first time participating on this forum, and as this subject seems to be of upmost importance.
First of all, I’d like to thank you for the project, as it is a tool that I’ve found extremely valuable during my growth from a student to becoming a sysadmin.
As someone who is using Foreman along with Katello, I think some of what I noticed during my experiments would be interesting for you.
I’ve read previously on this thread that the web UI looks more like it’s build around the database than the workflow. I think that this specific point is one of the source for most frustrations. To take the example I’ve read on the reddit post earlier, Puppet seems complicated to setup with Foreman. I found 2 different ways to have a working puppet installation, one linked to a content view, and one linked to hostgroup directly.
Something that has me still confused is building Puppet and / or Ansible snippets specifically for use with Foreman. There’s the possibility to use smart variables too. Maybe giving some “sample code” would be a great idea for that part ? I know it’s possible to find some guide on RedHat website, but centralizing it on a Puppet dedicated section would be probably helpful.
Also, something I think would be really great would be the ability to sync content from git or svn repository directly from the UI. I go into the CLI with no problem, but I still think it
Ansible, I had a horrible time around this one. If only I had known that Foreman was proxying through the loopback in order to then SSH into the remote hosts (that’s how I’ve witnessed the remote execution work), I would have spend less time debugging and wondering where were the logs.
I can also finalize my experience with the tool by the following : I was overwhelmed and have really been at some point (such as the example right above), at some point I even thought I’d need to dive into the rubygems to understand what was happening. I progressed by reading the RedHat Satellite documentation, which still lacked at some point.
4 Likes
I think it is if we don’t try to solve everything. See RFC: Make Foreman easy to deploy and maintain where also other topics such as documentation were discussed. Also look at GitHub - ares/spark: Alternative, light-weight and opinionated installer for Foreman, this is based on @ehelms modules (not sure how much effort was invested in them before) and it took 2-3 days to get this repo to a point, we can deploy Foreman + few critical plugins to be able to install more things using Foreman itself. We just need a wrapper to get 5-10 inputs from user. That should be the most that user can specify, all the rest should IMHO be configurable later.