Future of UI infrastructure and development in Foreman

Development
1

Lately I have noticed of a couple of seemingly unrelated issues with our UI infrastructure. It got me thinking about ways to fix those issues, and specifically about the lack of a broader direction for our UI.
I would like to present a vision doc that will describe how I see the Foreman UI in the future and what action we can take to get us there. One of my guiding principles for this doc is that I do understand that we we are all busy people and we don’t have the time to put a lot of effort in pure technical debt items without a clear benefit to our users. I want to show an evolutionary approach to the UI modernization effort, where we take one small step at a time to get our UI up to a higher standard. I would like to avoid a revolution approach where we change the UI drastically in a disruptive way.

I will start with the issues and requests I have identified already and then I will suggest a vision and a set of steps to get there.

Old infrastructure:

Multiple UI component technology stacks:
Currently our UI uses a lot of different UI technology stacks in different parts of the application. Currently we use ERB, jquery, angular (in Katello), PatternFly 3, PatternFly 5 and redux simultaneously. This creates a steeper learning curve for developers, complicates addition of new features and creates a non-unified visual experience.

UI testing infrastructure:
Current testing that involves UI elements is not standardized and has a lot of workarounds spread around the code. This makes the code unstable and difficult to read and modify. We are using Capybara for “integration” tests, and there is also the Robottelo project that is used for downstream testing. Both are suffering from instability and the amount of asynchronous UI processes that are part of the UI flow.

Future proofing:
While not strictly an existing problem, I think we should also mention two requirements here.

  • We want to lower the bar for creation of alternative UI projects, especially now with the whole AI and vibe coding trend.
  • We want to make sure it is easier to modernize our current UI stack. In easier, I mean laying the ground for better tests, so changes in the UI will require less rewrite also to our testing framework.

The vision:

We will be able to test the Foreman functionality by calling this JSON interface, instead of emulating UI interactions.
The UI deployment will become simpler, since we will have only one place to serve the whole our UI, as opposed to the current state, where we have static objects, webpack-compiled objects and dynamic HTML pages.
The MVVM pattern seem like a good fit for our UI, since it is about highly-interactive and interconnected components.
In case we will want to perform test that involve UI, we will be able to build an abstraction layer above the PatternFly components to make the access to our UI elements more streamlined and less prone to errors and reinventing the wheel. In my opinion, we will also be able to create more robust tests for both the business logic and the UI if we also integrate the idea of testable snapshots as described here: Enhancing our tests with stable snapshots for UI testing .

The way to get there:
First I will describe the desired end state. We want to have a page built on top of latest PatternFly components. This page will use state and custom hooks to manage the API requests/responses (the view model). For the model we can either create dedicated objects or use the API responses directly (similar to our current state in redux). The model will be retrieved by a dedicated endpoint on the server side that can be different from the one used by the “official” API.
I think we should not use the same endpoint for the “official” API and for the API calls performed by the UI. The UI is stateful, and does not face the user directly. This will enable us to differentiate our login and security mechanisms for example. We also should make sure we have an autogenerated description for the API, so discovery services will be able to “learn” our available endpoints.

Given that this is the end state, the way to get there is different for each component and it’s complexities.

I think we should consider this doc as a guideline for future UI refactorings and make sure we follow it.

I would like to hear opinions about this strategy and see if I missed some nuances in this approach that would prevent this from happening.

4 Likes
2

I love this plan, Shim. This is going to make the lives of so many people so much easier once implemented. What are the current thoughts on the project timeline? Are we considering React 19 + PF 6 or are we sticking with the current React + PF 5 to reuse as much as possible?

3

disclaimer: I am not a frontend person by any means so I’m looking at this as an external observer and I mean well by this

The proposal seems to imply that we want to stick with react or eventually have another standalone frontend application. I could understand the appeal behind doing as much as we can on the client in situations where a server has to serve many (read hundreds or thousands of) clients, but that doesn’t seem to be the situation we’re in. Without having any data to back this claim, I would say an average foreman instance has to serve at most 10 user sessions at once and then this starts to feel like too big of a hammer. Yes, you also get interactivity, but that can be accomplished with much simpler backend-focused libraries and frameworks like hotwire (default in rails since 7) or datastar or even htmx. Additionally, if my memory serves me, react was originally proposed and brought in by a group of people, most of whom have moved on to other projects since then. With that being said, are we sure this is what we really want? What are the benefits we get out of it and are they worth all the work that’s being put into it?

I would add a couple more things to that list:

  1. We still have leftovers from the time when react best practices were being changed weekly scattered all over the code base where conceptually similar things are implemented in wildly different ways in different places, even though it is all technically a single ui component technology stack - just react
  2. Often, new react-based pages and features take way too long to implement and are often implemented in seemingly unmaintainably high numbers of lines of code

As much as I’d like to believe this is the right way, we set out on a journey to be all-react some years ago (the oldest commit in foreman which mentions react is from 2016) and we’re still not there. Maybe that’s a sign that the evolutionary approach is hindering our attempts to do things right by forcing us to constantly fight with bending the new to fit with the old?

I was always quite fond of the katello model (rest api first, ui uses the rest api) and I’d prefer a single API (api v3 maybe?) over multiple separate APIs. While there are some benefits, I’m not sure if those outweigh the cost of having to maintain the feature parity between two separate APIs.

4 Likes
4

I would like to say, that we aim to not even use too much custom things in simple components (example of the modals index page, that passes just column names and an api path) - and to create and maintain a generic table for index pages (and maybe a generic show page component in later stages?
And for some REX pages, we had to create custom api endpoints, which I agree makes more sense.

We are now in the process of converting pf3 to pf5 so we can later convert them together to pf6.
We also are in the process of rewriting the Enzyme tests so we can upgrade to a newer React version.

I assume you ask if React is what we want, since we use Patternfly, the answer is probably yes, as PF supports React and HTML, and we already have React, and probably dont want to write all the JS that will be needed for the pure html version.

I generally agree and I think since we work very individually on the UI, its written in a less optimized way as we dont have many (or any?) discussions with as a group of people who write FE on how to approach tasks, and where can we do better. I try to update some of the dev/contribution docs to be more updated, but I cant assume they are read or understood, and we probably dont update them fast enough.

And for the API I would like to connect this maybe to this effort: Foreman REST API version 3 initial thoughts

2 Likes