Starting on a new project (creating a reusable Modal component) and realizing that everything in Foreman uses patternfly-react which is based on Patternfly 3.
Just to see what it’s like, I made a component using the Modal provided by @patternfly/react-core which is Patternfly 4. I put some very quick-and-dirty markup inside (based on one of the Katello modals) and used Patternfly 4 components.
I think this has a more modern look and is easier to read. Additionally, for the most part the React components provided in @patternfly/react-core seem to have a better syntax (more composable with JSX vs. making you construct complicated objects and render props). Just look at some of our code for constructing Patternfly tables, and you’ll see the pain caused by these awkward PF3 components.
What are people’s thoughts on switching to Patternfly 4 in Foreman? It would be a big effort, but I think it would totally improve the way customers feel about Foreman’s UI and UX. It would also make developers happier, and ensure that the product is keeping up with the latest in design standards. To me the question is not whether we should do it, but how should we approach it. Is there a way of going about it that will cause the least pain for everyone involved? Is it better to do it gradually or all at once? Would love to talk things through here.
@jeremylenz I think its a good idea and I’m sure we could benefit from the newer components. Any thoughts on the upgrade path? In their docs they hint at running 3 and 4 together. If that is something that works for our app, it would ease the transition to have both 3 and 4 available provided there are no major technical downsides to doing so.
There has been some discussion in hallways. Given Katello is still partially Angular, there’s a lot of Patternfly 3 and also some older stuff a big concern was adding yet another (incompatible) stack.
One suggestion was starting a full fresh interface based on just calling APIs treating Foreman as a pure API application. We’re developing a GrapHQL and while it’s still early days, that might be easier than trying to upgrade the existing UI code.
It would also allow us to get rid of the very complicated hybrid stack we have where part is rendered in ERB and other in Javascript.
I would love to see a small proof of concept that starts from scratch to see how realistic it would be.
Disclaimer: I am aware of the second system syndrome but our iterative approach is also hitting its limits since we’re not actually removing the old code in time.
That’s what I was thinking too, run 3 and 4 together during the transition period. Based on my Storybook experimentation, the CSS for PF4 seems to affect some of the old components too, but it seems like it makes them look better so I don’t think it’s a problem.
I love the idea of moving on from ERB and moving to a pure API application. I’m not used to GraphQL yet but that may be a different discussion. What do you think a good starting point would be for a POC?
As someone new to the Satellite team, it’s still very difficult to get my head around the way Foreman is hybridizing all of the different JS technologies (Angular, React, jQuery, etc.) and forcing them to work together. I agree that old code isn’t being removed in time, and some is even still being added.
I think it would be awesome if we came up with a plan to start over from scratch with a consistent stack. But I also realize this would be a huge job. So if we continue to use an iterative approach instead, there should be safeguards in place to ensure we don’t regress toward having more legacy code. (But again, this is probably a different discussion as well)
It has been well over 3 years now since we started on the journey of renewing the front-end side of Foreman. Despite a lot of effort, and having a dedicated UI team for the past 2 years as well as many other developers involved, we are only now at a point where we have some of the groundwork for working with react - basic things like allowing plugins to extend core pages, the ability to share modules between core and plugins in a reliable way, and some of the base components that can be composed to build actual pages.
While some progress has been made, we still have a lot of places that use one of several “legacy” implementations. This means we already have many inconstant pages that look “out of place”. If we start adding PF4 to the mix, that would mean even more such inconsistencies.
I think that our users will not be happy to have a mixed UI between the two - especially since it looks like PF4 is completely different in its styling to PF3. I also think you are underestimating the effort it would take to get there.
I would not be happy having to rewrite pages again to PF4, and I’d be concerned that by the time we are half-way there PF5 (or some other thing) would come out and we would need to start again.
I think a full rewrite is not feasible. We would still need to support the existing UI until the new UI will be ready, which I don’t expect will take less than a couple of years at best. All of this time, while having multiple developers working on recreating the UI with no value to our users. I believe some of the longer-time members of the @katello team can speak to the effort they undertook to rewrite Katello in angular (or take a look at Foreman and katello UI future from 5 years ago).
I don’t. Many pages render just fine with erb and are much simpler to create that way. There is no inherent benefit in doing things on the front-end, unless there is for example some extra value added by using react components that allow for better interactivity. Just take a look at the amount of changes it took just to get a fairly simple page from ERB to react - from ~25 lines ERB that are quite simple to understand, it became several hundreds of lines spread over dozens of files, not counting the tests. I’m not saying we shouldn’t rewrite things to React where it makes sense, but I don’t think our priority should be moving everything to the client side where there is minimal value in doing so.
Please keep in mind that a lot of the “Best Practices” around client side rendering come from the world of SaaS, where you want to minimize your server and network costs and offload as much as possible to the clients, while handling thousands of concurrent users. We are not in this world - the biggest foreman installations will have dozens of users at most, and they will mostly be connected on a very high speed connection to the foreman server. This means that many of the considerations that are big drivers elsewhere are irrelevant for us.
I think this is the biggest issue. I would not want to introduce yet another technology to the stack while we still haven’t finished removing things we wanted to get rid of 3 years ago.
Both flows, in my opinion, will take significantly longer than you expect them to, while providing very little value to our users in the mean time. I would love to be proven wrong, but if we still haven’t gotten rid of things like jQuery, jQuery-ui, deface, angular etc. (and aren’t planning to invest in doing so) - I don’t see how this doesn’t end up being yet huge effort undertaken and end up with yet another different way to do things that will in turn be abandoned when the next thing comes up.
As a final note, I think Patternfly team here is making a mistake in not having a clear and easy to perform upgrade path - which is exactly the second system syndrome that @ekohl was referring to.
Sorry for the long post and I don’t want to sound discouraging but there are some considerations that you are likely not familiar with yet as a new member on the team. I hope I shed some light on those.
I can see where you’re coming from. I can write a patch for ERB pages if there’s an issue but I’m not going to touch anything in webpack because I don’t have the time to figure out how the stack works and what obscure thing I need to change for it to show up the way I way want it. As someone who works on the platform and backend, I just can’t keep up with Javascript.
That doesn’t have to be a problem if we have sufficient people working on the UI and look at the incoming bugs. Currently this may be a problem though and having various stacks makes his harder.
I would actually say that removing deface should be a priority. We have a replacement (facets) that should be used. Once plugins use that, we can at least stop thinking about that. It will make future migrations easier IMHO.
We’re already at the point where we don’t want to touch some parts (host form) because it’s so likely to break. Given how central it is, we can quickly get to the point where it’s impeding the progress of the entire project.
As a project that’s 10 years old now, we know there’s a lot of technical debt and my impression is that the UI is gathering technical debt faster than it’s paying it off.
Killing technical debt is a complicated thing, but a great way to learn many odd quirks and the depths of the application (stack). There are various old timers here that know reasons why something was built a certain way and things you should be aware of.
Would this be a good moment to put a (beer) bounty on killing deface?
I got mixed feelings about pf-4 and the directions they took, they give us no choice but do a gradual migration (that will look bad for a while, and none can guaranty how long) or start a new UI from scratch (it will put the foreman UI on hold for a while, and again none can guaranty how long).
After we finish the migration, none can guaranty pf-5 won’t show up to bring back the same issues.
Another option is to keep using pf-3, which works well for us, but then we are taking the risk of using an unmaintained library (eventually).
Switch to a different library is also an option
+1
I agree a full rewrite is not feasible, but what if this version can be leaner and more opinionated so instead of rewriting we can try to redesign the flows from scratch, it might allow us to create a simpler UI in less time.
Of curse, more research needs to be done to understand if this idea can survive the complicated reality.
I agree with ease of use when you work with the current “erb stuck”.
As a developer, the main thing in my mind is how I can create tools so other people can do great things using them.
Therefore I feel like I have two different goals here:
Build tools for people who use foreman.
Build tools for developers who build the foreman or foreman plugins so they can tools for people who use foreman.
For a while, the UI team focused on goal number 1 so we can showcase how we can deliver a better experience to the users using a different stuck.
Now we much more focused on goal 2 than before, we are aiming to deliver a better experience to the full-stack developers
and we believe we can turn the new stuck to be easier to use than the “erb stuck”.
Time will tell, but IMO we will turn it in a way that you will choose to use webpack because it is more straight forward.
I think this is another side effect when you gradually migrate a project like the foreman, our biggest technical debts are because we keep more than one implementation for the same features. The only way to finish with the debt is to finish migrating and replacing current pages also when it doesn’t bring new features.
Rewriting means we can more easily get rid of those debts.
IMO it should be slot&fill, when slot&fill is not giving you the right feature you need, contribute to slot&fill.
Based on what I’ve heard today from @amirfefer, it’s not there yet and it’s not the aim for ERB. If the outcome of this thread is full react rewrite (which we rejected in the past, May Brno meetup most recently), we need a solution for ERB. Pagelets give us a way to access server side data when rendering the UI. Based on @amirfefer’s words, to use slot/fill in this case, it means converting the extended thing to react first.
Amir please correct me if I mixed up things. But it feels if we want to kill deface, it means using both for now.
I have no real preference and I’d propose be pragmatic by choosing the quickest way. My (limited) understanding is that deface is now used in ERB pages and conflicts with React due to the implementation. That makes me think that Pagelets are the quickest way. Even if some would still consider it technical debt, I’d prefer having one kind of technical debt rather than 2. However, there may be pages where slot&fill is easier to implement and if the dev prefers that, by all means.
I’m arguing that that’s ok. Having 2 methods is better than 3.
Rewriting is one way, but my concern is that historically we’ve been very good at implementing new solutions but had limited success in deprecating the old solution and actual removal. Of course I can understand why, but if we don’t (UI) development will hit a wall where nobody wants to make a change. And I’m pretty sure we’re in agreement that we should avoid that.
I understand why we’re bad at removing, but at the very least we should try not to regress. For that, automation could be really helpful. For example, I’ve definitely been writing a lot more JS tests since Coveralls started enforcing the “no decrease in coverage” rules.
What if we were to make an effort not to introduce any new features in jQuery or Angular, or anything else we’ve decided should be deprecated? Can Travis or Jenkins note when we’re adding too many lines of new old code and bug us about it?
Me too, but pure ERB with TurboLinks with zero to minimal JavaScript. This would need the host form to be redesigned to wizard that is generating pages on the fly with less or minimal dependencies between components on the client side. Most of our pages were/are totally fine being rendered via ERB, it was just few bad examples (host form) which had issues.
I don’t do much of UI so my voice is (and should be) minimal, however I had to post it again. Foreman is not a web application for the masses, we are not Google or Apple, we should be able to deliver on ERB with almost zero JavaScript just fine.
Yes but the tricky part is where the line is between a simple bugfix and a substantive change. If you have a “simple bugfix” that ends up as a new Angular page or new implementation with dozens of lines of legacy JS, it seems people don’t really question it. (And I don’t really blame them either, because who wants to rewrite a page when you can do less than that and it works just fine.) And yet it still results in a net regression. Take this times hundreds of PR’s per year and you get a very fragmented codebase.
AFAIK the UI team didn’t want to rewrite all pages and didn’t put it on their roadmap. It is understandable because they simply don’t have the capacity to do this. A lot of developers from outside the UI team are more familiar with using ERB and simple JS so will not rewrite their pages either. This does lead to a fragmented project and IMHO it is a problem.
One solution would be that everyone learns to write React and modern JS but it’s naive to assume this will work. Personally just today I debated on submitting a bugfix in translation because it meant I’d have to spin up a development environment which meant running npm install. I may not be the typical developer because I mostly work on the installer, packaging and releasing but I suspect I’m not alone in my hatred for npm. The fact I did submit https://github.com/theforeman/foreman/pull/7044 means I could get over it, but we are at risk of splitting the Foreman developer community into a backend and frontend community. A pure JS frontend and a pure Ruby backend may be beneficial for some but it also means that various developers will have a harder time of building a feature end-to-end.
Note that personally I’d like to see a proof of concept with a pure JS frontend that just uses APIs to see if the productivity trade off is worth it. Perhaps it’s so much easier to develop the UI without the fragmented codebase. On the other hand I am skeptical. The current code base is complicated because we solve real world problems which are always messy. There are workaround that you’ll need to have anyway. That also brings a big risk of regressions. Perhaps the only way to find out where the balance is, is to attempt it?
We do have several pages that have been completely rewritten to js. in core, there is the Audits page which was completely redesigned as part of the rewrite, Statistics page which was reimplemented, and Models page which is technically still a mix but in fact could be easily converted to fully react (it is almost only erb loading the react component). In katello there are also several pages in react - Content page, Subscriptions page and several others.
thinking about the future of Foreman,
I believe we should create a different repo for front-end, like in many modern projects
which will consume API or controllers json response, but preferably API.
In this way we could really start fresh, transition pages into cutting-edge front-end technologies easily without breaking the whole application.
In the new front-end application, I agree we should use PF4 which will take a few months for the existing React component in my opinion, and more time for creating new pages that exist now only as .erb.
I think we should provide developers time and courses to get familiar with React and Redux and help them to get into front-end to help and give their opinion in such a transition.
but there are some considerations that you are likely not familiar with yet as a new member on the team. I hope I shed some light on those.