Turbolinks - The react way

As you may know, foreman doesn’t use turbolinks anymore, therefore on each page-transition (excluding react pages) a full page reload occurs and leads to:

  • Full rerendering of all HTML content
  • Reinitialize redux’s store - stateless state across pages
  • remounting vertical navigation and topbar component on every transition

All of the above cause a performance drawback, plus pages load without an indication (such as spinner or loading state), white flashes might happen due to layout rerendering and overall all this affects user experience.

Recently we have added React Router to foreman, which is basically a client routing solution and it needed for a clean transition between react pages. We do use it also for every page transition because react-router manages the history and the location objects of the browser, it keeps these objects as one source of truth, and of course, it is much more convenient to handle it in one place. Turbolinks also mutated the history object, which caused some issues when we combined it with react-router.

At the moment we have a fallback route mechanism, react-router detects that the requested page is not a react page, it triggers a full page reload (instead of Turbolinks.visit) with the new path. What if instead of full page reload, react-router can render a Legacy component which will be responsible for its content via ajax request. With that foreman’s store remains and we gain a true stateful application, with almost every page. I’ve opened a PR for this new mechanism.

Let’s dive in for some snippets from react-router switcher which determines the current page rendering, this also shows the simplicity of using a react component instead of a full-page reload for all contents.

The current react-router switcher:

As you can see handleFallbackRoute is responsible for a full-page reload and it keeps the location with a workaround (for making it stateful across pages)

 const handleFallbackRoute = () => {
   const nextPath = window.location.pathname;
   if (currentPath !== nextPath) {
     updateCurrentPath();
     visit(nextPath);
   }
   return null;
 };

 return (
   <Switch>
     {routes.map(({ render: Component, path, ...routeProps }) => (
       <Route
         path={path}
         key={path}
         {...routeProps}
         render={props => handleRoute(Component, props)}
       />
     ))}
     <Route render={handleFallbackRoute} />
   </Switch>

react-router with Legacy content:

The fallback route just rendering Legacy component - the “React Way”

    <Switch>
      {routes.map(({ render: Component, path, ...routeProps }) => (
        <Route
          path={path}
          key={path}
          {...routeProps}
          render={props => handleReactRoute(Component, props)}
        />
      ))}
      <Route render={props => <Legacy {...props} />} />
    </Switch>

Demonstration and User experience

Disclaimer: The next videos and data were taken from compiled javascript environment (no webpack dev-server) and with production flag

When the loading time is above 250 ms, a loading screen will be shown

As you can see, the feedback is significantly slower.

Performance

On the same scenario from above I’ve also captured performance profiles via performence tab in chrome’s devtools

Current

current-state

With Legacy Component

performance-react-router

Stateless vs Stateful (Redux’s store)

Notice the redux recreation, it’s an expansive operation

VS

And of course the PR is ready for review :slight_smile: and you are more than welcome to test it by yourself

6 Likes

Thanks @amirfefer for detailed writeup, I must say it is a visible improvement and sounds like a good change. I’d like to hear from others who understands the implications more than I do, if there are any concerns with this aproach.

I’ve heard two things recently that comes to my mind

what are your thoughts on this?

This is a great example of issues and bugs that might occur due to the location workaround which I presented here, The PR fixes this by the way :slight_smile:

I’m not sure what you meant here, can you please explain a bit further?

I can express my feeling on the issue.
We are introducing a lot of custom infrastructure code, that is driving me nuts, because every custom code is additional maintanance. I’d expect someone out there has been going through the same we do now and have already solved this.

I don’t know enough about the issues with Turbolinks vs ReactRouter to judge this particular case and I don’t really know if someone was solving this with React, but my feelings are we should probably just toss our current UI instead of going through all this hasstle of creating custom solutions or choose different technology :slight_smile:

Don’t get me wrong, I am in favor of this solution if there is really not something on the market, that solves that already :slight_smile: especially if it get us to drop the backend full page rendering completely and thus not do the hybrid anymore, that way we could finally profit from react.
We have a lot of content in the head that is changing through the pages though and that could cause issues. Do we know how to approach that?

I totally agree with you, that’s my view for the long term. Creating a new UI is a long process, especially due to the fact that we migrate to an extremely different approach (server-side rendering vs client-side rendering) and that change might take a long time. On the other hand, I’d like to bring our user a much better experience now, and we can achieve that with the existing infrastructure.

Foreman with its plugins has a unique architecture, with tones of different techonolligs working together which leads to technical debts, it’s not surprising, foreman created more than 10 years ago, while web development aspects have changed rapidly.
Rendering HTML content on top of a react component exists thanks to the dangerouslySetInnerHTML feature, so it’s not something special in foreman. I haven’t reinvented the wheel here. I agree that it might create some issues, but merging it just after branching increases the reliability over time.

I wish, but there is no instant solution, foreman is still hybrid and will be for the near future. I think it is possible to deliver a better UI now with this hybrid stack and at the same time developing the next UI generation.

what I heard is pretty much aligned with what @ezr-ondrej mentioned in his post.

Few rather disturbing questions :slight_smile: If there’s no existing libraries for what we’re trying to achieve (hybrid), is it a good idea to pioneer it ourselves? Are there other stacks/libraries better fitting hybrid UIs? Should we deemphasize investing to hybrid and accelarate the transition instead? Should we invest more to hybrid? Are we OK with tooling we have for now and accept rough edges until we get to the new stack?

dislaimer: I don’t expect answers or concensus to be found here, I just want to hear devs thoughts

I personally feel like the hybrid approach has failed. The UI feels worse as a user (constant regressions, no new features).

IMHO the new tools that were supposed to make life easier and more productive have only made it harder and less productive. It has created a lot of friction between developers.

However, I also think that a fresh UI from scratch will set back the project for another year where no progress is made.

It feels like we’re stuck between a rock and a hard place. I don’t see an easy way out.

5 Likes

In these situations the only way forward is step by step by step.
Which may well entail things getting worse before they get better.
At such times one should try not to lose sight of the long term goal (transitioning the entire UI to client side rendering using react).

2 Likes

I think we have an all-or-nothing mentality and in reality we will need to live with some sort of hybrid approach for a long time while we make significant changes, no matter which approach we take technically. I wouldn’t look at it as delaying the project.

I will advocate for completely separating the UI from the backend, even if this means we go page-by-page and start out with small pages. Some benefits: using our own API and ensuring it stays up-to-date and also there is a large testing benefit. A lot of the pain points of the “new” tools are dealing with them in the larger tech stack.

For testing an SPA, we can do so at the PR level with a tool like cypress and a CI server like github actions, mocking out the API calls. This would run quickly and prevent a lot of regressions before code is even merged, testing the code by using the UI like a user. I think testing is something that we should consider in our approaches, our UI testing has room for improvement and currently it is hard, if not impossible, to test it in isolation from the backend code.

We also have the chance to build more workflow-based UIs and do so with feedback from actual user workflows. The conversations in this area have been very promising.

If we go this SPA route, we will have to accept that it will take a very long time to convert pages over and we can’t believe that a hybrid approach will be “temporary”, it will likely stay with us for years. I don’t see a huge issue with that if we handle it the right way. This means a way of sharing authentication and handling routing to the new pages so assets aren’t double-loaded. It sounds like this is all being considered by those involved.

I would hate to see us do things half-way and appease nobody, if we want to make significant changes lets use a proper SPA and get all the benefits of having decoupled UI code, while also looking at ways to improve the existing code like @amirfefer has here.

Just my two-cents :slight_smile:

1 Like

You’re talking about evolving the hybrid approach sufficiently so it actually becomes decoupled, which is usually the logical thing to avoid Second System Effect. That makes sense to me, but I was worried about an effort where a new UI from scratch was written.

On a technical level I agree, but in the past features have gotten lost in rewrites/refactors. One that still bugs me is Bug #24351: Chart items on the dashboard are no longer links - Foreman. We need to be careful in handling this.

Here there are two different views. On the one side there’s Foreman which is more like a database. Personally I like this because I’ve always seen Foreman as a CMDB. On the other side there’s Katello which has many workflow based UIs. Personally I’ve never used it because the workflows didn’t fit my mental model. This is also the feedback I’ve gotten from various users who deploy Foreman + standalone Pulp: Katello appears to complicated. When talking to them, the Library in Katello would probably have worked well for them but the workflows scared them away. IMHO Foreman’s strength has always been that you can build your own workflows on top of it. These are hard to combine and worth of a separate discussion because it’s not really tied to the SPA implementation details.

I think right now we are doing it half-way and making everyone unhappy. I’d like to hear more opinions about this because maybe I’m wrong but if I’m right, it’s a large risk for the long term health of the project.

1 Like