Redesign the webpack build

Development RFCs
1

Hello,

I would like to push forward the redesign process of the webpack build.
See the previous post about the webpack redesign: Webpacking plugins, let's do it right
See the post about the full JS redesign: Redesigning the javascript stack

Issues

  • RM #27195 - Plugins js/css build with webpack cause to copy the full js/css of the core to each plugin
  • It is difficult to understand the build process.
  • Plugins can’t build themselves, they need Foreman to get build. (more steps to do before you can build your plugin).
  • We need to rebuild plugins after each core build.
    Most of it solved by the vendor but from time to time we will still see this issue because
    • Some deps still exists outside of the vendor
    • We use an alias for plugins so they can reuse code from the react_app folder.
  • The build process creates an importing alias for plugins from foremanReact to the core react_app folder. While it works when running foreman itself, it doesn’t work in the plugins dev environment (test, lint, storybook).
    Plugins find themselves mocking each piece of code they are using from core, see: katello mocks

Goals

  • Fix all the issues above
  • Upgrade webpack from v3 to v4
  • Remove the webpack-rails gem
  • Make it simple and reusable

@theforeman/builder

A node module to provide the ability to build production and development bundle files for foreman core and plugins.

For now, it contains the babel shared configuration that webpack and other tools uses.
This PR is a WIP to add the building abilities to @theforeman/builder.

How I expect it to be consumed?

  1. The consumer would install @theforeman/builder from npm:
npm install --save-dev @theforeman/builder
  1. Create a config file with the minimal needed configurations:
// foreman/config/tfm-build.config.js
module.exports = {
  entry: {
    foreman: require.resolve('../webpack/assets/javascripts/bundle.js'),
  },
};

// katello/config/tfm-build.config.js
module.exports = {
  entry: {
    katello: require.resolve('../webpack/index.js'),
    katelloFills: require.resolve('../webpack/index_fills.js'),
    katelloCommon: require.resolve('../webpack/index_common.js'),
  },
};
  1. Use the needed executable (tfm-build, tfm-build-analyze, tfm-dev-server) and provide your config file (e.g. tfm-build config/tfm-build.config.js).

User Stories

  • Foreman Developer

    • Developers should be able to build foreman
      tfm-build config/tfm-build.config.js
    • Developers should be able to run a development server with the foreman and the installed plugins
      tfm-dev-server config/tfm-build.config.js
    • Developers should be able to analyze foreman build
      tfm-build-analyze config/tfm-build.config.js

  • Plugin Developer

    • Developers should be able to build their plugins
      tfm-build --plugin config/tfm-build.config.js
    • Developers should be able to analyze their plugins build
      tfm-build-analyze --plugin config/tfm-build.config.js

Solving the react_app/foremanReact issue

Fixing - RM #27195 - Plugins js/css build with webpack cause to copy the full js/css of the core to each plugin by creating a new separate process to build foremanReact/react_app into a standard library using webpack-4.

// foreman/config/tfm-build.config.js
module.exports = {
  library: 'foremanReact', // build it into a library
  entry: {
    foreman: require.resolve('../webpack/assets/javascripts/react_app/index.js'),
  },
};

tfm-build config/foreman-react.tfm-build.config.js

How to build a library with webpack:

Webpack Externals:

There is already a PR exists in foreman-js to put all the needed webpack configurations in @theforeman/builder and adding the tfm-build executable:

The goal is to add the library option so it would add some configurations that would build your entry to a library.

parallel-webpack

Using this library would allow us to build and run a development server with core, foremanReact, and plugins while they get to build in parallel.
It would allow us to watch and only build the parts that need to get rebuild.

TODO

Feel welcome to comment on anything, suggest changes and contribute to the effort.

Cheers!

1 Like
2

I am confused. I recall a conversation back in May 2019 about this topic and was under the assumption, this was the priority and was supposed to be resolved in last 6 months.

Few links:

I understand the vendor package was supposed to help with this issue and there may be still some edge cases not yet resolved, however this post suggests we haven’t started with the redesign and plugins are not yet ready to consume the react. So my questions are:

  • how long such redesign would take
  • what was implemented in last 6 months that had a higher priority than this?
  • what is the guarantee that this time, it will fix the issue

The last question is driven by my frustration, caused by seeing the same issues in nightlies, like a year ago. We haven’t resolved this yet, although react was introduced more than 3 years after to our UI stack. We’re trying to solve this since 2017. It almost feels it’s not possible to resolve it and we should change the direction. There are many other options I believe.

3

I would agree the process is not perfect and we can optimize it quite a lot. But I would suggest we can postpone this and move to more pressing issues like redesign the provisioning scenarios to make more sense and move it all to react.

So I think the main question is: can we live without this for a while?
I believe all the described issues can be lived with for some time, right?