Updating Foreman to 3.6.1, and webpack assets are now missing


The web interface does not fully render after upgrading from 3.5.x to 3.6.x, Opening Developer mode in Chrome shows a large number of assets missing (404 error) predominately webpack and .css files.

Expected outcome:

The login page appears properly

Foreman and Proxy versions:


Foreman and Proxy plugin versions:

Plugins have been removed except foreman-puppet as shipped with 3.6.1
Distribution and version:

Other relevant data:

 a supported stylesheet MIME type, and strict MIME checking is enabled.
login:1 Refused to apply style from 'https://puppet/assets/application-541657e7a4cc7cfe855f497ccf2403ff1e9757a96e172ca760559ccc296ad502.css' because its MIME type ('text/html') is not a supported stylesheet MIME type, and strict MIME checking is enabled.
login:30          GET https://puppet/assets/locale/en_GB/app-c0a40cd61b600869b4f1884a9af43f9ed27cee488a6a25a8f689d6ee7458446b.js net::ERR_ABORTED 404 (Not Found)
5Refused to execute script from '<URL>' because its MIME type ('text/html') is not executable, and strict MIME type checking is enabled.
login:31          GET https://puppet/webpack/foreman-vendor.bundle-v12.0.1-production-c1237d7a8fb1dce8773c.js net::ERR_ABORTED 404 (Not Found)
login:33          GET https://puppet/webpack/bundle-e4b0eeaf0651c6dd6fe1.js net::ERR_ABORTED 404 (Not Found)
login:34          GET https://puppet/assets/application-4846c4ac6b99dad645abcdb60bcde346f0db1137a5bfc7593bc1dfe752b0f4a3.js net::ERR_ABORTED 404 (Not Found)
login:32          GET https://puppet/webpack/vendor-950b729ed5965c2a80e1.js net::ERR_ABORTED 404 (Not Found)
foreman_puppet:global-58696924c8e93c84fd4d.js:1 Uncaught ReferenceError: webpackJsonp is not defined
    at foreman_puppet:global-58696924c8e93c84fd4d.js:1:1

How did you install/upgrade Foreman?
Did you run the installer after the upgrade?

Yes to both, and even re-ran the installer for kicks

When you try to load that in your browser directly (or cURL or whatever), what is the HTML output.

When you do this, do you see any output in /var/log/foreman/production.log with that URL?

ActionController::RoutingError (No route matches [GET] "/assets/application-541657e7a4cc7cfe855f497ccf2403ff1e9757a96e172ca760559ccc296ad502.css"):

sorry to clarify, followed the instructions listed in the manual, stop service, upgrade release, clean, and upgrade

Your /etc/httpd/conf.d/05-foreman-ssl.conf is missing the following two lines:

  ProxyPass /webpack !
  ProxyPass /assets !

The installer should have set those (via Fixes #33956 - serve static assets directly via Apache · theforeman/puppet-foreman@215db65 · GitHub)

Did you configure foreman::config::apache::proxy_no_proxy_uris in custom-hiera or foreman::config::apache::proxy_assets: true?

No deviations from defaults have been made

  ## Proxy rules
  ProxyRequests Off
  ProxyPreserveHost On
  ProxyAddHeaders On
  ProxyPass /pulp !
  ProxyPass /pub !
  ProxyPass /icons !
  ProxyPass /server-status !
  ProxyPass /webpack !
  ProxyPass /assets !


But how does the request end up in Puma/Rails, when it’s set not to be proxied (and thus served locally) by Apache?!

The workaround found be to set

config.public_file_server.enabled = true

in /usr/share/foreman/config/environments/production.rb and restart Foreman (systemctl restart foreman)

But that’s a really ugly workaround.

Also, you’re saying the same setup was working with 3.5? Because all of the above changes are in 3.5 already and should affect you identically there.

1 Like

Old value was as follows, forcing that to true and restarting does indeed fix the issue. Guess I won’t be upgrading for a while as I’m guessing this will break during upgrades.

#config.public_file_server.enabled = ENV.fetch('RAILS_SERVE_STATIC_FILES', false) == 'true'

You can apply the same workaround after every upgrade… But yeah, not pretty.

Could you get me a sos report of that machine somehow? (evgeni at redhat dot com)

There is something wrong with the setup, I just don’t see what :frowning:

Sure the update was part of a migration to a new VM to save upgrading post move, but I’ll get a report thrown over before I trash the current one.

1 Like