Improving Rails boot time

Development
1

Hello,

we have slightly touched the topic of slow app boot times. I mentioned it was worth knowing which initializers are slow because I had few candidates on my mind that could be makings things slow. So I created a small patch which measures the slowest initializers and list them in Rails debug logger:

However both on my dev and prod setup, I don’t see particularly slow initializers. Well, fast_gettext is definitely slow (0.5 - 2 seconds) since it is only dealing with a megabyte of strings, however nothing contributes to what some customers see - a minute booting time.

So it looks like it has to be Ruby or Rails what’s slow. There is a nice project called Bootsnap which makes Rails boot faster by up to 75% which is crazy. But there is one snag - bootsnap heavily relies on bundler. We use bundler for development and on debian, but for RPM installations we install a stub called bundler_ext which essentially turns off bundler and simply loads all dependencies from the default Ruby directory. Bunder_ext does not support installation of multiple versions of the same library, it does not track versions, RPM database is what makes sure all rubygems are in the correct versions.

This is a problem, if we wanted to use bootsnap to speed up boottime, it would not work:

yum install rh-ruby25-ruby-devel
scl enable tfm -- gem install bootsnap
cd /usr/share/foreman

Now make sure the require is called no matter of the environment:

# grep bootsnap config/boot.rb 
  require 'bootsnap/setup'
  require('bootsnap/setup') if early_env == "development" && File.exist?(ENV['BUNDLE_GEMFILE']) && !Gem::Specification.stubs_for("bootsnap").empty?

Then precompile files. This is where it gets dirty, this command has an argument --gemfile to pick all required gems, but on EL there is no Gemfile. We actually have a copy of gemfile named Gemfile.in but this is not used to load gems. Gems are simply loaded via simple Ruby require.

scl enable tfm -- bootsnap precompile app/ lib/ /opt/theforeman/tfm/root/usr/share/gems/gems/

Even if I try to explicitly tell the directory to walk:

scl enable tfm -- bootsnap precompile app/ lib/ /opt/theforeman/tfm/root/usr/share/gems/gems/

It does not work. I can see that about 140MB of cached bytecode is put into the cache directory however not matter of what I do, I still boot console in 12 seconds:

# time echo "puts :hello" | foreman-rake console
API controllers newer than Apipie cache! Run apipie:cache rake task to regenerate cache.
Loading production environment (Rails 6.0.3.4)
Switch to inspect mode.
puts :hello
hello
nil

real	0m12.057s
user	0m10.311s
sys	0m1.468s

I think if we want to use bootsnap, we need to figure this out. Maybe start using bundler on RPM installs, I am not sure if it’s used for DEB deployments or not.

1 Like
2

I think that the main reason we used bundler_ext was to allow using a version different from what’s in the Gemfile. That’s because patches were backported in the past. In practice we don’t really use that anymore and actually copy over the requirements in most places. We also don’t really cherry pick bugfixes or security fixes to older versions but rather update. Perhaps it’s time to reconsider bundler_ext.

3

Indeed that was the main reason.

Absolutely, agreed. It’s an ugly hack.

Since I was digging in the boot codebase, I’ve shaved 2 seconds out of my 12 seconds boot time of Foreman in development setup. It was because we initialize (thus load) all fast_gettext domains to translate language names. I made a patch which makes this lazy-loaded (but in production it still uses eager loading):

4

Related issue:
https://projects.theforeman.org/issues/31679