PSA: Rubygem psych 4.0 is out and it's a breaking change

lzap · May 17, 2021, 6:56am

Hello,

this is a warning if your plugin has an open ended dependency for psych, the YAML serializer. Version 4.0 was pushed out last week and it introduces a breaking change - YAML is by default loaded in safe mode, meaning that Ruby objects will not be deserialized.

github.com/ruby/psych

Use Psych.safe_load by default

ruby:master ← ruby:default-unsafe

opened 04:52PM - 10 May 21 UTC

tenderlove

+56 -129

Psych.load is not safe for use with untrusted data. Too many applications make… the mistake of using `Psych.load` with untrusted data and that ends up with some kind of security vulnerability. This commit changes the default `Psych.load` to use `safe_load`. Users that want to parse trusted data can use Psych.unsafe_load. `Psych.load` or `YAML.load` are susceptible to a Remote Code Execution (or RCE) flaw. The problem isn't with the load functions themselves, but they can be used to abuse other objects that are in the system to escalate to an RCE. [This blog post](https://staaldraad.github.io/post/2019-03-02-universal-rce-ruby-yaml-load/) goes in to details about how YAML can be used in conjunction with RubyGems to execute arbitrary code. If someone tries to load YAML from an untrusted source, bad actors could use this issue to execute arbitrary code in the target system. Psych provides a method called "safe_load" which can't be used to launch such an attack. The proposal here is to change the default such that "load" is now "safe_load". Of course the drawback is that "safe_load" is much more limited than "load", and making this change could break existing code. Also it's probably worth mentioning that the YAML load functions could be used with other objects in the system besides RubyGems to escalate to an RCE. Also worth mentioning that anything that can be used to load arbitrary objects (like `Marshal.load`) can be abused in the same way. So neither changing RubyGems nor changing to a different serialization scheme (like Marshal) would plug this hole.

This is why Katello CI is failing at the moment:

Psych::DisallowedClass: Tried to load unspecified class: Symbol
/usr/local/rvm/gems/ruby-2.5.1@test_develop_pr_katello-1/gems/psych-4.0.0/lib/psych/class_loader.rb:99:in `find'
/usr/local/rvm/gems/ruby-2.5.1@test_develop_pr_katello-1/gems/psych-4.0.0/lib/psych/class_loader.rb:28:in `load'
(eval):2:in `symbol'
/usr/local/rvm/gems/ruby-2.5.1@test_develop_pr_katello-1/gems/psych-4.0.0/lib/psych/class_loader.rb:32:in `symbolize'
/usr/local/rvm/gems/ruby-2.5.1@test_develop_pr_katello-1/gems/psych-4.0.0/lib/psych/class_loader.rb:84:in `symbolize'
/usr/local/rvm/gems/ruby-2.5.1@test_develop_pr_katello-1/gems/psych-4.0.0/lib/psych/scalar_scanner.rb:74:in `tokenize'
/usr/local/rvm/gems/ruby-2.5.1@test_develop_pr_katello-1/gems/psych-4.0.0/lib/psych/visitors/to_ruby.rb:65:in `deserialize'
/usr/local/rvm/gems/ruby-2.5.1@test_develop_pr_katello-1/gems/psych-4.0.0/lib/psych/visitors/to_ruby.rb:128:in `visit_Psych_Nodes_Scalar'
/usr/local/rvm/gems/ruby-2.5.1@test_develop_pr_katello-1/gems/psych-4.0.0/lib/psych/visitors/visitor.rb:30:in `visit'

I will push a fix shortly, unless someone else is faster. Take care!

lzap · May 17, 2021, 7:04am

Here is my patch:

lzap · May 17, 2021, 1:00pm

Looks like I have to pin psych to 3.x version because vcr is actually also pinned. I wonder why it failed in the first place.

Justin_Sherrill · May 19, 2021, 7:58pm

Sadly this seems to have broken our local devel environments for running
tests. It seems (at least when running 1 test), that the psych
installed from a gem was conflicting with the psych installed from
ruby. I went trying to hunt down why psych was even being installed as
a gem and discovered that it was being added to the bundler.d directory
here:

github.com

theforeman/jenkins-jobs/blob/f42f50d9ae5a171f9493c33eeb12b2016d9e86a4/theforeman.org/scripts/test/test_katello.sh#L48


      
          gemset=$(echo ${JOB_NAME} | cut -d/ -f1)-${EXECUTOR_NUMBER}
          rvm use ruby-${ruby}@${gemset} --create
          rvm gemset empty --force
          set -x
          
          gem update --no-document
          gem install bundler -v '< 2.0' --no-document
          
          # Now let's introduce the plugin
          echo "gemspec :path => '${PLUGIN_ROOT}', :development_group => :katello_dev" >> bundler.d/Gemfile.local.rb
          echo "gem 'psych'" >> bundler.d/Gemfile.local.rb
          
          # Install dependencies
          bundle update --jobs=5 --retry=5
          
          # Database environment
          (
            sed "s/^test:/development:/; s/database:.*/database: ${gemset}-dev/" $HOME/${database}.db.yaml
            echo
            sed "s/database:.*/database: ${gemset}-test/" $HOME/${database}.db.yaml
          ) > $APP_ROOT/config/database.yml

Digging further, i found this pr:

Those responsible have been sacked!

So i’ve reverted the katello pr here:

And removed this gem addition here:

This was done ~4 years ago and whatever problem it was trying to address
is likely not even around anymore.

Justin

lzap · May 20, 2021, 6:11am

Damn, I had no popcorn in the house. Well, sorry about that.