Smart_proxy_openscap tests fail with OpenSCAP >= 1.2.11

Ohai,

the tests for smart_proxy currently fail when executed with OpenSCAP
>= 1.2.11 (like in EL6 or Fedora 25):

Error: test_scap_content_guide(ScapContentParserApiTest):
JSON::ParserError: 784: unexpected token at 'Error occurred:
Internal error: Could not acquire handle to xccdf.xml source.
[ds_sds_session.c:357]
'
/home/remote/egolov/.gem/ruby/gems/json-1.8.6/lib/json/common.rb:155:in parse' /home/remote/egolov/.gem/ruby/gems/json-1.8.6/lib/json/common.rb:155:inparse'
/tmp/smart_proxy_openscap/test/scap_content_parser_api_test.rb:50:in
`test_scap_content_guide'
47:
48: def test_scap_content_guide
49: post
'/scap_content/guide/xccdf_org.ssgproject.content_profile_rht-ccp',
@scap_content, 'CONTENT_TYPE' => 'text/xml'
=> 50: result = JSON.parse(last_response.body)
51: assert(result['html'].start_with?('<!DOCTYPE html>'))
52: assert(last_response.successful?)
53: end

Seems that Marek did hit that issue at some point too:
http://projects.theforeman.org/issues/17839

It works fine with OpenSCAP 1.2.9 (Debian Stretch) and 1.2.10 (EL7),
but will terribly fail on EL6 (1.2.13) and Fedora 25 (1.2.14).
Pulling builds from Koji I tracked that down to 1.2.11 being the
culprit, a proper bisect between 1.2.10 and 1.2.11 is yet outstanding.
And I guess an OpenSCAP update in EL7 is coming at some point too.

I am yet unsure if that is us doing something fishy, or whether it is
an actual bug in OpenSCAP.

Cheers
Evgeni

··· -- Beste Grüße/Kind regards,

Evgeni Golov
Software Engineer


Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael
O’Neill, Eric Shander

Trivial reproducer, without our code:

require 'openscap'
require 'openscap/source'
require 'openscap/ds/arf'
require 'openscap/xccdf/benchmark'

scap_file= '/tmp/ruby-openscap/test/data/sds-complex.xml'

OpenSCAP.oscap_init
@source = OpenSCAP::Source.new(scap_file)

sds = OpenSCAP::DS::Sds.new @source
sds.select_checklist
html = sds.html_guide

(the file is https://github.com/OpenSCAP/ruby-openscap/blob/master/test/data/sds-complex.xml)

Running this with OpenSCAP 1.2.10 results in the following warning printed:
WARNING: Processing an unresolved XCCDF document. This may have
unexpected results.
You can resolve the document using "oscap xccdf resolve -o
resolved-xccdf.xml xccdf.xml"

This warning is missing when executing the above with 1.2.11, instead
you get the already known:
Internal error: Could not acquire handle to xccdf.xml source.
[ds_sds_session.c:339] (OpenSCAP::OpenSCAPError)

I am calling it a day, and will dig deeper tomorrow.

··· On Wed, May 17, 2017 at 4:41 PM, Evgeni Golov wrote: > Ohai, > > the tests for smart_proxy currently fail when executed with OpenSCAP >>= 1.2.11 (like in EL6 or Fedora 25): > > Error: test_scap_content_guide(ScapContentParserApiTest): > JSON::ParserError: 784: unexpected token at 'Error occurred: > Internal error: Could not acquire handle to xccdf.xml source. > [ds_sds_session.c:357] > ' > /home/remote/egolov/.gem/ruby/gems/json-1.8.6/lib/json/common.rb:155:in `parse' > /home/remote/egolov/.gem/ruby/gems/json-1.8.6/lib/json/common.rb:155:in `parse' > /tmp/smart_proxy_openscap/test/scap_content_parser_api_test.rb:50:in > `test_scap_content_guide' > 47: > 48: def test_scap_content_guide > 49: post > '/scap_content/guide/xccdf_org.ssgproject.content_profile_rht-ccp', > @scap_content, 'CONTENT_TYPE' => 'text/xml' > => 50: result = JSON.parse(last_response.body) > 51: assert(result['html'].start_with?('')) > 52: assert(last_response.successful?) > 53: end > > Seems that Marek did hit that issue at some point too: > http://projects.theforeman.org/issues/17839 > > It works fine with OpenSCAP 1.2.9 (Debian Stretch) and 1.2.10 (EL7), > but will terribly fail on EL6 (1.2.13) and Fedora 25 (1.2.14). > Pulling builds from Koji I tracked that down to 1.2.11 being the > culprit, a proper bisect between 1.2.10 and 1.2.11 is yet outstanding. > And I guess an OpenSCAP update in EL7 is coming at some point too. > > I am yet unsure if that is us doing something fishy, or whether it is > an actual bug in OpenSCAP. > > Cheers > Evgeni > > -- > Beste Grüße/Kind regards, > > Evgeni Golov > Software Engineer > ________________________________________________________________________ > Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn, > Commercial register: Amtsgericht Muenchen, HRB 153243, > Managing Directors: Charles Cachera, Michael Cunningham, Michael > O'Neill, Eric Shander


Beste Grüße/Kind regards,

Evgeni Golov
Software Engineer


Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael
O’Neill, Eric Shander

So, continuing talking to myself :wink:

I looked into upstream tests
(https://github.com/OpenSCAP/ruby-openscap/blob/master/test/ds/sds_test.rb#L43-L54)
and they are supposed to catch that, but they do not, as their loop
never executes with their DS file.
Using our file I get the exact same error as in my fist mail. And
using our file I don't get the warning with my code above either.
So it's seems that the issue is in OpenSCAP itself.
I'll bring that up on their issue tracker tomorrow, let's see what they think.

Thanks for listening.
Evgeni

··· On Wed, May 17, 2017 at 5:53 PM, Evgeni Golov wrote: > Trivial reproducer, without our code: > > require 'openscap' > require 'openscap/source' > require 'openscap/ds/arf' > require 'openscap/xccdf/benchmark' > > scap_file= '/tmp/ruby-openscap/test/data/sds-complex.xml' > > OpenSCAP.oscap_init > @source = OpenSCAP::Source.new(scap_file) > > sds = OpenSCAP::DS::Sds.new @source > sds.select_checklist > html = sds.html_guide > > (the file is https://github.com/OpenSCAP/ruby-openscap/blob/master/test/data/sds-complex.xml) > > Running this with OpenSCAP 1.2.10 results in the following warning printed: > WARNING: Processing an unresolved XCCDF document. This may have > unexpected results. > You can resolve the document using "oscap xccdf resolve -o > resolved-xccdf.xml xccdf.xml" > > This warning is missing when executing the above with 1.2.11, instead > you get the already known: > Internal error: Could not acquire handle to xccdf.xml source. > [ds_sds_session.c:339] (OpenSCAP::OpenSCAPError) > > I am calling it a day, and will dig deeper tomorrow. > > On Wed, May 17, 2017 at 4:41 PM, Evgeni Golov wrote: >> Ohai, >> >> the tests for smart_proxy currently fail when executed with OpenSCAP >>>= 1.2.11 (like in EL6 or Fedora 25): >> >> Error: test_scap_content_guide(ScapContentParserApiTest): >> JSON::ParserError: 784: unexpected token at 'Error occurred: >> Internal error: Could not acquire handle to xccdf.xml source. >> [ds_sds_session.c:357] >> ' >> /home/remote/egolov/.gem/ruby/gems/json-1.8.6/lib/json/common.rb:155:in `parse' >> /home/remote/egolov/.gem/ruby/gems/json-1.8.6/lib/json/common.rb:155:in `parse' >> /tmp/smart_proxy_openscap/test/scap_content_parser_api_test.rb:50:in >> `test_scap_content_guide' >> 47: >> 48: def test_scap_content_guide >> 49: post >> '/scap_content/guide/xccdf_org.ssgproject.content_profile_rht-ccp', >> @scap_content, 'CONTENT_TYPE' => 'text/xml' >> => 50: result = JSON.parse(last_response.body) >> 51: assert(result['html'].start_with?('')) >> 52: assert(last_response.successful?) >> 53: end >> >> Seems that Marek did hit that issue at some point too: >> http://projects.theforeman.org/issues/17839 >> >> It works fine with OpenSCAP 1.2.9 (Debian Stretch) and 1.2.10 (EL7), >> but will terribly fail on EL6 (1.2.13) and Fedora 25 (1.2.14). >> Pulling builds from Koji I tracked that down to 1.2.11 being the >> culprit, a proper bisect between 1.2.10 and 1.2.11 is yet outstanding. >> And I guess an OpenSCAP update in EL7 is coming at some point too. >> >> I am yet unsure if that is us doing something fishy, or whether it is >> an actual bug in OpenSCAP. >> >> Cheers >> Evgeni >> >> -- >> Beste Grüße/Kind regards, >> >> Evgeni Golov >> Software Engineer >> ________________________________________________________________________ >> Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn, >> Commercial register: Amtsgericht Muenchen, HRB 153243, >> Managing Directors: Charles Cachera, Michael Cunningham, Michael >> O'Neill, Eric Shander > > > > -- > Beste Grüße/Kind regards, > > Evgeni Golov > Software Engineer > ________________________________________________________________________ > Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn, > Commercial register: Amtsgericht Muenchen, HRB 153243, > Managing Directors: Charles Cachera, Michael Cunningham, Michael > O'Neill, Eric Shander


Beste Grüße/Kind regards,

Evgeni Golov
Software Engineer


Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael
O’Neill, Eric Shander

This has been identified as an upstream regression:
https://github.com/OpenSCAP/openscap/issues/744
And a patch has been already merged:

Should ship in OpenSCAP 1.2.15

··· On Thu, May 18, 2017 at 6:15 PM, Evgeni Golov wrote: > So, continuing talking to myself ;) > > I looked into upstream tests > (https://github.com/OpenSCAP/ruby-openscap/blob/master/test/ds/sds_test.rb#L43-L54) > and they are supposed to catch that, but they do not, as their loop > never executes with their DS file. > Using our file I get the exact same error as in my fist mail. And > using our file I don't get the warning with my code above either. > So it's seems that the issue is in OpenSCAP itself. > I'll bring that up on their issue tracker tomorrow, let's see what they think. > > Thanks for listening. > Evgeni > > On Wed, May 17, 2017 at 5:53 PM, Evgeni Golov wrote: >> Trivial reproducer, without our code: >> >> require 'openscap' >> require 'openscap/source' >> require 'openscap/ds/arf' >> require 'openscap/xccdf/benchmark' >> >> scap_file= '/tmp/ruby-openscap/test/data/sds-complex.xml' >> >> OpenSCAP.oscap_init >> @source = OpenSCAP::Source.new(scap_file) >> >> sds = OpenSCAP::DS::Sds.new @source >> sds.select_checklist >> html = sds.html_guide >> >> (the file is https://github.com/OpenSCAP/ruby-openscap/blob/master/test/data/sds-complex.xml) >> >> Running this with OpenSCAP 1.2.10 results in the following warning printed: >> WARNING: Processing an unresolved XCCDF document. This may have >> unexpected results. >> You can resolve the document using "oscap xccdf resolve -o >> resolved-xccdf.xml xccdf.xml" >> >> This warning is missing when executing the above with 1.2.11, instead >> you get the already known: >> Internal error: Could not acquire handle to xccdf.xml source. >> [ds_sds_session.c:339] (OpenSCAP::OpenSCAPError) >> >> I am calling it a day, and will dig deeper tomorrow. >> >> On Wed, May 17, 2017 at 4:41 PM, Evgeni Golov wrote: >>> Ohai, >>> >>> the tests for smart_proxy currently fail when executed with OpenSCAP >>>>= 1.2.11 (like in EL6 or Fedora 25): >>> >>> Error: test_scap_content_guide(ScapContentParserApiTest): >>> JSON::ParserError: 784: unexpected token at 'Error occurred: >>> Internal error: Could not acquire handle to xccdf.xml source. >>> [ds_sds_session.c:357] >>> ' >>> /home/remote/egolov/.gem/ruby/gems/json-1.8.6/lib/json/common.rb:155:in `parse' >>> /home/remote/egolov/.gem/ruby/gems/json-1.8.6/lib/json/common.rb:155:in `parse' >>> /tmp/smart_proxy_openscap/test/scap_content_parser_api_test.rb:50:in >>> `test_scap_content_guide' >>> 47: >>> 48: def test_scap_content_guide >>> 49: post >>> '/scap_content/guide/xccdf_org.ssgproject.content_profile_rht-ccp', >>> @scap_content, 'CONTENT_TYPE' => 'text/xml' >>> => 50: result = JSON.parse(last_response.body) >>> 51: assert(result['html'].start_with?('')) >>> 52: assert(last_response.successful?) >>> 53: end >>> >>> Seems that Marek did hit that issue at some point too: >>> http://projects.theforeman.org/issues/17839 >>> >>> It works fine with OpenSCAP 1.2.9 (Debian Stretch) and 1.2.10 (EL7), >>> but will terribly fail on EL6 (1.2.13) and Fedora 25 (1.2.14). >>> Pulling builds from Koji I tracked that down to 1.2.11 being the >>> culprit, a proper bisect between 1.2.10 and 1.2.11 is yet outstanding. >>> And I guess an OpenSCAP update in EL7 is coming at some point too. >>> >>> I am yet unsure if that is us doing something fishy, or whether it is >>> an actual bug in OpenSCAP. >>> >>> Cheers >>> Evgeni >>> >>> -- >>> Beste Grüße/Kind regards, >>> >>> Evgeni Golov >>> Software Engineer >>> ________________________________________________________________________ >>> Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn, >>> Commercial register: Amtsgericht Muenchen, HRB 153243, >>> Managing Directors: Charles Cachera, Michael Cunningham, Michael >>> O'Neill, Eric Shander >> >> >> >> -- >> Beste Grüße/Kind regards, >> >> Evgeni Golov >> Software Engineer >> ________________________________________________________________________ >> Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn, >> Commercial register: Amtsgericht Muenchen, HRB 153243, >> Managing Directors: Charles Cachera, Michael Cunningham, Michael >> O'Neill, Eric Shander > > > > -- > Beste Grüße/Kind regards, > > Evgeni Golov > Software Engineer > ________________________________________________________________________ > Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn, > Commercial register: Amtsgericht Muenchen, HRB 153243, > Managing Directors: Charles Cachera, Michael Cunningham, Michael > O'Neill, Eric Shander


Beste Grüße/Kind regards,

Evgeni Golov
Software Engineer


Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael
O’Neill, Eric Shander