OpenSCAP plugin with ansible - role/variables import failed

Support
, ,
1

Problem:

We’re using foreman + ansible, I’m trying to install/configure the OpenSCAP plugin as written on Foreman :: Plugin Manuals and when I’m trying to import the variables from ansible I obtain an “undefined method `map’ for nil:NilClass”

The installation steps to reproduce are:

foreman-installer --enable-foreman-plugin-openscap
foreman-installer --enable-foreman-proxy-plugin-openscap
edit the /etc/foreman-proxy/settings.d/openscap.yml
yum install ansiblerole-foreman_scap_client.noarch

Move to Configure -> Ansible -> Roles -> import role and the role “theforeman.foreman_scap_client” is present/loaded:

ls -ltr /etc/ansible/roles/
lrwxrwxrwx. 1 root root   56 Nov 14 14:08 scap -> /usr/share/ansible/roles/theforeman.foreman_scap_client/

Move to Configure -> Ansible -> Variables -> import variables and an error appear (see trace below).
i tried also to create a new policy with ansible but, of course, I have the error:

The following Ansible variables were missing for theforeman.foreman_scap_client: 
foreman_scap_client_policies, foreman_scap_client_port, foreman_scap_client_server.
Make sure they are imported before proceeding.

I think I’m missing something or there are some misconfiguration…any tip is really welcome.

Expected outcome:

importing variables and roles properly

Foreman and Proxy versions:

foreman 1.23.1

Foreman and Proxy plugin versions:

katello 3.13.3
openscap plugin 1.0.9

Distribution and version:

RHEL7
ansible 2.9.1
ansiblerole-foreman_scap_client 0.0.3-1.el7
tfm-rubygem-foreman_ansible 3.0.9-1.fm1_23.el7
tfm-rubygem-foreman_ansible_core 3.0.1-1.fm1_23.el7

Other relevant data:

Error on importing variables:

*NoMethodError*
**undefined method `map' for nil:NilClass**
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_ansible-3.0.9/app/services/foreman_ansible/variables_importer.rb:48:in `initialize_variables'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_ansible-3.0.9/app/services/foreman_ansible/variables_importer.rb:34:in `block in import_variables'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_ansible-3.0.9/app/services/foreman_ansible/variables_importer.rb:31:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_ansible-3.0.9/app/services/foreman_ansible/variables_importer.rb:31:in `map'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_ansible-3.0.9/app/services/foreman_ansible/variables_importer.rb:31:in `import_variables'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_ansible-3.0.9/app/services/foreman_ansible/variables_importer.rb:25:in `import_variable_names'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_ansible-3.0.9/app/controllers/ansible_variables_controller.rb:37:in `import'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_controller/metal/basic_implicit_render.rb:6:in `send_action'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/abstract_controller/base.rb:194:in `process_action'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_controller/metal/rendering.rb:30:in `process_action'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/abstract_controller/callbacks.rb:42:in `block in process_action'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:109:in `block in run_callbacks'
/usr/share/foreman/app/controllers/concerns/foreman/controller/timezone.rb:10:in `set_timezone'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/usr/share/foreman/app/models/concerns/foreman/thread_session.rb:32:in `clear_thread'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/usr/share/foreman/app/controllers/concerns/foreman/controller/topbar_sweeper.rb:12:in `set_topbar_sweeper_controller'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/opt/theforeman/tfm/root/usr/share/gems/gems/audited-4.7.1/lib/audited/sweeper.rb:14:in `around'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/opt/theforeman/tfm/root/usr/share/gems/gems/audited-4.7.1/lib/audited/sweeper.rb:14:in `around'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:136:in `run_callbacks'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/abstract_controller/callbacks.rb:41:in `process_action'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_controller/metal/rescue.rb:22:in `process_action'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_controller/metal/instrumentation.rb:34:in `block in process_action'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/notifications.rb:168:in `block in instrument'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/notifications/instrumenter.rb:23:in `instrument'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/notifications.rb:168:in `instrument'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_controller/metal/instrumentation.rb:32:in `process_action'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_controller/metal/params_wrapper.rb:256:in `process_action'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/railties/controller_runtime.rb:24:in `process_action'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/abstract_controller/base.rb:134:in `process'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionview-5.2.1/lib/action_view/rendering.rb:32:in `process'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_controller/metal.rb:191:in `dispatch'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_controller/metal.rb:252:in `dispatch'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/routing/route_set.rb:52:in `dispatch'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/routing/route_set.rb:34:in `serve'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/journey/router.rb:52:in `block in serve'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/journey/router.rb:35:in `each'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/journey/router.rb:35:in `serve'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/routing/route_set.rb:840:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.5.14/lib/apipie/static_dispatcher.rb:65:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.5.14/lib/apipie/extractor/recorder.rb:137:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call'
/usr/share/foreman/lib/foreman/middleware/telemetry.rb:10:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.5.14/lib/apipie/middleware/checksum_in_headers.rb:27:in `call'
/usr/share/foreman/lib/foreman/middleware/catch_json_parse_errors.rb:9:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/tempfile_reaper.rb:15:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/etag.rb:25:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/conditional_get.rb:25:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/head.rb:12:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/http/content_security_policy.rb:18:in `call'
/usr/share/foreman/lib/foreman/middleware/logging_context_session.rb:22:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/session/abstract/id.rb:232:in `context'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/session/abstract/id.rb:226:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/cookies.rb:670:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/callbacks.rb:28:in `block in call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:98:in `run_callbacks'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/callbacks.rb:26:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/debug_exceptions.rb:61:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/railties-5.2.1/lib/rails/rack/logger.rb:38:in `call_app'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/railties-5.2.1/lib/rails/rack/logger.rb:28:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/sprockets-rails-3.2.1/lib/sprockets/rails/quiet_assets.rb:13:in `call'
/usr/share/foreman/lib/foreman/middleware/logging_context_request.rb:11:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/request_id.rb:27:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.13.3/lib/katello/prevent_json_parsing.rb:12:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/method_override.rb:22:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/runtime.rb:22:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/cache/strategy/local_cache_middleware.rb:29:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/executor.rb:14:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/sendfile.rb:111:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/secure_headers-6.0.0/lib/secure_headers/middleware.rb:13:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/railties-5.2.1/lib/rails/engine.rb:524:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/railties-5.2.1/lib/rails/railtie.rb:190:in `public_send'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/railties-5.2.1/lib/rails/railtie.rb:190:in `method_missing'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/urlmap.rb:68:in `block in call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/urlmap.rb:53:in `each'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/urlmap.rb:53:in `call'
/usr/share/passenger/phusion_passenger/rack/thread_handler_extension.rb:74:in `process_request'
/usr/share/passenger/phusion_passenger/request_handler/thread_handler.rb:141:in `accept_and_process_next_request'
/usr/share/passenger/phusion_passenger/request_handler/thread_handler.rb:109:in `main_loop'
/usr/share/passenger/phusion_passenger/request_handler.rb:455:in `block (3 levels) in start_threads'
/opt/theforeman/tfm/root/usr/share/gems/gems/logging-2.2.2/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'
2

Hi, thanks for reporting this. Unfortunately, I haven’t been able to reproduce so far, I am setting up a new instance to see if the error appears.

It seems like there is some sort of problem with the variables that proxy sends to foreman. What is your version of smart_proxy_ansible?

Is the error present when importing variables for other roles as well or is this only caused by foreman_scap_client role?

3

The smart_proxy_ansible is:
rubygem-smart_proxy_ansible.noarch 3.0.1-1.fm1_23.el7

I double checked with a new role who have 3 variables and the problem is present too, the ansible feature is enabled on smart proxy and I’m able to create a variable if necessary.

If necessary, my ansible config is really simple:

[defaults]
callback_whitelist = foreman
[callback_foreman]
url = 'https://myforeman'
ssl_cert = /etc/foreman-proxy/foreman_ssl_cert.pem
ssl_key = /etc/foreman-proxy/foreman_ssl_key.pem
verify_certs = /etc/foreman-proxy/foreman_ssl_ca.pem
4

Thanks, I have no luck reproducing it. You can try turning log level to both foreman and smart proxy to debug and observe if anything interesting appears when you try to import variables. Knowing what proxy returns when asked to import variables would at least let us know whether there is a problem on the proxy side, unfortunately we do not log the response body.

5

Thanks!
The debug log for production.log is still the same of the one posted on first post, the foreman-proxy one is:

2019-12-16T12:18:41  [D] accept: ::ffff:10.140.16.230:50680
2019-12-16T12:18:41  [D] Rack::Handler::WEBrick is invoked.
2019-12-16T12:18:41 1f66fdf2 [I] Started GET /ansible/roles
2019-12-16T12:18:41 1f66fdf2 [I] Finished GET /ansible/roles with 200 (2.12 ms)
2019-12-16T12:18:41  [D] close: ::ffff:10.140.16.230:50680
2019-12-16T12:18:41  [D] accept: ::ffff:10.140.16.230:50682
2019-12-16T12:18:41  [D] Rack::Handler::WEBrick is invoked.
2019-12-16T12:18:41 1f66fdf2 [I] Started GET /ansible/roles/variables
2019-12-16T12:18:41 1f66fdf2 [I] Finished GET /ansible/roles/variables with 200 (9.18 ms)
2019-12-16T12:18:41  [D] close: ::ffff:10.140.16.230:50682
2019-12-16T12:18:41  [D] accept: ::ffff:10.140.16.230:50684
2019-12-16T12:18:41  [D] Rack::Handler::WEBrick is invoked.
2019-12-16T12:18:41 67a243a3 [I] Started GET /ansible/roles
2019-12-16T12:18:41 67a243a3 [I] Finished GET /ansible/roles with 200 (1.08 ms)
2019-12-16T12:18:41  [D] close: ::ffff:10.140.16.230:50684
2019-12-16T12:18:41  [D] accept: ::ffff:10.140.16.230:50686
2019-12-16T12:18:41  [D] Rack::Handler::WEBrick is invoked.
2019-12-16T12:18:41 67a243a3 [I] Started GET /ansible/roles/variables
2019-12-16T12:18:41 67a243a3 [I] Finished GET /ansible/roles/variables with 200 (5.34 ms)
2019-12-16T12:18:41  [D] close: ::ffff:10.140.16.230:50686

and 10.140.16.230 is the proxy.
Just a stupid question (who can help me to have a brainstorming :slight_smile: ): is the smart proxy checking for a direcotry named “variables” or “/plugin/variables” into /etc/ansible?
The rpm or the git of openscap doesn’t have it:

[ /etc/ansible/roles/theforeman.foreman_scap_client]# du -ah
4.0K    ./.flake8
4.0K    ./.gitignore
4.0K    ./.travis.yml
32K     ./LICENSE
8.0K    ./README.md
4.0K    ./defaults/main.yml
4.0K    ./defaults
4.0K    ./files/foreman_scap_client_facts.rb
4.0K    ./files
4.0K    ./meta/main.yml
4.0K    ./meta/version.yml
4.0K    ./meta/.galaxy_install_info
12K     ./meta
4.0K    ./molecule/default/molecule.yml
4.0K    ./molecule/default/playbook.yml
4.0K    ./molecule/default/tests/test_default.py
4.0K    ./molecule/default/tests
4.0K    ./molecule/default/yaml-lint.yml
16K     ./molecule/default
16K     ./molecule
4.0K    ./requirements.txt
4.0K    ./tasks/main.yml
4.0K    ./tasks
4.0K    ./templates/config.yaml.j2
4.0K    ./templates/cron.j2
8.0K    ./templates
4.0K    ./variables

and the variables are into defaults/main.yml.
Thanks

6

I’ll post also the /etc/foreman-proxy/ansible.cfg:

[defaults]
callback_whitelist = foreman
local_tmp = /tmp
host_key_checking = False
stdout_callback = yaml
roles_path = /etc/ansible/roles:/usr/share/ansible/roles

[callback_foreman]
url = https://myforeman
ssl_cert = /etc/foreman-proxy/foreman_ssl_cert.pem
ssl_key = /etc/foreman-proxy/foreman_ssl_key.pem
verify_certs = /etc/foreman-proxy/foreman_ssl_ca.pem

[ssh_connection]
ssh_args = -o ProxyCommand=none
7

/ansible/roles/variables is just an API endpoint of the proxy that gets called and is supposed to return variables for roles in roles_path. Proxy looks at roles_path and for each role scans defaults dir for files containing variables.

8

I also am getting this error for what it’s worth.

I found this code in the file at the top of the stack trace:

/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_ansible-3.0.9/app/services/foreman_ansible/variables_importer.rb:29

    def import_variables(role_variables, new_roles)
  detect_changes(
    role_variables.map do |role_name, variables|
      role = import_new_role(role_name, new_roles)
      next if role.blank?
      initialize_variables(variables, role)
    end.select(&:present?).flatten.compact
  )
end

I added the following, which fixed the issue for me:

          next if variables.blank?

Hope this helps you!

9

Thanks! Where have you added this line? Can you paste your final code? Thanks a lot!

10

A PR with the change would be great:

11

@kraba Here you go!

@Ondrej_Prazak I don’t know if this is a proper fix or if I am just masking a different error with this patch. But for my use case, it definitely fixed the problem.

12

I confirm is working, I can import variables now

13

@kraba Merged to master!