Role Filter of type "host" with filter for "compute_attribute" breaks Facts-Search

Problem:
If I use a userrole to limit the user access to specific hosts on a given compute_resource, it is no longer possible to search for hosts using the syntax facts.certname ~ test (or any other fact key/value).
The user gets a database error like:

Oops, we're sorry but something went wrong PG::SyntaxError: ERROR: subquery has too many 
 columns LINE 16: ...(

Steps to reproduce:

  1. Create a user role with the resource “Host” with any permissions and the search string of `
    compute_resource = “Test”``
  2. Assign the role to a user
  3. Impersonate/login to the user
  4. Search for some fact, like facts.certname ~ test

Expected outcome:
The user should see the hosts matching its permissions

Foreman and Proxy versions:
Foreman 1.24.2

Foreman and Proxy plugin versions:
No plugins

Distribution and version:

  • Local dev environment (git foreman-1.24-stable branch) with SQLite
  • RHEL 7.7

Other relevant data:
The log does not contain an actual error besides 500 Internal server error

I forgot to attach the stack trace thrown:

Extracted source (around line #50):
              
        @hosts = search.includes(included_associations).paginate(:page => params[:page], :per_page => params[:per_page])
        # SQL optimizations queries
        @last_report_ids = ConfigReport.where(:host_id => @hosts.map(&:id)).group(:host_id).maximum(:id)
        @last_reports = ConfigReport.where(:id => @last_report_ids.values)
        # rendering index page for non index page requests (out of sync hosts etc)
        @hostgroup_authorizer = Authorizer.new(User.current, :collection => @hosts.map(&:hostgroup_id).compact.uniq)

Rails.root: /Users/<user>/_dev/github.com/theforeman/foreman

Application Trace | Framework Trace | Full Trace
vendor/ruby/2.6.0/gems/sqlite3-1.3.13/lib/sqlite3/database.rb:91:in `initialize'
vendor/ruby/2.6.0/gems/sqlite3-1.3.13/lib/sqlite3/database.rb:91:in `new'
vendor/ruby/2.6.0/gems/sqlite3-1.3.13/lib/sqlite3/database.rb:91:in `prepare'
vendor/ruby/2.6.0/gems/activerecord-5.2.1/lib/active_record/connection_adapters/sqlite3_adapter.rb:216:in `block (2 levels) in exec_query'
vendor/ruby/2.6.0/gems/activesupport-5.2.1/lib/active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads'
vendor/ruby/2.6.0/gems/activesupport-5.2.1/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares'
vendor/ruby/2.6.0/gems/activesupport-5.2.1/lib/active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads'
vendor/ruby/2.6.0/gems/activerecord-5.2.1/lib/active_record/connection_adapters/sqlite3_adapter.rb:213:in `block in exec_query'
vendor/ruby/2.6.0/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract_adapter.rb:579:in `block (2 levels) in log'
/Users/<user>/.rvm/rubies/ruby-2.6.3/lib/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'
vendor/ruby/2.6.0/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract_adapter.rb:578:in `block in log'
vendor/ruby/2.6.0/gems/activesupport-5.2.1/lib/active_support/notifications/instrumenter.rb:23:in `instrument'
vendor/ruby/2.6.0/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract_adapter.rb:569:in `log'
vendor/ruby/2.6.0/gems/activerecord-5.2.1/lib/active_record/connection_adapters/sqlite3_adapter.rb:212:in `exec_query'
vendor/ruby/2.6.0/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:465:in `select'
vendor/ruby/2.6.0/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:57:in `select_all'
vendor/ruby/2.6.0/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/query_cache.rb:99:in `block in select_all'
vendor/ruby/2.6.0/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/query_cache.rb:122:in `block in cache_sql'
/Users/<user>/.rvm/rubies/ruby-2.6.3/lib/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'
vendor/ruby/2.6.0/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/query_cache.rb:108:in `cache_sql'
vendor/ruby/2.6.0/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/query_cache.rb:99:in `select_all'
vendor/ruby/2.6.0/gems/activerecord-5.2.1/lib/active_record/relation/finder_methods.rb:408:in `block in limited_ids_for'
vendor/ruby/2.6.0/gems/activerecord-5.2.1/lib/active_record/relation.rb:578:in `skip_query_cache_if_necessary'
vendor/ruby/2.6.0/gems/activerecord-5.2.1/lib/active_record/relation/finder_methods.rb:408:in `limited_ids_for'
vendor/ruby/2.6.0/gems/activerecord-5.2.1/lib/active_record/relation/finder_methods.rb:387:in `apply_join_dependency'
vendor/ruby/2.6.0/gems/activerecord-5.2.1/lib/active_record/relation.rb:544:in `block in exec_queries'
vendor/ruby/2.6.0/gems/activerecord-5.2.1/lib/active_record/relation.rb:578:in `skip_query_cache_if_necessary'
vendor/ruby/2.6.0/gems/activerecord-5.2.1/lib/active_record/relation.rb:541:in `exec_queries'
vendor/ruby/2.6.0/gems/activerecord-5.2.1/lib/active_record/relation.rb:414:in `load'
vendor/ruby/2.6.0/gems/activerecord-5.2.1/lib/active_record/relation.rb:200:in `records'
vendor/ruby/2.6.0/gems/bullet-6.0.2/lib/bullet/active_record52.rb:43:in `records'
vendor/ruby/2.6.0/gems/activerecord-5.2.1/lib/active_record/relation/delegation.rb:41:in `each'
app/controllers/hosts_controller.rb:50:in `map'
app/controllers/hosts_controller.rb:50:in `block (2 levels) in index'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/action_controller/metal/mime_responds.rb:203:in `respond_to'
app/controllers/hosts_controller.rb:46:in `index'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/action_controller/metal/basic_implicit_render.rb:6:in `send_action'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/abstract_controller/base.rb:194:in `process_action'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/action_controller/metal/rendering.rb:30:in `process_action'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/abstract_controller/callbacks.rb:42:in `block in process_action'
vendor/ruby/2.6.0/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:109:in `block in run_callbacks'
app/controllers/concerns/foreman/controller/timezone.rb:10:in `set_timezone'
vendor/ruby/2.6.0/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
app/models/concerns/foreman/thread_session.rb:32:in `clear_thread'
vendor/ruby/2.6.0/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
app/controllers/concerns/foreman/controller/topbar_sweeper.rb:12:in `set_topbar_sweeper_controller'
vendor/ruby/2.6.0/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
vendor/ruby/2.6.0/gems/audited-4.9.0/lib/audited/sweeper.rb:14:in `around'
vendor/ruby/2.6.0/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
vendor/ruby/2.6.0/gems/audited-4.9.0/lib/audited/sweeper.rb:14:in `around'
vendor/ruby/2.6.0/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
vendor/ruby/2.6.0/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:136:in `run_callbacks'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/abstract_controller/callbacks.rb:41:in `process_action'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/action_controller/metal/rescue.rb:22:in `process_action'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/action_controller/metal/instrumentation.rb:34:in `block in process_action'
vendor/ruby/2.6.0/gems/activesupport-5.2.1/lib/active_support/notifications.rb:168:in `block in instrument'
vendor/ruby/2.6.0/gems/activesupport-5.2.1/lib/active_support/notifications/instrumenter.rb:23:in `instrument'
vendor/ruby/2.6.0/gems/activesupport-5.2.1/lib/active_support/notifications.rb:168:in `instrument'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/action_controller/metal/instrumentation.rb:32:in `process_action'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/action_controller/metal/params_wrapper.rb:256:in `process_action'
vendor/ruby/2.6.0/gems/activerecord-5.2.1/lib/active_record/railties/controller_runtime.rb:24:in `process_action'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/abstract_controller/base.rb:134:in `process'
vendor/ruby/2.6.0/gems/actionview-5.2.1/lib/action_view/rendering.rb:32:in `process'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/action_controller/metal.rb:191:in `dispatch'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/action_controller/metal.rb:252:in `dispatch'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/action_dispatch/routing/route_set.rb:52:in `dispatch'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/action_dispatch/routing/route_set.rb:34:in `serve'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/action_dispatch/journey/router.rb:52:in `block in serve'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/action_dispatch/journey/router.rb:35:in `each'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/action_dispatch/journey/router.rb:35:in `serve'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/action_dispatch/routing/route_set.rb:840:in `call'
vendor/ruby/2.6.0/gems/apipie-rails-0.5.16/lib/apipie/static_dispatcher.rb:66:in `call'
vendor/ruby/2.6.0/gems/bullet-6.0.2/lib/bullet/rack.rb:15:in `call'
vendor/ruby/2.6.0/gems/apipie-rails-0.5.16/lib/apipie/extractor/recorder.rb:137:in `call'
lib/foreman/middleware/telemetry.rb:10:in `call'
vendor/ruby/2.6.0/gems/apipie-rails-0.5.16/lib/apipie/middleware/checksum_in_headers.rb:27:in `call'
lib/foreman/middleware/catch_json_parse_errors.rb:9:in `call'
vendor/ruby/2.6.0/gems/rack-openid-1.4.2/lib/rack/openid.rb:98:in `call'
vendor/ruby/2.6.0/gems/rack-2.0.7/lib/rack/tempfile_reaper.rb:15:in `call'
vendor/ruby/2.6.0/gems/rack-2.0.7/lib/rack/etag.rb:25:in `call'
vendor/ruby/2.6.0/gems/rack-2.0.7/lib/rack/conditional_get.rb:25:in `call'
vendor/ruby/2.6.0/gems/rack-2.0.7/lib/rack/head.rb:12:in `call'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/action_dispatch/http/content_security_policy.rb:18:in `call'
lib/foreman/middleware/logging_context_session.rb:22:in `call'
vendor/ruby/2.6.0/gems/rack-2.0.7/lib/rack/session/abstract/id.rb:232:in `context'
vendor/ruby/2.6.0/gems/rack-2.0.7/lib/rack/session/abstract/id.rb:226:in `call'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/action_dispatch/middleware/cookies.rb:670:in `call'
vendor/ruby/2.6.0/gems/activerecord-5.2.1/lib/active_record/migration.rb:559:in `call'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/action_dispatch/middleware/callbacks.rb:28:in `block in call'
vendor/ruby/2.6.0/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:98:in `run_callbacks'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/action_dispatch/middleware/callbacks.rb:26:in `call'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/action_dispatch/middleware/executor.rb:14:in `call'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/action_dispatch/middleware/debug_exceptions.rb:61:in `call'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
vendor/ruby/2.6.0/gems/railties-5.2.1/lib/rails/rack/logger.rb:38:in `call_app'
vendor/ruby/2.6.0/gems/railties-5.2.1/lib/rails/rack/logger.rb:28:in `call'
vendor/ruby/2.6.0/gems/sprockets-rails-3.2.1/lib/sprockets/rails/quiet_assets.rb:13:in `call'
lib/foreman/middleware/logging_context_request.rb:11:in `call'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/action_dispatch/middleware/request_id.rb:27:in `call'
vendor/ruby/2.6.0/gems/rack-2.0.7/lib/rack/method_override.rb:22:in `call'
vendor/ruby/2.6.0/gems/rack-2.0.7/lib/rack/runtime.rb:22:in `call'
vendor/ruby/2.6.0/gems/activesupport-5.2.1/lib/active_support/cache/strategy/local_cache_middleware.rb:29:in `call'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/action_dispatch/middleware/executor.rb:14:in `call'
vendor/ruby/2.6.0/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call'
vendor/ruby/2.6.0/gems/rack-2.0.7/lib/rack/sendfile.rb:111:in `call'
vendor/ruby/2.6.0/gems/secure_headers-6.3.0/lib/secure_headers/middleware.rb:11:in `call'
vendor/ruby/2.6.0/gems/railties-5.2.1/lib/rails/engine.rb:524:in `call'
vendor/ruby/2.6.0/gems/railties-5.2.1/lib/rails/railtie.rb:190:in `public_send'
vendor/ruby/2.6.0/gems/railties-5.2.1/lib/rails/railtie.rb:190:in `method_missing'
vendor/ruby/2.6.0/gems/rack-2.0.7/lib/rack/urlmap.rb:68:in `block in call'
vendor/ruby/2.6.0/gems/rack-2.0.7/lib/rack/urlmap.rb:53:in `each'
vendor/ruby/2.6.0/gems/rack-2.0.7/lib/rack/urlmap.rb:53:in `call'
vendor/ruby/2.6.0/gems/puma-4.3.0/lib/puma/configuration.rb:228:in `call'
vendor/ruby/2.6.0/gems/puma-4.3.0/lib/puma/server.rb:667:in `handle_request'
vendor/ruby/2.6.0/gems/puma-4.3.0/lib/puma/server.rb:470:in `process_client'
vendor/ruby/2.6.0/gems/puma-4.3.0/lib/puma/server.rb:328:in `block in run'
vendor/ruby/2.6.0/gems/puma-4.3.0/lib/puma/thread_pool.rb:134:in `block in spawn_thread'
vendor/ruby/2.6.0/gems/logging-2.2.2/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'

Note: The original error message is the one thrown on our production Foreman with PostgreSQL backend and the following data (stacktrace and this query) is from my local dev environment with a SQLite database.

That’s the actual query shown when using a blank Foreman 1.24 instance with SQLite backend (local dev environment):

SQLite3::SQLException: sub-select returns 59 columns - expected 1: SELECT DISTINCT “hosts”.“id” FROM “hosts” LEFT OUTER JOIN “compute_resources” ON “compute_resources”.“id” = “hosts”.“compute_resource_id” AND (1=0) LEFT OUTER JOIN “fact_values” ON “fact_values”.“host_id” = “hosts”.“id” AND “fact_values”.“fact_name_id” = ? LEFT OUTER JOIN “hostgroups” ON “hostgroups”.“id” = “hosts”.“hostgroup_id” AND (1=0) LEFT OUTER JOIN “operatingsystems” ON “operatingsystems”.“id” = “hosts”.“operatingsystem_id” LEFT OUTER JOIN “environments” ON “environments”.“id” = “hosts”.“environment_id” AND (environments.id IN (1)) LEFT OUTER JOIN “models” ON “models”.“id” = “hosts”.“model_id” LEFT OUTER JOIN “host_status” ON “host_status”.“host_id” = “hosts”.“id” LEFT OUTER JOIN “tokens” ON “tokens”.“host_id” = “hosts”.“id” AND “tokens”.“type” IN (‘Token::Build’) WHERE “hosts”.“type” IN (‘Host::Managed’) AND “hosts”.“organization_id” = ? AND “hosts”.“location_id” = ? AND (((“compute_resources”.“name” = ‘Test’) AND (“hosts”.“organization_id” = ‘1’) AND (“hosts”.“location_id” = ‘2’))) AND ((hosts.id in (SELECT “hosts”.“id”, “hosts”.“id” AS t0_r0, “hosts”.“name” AS t0_r1, “hosts”.“last_compile” AS t0_r2, “hosts”.“last_report” AS t0_r3, “hosts”.“updated_at” AS t0_r4, “hosts”.“created_at” AS t0_r5, “hosts”.“root_pass” AS t0_r6, “hosts”.“architecture_id” AS t0_r7, “hosts”.“operatingsystem_id” AS t0_r8, “hosts”.“environment_id” AS t0_r9, “hosts”.“ptable_id” AS t0_r10, “hosts”.“medium_id” AS t0_r11, “hosts”.“build” AS t0_r12, “hosts”.“comment” AS t0_r13, “hosts”.“disk” AS t0_r14, “hosts”.“installed_at” AS t0_r15, “hosts”.“model_id” AS t0_r16, “hosts”.“hostgroup_id” AS t0_r17, “hosts”.“owner_id” AS t0_r18, “hosts”.“owner_type” AS t0_r19, “hosts”.“enabled” AS t0_r20, “hosts”.“puppet_ca_proxy_id” AS t0_r21, “hosts”.“managed” AS t0_r22, “hosts”.“use_image” AS t0_r23, “hosts”.“image_file” AS t0_r24, “hosts”.“uuid” AS t0_r25, “hosts”.“compute_resource_id” AS t0_r26, “hosts”.“puppet_proxy_id” AS t0_r27, “hosts”.“certname” AS t0_r28, “hosts”.“image_id” AS t0_r29, “hosts”.“organization_id” AS t0_r30, “hosts”.“location_id” AS t0_r31, “hosts”.“type” AS t0_r32, “hosts”.“otp” AS t0_r33, “hosts”.“realm_id” AS t0_r34, “hosts”.“compute_profile_id” AS t0_r35, “hosts”.“provision_method” AS t0_r36, “hosts”.“grub_pass” AS t0_r37, “hosts”.“expired_on” AS t0_r38, “hosts”.“global_status” AS t0_r39, “hosts”.“lookup_value_matcher” AS t0_r40, “hosts”.“pxe_loader” AS t0_r41, “hosts”.“initiated_at” AS t0_r42, “hosts”.“build_errors” AS t0_r43, “compute_resources”.“id” AS t1_r0, “compute_resources”.“name” AS t1_r1, “compute_resources”.“description” AS t1_r2, “compute_resources”.“url” AS t1_r3, “compute_resources”.“user” AS t1_r4, “compute_resources”.“password” AS t1_r5, “compute_resources”.“uuid” AS t1_r6, “compute_resources”.“type” AS t1_r7, “compute_resources”.“created_at” AS t1_r8, “compute_resources”.“updated_at” AS t1_r9, “compute_resources”.“attrs” AS t1_r10, “compute_resources”.“caching_enabled” AS t1_r11, “compute_resources”.“domain” AS t1_r12, “compute_resources”.“http_proxy_id” AS t1_r13 FROM “hosts” INNER JOIN “fact_values” ON “fact_values”.“host_id” = “hosts”.“id” INNER JOIN “fact_names” ON “fact_names”.“id” = “fact_values”.“fact_name_id” LEFT OUTER JOIN “compute_resources” ON “compute_resources”.“id” = “hosts”.“compute_resource_id” AND (1=0) WHERE “hosts”.“type” IN (‘Host::Managed’) AND “hosts”.“organization_id” = 1 AND “hosts”.“location_id” = 2 AND (((“compute_resources”.“name” = ‘Test’) AND (“hosts”.“organization_id” = ‘1’) AND (“hosts”.“location_id” = ‘2’))) AND (fact_names.name = ‘app_teir’) AND (fact_values.value = ‘test’)))) ORDER BY “hosts”.“name” ASC LIMIT ? OFFSET ?

Thanks to the live debugging session during the Foreman Construction Day, this should now be fixed by https://github.com/theforeman/foreman/pull/7417

2 Likes