Searching On puppetclass with hammer and webui fails

Problem:
When trying to search on hosts based on puppetclass an error is returned indicating “Internal Server Error: the server was unable to finish the request. This may be caused by unavailability of some required service, incorrect API call or a server-side bug. There may be more information in the server’s logs.”. This happens both in the web UI and when using hammer.

This seems to be an issue in the foreman_rubygem Release Release 6.1.0 · theforeman/foreman_puppet · GitHub, however, even after updating that using the nightly version from Index of /plugins/nightly/el8/x86_64 this still doesn’t work. For reference I’m currently using a fresh RHEL 8.8 system. Additionally this seems to have been broken all the way back to 3.2.1 based on the versions I tested (3.2.1, 3.5.2, 3.6.2, 3.7, and 3.8).

Is there any recommendation on how to either correct or workaround this issue?

Expected outcome:
Query should return correctly without the error.

Foreman and Proxy versions:
foreman.noarch 3.7.0-1.el8
foreman-cli.noarch 3.7.0-1.el8
foreman-debug.noarch 3.7.0-1.el8
foreman-dynflow-sidekiq.noarch 3.7.0-1.el8
foreman-ec2.noarch 3.7.0-1.el8
foreman-installer.noarch 1:3.7.0-1.el8
foreman-libvirt.noarch 3.7.0-1.el8
foreman-postgresql.noarch 3.7.0-1.el8
foreman-proxy.noarch 3.7.0-1.el8
foreman-release.noarch 3.7.0-1.el8
foreman-selinux.noarch 3.7.0-1.el8
foreman-service.noarch 3.7.0-1.el8

Foreman and Proxy plugin versions:
rubygem-acts_as_list.noarch 1.0.3-2.el8
rubygem-deface.noarch 1.5.3-3.el8
rubygem-et-orbi.noarch 1.2.7-1.el8
rubygem-foreman-tasks.noarch 8.1.1-1.fm3_7.el8
rubygem-foreman_ansible.noarch 12.0.6-1.fm3_7.el8
rubygem-foreman_hooks.noarch 0.3.17-3.fm3_3.el8
rubygem-foreman_openscap.noarch 7.0.0-1.fm3_7.el8
rubygem-foreman_puppet.noarch 6.1.0-1.fm3_9.el8
rubygem-foreman_remote_execution.noarch 10.0.7-1.fm3_7.el8
rubygem-fugit.noarch 1.8.1-1.el8
rubygem-hammer_cli_foreman_ansible.noarch 0.5.0-1.fm3_7.el8
rubygem-hammer_cli_foreman_openscap.noarch 0.1.13-2.fm3_5.el8
rubygem-hammer_cli_foreman_puppet.noarch 0.0.6-1.fm3_4.el8
rubygem-hammer_cli_foreman_remote_execution.noarch 0.2.3-1.fm3_7.el8
rubygem-hammer_cli_foreman_tasks.noarch 0.0.19-1.fm3_7.el8
rubygem-openscap.noarch 0.4.9-9.el8
rubygem-openscap_parser.noarch 1.0.2-2.el8
rubygem-polyglot.noarch 0.3.5-3.el8
rubygem-raabro.noarch 1.4.0-1.el8
rubygem-rainbow.noarch 2.2.2-1.el8
rubygem-smart_proxy_openscap.noarch 0.9.2-1.fm3_4.el8

Distribution and version:
Red Hat Enterprise Linux release 8.8 (Ootpa)

Other relevant data:

2023-10-05T21:02:47 [I|app|963227ac] Started GET "/api/hosts?search=not+has+hostgroup+AND+not+has+puppetclass&page=1&per_page=1000" for 192.168.10.193 at 2023-10-05 21:02:47 -0400
2023-10-05T21:02:47 [I|app|963227ac] Processing by Api::V2::HostsController#index as JSON
2023-10-05T21:02:47 [I|app|963227ac]   Parameters: {"search"=>"not has hostgroup AND not has puppetclass", "page"=>"1", "per_page"=>"1000", "apiv"=>"v2", "host"=>{}}
2023-10-05T21:02:47 [I|app|963227ac] Authorized user admin(Admin User)
2023-10-05T21:02:47 [W|app|963227ac] Action failed
2023-10-05T21:02:47 [I|app|963227ac] Backtrace for 'Action failed' error (NoMethodError): undefined method `klass' for nil:NilClass
 963227ac | /usr/share/gems/gems/scoped_search-4.1.10/lib/scoped_search/definition.rb:99:in `klass'
 963227ac | /usr/share/gems/gems/scoped_search-4.1.10/lib/scoped_search/query_builder.rb:335:in `to_sql'
 963227ac | /usr/share/gems/gems/scoped_search-4.1.10/lib/scoped_search/query_builder.rb:481:in `to_null_sql'
 963227ac | /usr/share/gems/gems/scoped_search-4.1.10/lib/scoped_search/query_builder.rb:524:in `to_sql'
 963227ac | /usr/share/gems/gems/scoped_search-4.1.10/lib/scoped_search/query_builder.rb:587:in `to_not_sql'
 963227ac | /usr/share/gems/gems/scoped_search-4.1.10/lib/scoped_search/query_builder.rb:522:in `to_sql'
 963227ac | /usr/share/gems/gems/scoped_search-4.1.10/lib/scoped_search/query_builder.rb:548:in `block in to_sql'
 963227ac | /usr/share/gems/gems/scoped_search-4.1.10/lib/scoped_search/query_builder.rb:548:in `map'
 963227ac | /usr/share/gems/gems/scoped_search-4.1.10/lib/scoped_search/query_builder.rb:548:in `to_sql'
 963227ac | /usr/share/gems/gems/scoped_search-4.1.10/lib/scoped_search/query_builder.rb:56:in `build_find_params'
 963227ac | /usr/share/gems/gems/scoped_search-4.1.10/lib/scoped_search/query_builder.rb:22:in `build_query'
 963227ac | /usr/share/gems/gems/scoped_search-4.1.10/lib/scoped_search/definition.rb:328:in `block in register_named_scope!'
 963227ac | /usr/share/gems/gems/activerecord-6.1.7.3/lib/active_record/relation/delegation.rb:108:in `public_send'
 963227ac | /usr/share/gems/gems/activerecord-6.1.7.3/lib/active_record/relation/delegation.rb:108:in `block in method_missing'
 963227ac | /usr/share/gems/gems/activerecord-6.1.7.3/lib/active_record/relation.rb:406:in `block in scoping'
 963227ac | /usr/share/gems/gems/activerecord-6.1.7.3/lib/active_record/relation.rb:804:in `_scoping'
 963227ac | /usr/share/gems/gems/activerecord-6.1.7.3/lib/active_record/relation.rb:406:in `scoping'
 963227ac | /usr/share/gems/gems/activerecord-6.1.7.3/lib/active_record/relation/delegation.rb:108:in `method_missing'
 963227ac | /usr/share/foreman/app/controllers/api/base_controller.rb:114:in `resource_scope_for_index'
 963227ac | /usr/share/foreman/app/controllers/api/v2/hosts_controller.rb:40:in `index'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_controller/metal/basic_implicit_render.rb:6:in `send_action'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/abstract_controller/base.rb:228:in `process_action'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_controller/metal/rendering.rb:30:in `process_action'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/abstract_controller/callbacks.rb:42:in `block in process_action'
 963227ac | /usr/share/gems/gems/activesupport-6.1.7.3/lib/active_support/callbacks.rb:117:in `block in run_callbacks'
 963227ac | /usr/share/foreman/app/controllers/concerns/foreman/controller/timezone.rb:10:in `set_timezone'
 963227ac | /usr/share/gems/gems/activesupport-6.1.7.3/lib/active_support/callbacks.rb:126:in `block in run_callbacks'
 963227ac | /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:32:in `clear_thread'
 963227ac | /usr/share/gems/gems/activesupport-6.1.7.3/lib/active_support/callbacks.rb:126:in `block in run_callbacks'
 963227ac | /usr/share/foreman/app/controllers/concerns/foreman/controller/topbar_sweeper.rb:12:in `set_topbar_sweeper_controller'
 963227ac | /usr/share/gems/gems/activesupport-6.1.7.3/lib/active_support/callbacks.rb:126:in `block in run_callbacks'
 963227ac | /usr/share/gems/gems/audited-5.3.3/lib/audited/sweeper.rb:16:in `around'
 963227ac | /usr/share/gems/gems/activesupport-6.1.7.3/lib/active_support/callbacks.rb:126:in `block in run_callbacks'
 963227ac | /usr/share/gems/gems/audited-5.3.3/lib/audited/sweeper.rb:16:in `around'
 963227ac | /usr/share/gems/gems/activesupport-6.1.7.3/lib/active_support/callbacks.rb:126:in `block in run_callbacks'
 963227ac | /usr/share/gems/gems/activesupport-6.1.7.3/lib/active_support/callbacks.rb:137:in `run_callbacks'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/abstract_controller/callbacks.rb:41:in `process_action'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_controller/metal/rescue.rb:22:in `process_action'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_controller/metal/instrumentation.rb:34:in `block in process_action'
 963227ac | /usr/share/gems/gems/activesupport-6.1.7.3/lib/active_support/notifications.rb:203:in `block in instrument'
 963227ac | /usr/share/gems/gems/activesupport-6.1.7.3/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
 963227ac | /usr/share/gems/gems/activesupport-6.1.7.3/lib/active_support/notifications.rb:203:in `instrument'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_controller/metal/instrumentation.rb:33:in `process_action'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_controller/metal/params_wrapper.rb:249:in `process_action'
 963227ac | /usr/share/gems/gems/activerecord-6.1.7.3/lib/active_record/railties/controller_runtime.rb:27:in `process_action'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/abstract_controller/base.rb:165:in `process'
 963227ac | /usr/share/gems/gems/actionview-6.1.7.3/lib/action_view/rendering.rb:39:in `process'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_controller/metal.rb:190:in `dispatch'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_controller/metal.rb:254:in `dispatch'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_dispatch/routing/route_set.rb:50:in `dispatch'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_dispatch/routing/route_set.rb:33:in `serve'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_dispatch/routing/mapper.rb:19:in `block in <class:Constraints>'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_dispatch/routing/mapper.rb:49:in `serve'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_dispatch/journey/router.rb:50:in `block in serve'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_dispatch/journey/router.rb:32:in `each'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_dispatch/journey/router.rb:32:in `serve'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_dispatch/routing/route_set.rb:842:in `call'
 963227ac | /usr/share/gems/gems/apipie-dsl-2.5.0/lib/apipie_dsl/static_dispatcher.rb:67:in `call'
 963227ac | /usr/share/gems/gems/apipie-rails-1.1.0/lib/apipie/static_dispatcher.rb:68:in `call'
 963227ac | /usr/share/gems/gems/apipie-rails-1.1.0/lib/apipie/extractor/recorder.rb:139:in `call'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_dispatch/middleware/static.rb:24:in `call'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_dispatch/middleware/static.rb:24:in `call'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_dispatch/middleware/static.rb:24:in `call'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_dispatch/middleware/static.rb:24:in `call'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_dispatch/middleware/static.rb:24:in `call'
 963227ac | /usr/share/foreman/lib/foreman/middleware/libvirt_connection_cleaner.rb:9:in `call'
 963227ac | /usr/share/foreman/lib/foreman/middleware/telemetry.rb:10:in `call'
 963227ac | /usr/share/gems/gems/apipie-rails-1.1.0/lib/apipie/middleware/checksum_in_headers.rb:27:in `call'
 963227ac | /usr/share/gems/gems/rack-2.2.7/lib/rack/tempfile_reaper.rb:15:in `call'
 963227ac | /usr/share/gems/gems/rack-2.2.7/lib/rack/etag.rb:27:in `call'
 963227ac | /usr/share/gems/gems/rack-2.2.7/lib/rack/conditional_get.rb:27:in `call'
 963227ac | /usr/share/gems/gems/rack-2.2.7/lib/rack/head.rb:12:in `call'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_dispatch/http/permissions_policy.rb:22:in `call'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_dispatch/http/content_security_policy.rb:19:in `call'
 963227ac | /usr/share/foreman/lib/foreman/middleware/logging_context_session.rb:22:in `call'
 963227ac | /usr/share/gems/gems/rack-2.2.7/lib/rack/session/abstract/id.rb:266:in `context'
 963227ac | /usr/share/gems/gems/rack-2.2.7/lib/rack/session/abstract/id.rb:260:in `call'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_dispatch/middleware/cookies.rb:697:in `call'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_dispatch/middleware/callbacks.rb:27:in `block in call'
 963227ac | /usr/share/gems/gems/activesupport-6.1.7.3/lib/active_support/callbacks.rb:98:in `run_callbacks'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_dispatch/middleware/callbacks.rb:26:in `call'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_dispatch/middleware/actionable_exceptions.rb:18:in `call'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_dispatch/middleware/debug_exceptions.rb:29:in `call'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
 963227ac | /usr/share/gems/gems/railties-6.1.7.3/lib/rails/rack/logger.rb:37:in `call_app'
 963227ac | /usr/share/gems/gems/railties-6.1.7.3/lib/rails/rack/logger.rb:28:in `call'
 963227ac | /usr/share/gems/gems/sprockets-rails-3.4.2/lib/sprockets/rails/quiet_assets.rb:13:in `call'
 963227ac | /usr/share/foreman/lib/foreman/middleware/logging_context_request.rb:11:in `call'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
 963227ac | /usr/share/gems/gems/request_store-1.5.1/lib/request_store/middleware.rb:19:in `call'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_dispatch/middleware/request_id.rb:26:in `call'
 963227ac | /usr/share/gems/gems/rack-2.2.7/lib/rack/method_override.rb:24:in `call'
 963227ac | /usr/share/gems/gems/rack-2.2.7/lib/rack/runtime.rb:22:in `call'
 963227ac | /usr/share/gems/gems/activesupport-6.1.7.3/lib/active_support/cache/strategy/local_cache_middleware.rb:29:in `call'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_dispatch/middleware/executor.rb:14:in `call'
 963227ac | /usr/share/gems/gems/rack-2.2.7/lib/rack/sendfile.rb:110:in `call'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_dispatch/middleware/ssl.rb:77:in `call'
 963227ac | /usr/share/gems/gems/actionpack-6.1.7.3/lib/action_dispatch/middleware/host_authorization.rb:142:in `call'
 963227ac | /usr/share/gems/gems/secure_headers-6.5.0/lib/secure_headers/middleware.rb:11:in `call'
 963227ac | /usr/share/gems/gems/railties-6.1.7.3/lib/rails/engine.rb:539:in `call'
 963227ac | /usr/share/gems/gems/railties-6.1.7.3/lib/rails/railtie.rb:207:in `public_send'
 963227ac | /usr/share/gems/gems/railties-6.1.7.3/lib/rails/railtie.rb:207:in `method_missing'
 963227ac | /usr/share/gems/gems/rack-2.2.7/lib/rack/urlmap.rb:74:in `block in call'
 963227ac | /usr/share/gems/gems/rack-2.2.7/lib/rack/urlmap.rb:58:in `each'
 963227ac | /usr/share/gems/gems/rack-2.2.7/lib/rack/urlmap.rb:58:in `call'
 963227ac | /usr/share/gems/gems/puma-6.2.2/lib/puma/configuration.rb:270:in `call'
 963227ac | /usr/share/gems/gems/puma-6.2.2/lib/puma/request.rb:98:in `block in handle_request'
 963227ac | /usr/share/gems/gems/puma-6.2.2/lib/puma/thread_pool.rb:340:in `with_force_shutdown'
 963227ac | /usr/share/gems/gems/puma-6.2.2/lib/puma/request.rb:97:in `handle_request'
 963227ac | /usr/share/gems/gems/puma-6.2.2/lib/puma/server.rb:431:in `process_client'
 963227ac | /usr/share/gems/gems/puma-6.2.2/lib/puma/server.rb:233:in `block in run'
 963227ac | /usr/share/gems/gems/puma-6.2.2/lib/puma/thread_pool.rb:147:in `block in spawn_thread'
 963227ac | /usr/share/gems/gems/logging-2.3.1/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'
2023-10-05T21:02:47 [I|app|963227ac]   Rendered api/v2/errors/custom_error.json.rabl within api/v2/layouts/error_layout (Duration: 0.3ms | Allocations: 271)
2023-10-05T21:02:47 [I|app|963227ac]   Rendered layout api/v2/layouts/error_layout.json.erb (Duration: 0.6ms | Allocations: 506)
2023-10-05T21:02:47 [I|app|963227ac] Completed 500 Internal Server Error in 105ms (Views: 2.0ms | ActiveRecord: 13.1ms | Allocations: 11535)

Hi Leah,
this is was indeed broken for hostgroups but should to be fixed with the new foreman_puppet 6.1.0 release that you mentioned. At least, if you enter something like puppetclass = xyz and then hit enter in UI, this should work. What does not work and has not yet been fixed is the autocomplete function and this is the one throwing the error messages if I remember correctly. Can you please double-check in the UI? I have not tested the API myself.

After looking into this a bit more things seem to be slightly broken but I think I’ve been able to find a way to work around it at least.

When doing a filter of “not has puppetclass” that still doesn’t work even in the web UI. Although that’s a valid search syntax on other fields such as hostgroup. If I change the search to something like “not puppetclass ~ role” that does seem to work.

Here is what should work: https://github.com/theforeman/foreman_puppet/blob/master/app/models/concerns/foreman_puppet/extensions/host.rb#L16