VMware intregration broken (again) Fog cant read folders (again)

abyss1 · December 18, 2019, 10:14am

Problem:
Unable to edit VVMware based hosts via Foreman if associated with a VMware cluster. Not able to browse Compute resource, Unable to associate VMs and Unble to look at associated VMs.

Expected outcome:
All three items should produce einter a Host listing from VMware, the Host detials when clicking edit in foreman or a number of asciciated hosts.

Foreman and Proxy versions:
as off foreman 1.20 through foreman 1.24

Foreman and Proxy plugin versions:
all currently updated to 1.24 repository

Distribution and version:
Debian Stretch
Vmware vSphere 6.7 u3 (updated 2 weeks ago)

Other relevant data:
Foreman UI error: There was an error listing VMs: undefined method `vmFolder’ for nil:NilClass

No additional logging shows up in the production.log or plugins folder (no debuggign enabled at the moment)

abyss1 · December 18, 2019, 11:47am

Under the full stack trace :

*NoMethodError*
**undefined method `vmFolder' for nil:NilClass**
/usr/share/foreman/vendor/ruby/2.3.0/gems/fog-vsphere-3.2.1/lib/fog/vsphere/compute.rb:733:in `public_send'
/usr/share/foreman/vendor/ruby/2.3.0/gems/fog-vsphere-3.2.1/lib/fog/vsphere/compute.rb:733:in `list_container_view'
/usr/share/foreman/vendor/ruby/2.3.0/gems/fog-vsphere-3.2.1/lib/fog/vsphere/requests/compute/list_virtual_machines.rb:64:in `block in raw_list_all_virtual_machines'
/usr/share/foreman/vendor/ruby/2.3.0/gems/fog-vsphere-3.2.1/lib/fog/vsphere/requests/compute/list_virtual_machines.rb:63:in `map'
/usr/share/foreman/vendor/ruby/2.3.0/gems/fog-vsphere-3.2.1/lib/fog/vsphere/requests/compute/list_virtual_machines.rb:63:in `raw_list_all_virtual_machines'
/usr/share/foreman/vendor/ruby/2.3.0/gems/fog-vsphere-3.2.1/lib/fog/vsphere/requests/compute/list_virtual_machines.rb:50:in `list_all_virtual_machines'
/usr/share/foreman/vendor/ruby/2.3.0/gems/fog-vsphere-3.2.1/lib/fog/vsphere/requests/compute/list_virtual_machines.rb:16:in `list_virtual_machines'
/usr/share/foreman/vendor/ruby/2.3.0/gems/fog-vsphere-3.2.1/lib/fog/vsphere/models/compute/servers.rb:27:in `all'
/usr/share/foreman/vendor/ruby/2.3.0/gems/fog-core-2.1.0/lib/fog/core/collection.rb:112:in `lazy_load'
/usr/share/foreman/vendor/ruby/2.3.0/gems/fog-core-2.1.0/lib/fog/core/collection.rb:16:in `each'
/usr/share/foreman/app/services/compute_resource_host_associator.rb:11:in `associate_hosts'
/usr/share/foreman/app/controllers/compute_resources_controller.rb:52:in `associate'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_controller/metal/basic_implicit_render.rb:6:in `send_action'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/abstract_controller/base.rb:194:in `process_action'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_controller/metal/rendering.rb:30:in `process_action'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/abstract_controller/callbacks.rb:42:in `block in process_action'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:109:in `block in run_callbacks'
/usr/share/foreman/app/controllers/concerns/foreman/controller/timezone.rb:10:in `set_timezone'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/usr/share/foreman/app/models/concerns/foreman/thread_session.rb:32:in `clear_thread'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/usr/share/foreman/app/controllers/concerns/foreman/controller/topbar_sweeper.rb:12:in `set_topbar_sweeper_controller'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/usr/share/foreman/vendor/ruby/2.3.0/gems/audited-4.9.0/lib/audited/sweeper.rb:14:in `around'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/usr/share/foreman/vendor/ruby/2.3.0/gems/audited-4.9.0/lib/audited/sweeper.rb:14:in `around'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:136:in `run_callbacks'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/abstract_controller/callbacks.rb:41:in `process_action'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_controller/metal/rescue.rb:22:in `process_action'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_controller/metal/instrumentation.rb:34:in `block in process_action'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.2.1/lib/active_support/notifications.rb:168:in `block in instrument'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.2.1/lib/active_support/notifications/instrumenter.rb:23:in `instrument'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.2.1/lib/active_support/notifications.rb:168:in `instrument'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_controller/metal/instrumentation.rb:32:in `process_action'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_controller/metal/params_wrapper.rb:256:in `process_action'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activerecord-5.2.1/lib/active_record/railties/controller_runtime.rb:24:in `process_action'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/abstract_controller/base.rb:134:in `process'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionview-5.2.1/lib/action_view/rendering.rb:32:in `process'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_controller/metal.rb:191:in `dispatch'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_controller/metal.rb:252:in `dispatch'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/routing/route_set.rb:52:in `dispatch'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/routing/route_set.rb:34:in `serve'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/journey/router.rb:52:in `block in serve'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/journey/router.rb:35:in `each'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/journey/router.rb:35:in `serve'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/routing/route_set.rb:840:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/apipie-rails-0.5.16/lib/apipie/static_dispatcher.rb:66:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/apipie-rails-0.5.16/lib/apipie/extractor/recorder.rb:137:in `call'
/usr/share/foreman/lib/foreman/middleware/telemetry.rb:10:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/apipie-rails-0.5.16/lib/apipie/middleware/checksum_in_headers.rb:27:in `call'
/usr/share/foreman/lib/foreman/middleware/catch_json_parse_errors.rb:9:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.0.7/lib/rack/tempfile_reaper.rb:15:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.0.7/lib/rack/etag.rb:25:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.0.7/lib/rack/conditional_get.rb:38:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.0.7/lib/rack/head.rb:12:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/http/content_security_policy.rb:18:in `call'
/usr/share/foreman/lib/foreman/middleware/logging_context_session.rb:22:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.0.7/lib/rack/session/abstract/id.rb:232:in `context'
/usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.0.7/lib/rack/session/abstract/id.rb:226:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/middleware/cookies.rb:670:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/middleware/callbacks.rb:28:in `block in call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:98:in `run_callbacks'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/middleware/callbacks.rb:26:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/middleware/debug_exceptions.rb:61:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/railties-5.2.1/lib/rails/rack/logger.rb:38:in `call_app'
/usr/share/foreman/vendor/ruby/2.3.0/gems/railties-5.2.1/lib/rails/rack/logger.rb:28:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/sprockets-rails-3.2.1/lib/sprockets/rails/quiet_assets.rb:13:in `call'
/usr/share/foreman/lib/foreman/middleware/logging_context_request.rb:11:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/middleware/request_id.rb:27:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.0.7/lib/rack/method_override.rb:22:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.0.7/lib/rack/runtime.rb:22:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.2.1/lib/active_support/cache/strategy/local_cache_middleware.rb:29:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/middleware/executor.rb:14:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.0.7/lib/rack/sendfile.rb:111:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/secure_headers-6.1.1/lib/secure_headers/middleware.rb:11:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/railties-5.2.1/lib/rails/engine.rb:524:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/railties-5.2.1/lib/rails/railtie.rb:190:in `public_send'
/usr/share/foreman/vendor/ruby/2.3.0/gems/railties-5.2.1/lib/rails/railtie.rb:190:in `method_missing'
/usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.0.7/lib/rack/urlmap.rb:68:in `block in call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.0.7/lib/rack/urlmap.rb:53:in `each'
/usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.0.7/lib/rack/urlmap.rb:53:in `call'
/usr/lib/ruby/vendor_ruby/phusion_passenger/rack/thread_handler_extension.rb:97:in `process_request'
/usr/lib/ruby/vendor_ruby/phusion_passenger/request_handler/thread_handler.rb:160:in `accept_and_process_next_request'
/usr/lib/ruby/vendor_ruby/phusion_passenger/request_handler/thread_handler.rb:113:in `main_loop'
/usr/lib/ruby/vendor_ruby/phusion_passenger/request_handler.rb:416:in `block (3 levels) in start_threads'
/usr/lib/ruby/vendor_ruby/phusion_passenger/utils.rb:113:in `block in create_thread_and_abort_on_exception'
/usr/share/foreman/vendor/ruby/2.3.0/gems/logging-2.2.2/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'

ekohl · December 18, 2019, 12:14pm

I’ve edited your reply to use code tags. See https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet#code as well to keep it readable.

Looking at the code, there’s no proper handling when it can’t find the datacenter object.

github.com

fog/fog-vsphere/blob/ae3c1d927374d0368563af03eadda5ae407dbc92/lib/fog/vsphere/compute.rb#L733


end


def list_container_view(datacenter_obj_or_name, type, container_object = nil)
  dc = if datacenter_obj_or_name.is_a?(String)
         find_raw_datacenter(datacenter_obj_or_name)
       else
         datacenter_obj_or_name
       end


  container = if container_object
                dc.public_send(container_object)
              else
                dc
              end


  container_view = connection.serviceContent.viewManager.CreateContainerView(
    container: dc,
    type: [type],
    recursive: true
  )

As a user I’d make sure the configured user has permission to the datacenter. On the fog-vsphere side I don’t know enough to give any recommendations. It either needs to return nil, an empty array or raise a proper exception.

abyss1 · January 22, 2020, 10:02am

Hi, ok maybe I should have clearafied this. The issue came after updating foreman against a long time running environment. Meaning 1.20 could talk to my VMware environment and through updates this quit working.

Now there are 2 factors in this 1 I am now runnign Foreman 1.24.1 (as of today) and VMware vsphere 6.7U3.xxxxx since late december 2019. Now I am pretty sure the combonation broke it as I am runnign a 6.7u2 (lower patch level due to update restrictions) which does still work.

I assume the fog implementation in foreman is to blame but not sure about this. Hense I ccome here to conforim and find a solution.

abyss1 · January 22, 2020, 10:09am

Ok so I stand corrected …apperently there was a dependecy with locations which prevented this from loading after I added all locations and organazations to the VMware clussters(all 15 of them) it started working…

Seems like there was a change there which I didnt realise.