No longer able to provision Foreman 3.10/Katello 4.12 / Xen-server

Problem:
I’m unable to provision vms with Foreman 3.10 /Katello 4.12 on a Xen cluster

Expected outcome:
Deploy provisioned Vms

Foreman and Proxy versions:
3.10

Foreman and Proxy plugin versions:
Katello 4.12

Distribution and version:
AlmaLinux 8.9

Other relevant data:

I think there is a bug somewhere in the UI, if I remembered correctly there was an option to select available templates for different OS in the “Operating system” tab.
For some reason that form is no longer available and I get an error when I try to provision.

2024-04-08T11:07:56 [I|app|545de6cd] /usr/share/gems/gems/foreman_xen-1.0.1/app/models/foreman_xen/xenserver.rb:298:in `create_vm'
 545de6cd | /usr/share/foreman/app/models/concerns/orchestration/compute.rb:107:in `setCompute'
 545de6cd | /usr/share/foreman/app/models/concerns/orchestration.rb:227:in `execute'
 545de6cd | /usr/share/foreman/app/models/concerns/orchestration.rb:152:in `block in process'
 545de6cd | /usr/share/foreman/app/models/concerns/orchestration.rb:144:in `each'
 545de6cd | /usr/share/foreman/app/models/concerns/orchestration.rb:144:in `process'
 545de6cd | /usr/share/foreman/app/models/concerns/orchestration.rb:44:in `around_save_orchestration'
 545de6cd | /usr/share/gems/gems/activesupport-6.1.7.7/lib/active_support/callbacks.rb:126:in `block in run_callbacks'
 545de6cd | /usr/share/gems/gems/activesupport-6.1.7.7/lib/active_support/callbacks.rb:137:in `run_callbacks'
 545de6cd | /usr/share/gems/gems/activesupport-6.1.7.7/lib/active_support/callbacks.rb:824:in `_run_save_callbacks'
 545de6cd | /usr/share/gems/gems/activerecord-6.1.7.7/lib/active_record/callbacks.rb:457:in `create_or_update'
 545de6cd | /usr/share/gems/gems/activerecord-6.1.7.7/lib/active_record/timestamp.rb:126:in `create_or_update'
 545de6cd | /usr/share/gems/gems/activerecord-6.1.7.7/lib/active_record/persistence.rb:474:in `save'
 545de6cd | /usr/share/gems/gems/activerecord-6.1.7.7/lib/active_record/validations.rb:47:in `save'
 545de6cd | /usr/share/gems/gems/activerecord-6.1.7.7/lib/active_record/transactions.rb:298:in `block in save'
 545de6cd | /usr/share/gems/gems/activerecord-6.1.7.7/lib/active_record/transactions.rb:354:in `block in with_transaction_returning_status'
 545de6cd | /usr/share/gems/gems/activerecord-6.1.7.7/lib/active_record/connection_adapters/abstract/database_statements.rb:320:in `block in transaction'
 545de6cd | /usr/share/gems/gems/activerecord-6.1.7.7/lib/active_record/connection_adapters/abstract/transaction.rb:319:in `block in within_new_transaction'
 545de6cd | /usr/share/gems/gems/activesupport-6.1.7.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:26:in `block (2 levels) in synchronize'
 545de6cd | /usr/share/gems/gems/activesupport-6.1.7.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
 545de6cd | /usr/share/gems/gems/activesupport-6.1.7.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
 545de6cd | /usr/share/gems/gems/activesupport-6.1.7.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
 545de6cd | /usr/share/gems/gems/activesupport-6.1.7.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
 545de6cd | /usr/share/gems/gems/activerecord-6.1.7.7/lib/active_record/connection_adapters/abstract/transaction.rb:317:in `within_new_transaction'
 545de6cd | /usr/share/gems/gems/activerecord-6.1.7.7/lib/active_record/connection_adapters/abstract/database_statements.rb:320:in `transaction'
 545de6cd | /usr/share/gems/gems/activerecord-6.1.7.7/lib/active_record/transactions.rb:350:in `with_transaction_returning_status'
 545de6cd | /usr/share/gems/gems/activerecord-6.1.7.7/lib/active_record/transactions.rb:298:in `save'
 545de6cd | /usr/share/gems/gems/activerecord-6.1.7.7/lib/active_record/suppressor.rb:44:in `save'
 545de6cd | /usr/share/foreman/app/models/concerns/foreman/sti.rb:26:in `save'
 545de6cd | /usr/share/foreman/app/controllers/hosts_controller.rb:100:in `create'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_controller/metal/basic_implicit_render.rb:6:in `send_action'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/abstract_controller/base.rb:228:in `process_action'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_controller/metal/rendering.rb:30:in `process_action'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/abstract_controller/callbacks.rb:42:in `block in process_action'
 545de6cd | /usr/share/gems/gems/activesupport-6.1.7.7/lib/active_support/callbacks.rb:117:in `block in run_callbacks'
 545de6cd | /usr/share/foreman/app/controllers/concerns/foreman/controller/timezone.rb:10:in `set_timezone'
 545de6cd | /usr/share/gems/gems/activesupport-6.1.7.7/lib/active_support/callbacks.rb:126:in `block in run_callbacks'
 545de6cd | /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:32:in `clear_thread'
 545de6cd | /usr/share/gems/gems/activesupport-6.1.7.7/lib/active_support/callbacks.rb:126:in `block in run_callbacks'
 545de6cd | /usr/share/foreman/app/controllers/concerns/foreman/controller/topbar_sweeper.rb:12:in `set_topbar_sweeper_controller'
 545de6cd | /usr/share/gems/gems/activesupport-6.1.7.7/lib/active_support/callbacks.rb:126:in `block in run_callbacks'
 545de6cd | /usr/share/gems/gems/audited-5.4.3/lib/audited/sweeper.rb:16:in `around'
 545de6cd | /usr/share/gems/gems/activesupport-6.1.7.7/lib/active_support/callbacks.rb:126:in `block in run_callbacks'
 545de6cd | /usr/share/gems/gems/audited-5.4.3/lib/audited/sweeper.rb:16:in `around'
 545de6cd | /usr/share/gems/gems/activesupport-6.1.7.7/lib/active_support/callbacks.rb:126:in `block in run_callbacks'
 545de6cd | /usr/share/gems/gems/activesupport-6.1.7.7/lib/active_support/callbacks.rb:137:in `run_callbacks'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/abstract_controller/callbacks.rb:41:in `process_action'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_controller/metal/rescue.rb:22:in `process_action'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_controller/metal/instrumentation.rb:34:in `block in process_action'
 545de6cd | /usr/share/gems/gems/activesupport-6.1.7.7/lib/active_support/notifications.rb:203:in `block in instrument'
 545de6cd | /usr/share/gems/gems/activesupport-6.1.7.7/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
 545de6cd | /usr/share/gems/gems/activesupport-6.1.7.7/lib/active_support/notifications.rb:203:in `instrument'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_controller/metal/instrumentation.rb:33:in `process_action'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_controller/metal/params_wrapper.rb:249:in `process_action'
 545de6cd | /usr/share/gems/gems/activerecord-6.1.7.7/lib/active_record/railties/controller_runtime.rb:27:in `process_action'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/abstract_controller/base.rb:165:in `process'
 545de6cd | /usr/share/gems/gems/actionview-6.1.7.7/lib/action_view/rendering.rb:39:in `process'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_controller/metal.rb:190:in `dispatch'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_controller/metal.rb:254:in `dispatch'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/routing/route_set.rb:50:in `dispatch'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/routing/route_set.rb:33:in `serve'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/journey/router.rb:50:in `block in serve'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/journey/router.rb:32:in `each'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/journey/router.rb:32:in `serve'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/routing/route_set.rb:842:in `call'
 545de6cd | /usr/share/gems/gems/katello-4.12.0/lib/katello/middleware/organization_created_enforcer.rb:18:in `call'
 545de6cd | /usr/share/gems/gems/katello-4.12.0/lib/katello/middleware/event_daemon.rb:10:in `call'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/middleware/static.rb:24:in `call'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/middleware/static.rb:24:in `call'
 545de6cd | /usr/share/gems/gems/apipie-dsl-2.6.2/lib/apipie_dsl/static_dispatcher.rb:67:in `call'
 545de6cd | /usr/share/gems/gems/apipie-rails-1.3.0/lib/apipie/static_dispatcher.rb:74:in `call'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/middleware/static.rb:24:in `call'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/middleware/static.rb:24:in `call'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/middleware/static.rb:24:in `call'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/middleware/static.rb:24:in `call'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/middleware/static.rb:24:in `call'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/middleware/static.rb:24:in `call'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/middleware/static.rb:24:in `call'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/middleware/static.rb:24:in `call'
 545de6cd | /usr/share/foreman/lib/foreman/middleware/libvirt_connection_cleaner.rb:9:in `call'
 545de6cd | /usr/share/foreman/lib/foreman/middleware/telemetry.rb:10:in `call'
 545de6cd | /usr/share/gems/gems/apipie-rails-1.3.0/lib/apipie/middleware/checksum_in_headers.rb:27:in `call'
 545de6cd | /usr/share/gems/gems/rack-2.2.8/lib/rack/tempfile_reaper.rb:15:in `call'
 545de6cd | /usr/share/gems/gems/rack-2.2.8/lib/rack/etag.rb:27:in `call'
 545de6cd | /usr/share/gems/gems/rack-2.2.8/lib/rack/conditional_get.rb:40:in `call'
 545de6cd | /usr/share/gems/gems/rack-2.2.8/lib/rack/head.rb:12:in `call'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/http/permissions_policy.rb:22:in `call'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/http/content_security_policy.rb:19:in `call'
 545de6cd | /usr/share/foreman/lib/foreman/middleware/logging_context_session.rb:22:in `call'
 545de6cd | /usr/share/gems/gems/rack-2.2.8/lib/rack/session/abstract/id.rb:266:in `context'
 545de6cd | /usr/share/gems/gems/rack-2.2.8/lib/rack/session/abstract/id.rb:260:in `call'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/middleware/cookies.rb:697:in `call'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/middleware/callbacks.rb:27:in `block in call'
 545de6cd | /usr/share/gems/gems/activesupport-6.1.7.7/lib/active_support/callbacks.rb:98:in `run_callbacks'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/middleware/callbacks.rb:26:in `call'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/middleware/actionable_exceptions.rb:18:in `call'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/middleware/debug_exceptions.rb:29:in `call'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
 545de6cd | /usr/share/gems/gems/railties-6.1.7.7/lib/rails/rack/logger.rb:37:in `call_app'
 545de6cd | /usr/share/gems/gems/railties-6.1.7.7/lib/rails/rack/logger.rb:28:in `call'
 545de6cd | /usr/share/gems/gems/sprockets-rails-3.4.2/lib/sprockets/rails/quiet_assets.rb:13:in `call'
 545de6cd | /usr/share/foreman/lib/foreman/middleware/logging_context_request.rb:11:in `call'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
 545de6cd | /usr/share/gems/gems/request_store-1.6.0/lib/request_store/middleware.rb:19:in `call'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/middleware/request_id.rb:26:in `call'
 545de6cd | /usr/share/gems/gems/katello-4.12.0/lib/katello/prevent_json_parsing.rb:12:in `call'
 545de6cd | /usr/share/gems/gems/rack-2.2.8/lib/rack/method_override.rb:24:in `call'
 545de6cd | /usr/share/gems/gems/rack-2.2.8/lib/rack/runtime.rb:22:in `call'
 545de6cd | /usr/share/gems/gems/activesupport-6.1.7.7/lib/active_support/cache/strategy/local_cache_middleware.rb:29:in `call'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/middleware/executor.rb:14:in `call'
 545de6cd | /usr/share/gems/gems/rack-2.2.8/lib/rack/sendfile.rb:110:in `call'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/middleware/ssl.rb:77:in `call'
 545de6cd | /usr/share/gems/gems/actionpack-6.1.7.7/lib/action_dispatch/middleware/host_authorization.rb:142:in `call'
 545de6cd | /usr/share/gems/gems/secure_headers-6.5.0/lib/secure_headers/middleware.rb:11:in `call'
 545de6cd | /usr/share/gems/gems/railties-6.1.7.7/lib/rails/engine.rb:539:in `call'
 545de6cd | /usr/share/gems/gems/railties-6.1.7.7/lib/rails/railtie.rb:207:in `public_send'
 545de6cd | /usr/share/gems/gems/railties-6.1.7.7/lib/rails/railtie.rb:207:in `method_missing'
 545de6cd | /usr/share/gems/gems/rack-2.2.8/lib/rack/urlmap.rb:74:in `block in call'
 545de6cd | /usr/share/gems/gems/rack-2.2.8/lib/rack/urlmap.rb:58:in `each'
 545de6cd | /usr/share/gems/gems/rack-2.2.8/lib/rack/urlmap.rb:58:in `call'
 545de6cd | /usr/share/gems/gems/puma-6.4.2/lib/puma/configuration.rb:272:in `call'
 545de6cd | /usr/share/gems/gems/puma-6.4.2/lib/puma/request.rb:100:in `block in handle_request'
 545de6cd | /usr/share/gems/gems/puma-6.4.2/lib/puma/thread_pool.rb:378:in `with_force_shutdown'
 545de6cd | /usr/share/gems/gems/puma-6.4.2/lib/puma/request.rb:99:in `handle_request'
 545de6cd | /usr/share/gems/gems/puma-6.4.2/lib/puma/server.rb:464:in `process_client'
 545de6cd | /usr/share/gems/gems/puma-6.4.2/lib/puma/server.rb:245:in `block in run'
 545de6cd | /usr/share/gems/gems/puma-6.4.2/lib/puma/thread_pool.rb:155:in `block in spawn_thread'
 545de6cd | /usr/share/gems/gems/logging-2.3.1/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'
2024-04-08T11:07:56 [W|app|545de6cd] Failed to create a compute XCP_prod (Xenserver) instance bart.xxx.lan: Template not found
1 Like

@lstejska do you know about the UI bug/change that the user mentions above?

I am not aware of such an issue, but I would check the following:

  1. Check if the template is assigned to the OS
  2. Go to OS and select the template

If this is not available, then it could be a bug.

Hi @lstejska & @iballou, thanks for looking into it.

The template is asigned to OS and selected in the templates menu from OS.

What I was referring and maybe I wasn’t very clear, there was a drop down menu in the Virtual Machine tab in the provisioning menu.

After few days of testing and debugging I think is related to the package of Xen connector provider, the project is archived from last year, but there were some PR uncommitted in the main branch and I manually patched it.

Now, I can provision, I just tested a VM few minutes ago.

My question is, how do you see the situation with Xen connector in the future?

Thanks.

1 Like

It looks like contributions for the xen plugin have gone quiet, along with contribution for fog-xenserver that powers the connection between Foreman and Xen.

Maybe something with the templates view broke? foreman-xen/app/views/compute_resources_vms/form/_templates.html.erb at master · theforeman/foreman-xen · GitHub – Just a guess.

1 Like

Hello. Here’s the diff to fix the issue: Removing 'display:none' from the 'network_fields' div so the VM Templ… · bnerickson/foreman-xen@c157ce3 · GitHub

If you apply the patch above to your local app/views/compute_resources_vms/form/_network_provisioning.html.erb file and restart the foreman services (foreman-maintain service restart) the drop-down SHOULD reappear.

I still use Xenserver and provide sporadic bugfixes to my own fork where that commit is listed, and there are a few more commits in that fork to fix some other issues that aren’t in the old plugin. I wouldn’t mind maintaining the foreman_xen plugin, though in all honesty I have no Ruby knowledge whatsoever and just apply fixes as necessary.

2 Likes

As @iballou said, the owner archived the foreman_xen repository on August 8, 2023. It is now read-only.

1 Like

@bnerickson perhaps we could start with updating the archived foreman-xen’s README to point to your own fork. If you get more contributors over time, then it might make sense to add your fork under the Foreman organization.

Sure, that works for me.

Presumably, because the repository is under “theforeman” umbrella someone within the the Foreman org will need to write/push the commit?

1 Like

Yep, once I get a second opinion from someone else in the Foreman org I can probably unarchive the repo temporarily and edit the README.

1 Like

Hi @bnerickson and thanks for your contributions :metal: :saluting_face:
I found your fork of the plugin on GitHub and I patched my files manually.

We are still using Xen server extensively and it’s a bummer that the plugin is not maintained.

@iballou & @lstejska , thanks for look into this post as well

1 Like

@ekohl or @evgeni , are you able to help with my idea above? We’d just need to temporarily unarchive the repo (GitHub - theforeman/foreman-xen: Provision and manage Xen Server in Foreman) and add a link to GitHub - bnerickson/foreman-xen: Provision and manage Xen Server in Foreman

I’ve unarchived the repo and opened link to bnerickson/foreman-xen by evgeni · Pull Request #74 · theforeman/foreman-xen · GitHub
WDYT?