Problem creating product

Problem: When I try to create a new product, I get an error message in the WebGUI:

Error
An error occurred while saving the Product: 0 The Dynflow world was not initialized yet. If your plugin uses it, make sure to call Rails.application.dynflow.require! in some initializer

Expected outcome: A product can be successfully created.

Foreman and Proxy versions: Foreman 3.0.1 with katello 4.2

Foreman and Proxy plugin versions:

Distribution and version:
Centos 8 stream

Other relevant data:
With foreman 3.0 all worked fine, the problem arised after the update to 3.0.1 .
Also I cannot access the existing products: They will be listed, but when clicking on it also an error message is printed in the WebGUI:

undefined method `[]' for nil:NilClass

Again this problem did not exists in 3.0 and starts with 3.0.1 .

Hi @martux69

Does the output of foreman-maintain service status look okay? If not, you may try foreman-maintain service restart…

also, it would be helpful to have the full backtrace from both those errors. It should be in the production.log file.

thanks!

1 Like

Hi @jeremylenz ,
sorry for delay in answering.
foreman-maintain says : | All services are running [OK]
In the production.log are a lot of entries when I click on product, here is an extraction:

2021-12-15T16:08:13 [I|app|5a40386c] Processing by Katello::Api::V2::ProductsController#show as HTML
2021-12-15T16:08:13 [I|app|5a40386c]   Parameters: {"organization_id"=>"3", "api_version"=>"v2", "id"=>"4"}
2021-12-15T16:08:13 [I|app|5a40386c]   Rendering /usr/share/gems/gems/katello-4.3.0.rc3/app/views/katello/api/v2/products/show.json.rabl within katello/api/v2/layouts/resource
2021-12-15T16:08:13 [I|app|72ea25ab] Started GET "/layouts/partials/page-loading-indicator.html" for 2a02:8108:1680:6700:1588:74cc:a415:292d at 2021-12-15 16:08:13 +0100
2021-12-15T16:08:13 [W|for|5a40386c] Could not load execution plan 6e679efd-56dd-4819-bedb-a0838ea7f568 for task 49443278-bd86-4fa3-8c36-f6048e8bede1
2021-12-15T16:08:13 [I|for|5a40386c] Backtrace for 'Could not load execution plan 6e679efd-56dd-4819-bedb-a0838ea7f568 for task 49443278-bd86-4fa3-8c36-f6048e8bede1' error (RuntimeError): The Dynflow world was not initialized yet. If your plugin uses it, make sure to call Rails.application.dynflow.require! in some initializer
 5a40386c | /usr/share/gems/gems/dynflow-1.6.2/lib/dynflow/rails.rb:75:in `world'
 5a40386c | /usr/share/gems/gems/foreman-tasks-5.2.0/app/models/foreman_tasks/task/dynflow_task.rb:55:in `execution_plan'
 5a40386c | /usr/share/gems/gems/foreman-tasks-5.2.0/app/models/foreman_tasks/task/dynflow_task.rb:167:in `active_job?'
 5a40386c | /usr/share/gems/gems/foreman-tasks-5.2.0/app/models/foreman_tasks/task/dynflow_task.rb:87:in `input'
 5a40386c | /usr/share/gems/gems/katello-4.3.0.rc3/app/models/katello/glue/pulp/repos.rb:95:in `block in sync_state_aggregated'
 5a40386c | /usr/share/gems/gems/katello-4.3.0.rc3/app/models/katello/glue/pulp/repos.rb:95:in `transform_values'
 5a40386c | /usr/share/gems/gems/katello-4.3.0.rc3/app/models/katello/glue/pulp/repos.rb:95:in `sync_state_aggregated'
 5a40386c | /usr/share/gems/gems/rabl-0.14.3/lib/rabl/helpers.rb:16:in `data_object_attribute'
 5a40386c | /usr/share/gems/gems/rabl-0.14.3/lib/rabl/builder.rb:145:in `attribute'
 5a40386c | /usr/share/gems/gems/rabl-0.14.3/lib/rabl/builder.rb:116:in `block in compile_settings'
 5a40386c | /usr/share/gems/gems/rabl-0.14.3/lib/rabl/builder.rb:115:in `each'
 5a40386c | /usr/share/gems/gems/rabl-0.14.3/lib/rabl/builder.rb:115:in `compile_settings'
 5a40386c | /usr/share/gems/gems/rabl-0.14.3/lib/rabl/builder.rb:55:in `block in to_hash'
 5a40386c | /usr/share/gems/gems/rabl-0.14.3/lib/rabl/builder.rb:252:in `cache_results'
 5a40386c | /usr/share/gems/gems/rabl-0.14.3/lib/rabl/builder.rb:51:in `to_hash'
 5a40386c | /usr/share/gems/gems/rabl-0.14.3/lib/rabl/engine.rb:90:in `to_hash'
 5a40386c | /usr/share/gems/gems/rabl-0.14.3/lib/rabl/engine.rb:103:in `to_dumpable'
 5a40386c | /usr/share/gems/gems/rabl-0.14.3/lib/rabl/engine.rb:112:in `to_json'
 5a40386c | /usr/share/gems/gems/rabl-0.14.3/lib/rabl/engine.rb:50:in `block in render'
 5a40386c | /usr/share/gems/gems/rabl-0.14.3/lib/rabl/engine.rb:385:in `cache_results'
 5a40386c | /usr/share/gems/gems/rabl-0.14.3/lib/rabl/engine.rb:49:in `render'
 5a40386c | /usr/share/foreman/config/initializers/rabl_init.rb:49:in `render'
 5a40386c | /usr/share/gems/gems/katello-4.3.0.rc3/app/views/katello/api/v2/products/show.json.rabl:3:in `_fa48dbe2890240839998e1cf8db47f33'
 5a40386c | /usr/share/gems/gems/actionview-6.0.3.7/lib/action_view/base.rb:274:in `_run'
 5a40386c | /usr/share/gems/gems/actionview-6.0.3.7/lib/action_view/template.rb:185:in `block in render'
 5a40386c | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/notifications.rb:182:in `instrument'
 5a40386c | /usr/share/gems/gems/actionview-6.0.3.7/lib/action_view/template.rb:385:in `instrument_render_template'
 5a40386c | /usr/share/gems/gems/actionview-6.0.3.7/lib/action_view/template.rb:183:in `render'
 5a40386c | /usr/share/gems/gems/deface-1.5.3/lib/deface/action_view_extensions.rb:43:in `render'
 5a40386c | /usr/share/gems/gems/actionview-6.0.3.7/lib/action_view/renderer/template_renderer.rb:58:in `block (2 levels) in render_template'
 5a40386c | /usr/share/gems/gems/actionview-6.0.3.7/lib/action_view/renderer/abstract_renderer.rb:88:in `block in instrument'
 5a40386c | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/notifications.rb:180:in `block in instrument'
 5a40386c | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
 5a40386c | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/notifications.rb:180:in `instrument'
 5a40386c | /usr/share/gems/gems/actionview-6.0.3.7/lib/action_view/renderer/abstract_renderer.rb:87:in `instrument'
 5a40386c | /usr/share/gems/gems/actionview-6.0.3.7/lib/action_view/renderer/template_renderer.rb:57:in `block in render_template'
 5a40386c | /usr/share/gems/gems/actionview-6.0.3.7/lib/action_view/renderer/template_renderer.rb:65:in `render_with_layout'
 5a40386c | /usr/share/gems/gems/actionview-6.0.3.7/lib/action_view/renderer/template_renderer.rb:56:in `render_template'
 5a40386c | /usr/share/gems/gems/actionview-6.0.3.7/lib/action_view/renderer/template_renderer.rb:13:in `render'
 5a40386c | /usr/share/gems/gems/actionview-6.0.3.7/lib/action_view/renderer/renderer.rb:61:in `render_template_to_object'
 5a40386c | /usr/share/gems/gems/actionview-6.0.3.7/lib/action_view/renderer/renderer.rb:29:in `render_to_object'
 5a40386c | /usr/share/gems/gems/actionview-6.0.3.7/lib/action_view/rendering.rb:117:in `block in _render_template'
 5a40386c | /usr/share/gems/gems/actionview-6.0.3.7/lib/action_view/base.rb:304:in `in_rendering_context'
 5a40386c | /usr/share/gems/gems/actionview-6.0.3.7/lib/action_view/rendering.rb:116:in `_render_template'
 5a40386c | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/streaming.rb:218:in `_render_template'
 5a40386c | /usr/share/gems/gems/actionview-6.0.3.7/lib/action_view/rendering.rb:103:in `render_to_body'
 5a40386c | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/rendering.rb:52:in `render_to_body'
 5a40386c | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/renderers.rb:142:in `render_to_body'
 5a40386c | /usr/share/gems/gems/actionpack-6.0.3.7/lib/abstract_controller/rendering.rb:25:in `render'
 5a40386c | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/rendering.rb:36:in `render'
 5a40386c | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/instrumentation.rb:44:in `block (2 levels) in render'
 5a40386c | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/core_ext/benchmark.rb:14:in `block in ms'
 5a40386c | /usr/share/ruby/benchmark.rb:308:in `realtime'
 5a40386c | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/core_ext/benchmark.rb:14:in `ms'
 5a40386c | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/instrumentation.rb:44:in `block in render'
 .....
  5a40386c | /usr/share/gems/gems/logging-2.3.0/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'
2021-12-15T16:08:13 [I|app|5a40386c]   Rendered /usr/share/gems/gems/katello-4.3.0.rc3/app/views/katello/api/v2/products/show.json.rabl within katello/api/v2/layouts/resource (Duration: 61.0ms | Allocations: 8410)
2021-12-15T16:08:13 [E|app|5a40386c] ActionView::Template::Error: undefined method `[]' for nil:NilClass
 5a40386c | /usr/share/gems/gems/katello-4.3.0.rc3/app/models/katello/glue/pulp/repos.rb:95:in `block in sync_state_aggregated'
 5a40386c | /usr/share/gems/gems/katello-4.3.0.rc3/app/models/katello/glue/pulp/repos.rb:95:in `transform_values'
 5a40386c | /usr/share/gems/gems/katello-4.3.0.rc3/app/models/katello/glue/pulp/repos.rb:95:in `sync_state_aggregated'
 5a40386c | /usr/share/gems/gems/rabl-0.14.3/lib/rabl/helpers.rb:16:in `data_object_attribute'
 5a40386c | /usr/share/gems/gems/rabl-0.14.3/lib/rabl/builder.rb:145:in `attribute'
 5a40386c | /usr/share/gems/gems/rabl-0.14.3/lib/rabl/builder.rb:116:in `block in compile_settings'
 5a40386c | /usr/share/gems/gems/rabl-0.14.3/lib/rabl/builder.rb:115:in `each'
 5a40386c | /usr/share/gems/gems/rabl-0.14.3/lib/rabl/builder.rb:115:in `compile_settings'
 5a40386c | /usr/share/gems/gems/rabl-0.14.3/lib/rabl/builder.rb:55:in `block in to_hash'
 5a40386c | /usr/share/gems/gems/rabl-0.14.3/lib/rabl/builder.rb:252:in `cache_results'
 5a40386c | /usr/share/gems/gems/rabl-0.14.3/lib/rabl/builder.rb:51:in `to_hash'
 5a40386c | /usr/share/gems/gems/rabl-0.14.3/lib/rabl/engine.rb:90:in `to_hash'
 5a40386c | /usr/share/gems/gems/rabl-0.14.3/lib/rabl/engine.rb:103:in `to_dumpable'
 5a40386c | /usr/share/gems/gems/rabl-0.14.3/lib/rabl/engine.rb:112:in `to_json'
 5a40386c | /usr/share/gems/gems/rabl-0.14.3/lib/rabl/engine.rb:50:in `block in render'
 5a40386c | /usr/share/gems/gems/rabl-0.14.3/lib/rabl/engine.rb:385:in `cache_results'
 5a40386c | /usr/share/gems/gems/rabl-0.14.3/lib/rabl/engine.rb:49:in `render'
 5a40386c | /usr/share/foreman/config/initializers/rabl_init.rb:49:in `render'
 5a40386c | /usr/share/gems/gems/katello-4.3.0.rc3/app/views/katello/api/v2/products/show.json.rabl:3:in `_fa48dbe2890240839998e1cf8db47f33'

Some additional info: On the CLI all seems fine.

[root@scotty ~]# hammer product list --organization "Petersen"
---|-----------------|---------------------------|--------------|--------------|------------------
ID | NAME            | DESCRIPTION               | ORGANIZATION | REPOSITORIES | SYNC STATE       
---|-----------------|---------------------------|--------------|--------------|------------------
4  | Centos 8 Stream |                           | Petersen     | 2            | Syncing Complete.
2  | custom          | Custom packages           | Petersen     | 3            | Syncing Complete.
3  | Ubuntu Focal    | Local Ubuntu Focal mirror | Petersen     | 4            | Sync Incomplete  
---|-----------------|---------------------------|--------------|--------------|------------------
[root@scotty ~]# hammer repository list --organization "Petersen"        
---|----------------------|-----------------|--------------|---------------------------------------------------------------
ID | NAME                 | PRODUCT         | CONTENT TYPE | URL                                                           
---|----------------------|-----------------|--------------|---------------------------------------------------------------
12 | appstream            | Centos 8 Stream | yum          | http://mirror.centos.org/centos/8-stream/AppStream/x86_64/os/ 
11 | base                 | Centos 8 Stream | yum          | http://mirror.centos.org/centos/8-stream/BaseOS/x86_64/os/    
6  | checkmk-plugins      | custom          | file         | http://kim.home.petersen20.de/monitor/check_mk/agents/plugins/
4  | focal                | custom          | deb          |                                                               
7  | main                 | Ubuntu Focal    | deb          | http://de.archive.ubuntu.com/ubuntu/                          
5  | rhel8                | custom          | yum          |                                                               
8  | security             | Ubuntu Focal    | deb          | http://de.archive.ubuntu.com/ubuntu                           
9  | subscription-manager | Ubuntu Focal    | deb          | https://apt.atix.de/Ubuntu20LTS/                              
10 | updates              | Ubuntu Focal    | deb          | http://de.archive.ubuntu.com/ubuntu                           
---|----------------------|-----------------|--------------|---------------------------------------------------------------
[root@scotty ~]# hammer repository info --organization "Petersen" --name "main" --product "Ubuntu Focal"
Id:                 7
Name:               main
Label:              main
Description:        
Organization:       Petersen
Red Hat Repository: no
Content Type:       deb
Mirror on Sync:     yes
Url:                http://de.archive.ubuntu.com/ubuntu/
Publish Via HTTP:   yes
Published At:       https://scotty.home.petersen20.de/pulp/content/petersen/Library/custom/Ubuntu_Focal/main/
Relative Path:      petersen/Library/custom/Ubuntu_Focal/main
Download Policy:    
HTTP Proxy:         
    HTTP Proxy Policy: global_default_http_proxy
Product:            
    Id:   3
    Name: Ubuntu Focal
GPG Key:            

Sync:               
    Status:         Warning
    Last Sync Date: about 15 hours
Created:            2021/09/21 12:08:39
Updated:            2021/09/29 07:15:17
Content Counts:

[root@scotty ~]#

BUT :face_with_raised_eyebrow: synchronization from the cli will not work:

hammer product synchronize --organization "Petersen" --name "Ubuntu Focal"
Could not synchronize the product repositories:
  The Dynflow world was not initialized yet. If your plugin uses it, make sure to call Rails.application.dynflow.require! in some initializer

The same error as the GUI call ??

Could you please post the full backtrace from this error so we may get some idea what was going on here?
Thanks.

Sure, here it is:

2021-12-16T17:52:09 [I|app|c568a13d] Started GET "/katello/api/organizations?search=name+%3D+%22Petersen%22&per_page=1000&page=1" for 2a02:8108:1680:6700:29ff:19a2:8a9f:6ff5 at 2021-12-16 17:52:09 +0100
2021-12-16T17:52:09 [I|app|c568a13d] Processing by Katello::Api::V2::OrganizationsController#index as JSON
2021-12-16T17:52:09 [I|app|c568a13d]   Parameters: {"search"=>"name = \"Petersen\"", "per_page"=>"1000", "page"=>"1", "api_version"=>"v2", "organization"=>{}}
2021-12-16T17:52:09 [I|app|c568a13d] Authorized user admin(Admin User)
2021-12-16T17:52:09 [I|app|c568a13d]   Rendering /usr/share/gems/gems/katello-4.3.0.rc3/app/views/katello/api/v2/organizations/index.json.rabl within api/v2/layouts/index_layout
2021-12-16T17:52:09 [I|app|c568a13d]   Rendered /usr/share/gems/gems/katello-4.3.0.rc3/app/views/katello/api/v2/organizations/index.json.rabl within api/v2/layouts/index_layout (Duration: 11.6ms | Allocations: 2995)
2021-12-16T17:52:09 [I|app|c568a13d] Completed 200 OK in 200ms (Views: 19.8ms | ActiveRecord: 43.9ms | Allocations: 15350)
2021-12-16T17:52:09 [I|app|c9530242] Started POST "/katello/api/products" for 2a02:8108:1680:6700:29ff:19a2:8a9f:6ff5 at 2021-12-16 17:52:09 +0100
2021-12-16T17:52:09 [I|app|c9530242] Processing by Katello::Api::V2::ProductsController#create as JSON
2021-12-16T17:52:09 [I|app|c9530242]   Parameters: {"organization_id"=>3, "name"=>"cli test", "api_version"=>"v2", "product"=>{"name"=>"cli test", "organization_id"=>3}}
2021-12-16T17:52:09 [I|app|c9530242] Authorized user admin(Admin User)
2021-12-16T17:52:09 [E|app|c9530242] RuntimeError: The Dynflow world was not initialized yet. If your plugin uses it, make sure to call Rails.application.dynflow.require! in some initializer
 c9530242 | /usr/share/gems/gems/dynflow-1.6.2/lib/dynflow/rails.rb:75:in `world'
 c9530242 | /usr/share/gems/gems/foreman-tasks-5.2.0/lib/foreman_tasks.rb:20:in `trigger'
 c9530242 | /usr/share/gems/gems/foreman-tasks-5.2.0/lib/foreman_tasks.rb:26:in `block in trigger_task'
 c9530242 | /usr/share/gems/gems/foreman-tasks-5.2.0/lib/foreman_tasks.rb:46:in `block in rails_safe_trigger_task'
 c9530242 | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads'
 c9530242 | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares'
 c9530242 | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads'
 c9530242 | /usr/share/gems/gems/foreman-tasks-5.2.0/lib/foreman_tasks.rb:45:in `rails_safe_trigger_task'
 c9530242 | /usr/share/gems/gems/foreman-tasks-5.2.0/lib/foreman_tasks.rb:24:in `trigger_task'
 c9530242 | /usr/share/gems/gems/foreman-tasks-5.2.0/lib/foreman_tasks.rb:55:in `sync_task'
 c9530242 | /usr/share/gems/gems/foreman-tasks-5.2.0/lib/foreman_tasks/triggers.rb:22:in `sync_task'
 c9530242 | /usr/share/gems/gems/katello-4.3.0.rc3/app/controllers/katello/api/v2/products_controller.rb:85:in `create'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/basic_implicit_render.rb:6:in `send_action'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/abstract_controller/base.rb:195:in `process_action'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/rendering.rb:30:in `process_action'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/abstract_controller/callbacks.rb:42:in `block in process_action'
 c9530242 | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
 c9530242 | /usr/share/foreman/app/controllers/concerns/foreman/controller/timezone.rb:10:in `set_timezone'
 c9530242 | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
 c9530242 | /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:32:in `clear_thread'
 c9530242 | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
 c9530242 | /usr/share/foreman/app/controllers/concerns/foreman/controller/topbar_sweeper.rb:12:in `set_topbar_sweeper_controller'
 c9530242 | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
 c9530242 | /usr/share/gems/gems/audited-4.9.0/lib/audited/sweeper.rb:14:in `around'
 c9530242 | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
 c9530242 | /usr/share/gems/gems/audited-4.9.0/lib/audited/sweeper.rb:14:in `around'
 c9530242 | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
 c9530242 | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:139:in `run_callbacks'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/abstract_controller/callbacks.rb:41:in `process_action'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/rescue.rb:22:in `process_action'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/instrumentation.rb:33:in `block in process_action'
 c9530242 | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/notifications.rb:180:in `block in instrument'
 c9530242 | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
 c9530242 | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/notifications.rb:180:in `instrument'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/instrumentation.rb:32:in `process_action'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/params_wrapper.rb:245:in `process_action'
 c9530242 | /usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/railties/controller_runtime.rb:27:in `process_action'
 c9530242 | /usr/share/gems/gems/katello-4.3.0.rc3/app/controllers/katello/concerns/api/api_controller.rb:50:in `process_action'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/abstract_controller/base.rb:136:in `process'
 c9530242 | /usr/share/gems/gems/actionview-6.0.3.7/lib/action_view/rendering.rb:39:in `process'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal.rb:190:in `dispatch'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal.rb:254:in `dispatch'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/routing/route_set.rb:50:in `dispatch'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/routing/route_set.rb:33:in `serve'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/routing/mapper.rb:18:in `block in <class:Constraints>'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/routing/mapper.rb:48:in `serve'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/journey/router.rb:49:in `block in serve'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/journey/router.rb:32:in `each'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/journey/router.rb:32:in `serve'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/routing/route_set.rb:834:in `call'
 c9530242 | /usr/share/gems/gems/railties-6.0.3.7/lib/rails/engine.rb:527:in `call'
 c9530242 | /usr/share/gems/gems/railties-6.0.3.7/lib/rails/railtie.rb:190:in `public_send'
 c9530242 | /usr/share/gems/gems/railties-6.0.3.7/lib/rails/railtie.rb:190:in `method_missing'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/routing/mapper.rb:19:in `block in <class:Constraints>'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/routing/mapper.rb:48:in `serve'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/journey/router.rb:49:in `block in serve'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/journey/router.rb:32:in `each'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/journey/router.rb:32:in `serve'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/routing/route_set.rb:834:in `call'
 c9530242 | /usr/share/gems/gems/katello-4.3.0.rc3/lib/katello/middleware/organization_created_enforcer.rb:18:in `call'
 c9530242 | /usr/share/gems/gems/katello-4.3.0.rc3/lib/katello/middleware/event_daemon.rb:10:in `call'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in `call'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in `call'
 c9530242 | /usr/share/gems/gems/apipie-dsl-2.4.0/lib/apipie_dsl/static_dispatcher.rb:67:in `call'
 c9530242 | /usr/share/gems/gems/apipie-rails-0.5.17/lib/apipie/static_dispatcher.rb:66:in `call'
 c9530242 | /usr/share/gems/gems/apipie-rails-0.5.17/lib/apipie/extractor/recorder.rb:137:in `call'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in `call'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in `call'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in `call'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in `call'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in `call'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in `call'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in `call'
 c9530242 | /usr/share/foreman/lib/foreman/middleware/libvirt_connection_cleaner.rb:9:in `call'
 c9530242 | /usr/share/foreman/lib/foreman/middleware/telemetry.rb:10:in `call'
 c9530242 | /usr/share/gems/gems/apipie-rails-0.5.17/lib/apipie/middleware/checksum_in_headers.rb:27:in `call'
 c9530242 | /usr/share/foreman/lib/foreman/middleware/catch_json_parse_errors.rb:9:in `call'
 c9530242 | /usr/share/gems/gems/rack-2.2.3/lib/rack/tempfile_reaper.rb:15:in `call'
 c9530242 | /usr/share/gems/gems/rack-2.2.3/lib/rack/etag.rb:27:in `call'
 c9530242 | /usr/share/gems/gems/rack-2.2.3/lib/rack/conditional_get.rb:40:in `call'
 c9530242 | /usr/share/gems/gems/rack-2.2.3/lib/rack/head.rb:12:in `call'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/http/content_security_policy.rb:18:in `call'
 c9530242 | /usr/share/foreman/lib/foreman/middleware/logging_context_session.rb:22:in `call'
 c9530242 | /usr/share/gems/gems/rack-2.2.3/lib/rack/session/abstract/id.rb:266:in `context'
 c9530242 | /usr/share/gems/gems/rack-2.2.3/lib/rack/session/abstract/id.rb:260:in `call'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/cookies.rb:648:in `call'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/callbacks.rb:27:in `block in call'
 c9530242 | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:101:in `run_callbacks'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/callbacks.rb:26:in `call'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/actionable_exceptions.rb:18:in `call'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/debug_exceptions.rb:32:in `call'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
 c9530242 | /usr/share/gems/gems/railties-6.0.3.7/lib/rails/rack/logger.rb:37:in `call_app'
 c9530242 | /usr/share/gems/gems/railties-6.0.3.7/lib/rails/rack/logger.rb:28:in `call'
 c9530242 | /usr/share/gems/gems/sprockets-rails-3.2.1/lib/sprockets/rails/quiet_assets.rb:13:in `call'
 c9530242 | /usr/share/foreman/lib/foreman/middleware/logging_context_request.rb:11:in `call'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/request_id.rb:27:in `call'
 c9530242 | /usr/share/gems/gems/katello-4.3.0.rc3/lib/katello/prevent_json_parsing.rb:12:in `call'
 c9530242 | /usr/share/gems/gems/rack-2.2.3/lib/rack/method_override.rb:24:in `call'
 c9530242 | /usr/share/gems/gems/rack-2.2.3/lib/rack/runtime.rb:22:in `call'
 c9530242 | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/cache/strategy/local_cache_middleware.rb:29:in `call'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/executor.rb:14:in `call'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in `call'
 c9530242 | /usr/share/gems/gems/rack-2.2.3/lib/rack/sendfile.rb:110:in `call'
 c9530242 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/host_authorization.rb:76:in `call'
 c9530242 | /usr/share/gems/gems/secure_headers-6.3.0/lib/secure_headers/middleware.rb:11:in `call'
 c9530242 | /usr/share/gems/gems/railties-6.0.3.7/lib/rails/engine.rb:527:in `call'
 c9530242 | /usr/share/gems/gems/railties-6.0.3.7/lib/rails/railtie.rb:190:in `public_send'
 c9530242 | /usr/share/gems/gems/railties-6.0.3.7/lib/rails/railtie.rb:190:in `method_missing'
 c9530242 | /usr/share/gems/gems/rack-2.2.3/lib/rack/urlmap.rb:74:in `block in call'
 c9530242 | /usr/share/gems/gems/rack-2.2.3/lib/rack/urlmap.rb:58:in `each'
 c9530242 | /usr/share/gems/gems/rack-2.2.3/lib/rack/urlmap.rb:58:in `call'
 c9530242 | /usr/share/gems/gems/puma-5.5.0/lib/puma/configuration.rb:249:in `call'
 c9530242 | /usr/share/gems/gems/puma-5.5.0/lib/puma/request.rb:77:in `block in handle_request'
 c9530242 | /usr/share/gems/gems/puma-5.5.0/lib/puma/thread_pool.rb:340:in `with_force_shutdown'
 c9530242 | /usr/share/gems/gems/puma-5.5.0/lib/puma/request.rb:76:in `handle_request'
 c9530242 | /usr/share/gems/gems/puma-5.5.0/lib/puma/server.rb:447:in `process_client'
 c9530242 | /usr/share/gems/gems/puma-5.5.0/lib/puma/thread_pool.rb:147:in `block in spawn_thread'
 c9530242 | /usr/share/gems/gems/logging-2.3.0/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'
2021-12-16T17:52:09 [I|app|c9530242] Completed 500 Internal Server Error in 182ms (Views: 0.9ms | ActiveRecord: 40.8ms | Allocations: 9297)

Thanks for posting the backtrace. But unfortunately it did not give me any clue.

Tested with a fresh Katello 4.2 and no issue to create a new product there.

Did you upgrade your environment from Foreman 3.0 to 3.0.1? Have you run into any issue/error during the upgrade?

Yes, I made the upgrade from 3.0 to 3.0.1, without any errors. Actually I’m using foreman 3.1 with katello 4.3rc4 and the problem still exists.
I will try to delete existing products and create new ones then.

I delete all products and try to create a new one by cli and gui, but still the same error:

Could not create the product:
  The Dynflow world was not initialized yet. If your plugin uses it, make sure to call Rails.application.dynflow.require! in some initializer

I also find out that the same problem occurs at several other actions, eg. creating a sync plan or deleting an environment in a library.
I suppose the problem come from the dynflow processes, but they are running and there are no syslog entries from with this services:

systemctl | grep dyn
dynflow-sidekiq@orchestrator.service                                                                             loaded active running   Foreman jobs daemon - orchestrator on sidekiq                                
dynflow-sidekiq@worker-1.service                                                                                 loaded active running   Foreman jobs daemon - worker-1 on sidekiq                                    
dynflow-sidekiq@worker-hosts-queue-1.service                                                                     loaded active running   Foreman jobs daemon - worker-hosts-queue-1 on sidekiq 

There are only a warning message:

systemctl status dynflow-sidekiq@worker-hosts-queue-1.service 
● dynflow-sidekiq@worker-hosts-queue-1.service - Foreman jobs daemon - worker-hosts-queue-1 on sidekiq
   Loaded: loaded (/usr/lib/systemd/system/dynflow-sidekiq@.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2021-12-20 16:14:43 CET; 10min ago
     Docs: https://theforeman.org
 Main PID: 20408 (sidekiq)
    Tasks: 16 (limit: 50031)
   Memory: 408.2M
   CGroup: /system.slice/system-dynflow\x2dsidekiq.slice/dynflow-sidekiq@worker-hosts-queue-1.service
           └─20408 sidekiq 5.2.7  [0 of 5 busy]

Dez 20 16:13:15 scotty.home.petersen20.de systemd[1]: Starting Foreman jobs daemon - worker-hosts-queue-1 on sidekiq...
Dez 20 16:13:19 scotty.home.petersen20.de dynflow-sidekiq@worker-hosts-queue-1[20408]: 2021-12-20T15:13:19.602Z 20408 TID-e98 INFO: GitLab reliable fetch activated!
Dez 20 16:13:19 scotty.home.petersen20.de dynflow-sidekiq@worker-hosts-queue-1[20408]: 2021-12-20T15:13:19.604Z 20408 TID-e8g INFO: Booting Sidekiq 5.2.7 with redis options {:id=>"Sidekiq-server-PID-20408", :url=>"redis://localhost:6379/0"}
Dez 20 16:13:28 scotty.home.petersen20.de dynflow-sidekiq@worker-hosts-queue-1[20408]: 'ErbParser' is ignored.
Dez 20 16:13:28 scotty.home.petersen20.de dynflow-sidekiq@worker-hosts-queue-1[20408]: 'RubyParser' is ignored.
Dez 20 16:13:38 scotty.home.petersen20.de dynflow-sidekiq@worker-hosts-queue-1[20408]: /usr/share/gems/gems/foreman_puppet-2.0.0/lib/foreman_puppet/register.rb:155: warning: already initialized constant Foreman::Plugin::RbacSupport::AUTO_EXT>
Dez 20 16:13:38 scotty.home.petersen20.de dynflow-sidekiq@worker-hosts-queue-1[20408]: /usr/share/foreman/app/registries/foreman/plugin/rbac_support.rb:5: warning: previous definition of AUTO_EXTENDED_ROLES was here
Dez 20 16:14:43 scotty.home.petersen20.de systemd[1]: Started Foreman jobs daemon - worker-hosts-queue-1 on sidekiq.

I’m at a loss :roll_eyes: I don’t know how to debug the dynflow process …

It seems like something is not working right around Dynflow.

If you go to Infrastructure > Smart Proxies, Dynflow should be listed under ‘Active features.’ Is it there? Do you see any errors on that page, particularly about Dynflow?

also can’t hurt to try the ‘Refresh features’ button…

Under smart proxies, all is fine even after a refresh:


hmm, We may need to appeal to the Dynflow masters @aruzicka and @ezr-ondrej…

Maybe it relates to another bug Bug #34027: "Schedule remote job -> run salt" fails with "Dynflow world not set" - Salt - Foreman ?

As it turns out, that particular Dynflow error can show up in many different situations, not necessarily related :slight_smile:

Hmm, ok :slight_smile: What does it mean to my situation? Is there a fairly chance to debug it or is a complete new fresh installation the better option?

It could still be a bug. I wouldn’t want you to wipe out everything if it won’t help. If you’re able to wait until after the holidays the Dynflow experts should be available again…

That’s ok for me :slight_smile: I would be also more interest in an analyse and a right/good solution if there is a chance for it.
I’ll wait then.
I wish you a merry christmas :evergreen_tree: :snowman_with_snow:

1 Like

I think, I found the reason und could solve the problem:
The amount of puma workers were zero. After setting it to two and restart of all foreman services all works fine again. Also the value of puma_threads_min was blank in the answers file.
It seems as if something went wrong with these parameters during the migration from 3.0 to 3.1?

2 Likes