Problem: while trying to create a host, the deployment stops at Generating ISO image -phase. Error message is “Failed to generate ISOimage for instance xxxxxx: end of file reached” where xxxxxx is the hostname of the host we’re trying to create.
Logs say the process is fetching vmlinuz file, but access logs of the http server say that the request never comes in. Similar setup worked in January/February with Foreman 2.2, but for some reason doesn’t seem to work with newer foreman and katello. I have also been unable to find out how to tell foreman-bootdisk plugin to increase its log level. Any ideas are welcome at this point.
Expected outcome: working host deployment with preconfigured iPXE bootdisk
Foreman and Proxy versions: Foreman 2.4.0 (+Katello 4.0.1), foreman-bootdisk 17.1
Distribution and version: Centos 7.9.2009
Other relevant data:
snip from /var/log/foreman/production.log
2021-05-27T23:49:47 [I|app|0c76fa45] Generating FULL HOST ISO image for [redacted]
2021-05-27T23:49:47 [I|for|0c76fa45] Fetching https://[redacted]/pulp/content/[redacted]/Library/custom/Oracle_Linux_8/Oracle_Linux_8_x86_64_BaseOS_Latest//images/pxeboot/vmlinuz
2021-05-27T23:49:47 [W|app|0c76fa45] Failed to generate ISO image for instance [redacted]: end of file reached
2021-05-27T23:49:47 [I|app|0c76fa45] Backtrace for 'Failed to generate ISO image for instance [redacted]: end of file reached' error (EOFError): end of file reached
0c76fa45 | /opt/rh/rh-ruby25/root/usr/share/ruby/net/protocol.rb:189:in `rbuf_fill'
0c76fa45 | /opt/rh/rh-ruby25/root/usr/share/ruby/net/protocol.rb:157:in `readuntil'
0c76fa45 | /opt/rh/rh-ruby25/root/usr/share/ruby/net/protocol.rb:167:in `readline'
0c76fa45 | /opt/rh/rh-ruby25/root/usr/share/ruby/net/http/response.rb:40:in `read_status_line'
0c76fa45 | /opt/rh/rh-ruby25/root/usr/share/ruby/net/http/response.rb:29:in `read_new'
0c76fa45 | /opt/rh/rh-ruby25/root/usr/share/ruby/net/http.rb:1497:in `block in transport_request'
0c76fa45 | /opt/rh/rh-ruby25/root/usr/share/ruby/net/http.rb:1494:in `catch'
0c76fa45 | /opt/rh/rh-ruby25/root/usr/share/ruby/net/http.rb:1494:in `transport_request'
0c76fa45 | /opt/rh/rh-ruby25/root/usr/share/ruby/net/http.rb:1467:in `request'
0c76fa45 | /usr/share/foreman/lib/foreman/http_proxy/net_http_extension.rb:13:in `request'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_bootdisk-17.1.0/app/services/foreman_bootdisk/iso_generator.rb:224:in `block (2 levels) in fetch'
0c76fa45 | /opt/rh/rh-ruby25/root/usr/share/ruby/net/http.rb:910:in `start'
0c76fa45 | /opt/rh/rh-ruby25/root/usr/share/ruby/net/http.rb:609:in `start'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_bootdisk-17.1.0/app/services/foreman_bootdisk/iso_generator.rb:221:in `block in fetch'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_bootdisk-17.1.0/app/services/foreman_bootdisk/iso_generator.rb:204:in `open'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_bootdisk-17.1.0/app/services/foreman_bootdisk/iso_generator.rb:204:in `fetch'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_bootdisk-17.1.0/app/services/foreman_bootdisk/iso_generator.rb:150:in `block in generate'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_bootdisk-17.1.0/app/services/foreman_bootdisk/iso_generator.rb:149:in `each'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_bootdisk-17.1.0/app/services/foreman_bootdisk/iso_generator.rb:149:in `generate'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_bootdisk-17.1.0/app/services/foreman_bootdisk/iso_generator.rb:30:in `generate_full_host'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_bootdisk-17.1.0/app/models/concerns/foreman_bootdisk/orchestration/compute.rb:47:in `bootdisk_generate_iso_image'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_bootdisk-17.1.0/app/models/concerns/foreman_bootdisk/orchestration/compute.rb:71:in `setGenerateIsoImage'
0c76fa45 | /usr/share/foreman/app/models/concerns/orchestration.rb:227:in `execute'
0c76fa45 | /usr/share/foreman/app/models/concerns/orchestration.rb:152:in `block in process'
0c76fa45 | /usr/share/foreman/app/models/concerns/orchestration.rb:144:in `each'
0c76fa45 | /usr/share/foreman/app/models/concerns/orchestration.rb:144:in `process'
0c76fa45 | /usr/share/foreman/app/models/concerns/orchestration.rb:44:in `around_save_orchestration'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/callbacks.rb:139:in `run_callbacks'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/callbacks.rb:825:in `_run_save_callbacks'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/callbacks.rb:327:in `create_or_update'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/timestamp.rb:128:in `create_or_update'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/persistence.rb:470:in `save'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/validations.rb:47:in `save'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/transactions.rb:314:in `block in save'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/transactions.rb:375:in `block in with_transaction_returning_status'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/connection_adapters/abstract/database_statements.rb:280:in `block in transaction'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/connection_adapters/abstract/transaction.rb:280:in `block in within_new_transaction'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/concurrency/load_interlock_aware_monitor.rb:26:in `block (2 levels) in synchronize'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/connection_adapters/abstract/transaction.rb:278:in `within_new_transaction'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/connection_adapters/abstract/database_statements.rb:280:in `transaction'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/transactions.rb:212:in `transaction'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/transactions.rb:366:in `with_transaction_returning_status'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/transactions.rb:314:in `save'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/suppressor.rb:44:in `save'
0c76fa45 | /usr/share/foreman/app/models/concerns/foreman/sti.rb:26:in `save'
0c76fa45 | /usr/share/foreman/app/controllers/hosts_controller.rb:98:in `create'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_controller/metal/basic_implicit_render.rb:6:in `send_action'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/abstract_controller/base.rb:195:in `process_action'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_controller/metal/rendering.rb:30:in `process_action'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/abstract_controller/callbacks.rb:42:in `block in process_action'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
0c76fa45 | /usr/share/foreman/app/controllers/concerns/foreman/controller/timezone.rb:10:in `set_timezone'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
0c76fa45 | /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:32:in `clear_thread'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
0c76fa45 | /usr/share/foreman/app/controllers/concerns/foreman/controller/topbar_sweeper.rb:12:in `set_topbar_sweeper_controller'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/audited-4.9.0/lib/audited/sweeper.rb:14:in `around'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/audited-4.9.0/lib/audited/sweeper.rb:14:in `around'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/callbacks.rb:139:in `run_callbacks'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/abstract_controller/callbacks.rb:41:in `process_action'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_controller/metal/rescue.rb:22:in `process_action'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_controller/metal/instrumentation.rb:33:in `block in process_action'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/notifications.rb:180:in `block in instrument'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/notifications.rb:180:in `instrument'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_controller/metal/instrumentation.rb:32:in `process_action'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_controller/metal/params_wrapper.rb:245:in `process_action'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/railties/controller_runtime.rb:27:in `process_action'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/abstract_controller/base.rb:136:in `process'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionview-6.0.3.4/lib/action_view/rendering.rb:39:in `process'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_controller/metal.rb:190:in `dispatch'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_controller/metal.rb:254:in `dispatch'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/routing/route_set.rb:50:in `dispatch'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/routing/route_set.rb:33:in `serve'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/journey/router.rb:49:in `block in serve'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/journey/router.rb:32:in `each'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/journey/router.rb:32:in `serve'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/routing/route_set.rb:834:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.0.1.1/lib/katello/middleware/event_daemon.rb:10:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/static.rb:126:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/static.rb:126:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-dsl-2.3.0/lib/apipie_dsl/static_dispatcher.rb:67:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.5.17/lib/apipie/static_dispatcher.rb:66:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.5.17/lib/apipie/extractor/recorder.rb:137:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/static.rb:126:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/static.rb:126:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/static.rb:126:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/static.rb:126:in `call'
0c76fa45 | /usr/share/foreman/lib/foreman/middleware/telemetry.rb:10:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.5.17/lib/apipie/middleware/checksum_in_headers.rb:27:in `call'
0c76fa45 | /usr/share/foreman/lib/foreman/middleware/catch_json_parse_errors.rb:9:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/tempfile_reaper.rb:15:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/etag.rb:27:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/conditional_get.rb:40:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/head.rb:12:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/http/content_security_policy.rb:18:in `call'
0c76fa45 | /usr/share/foreman/lib/foreman/middleware/logging_context_session.rb:22:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/session/abstract/id.rb:266:in `context'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/session/abstract/id.rb:260:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/cookies.rb:648:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/callbacks.rb:27:in `block in call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/callbacks.rb:101:in `run_callbacks'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/callbacks.rb:26:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/actionable_exceptions.rb:18:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/debug_exceptions.rb:32:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.4/lib/rails/rack/logger.rb:37:in `call_app'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.4/lib/rails/rack/logger.rb:28:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/sprockets-rails-3.2.1/lib/sprockets/rails/quiet_assets.rb:13:in `call'
0c76fa45 | /usr/share/foreman/lib/foreman/middleware/logging_context_request.rb:11:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/request_id.rb:27:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.0.1.1/lib/katello/prevent_json_parsing.rb:12:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/method_override.rb:24:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/runtime.rb:22:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/cache/strategy/local_cache_middleware.rb:29:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/executor.rb:14:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/static.rb:126:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/sendfile.rb:110:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/host_authorization.rb:76:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/secure_headers-6.3.0/lib/secure_headers/middleware.rb:11:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.4/lib/rails/engine.rb:527:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.4/lib/rails/railtie.rb:190:in `public_send'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.4/lib/rails/railtie.rb:190:in `method_missing'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/urlmap.rb:74:in `block in call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/urlmap.rb:58:in `each'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/urlmap.rb:58:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/puma-5.1.1/lib/puma/configuration.rb:246:in `call'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/puma-5.1.1/lib/puma/request.rb:76:in `block in handle_request'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/puma-5.1.1/lib/puma/thread_pool.rb:337:in `with_force_shutdown'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/puma-5.1.1/lib/puma/request.rb:75:in `handle_request'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/puma-5.1.1/lib/puma/server.rb:431:in `process_client'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/puma-5.1.1/lib/puma/thread_pool.rb:145:in `block in spawn_thread'
0c76fa45 | /opt/theforeman/tfm/root/usr/share/gems/gems/logging-2.3.0/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'
2021-05-27T23:49:47 [W|app|0c76fa45] Rolling back due to a problem: [#<Orchestration::Task:0x000000001ce99e10 @name="Generating ISO image for [redacted]", @id="Generating ISO image for [redacted]", @status="failed", @priority=5, @action=[#<Host::Managed id: nil, name: "[redacted]", last_compile: nil, last_report: nil, updated_at: nil, created_at: nil, root_pass: [FILTERED], architecture_id: 1, operatingsystem_id: 3, environment_id: 1, ptable_id: 123, medium_id: 13, build: true, comment: "", disk: "", installed_at: nil, model_id: nil, hostgroup_id: 1, owner_id: 6, owner_type: "User", enabled: true, puppet_ca_proxy_id: 1, managed: true, use_image: nil, image_file: nil, uuid: "50286201-9959-bffb-4e26-cacf6af56169", compute_resource_id: 1, puppet_proxy_id: 1, certname: nil, image_id: nil, organization_id: 1, location_id: 2, type: "Host::Managed", otp: nil, realm_id: nil, compute_profile_id: 1, provision_method: "bootdisk", grub_pass: "[redacted]", global_status: 0, lookup_value_matcher: [FILTERED], pxe_loader: "iPXE Chain BIOS", initiated_at: nil, build_errors: nil>, :setGenerateIsoImage], @created=1622148585.6266103, @timestamp=2021-05-27 20:49:47 UTC>]
2021-05-27T23:49:47 [I|app|0c76fa45] Removing Compute instance for [redacted]
2021-05-27T23:49:48 [I|app|0c76fa45] Processed 3 tasks from queue 'Host::Managed Main', completed 0/5
2021-05-27T23:49:48 [E|app|0c76fa45] Task 'Set up compute instance [redacted]' *rollbacked*
2021-05-27T23:49:48 [E|app|0c76fa45] Task 'Query instance details for [redacted]' *rollbacked*
2021-05-27T23:49:48 [E|app|0c76fa45] Task 'Generating ISO image for [redacted]' *failed*
2021-05-27T23:49:48 [E|app|0c76fa45] Task 'Upload ISO image to datastore for [redacted]' *canceled*
2021-05-27T23:49:48 [E|app|0c76fa45] Task 'Attach ISO image to CDROM drive for [redacted]' *canceled*
2021-05-27T23:49:48 [E|app|0c76fa45] Failed to save: Failed to generate ISO image for instance [redacted]: end of file reached