Hi there !
It’s me, and I think I found a bug
Problem:
When AWX (Ansible Tower) sync its inventory or when manually generating an “Ansible Inventory” Report template, with facts, if any of those facts contains a non ascii character and you try to export the template in Json, you’ll receive an error similar to :
(Encoding::UndefinedConversionError): “\xC3” from ASCII-8BIT to UTF-8
In my case, I had the following in one of my host :
ec2_userdata :
"then rm /bootstrap.donefi# Récupération du répertoire courant et du répertoire "
Which caused the problem.
Exporting in anything else but Json works without problem.
Expected outcome:
No error when generating the report
Foreman and Proxy versions:
1.24.1
Foreman and Proxy plugin versions:
1.24.1
Fix:
In my case, removing the user-data template from my EC2 host and running the following command to clean it up solved the problem :
# On Foreman:
echo "Host.find_by_name('my-ec2-host.fqdn').fact_values.destroy_all" | foreman-rake console
# On the host:
puppet agent -tv
You can see below, the debug log :
Other relevant data:
2020-01-24T14:16:37 [I|app|8146e58c] Started GET "/api/v2/status?per_page=250&page=1" for 172.31.11.118 at 2020-01-24 14:16:37 +0100
2020-01-24T14:16:37 [I|app|8146e58c] Processing by Api::V2::HomeController#status as JSON
2020-01-24T14:16:37 [I|app|8146e58c] Parameters: {"per_page"=>"250", "page"=>"1", "apiv"=>"v2"}
2020-01-24T14:16:37 [D|app|8146e58c] Authenticated user ansible-inventory against INTERNAL authentication source
2020-01-24T14:16:37 [I|app|8146e58c] Authorized user ansible-inventory(ansible-inventory)
2020-01-24T14:16:37 [D|tax|8146e58c] Current location set to none
2020-01-24T14:16:37 [D|tax|8146e58c] Current organization set to none
2020-01-24T14:16:37 [D|tax|8146e58c] Current location set to none
2020-01-24T14:16:37 [D|tax|8146e58c] Current organization set to none
2020-01-24T14:16:37 [I|app|8146e58c] Rendering api/v2/home/status.json.rabl
2020-01-24T14:16:37 [I|app|8146e58c] Rendered api/v2/home/status.json.rabl (0.9ms)
2020-01-24T14:16:37 [D|app|8146e58c] Body: {"result":"ok","status":200,"version":"1.24.1","api_version":2}
2020-01-24T14:16:37 [I|app|8146e58c] Completed 200 OK in 99ms (Views: 2.0ms | ActiveRecord: 6.8ms)
2020-01-24T14:16:37 [I|app|82f338ea] Started POST "/ansible/api/v2/ansible_inventories/schedule" for 172.31.11.118 at 2020-01-24 14:16:37 +0100
2020-01-24T14:16:37 [I|app|82f338ea] Processing by Api::V2::AnsibleInventoriesController#schedule as */*
2020-01-24T14:16:37 [I|app|82f338ea] Parameters: {"input_values"=>"[FILTERED]", "apiv"=>"v2", "ansible_inventory"=>{"input_values"=>"[FILTERED]"}}
2020-01-24T14:16:37 [D|tax|82f338ea] Current location set to none
2020-01-24T14:16:37 [D|tax|82f338ea] Current organization set to none
2020-01-24T14:16:37 [I|app|82f338ea] Enqueued TemplateRenderJob (Job ID: 04ce4e9c-0e75-440b-b2b9-00aa209c02f8) to Dynflow(default) with arguments: {"template_id"=>161, "gzip"=>false, "send_mail"=>false, "mail_to"=>nil, "format"=>"json", "input_values"=>{"74"=>{"value"=>"yes"}, "75"=>{"value"=>"yes"}, "76"=>{"value"=>"yes"}, "77"=>{"value"=>"no"}, "78"=>{"value"=>"no"}, "79"=>{"value"=>"yes"}, "80"=>{"value"=>"no"}, "81"=>{"value"=>"yes"}, "82"=>{"value"=>"no"}, "83"=>{"value"=>"yes"}, "84"=>{"value"=>"no"}, "85"=>{"value"=>"yes"}}}, {:user_id=>13}
2020-01-24T14:16:37 [D|app|82f338ea] Body: {"job_id":"04ce4e9c-0e75-440b-b2b9-00aa209c02f8","data_url":"/api/v2/report_templates/161-Ansible%20Inventory/report_data/04ce4e9c-0e75-440b-b2b9-00aa209c02f8"}
2020-01-24T14:16:37 [I|app|82f338ea] Completed 200 OK in 227ms (Views: 0.2ms | ActiveRecord: 56.9ms)
2020-01-24T14:16:38 [I|app|642fe66d] Started GET "/api/v2/report_templates/161-Ansible%20Inventory/report_data/04ce4e9c-0e75-440b-b2b9-00aa209c02f8" for 172.31.11.118 at 2020-01-24 14:16:38 +0100
2020-01-24T14:16:38 [I|app|642fe66d] Processing by Api::V2::ReportTemplatesController#report_data as JSON
2020-01-24T14:16:38 [I|app|642fe66d] Parameters: {"apiv"=>"v2", "id"=>"161-Ansible Inventory", "job_id"=>"04ce4e9c-0e75-440b-b2b9-00aa209c02f8"}
2020-01-24T14:16:38 [D|tax|642fe66d] Current location set to none
2020-01-24T14:16:38 [D|tax|642fe66d] Current organization set to none
2020-01-24T14:16:38 [D|app|642fe66d] Body:
2020-01-24T14:16:38 [I|app|642fe66d] Completed 204 No Content in 43ms (ActiveRecord: 1.7ms)
2020-01-24T14:16:48 [I|app|8faa608b] Started GET "/api/v2/report_templates/161-Ansible%20Inventory/report_data/04ce4e9c-0e75-440b-b2b9-00aa209c02f8" for 172.31.11.118 at 2020-01-24 14:16:48 +0100
2020-01-24T14:16:48 [I|app|8faa608b] Processing by Api::V2::ReportTemplatesController#report_data as JSON
2020-01-24T14:16:48 [I|app|8faa608b] Parameters: {"apiv"=>"v2", "id"=>"161-Ansible Inventory", "job_id"=>"04ce4e9c-0e75-440b-b2b9-00aa209c02f8"}
2020-01-24T14:16:48 [D|tax|8faa608b] Current location set to none
2020-01-24T14:16:48 [D|tax|8faa608b] Current organization set to none
2020-01-24T14:16:48 [D|app|8faa608b] Body:
2020-01-24T14:16:48 [I|app|8faa608b] Completed 204 No Content in 10ms (ActiveRecord: 1.3ms)
2020-01-24T14:16:58 [I|app|1e34f688] Started GET "/api/v2/report_templates/161-Ansible%20Inventory/report_data/04ce4e9c-0e75-440b-b2b9-00aa209c02f8" for 172.31.11.118 at 2020-01-24 14:16:58 +0100
2020-01-24T14:16:58 [I|app|1e34f688] Processing by Api::V2::ReportTemplatesController#report_data as JSON
2020-01-24T14:16:58 [I|app|1e34f688] Parameters: {"apiv"=>"v2", "id"=>"161-Ansible Inventory", "job_id"=>"04ce4e9c-0e75-440b-b2b9-00aa209c02f8"}
2020-01-24T14:16:58 [D|tax|1e34f688] Current location set to none
2020-01-24T14:16:58 [D|tax|1e34f688] Current organization set to none
2020-01-24T14:16:58 [W|app|1e34f688] Action failed
2020-01-24T14:16:58 [D|app|1e34f688] Backtrace for 'Action failed' error (Encoding::UndefinedConversionError): "\xC3" from ASCII-8BIT to UTF-8
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/core_ext/object/json.rb:38:in `encode'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/core_ext/object/json.rb:38:in `to_json'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/core_ext/object/json.rb:38:in `to_json'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/json/encoding.rb:57:in `to_json'
/opt/rh/rh-ruby25/root/usr/share/ruby/json/common.rb:224:in `generate'
/opt/rh/rh-ruby25/root/usr/share/ruby/json/common.rb:224:in `generate'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/json/encoding.rb:102:in `stringify'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/json/encoding.rb:35:in `encode'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/json/encoding.rb:22:in `encode'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/core_ext/object/json.rb:41:in `to_json'
/usr/share/foreman/app/controllers/api/v2/report_templates_controller.rb:185:in `report_data'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_controller/metal/basic_implicit_render.rb:6:in `send_action'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/abstract_controller/base.rb:194:in `process_action'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_controller/metal/rendering.rb:30:in `process_action'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/abstract_controller/callbacks.rb:42:in `block in process_action'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:109:in `block in run_callbacks'
/usr/share/foreman/app/controllers/api/v2/base_controller.rb:163:in `disable_json_root'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/usr/share/foreman/app/controllers/concerns/foreman/controller/timezone.rb:10:in `set_timezone'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/usr/share/foreman/app/models/concerns/foreman/thread_session.rb:32:in `clear_thread'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/usr/share/foreman/app/controllers/concerns/foreman/controller/topbar_sweeper.rb:12:in `set_topbar_sweeper_controller'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/opt/theforeman/tfm/root/usr/share/gems/gems/audited-4.9.0/lib/audited/sweeper.rb:14:in `around'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/opt/theforeman/tfm/root/usr/share/gems/gems/audited-4.9.0/lib/audited/sweeper.rb:14:in `around'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:136:in `run_callbacks'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/abstract_controller/callbacks.rb:41:in `process_action'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_controller/metal/rescue.rb:22:in `process_action'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_controller/metal/instrumentation.rb:34:in `block in process_action'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/notifications.rb:168:in `block in instrument'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/notifications/instrumenter.rb:23:in `instrument'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/notifications.rb:168:in `instrument'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_controller/metal/instrumentation.rb:32:in `process_action'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_controller/metal/params_wrapper.rb:256:in `process_action'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/railties/controller_runtime.rb:24:in `process_action'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/abstract_controller/base.rb:134:in `process'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionview-5.2.1/lib/action_view/rendering.rb:32:in `process'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_controller/metal.rb:191:in `dispatch'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_controller/metal.rb:252:in `dispatch'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/routing/route_set.rb:52:in `dispatch'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/routing/route_set.rb:34:in `serve'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/routing/mapper.rb:18:in `block in <class:Constraints>'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/routing/mapper.rb:48:in `serve'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/journey/router.rb:52:in `block in serve'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/journey/router.rb:35:in `each'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/journey/router.rb:35:in `serve'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/routing/route_set.rb:840:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.14.1/lib/katello/middleware/event_daemon.rb:10:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.5.14/lib/apipie/static_dispatcher.rb:65:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.5.14/lib/apipie/extractor/recorder.rb:137:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call'
/usr/share/foreman/lib/foreman/middleware/telemetry.rb:10:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.5.14/lib/apipie/middleware/checksum_in_headers.rb:27:in `call'
/usr/share/foreman/lib/foreman/middleware/catch_json_parse_errors.rb:9:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/tempfile_reaper.rb:15:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/etag.rb:25:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/conditional_get.rb:25:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/head.rb:12:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/http/content_security_policy.rb:18:in `call'
/usr/share/foreman/lib/foreman/middleware/logging_context_session.rb:22:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/session/abstract/id.rb:232:in `context'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/session/abstract/id.rb:226:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/cookies.rb:670:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/callbacks.rb:28:in `block in call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:98:in `run_callbacks'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/callbacks.rb:26:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/debug_exceptions.rb:61:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/railties-5.2.1/lib/rails/rack/logger.rb:38:in `call_app'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/railties-5.2.1/lib/rails/rack/logger.rb:28:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/sprockets-rails-3.2.1/lib/sprockets/rails/quiet_assets.rb:13:in `call'
/usr/share/foreman/lib/foreman/middleware/logging_context_request.rb:11:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/request_id.rb:27:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.14.1/lib/katello/prevent_json_parsing.rb:12:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/method_override.rb:22:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/runtime.rb:22:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/cache/strategy/local_cache_middleware.rb:29:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/executor.rb:14:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/sendfile.rb:111:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/secure_headers-6.0.0/lib/secure_headers/middleware.rb:13:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/railties-5.2.1/lib/rails/engine.rb:524:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/railties-5.2.1/lib/rails/railtie.rb:190:in `public_send'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/railties-5.2.1/lib/rails/railtie.rb:190:in `method_missing'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/urlmap.rb:68:in `block in call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/urlmap.rb:53:in `each'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/rack-2.0.6/lib/rack/urlmap.rb:53:in `call'
/usr/share/passenger/phusion_passenger/rack/thread_handler_extension.rb:74:in `process_request'
/usr/share/passenger/phusion_passenger/request_handler/thread_handler.rb:141:in `accept_and_process_next_request'
/usr/share/passenger/phusion_passenger/request_handler/thread_handler.rb:109:in `main_loop'
/usr/share/passenger/phusion_passenger/request_handler.rb:455:in `block (3 levels) in start_threads'
/opt/theforeman/tfm/root/usr/share/gems/gems/logging-2.2.2/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'
2020-01-24T14:16:58 [I|app|1e34f688] Rendering api/v2/errors/standard_error.json.rabl within api/v2/layouts/error_layout
2020-01-24T14:16:58 [I|app|1e34f688] Rendered api/v2/errors/standard_error.json.rabl within api/v2/layouts/error_layout (0.8ms)
2020-01-24T14:16:58 [I|app|1e34f688] Completed 500 Internal Server Error in 87ms (Views: 3.4ms | ActiveRecord: 43.3ms)
2020-01-24T14:16:58 [I|app|0faab079] Started GET "/api/v2/hosts?per_page=250&page=1" for 172.31.11.118 at 2020-01-24 14:16:58 +0100
2020-01-24T14:16:58 [I|app|0faab079] Processing by Api::V2::HostsController#index as JSON
2020-01-24T14:16:58 [I|app|0faab079] Parameters: {"per_page"=>"250", "page"=>"1", "apiv"=>"v2"}
2020-01-24T14:16:58 [D|tax|0faab079] Current location set to none
2020-01-24T14:16:58 [D|tax|0faab079] Current organization set to none
2020-01-24T14:16:59 [I|app|0faab079] Rendering api/v2/hosts/index.json.rabl within api/v2/layouts/index_layout
2020-01-24T14:17:04 [I|app|0faab079] Rendered api/v2/hosts/index.json.rabl within api/v2/layouts/index_layout (5250.6ms)
2020-01-24T14:17:04 [D|app|0faab079] Body: {
, I should have reported the issue in it myself with the forum post.