Foreman Discovery Validation failed: Conflict DHCP records Action failed

Problem:
We test the Foreman Discovery in one Foreman instance (1.18.3) with foreman_discovery (12.0.2).
If the DHCP records existed, the Web interface ask “if you are certain that they should be removed, please click on overwrite”. and I can provision the host with overwrite.

But we have the other Foreman instance (1.24.2) with foreman-discovery (16.0.1-1)
We try to use the Foreman Discovery, and it run fails without asking the overwrite option. The error.
Oops, we’re sorry but something went wrong Validation failed: Conflict DHCP records -00:00:00:00:00:00/1.1.1.1 already exists

ActiveRecord::RecordInvalid
Validation failed: Conflict DHCP records -00:00:00:00:00:00/1.1.1.1 already exists
/usr/share/foreman/vendor/ruby/2.3.0/gems/activerecord-5.2.1/lib/active_record/validations.rb:80:in `raise_validation_error'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activerecord-5.2.1/lib/active_record/validations.rb:52:in `save!'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activerecord-5.2.1/lib/active_record/transactions.rb:315:in `block in save!'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activerecord-5.2.1/lib/active_record/transactions.rb:387:in `block in with_transaction_returning_status'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:254:in `block in transaction'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/transaction.rb:239:in `block in within_new_transaction'
/usr/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/transaction.rb:236:in `within_new_transaction'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:254:in `transaction'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activerecord-5.2.1/lib/active_record/transactions.rb:212:in `transaction'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activerecord-5.2.1/lib/active_record/transactions.rb:385:in `with_transaction_returning_status'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activerecord-5.2.1/lib/active_record/transactions.rb:315:in `save!'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activerecord-5.2.1/lib/active_record/suppressor.rb:48:in `save!'
/usr/share/foreman/vendor/ruby/2.3.0/gems/foreman_discovery-16.0.1/app/services/foreman_discovery/host_converter.rb:54:in `block in unused_ip_for_host'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activerecord-5.2.1/lib/active_record/relation/delegation.rb:41:in `each'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activerecord-5.2.1/lib/active_record/relation/delegation.rb:41:in `each'
/usr/share/foreman/vendor/ruby/2.3.0/gems/foreman_discovery-16.0.1/app/services/foreman_discovery/host_converter.rb:47:in `unused_ip_for_host'
/usr/share/foreman/vendor/ruby/2.3.0/gems/foreman_discovery-16.0.1/app/controllers/discovered_hosts_controller.rb:81:in `block in perform_update'
/usr/share/foreman/app/models/taxonomy.rb:61:in `block in no_taxonomy_scope'
/usr/share/foreman/app/models/taxonomy.rb:68:in `block (2 levels) in as_taxonomy'
/usr/share/foreman/app/models/concerns/foreman/thread_session.rb:195:in `as_location'
/usr/share/foreman/app/models/taxonomy.rb:67:in `block in as_taxonomy'
/usr/share/foreman/app/models/concerns/foreman/thread_session.rb:154:in `as_org'
/usr/share/foreman/app/models/taxonomy.rb:66:in `as_taxonomy'
/usr/share/foreman/app/models/taxonomy.rb:60:in `no_taxonomy_scope'
/usr/share/foreman/vendor/ruby/2.3.0/gems/foreman_discovery-16.0.1/app/controllers/discovered_hosts_controller.rb:79:in `perform_update'
/usr/share/foreman/vendor/ruby/2.3.0/gems/foreman_discovery-16.0.1/app/controllers/discovered_hosts_controller.rb:75:in `update'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_controller/metal/basic_implicit_render.rb:6:in `send_action'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/abstract_controller/base.rb:194:in `process_action'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_controller/metal/rendering.rb:30:in `process_action'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/abstract_controller/callbacks.rb:42:in `block in process_action'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:109:in `block in run_callbacks'
/usr/share/foreman/app/controllers/concerns/foreman/controller/timezone.rb:10:in `set_timezone'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/usr/share/foreman/app/models/concerns/foreman/thread_session.rb:32:in `clear_thread'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/usr/share/foreman/app/controllers/concerns/foreman/controller/topbar_sweeper.rb:12:in `set_topbar_sweeper_controller'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/usr/share/foreman/vendor/ruby/2.3.0/gems/audited-4.9.0/lib/audited/sweeper.rb:14:in `around'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/usr/share/foreman/vendor/ruby/2.3.0/gems/audited-4.9.0/lib/audited/sweeper.rb:14:in `around'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:136:in `run_callbacks'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/abstract_controller/callbacks.rb:41:in `process_action'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_controller/metal/rescue.rb:22:in `process_action'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_controller/metal/instrumentation.rb:34:in `block in process_action'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.2.1/lib/active_support/notifications.rb:168:in `block in instrument'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.2.1/lib/active_support/notifications/instrumenter.rb:23:in `instrument'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.2.1/lib/active_support/notifications.rb:168:in `instrument'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_controller/metal/instrumentation.rb:32:in `process_action'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_controller/metal/params_wrapper.rb:256:in `process_action'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activerecord-5.2.1/lib/active_record/railties/controller_runtime.rb:24:in `process_action'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/abstract_controller/base.rb:134:in `process'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionview-5.2.1/lib/action_view/rendering.rb:32:in `process'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_controller/metal.rb:191:in `dispatch'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_controller/metal.rb:252:in `dispatch'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/routing/route_set.rb:52:in `dispatch'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/routing/route_set.rb:34:in `serve'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/journey/router.rb:52:in `block in serve'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/journey/router.rb:35:in `each'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/journey/router.rb:35:in `serve'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/routing/route_set.rb:840:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/apipie-rails-0.5.17/lib/apipie/static_dispatcher.rb:66:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/apipie-rails-0.5.17/lib/apipie/extractor/recorder.rb:137:in `call'
/usr/share/foreman/lib/foreman/middleware/telemetry.rb:10:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/apipie-rails-0.5.17/lib/apipie/middleware/checksum_in_headers.rb:27:in `call'
/usr/share/foreman/lib/foreman/middleware/catch_json_parse_errors.rb:9:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.1.1/lib/rack/tempfile_reaper.rb:17:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.1.1/lib/rack/etag.rb:27:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.1.1/lib/rack/conditional_get.rb:40:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.1.1/lib/rack/head.rb:14:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/http/content_security_policy.rb:18:in `call'
/usr/share/foreman/lib/foreman/middleware/logging_context_session.rb:22:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.1.1/lib/rack/session/abstract/id.rb:277:in `context'
/usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.1.1/lib/rack/session/abstract/id.rb:271:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/middleware/cookies.rb:670:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/middleware/callbacks.rb:28:in `block in call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:98:in `run_callbacks'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/middleware/callbacks.rb:26:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/middleware/debug_exceptions.rb:61:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/railties-5.2.1/lib/rails/rack/logger.rb:38:in `call_app'
/usr/share/foreman/vendor/ruby/2.3.0/gems/railties-5.2.1/lib/rails/rack/logger.rb:28:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/sprockets-rails-3.2.1/lib/sprockets/rails/quiet_assets.rb:13:in `call'
/usr/share/foreman/lib/foreman/middleware/logging_context_request.rb:11:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/middleware/request_id.rb:27:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.1.1/lib/rack/method_override.rb:24:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.1.1/lib/rack/runtime.rb:24:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/activesupport-5.2.1/lib/active_support/cache/strategy/local_cache_middleware.rb:29:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/middleware/executor.rb:14:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/actionpack-5.2.1/lib/action_dispatch/middleware/static.rb:127:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.1.1/lib/rack/sendfile.rb:113:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/secure_headers-6.2.0/lib/secure_headers/middleware.rb:11:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/railties-5.2.1/lib/rails/engine.rb:524:in `call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/railties-5.2.1/lib/rails/railtie.rb:190:in `public_send'
/usr/share/foreman/vendor/ruby/2.3.0/gems/railties-5.2.1/lib/rails/railtie.rb:190:in `method_missing'
/usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.1.1/lib/rack/urlmap.rb:77:in `block in call'
/usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.1.1/lib/rack/urlmap.rb:61:in `each'
/usr/share/foreman/vendor/ruby/2.3.0/gems/rack-2.1.1/lib/rack/urlmap.rb:61:in `call'
/usr/lib/ruby/vendor_ruby/phusion_passenger/rack/thread_handler_extension.rb:97:in `process_request'
/usr/lib/ruby/vendor_ruby/phusion_passenger/request_handler/thread_handler.rb:160:in `accept_and_process_next_request'
/usr/lib/ruby/vendor_ruby/phusion_passenger/request_handler/thread_handler.rb:113:in `main_loop'
/usr/lib/ruby/vendor_ruby/phusion_passenger/request_handler.rb:416:in `block (3 levels) in start_threads'
/usr/lib/ruby/vendor_ruby/phusion_passenger/utils.rb:113:in `block in create_thread_and_abort_on_exception'
/usr/share/foreman/vendor/ruby/2.3.0/gems/logging-2.2.2/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'

And I check the production.log

2020-07-21T06:40:26 [I|app|74951d82] Started PATCH "/discovered_hosts/140" for 10.10.10.10 at 2020-07-21 06:40:26 +0100
2020-07-21T06:40:26 [I|app|74951d82] Processing by DiscoveredHostsController#update as */*
2020-07-21T06:40:26 [I|app|74951d82] Parameters: {"utf8"=>"✓", "authenticity_token"=>"8Rs1m+4krAMnYEsjMITca+T73mOw6fd+tFK5NQXCgYaxpTV3m8E9othP+Qg7+m+J1BAISv/y3i51k4UMtXPx3g==", "host"=>{"name"=>"fdi-test", "organization_id"=>"3", "location_id"=>"2", "hostgroup_id"=>"2", "puppetclass_ids"=>[""], "managed"=>"true", "progress_report_id"=>"[FILTERED]", "type"=>"Host::Managed", "interfaces_attributes"=>{"0"=>{"_destroy"=>"0", "mac"=>"00:50:56:a8:9b:dd", "identifier"=>"ens160", "name"=>"cl-fdi-test", "domain_id"=>"3", "subnet_id"=>"4", "ip"=>"172.19.16.184", "ip6"=>"", "managed"=>"1", "primary"=>"1", "provision"=>"1", "tag"=>"", "attached_to"=>"", "id"=>"165"}}, "architecture_id"=>"1", "operatingsystem_id"=>"14", "build"=>"1", "medium_id"=>"10", "ptable_id"=>"143", "pxe_loader"=>"PXELinux BIOS", "disk"=>"", "is_owned_by"=>"", "enabled"=>"1", "model_id"=>"1", "comment"=>"", "overwrite"=>"false"}, "id"=>"140"}
2020-07-21T06:40:27 [W|app|74951d82] DHCP records -00:00:00:00:00:00/1.1.1.1 already exists
2020-07-21T06:40:27 [W|app|74951d82] Not queueing Nic::Managed: ["Conflict DHCP records -00:00:00:00:00:00/1.1.1.1 already exists"] and ["Conflict DHCP records -00:00:00:00:00:00/1.1.1.1 already exists"]
2020-07-21T06:40:27 [W|app|74951d82] Not queueing Nic::Managed: ["Conflict DHCP records -00:00:00:00:00:00/1.1.1.1 already exists"] and ["Conflict DHCP records -00:00:00:00:00:00/1.1.1.1 already exists"]
2020-07-21T06:40:27 [W|app|74951d82] Not queueing Nic::Managed: ["Conflict DHCP records -00:00:00:00:00:00/1.1.1.1 already exists"] and ["Conflict DHCP records -00:00:00:00:00:00/1.1.1.1 already exists"]
2020-07-21T06:40:27 [W|app|74951d82] Action failed
2020-07-21T06:40:27 [I|app|74951d82] Rendering common/500.html.erb
2020-07-21T06:40:27 [I|app|74951d82] Rendered common/500.html.erb (2.0ms)
2020-07-21T06:40:27 [I|app|74951d82] Completed 500 Internal Server Error in 908ms (Views: 6.3ms | ActiveRecord: 17.6ms)

Any suggestion? Thanks.

Expected outcome:
Ask the overwrite for the conflict DHCP records.

Foreman and Proxy versions:
Foreman 1.24.2

Foreman and Proxy plugin versions:
foreman-discovery 16.0.1-1

Distribution and version:

Other relevant data:

Well 00:00:00:00:00:00/1.1.1.1 does not look like a valid MAC and IP address. When you create a reservation, it will pass once. But any other host with this MAC or IP will fail. In other words, research why the host has these (incorrect) addresses.

Hi Lukas,

Thanks for your suggestion. It’s not the real MAC address / IP address.
We have the custom DNS/DHCP Smart Proxy integrated with the BlueCat. The smart proxy is working fine with Foreman Discovery in the production Foreman instance (1.18.3) and we will upgrade the Foreman from 1.18.3 to 1.24.2. In the testing Foreman instance (1.24.2) , it run fails without asking the overwrite option. I’m not sure maybe it’s related with out custom DNS/DHCP Smart Proxy. Any suggestion to check this issue? Thanks.

Best Regards,
Chun-Liang Lin

Please always state when you have a custom plugin.

We don’t see this behavior on ISC DHCP/DNS. It’s likely an issue with the plugin. There were some changes in the DHCP and DNS modules few releases back in regard to conflict resolution IIRC. Maybe that broke the plugin.

Hi Lukas,

Thanks for your suggestion and support.
It’s related with this foreman discovery change, https://github.com/theforeman/foreman_discovery/pull/306/files.The custom plugin has some problem in unused_ip, so we don’t have this problem before foreman_discovery 16.0.1.

Best Regards,
Chun-Liang Lin

I see, well we want to keep the new behavior, it was a long-standing bug and now it’s been fixed. Let me know if I can do something for you.