Finished POST /dhcp/10.2.1.0 with 400

Problem:

Hunting an issue when creating a host, so trying to dumb down the deployment to see if this is the only issue I have and ran into a problem with DHCP:

Finished POST /dhcp/10.2.1.0 with 400

Expected outcome:

Foreman creates a DHCP lease for a new host.

Foreman and Proxy versions:

Foreman & Smart Proxy v3.9.1

Foreman and Proxy plugin versions:

Name Version
foreman-tasks 9.0.1
foreman_fog_proxmox 0.15.0
foreman_puppet 6.1.1
foreman_remote_execution 12.0.5
katello 4.11.0
smart proxy plugin - DHCP 3.9.1
smart proxy plugin - DNS 3.9.1
smart proxy plugin - Dynflow 0.9.1
smart proxy plugin - External IPAM 0.1.4
smart proxy plugin - Content 3.2.0
smart proxy plugin - Realm 3.9.1
smart proxy plugin - Script 0.10.3
smart proxy plugin - TFTP 3.9.1

Distribution and version:

CentOS 8 stream

Other relevant data:
proxy.log:

2024-02-15T00:10:20  [I] Execution plan cleaner removing 0 execution plans.
2024-02-15T00:10:37 19e7c07a [I] Started GET /tftp/serverName
2024-02-15T00:10:37 19e7c07a [I] Finished GET /tftp/serverName with 200 (0.71 ms)
2024-02-15T00:10:37 19e7c07a [I] Started GET /tftp/serverName
2024-02-15T00:10:37 19e7c07a [I] Finished GET /tftp/serverName with 200 (0.61 ms)
2024-02-15T00:10:37 19e7c07a [I] Started GET /tftp/serverName
2024-02-15T00:10:37 19e7c07a [I] Finished GET /tftp/serverName with 200 (0.65 ms)
2024-02-15T00:10:37 19e7c07a [I] Started GET /tftp/serverName
2024-02-15T00:10:37 19e7c07a [I] Finished GET /tftp/serverName with 200 (0.46 ms)
2024-02-15T00:10:37 19e7c07a [I] Started POST /dhcp/10.2.1.0
2024-02-15T00:10:37 19e7c07a [E] Unable to parse postgres subnets: received invalid response to GSSAPI negotiation: q

2024-02-15T00:10:37 19e7c07a [E] Unable to parse postgres subnets: received invalid response to GSSAPI negotiation: q

2024-02-15T00:10:37 19e7c07a [W] Error details for Unable to parse postgres subnets: received invalid response to GSSAPI negotiation: q
: <Proxy::DHCP::Error>: Unable to parse postgres subnets: received invalid response to GSSAPI negotiation: q

/usr/local/share/gems/gems/smart_proxy_dhcp_kea-0.1.2/lib/smart_proxy_dhcp_kea/subnet_service_initializer.rb:52:in `rescue in parse_config_for_subnets'
/usr/local/share/gems/gems/smart_proxy_dhcp_kea-0.1.2/lib/smart_proxy_dhcp_kea/subnet_service_initializer.rb:30:in `parse_config_for_subnets'
/usr/local/share/gems/gems/smart_proxy_dhcp_kea-0.1.2/lib/smart_proxy_dhcp_kea/subnet_service_initializer.rb:24:in `initialized_subnet_service'
/usr/local/share/gems/gems/smart_proxy_dhcp_kea-0.1.2/lib/smart_proxy_dhcp_kea/plugin_configuration.rb:34:in `block in load_dependency_injection_wirings'
/usr/share/foreman-proxy/lib/proxy/dependency_injection.rb:8:in `instance'
/usr/share/foreman-proxy/lib/proxy/dependency_injection.rb:57:in `get_dependency'
/usr/local/share/gems/gems/smart_proxy_dhcp_kea-0.1.2/lib/smart_proxy_dhcp_kea/plugin_configuration.rb:42:in `block in load_dependency_injection_wirings'
/usr/share/foreman-proxy/lib/proxy/dependency_injection.rb:8:in `instance'
/usr/share/foreman-proxy/lib/proxy/dependency_injection.rb:57:in `get_dependency'
/usr/share/foreman-proxy/lib/proxy/dependency_injection.rb:74:in `block in inject_attr'
/usr/share/foreman-proxy/modules/dhcp/dhcp_api.rb:78:in `block in <class:DhcpApi>'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1697:in `call'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1697:in `block in compile!'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1030:in `block (3 levels) in route!'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1049:in `route_eval'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1030:in `block (2 levels) in route!'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1078:in `block in process_route'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1076:in `catch'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1076:in `process_route'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1028:in `block in route!'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1025:in `each'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1025:in `route!'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1147:in `block in dispatch!'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1119:in `block in invoke'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1119:in `catch'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1119:in `invoke'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1142:in `dispatch!'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:956:in `block in call!'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1119:in `block in invoke'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1119:in `catch'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1119:in `invoke'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:956:in `call!'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:945:in `call'
/usr/share/gems/gems/rack-2.2.8/lib/rack/method_override.rb:24:in `call'
/usr/share/foreman-proxy/lib/proxy/log.rb:101:in `call'
/usr/share/foreman-proxy/lib/proxy/request_id_middleware.rb:11:in `call'
/usr/share/gems/gems/rack-protection-2.2.4/lib/rack/protection/xss_header.rb:18:in `call'
/usr/share/gems/gems/rack-protection-2.2.4/lib/rack/protection/path_traversal.rb:16:in `call'
/usr/share/gems/gems/rack-protection-2.2.4/lib/rack/protection/json_csrf.rb:26:in `call'
/usr/share/gems/gems/rack-protection-2.2.4/lib/rack/protection/base.rb:50:in `call'
/usr/share/gems/gems/rack-protection-2.2.4/lib/rack/protection/base.rb:50:in `call'
/usr/share/gems/gems/rack-protection-2.2.4/lib/rack/protection/frame_options.rb:31:in `call'
/usr/share/gems/gems/rack-2.2.8/lib/rack/null_logger.rb:11:in `call'
/usr/share/gems/gems/rack-2.2.8/lib/rack/head.rb:12:in `call'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/show_exceptions.rb:22:in `call'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:218:in `call'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:2004:in `call'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1564:in `block in call'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1780:in `synchronize'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1564:in `call'
/usr/share/gems/gems/rack-2.2.8/lib/rack/urlmap.rb:74:in `block in call'
/usr/share/gems/gems/rack-2.2.8/lib/rack/urlmap.rb:58:in `each'
/usr/share/gems/gems/rack-2.2.8/lib/rack/urlmap.rb:58:in `call'
/usr/share/gems/gems/rack-2.2.8/lib/rack/builder.rb:244:in `call'
/usr/share/gems/gems/rack-2.2.8/lib/rack/handler/webrick.rb:95:in `service'
/usr/share/gems/gems/webrick-1.8.1/lib/webrick/httpserver.rb:140:in `service'
/usr/share/gems/gems/webrick-1.8.1/lib/webrick/httpserver.rb:96:in `run'
/usr/share/gems/gems/webrick-1.8.1/lib/webrick/server.rb:310:in `block in start_thread'
/usr/share/gems/gems/logging-2.3.1/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'
2024-02-15T00:10:37 19e7c07a [W] Unable to parse postgres subnets: received invalid response to GSSAPI negotiation: q
: <Proxy::DHCP::Error>: Unable to parse postgres subnets: received invalid response to GSSAPI negotiation: q

/usr/local/share/gems/gems/smart_proxy_dhcp_kea-0.1.2/lib/smart_proxy_dhcp_kea/subnet_service_initializer.rb:52:in `rescue in parse_config_for_subnets'
/usr/local/share/gems/gems/smart_proxy_dhcp_kea-0.1.2/lib/smart_proxy_dhcp_kea/subnet_service_initializer.rb:30:in `parse_config_for_subnets'
/usr/local/share/gems/gems/smart_proxy_dhcp_kea-0.1.2/lib/smart_proxy_dhcp_kea/subnet_service_initializer.rb:24:in `initialized_subnet_service'
/usr/local/share/gems/gems/smart_proxy_dhcp_kea-0.1.2/lib/smart_proxy_dhcp_kea/plugin_configuration.rb:34:in `block in load_dependency_injection_wirings'
/usr/share/foreman-proxy/lib/proxy/dependency_injection.rb:8:in `instance'
/usr/share/foreman-proxy/lib/proxy/dependency_injection.rb:57:in `get_dependency'
/usr/local/share/gems/gems/smart_proxy_dhcp_kea-0.1.2/lib/smart_proxy_dhcp_kea/plugin_configuration.rb:42:in `block in load_dependency_injection_wirings'
/usr/share/foreman-proxy/lib/proxy/dependency_injection.rb:8:in `instance'
/usr/share/foreman-proxy/lib/proxy/dependency_injection.rb:57:in `get_dependency'
/usr/share/foreman-proxy/lib/proxy/dependency_injection.rb:74:in `block in inject_attr'
/usr/share/foreman-proxy/modules/dhcp/dhcp_api.rb:78:in `block in <class:DhcpApi>'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1697:in `call'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1697:in `block in compile!'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1030:in `block (3 levels) in route!'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1049:in `route_eval'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1030:in `block (2 levels) in route!'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1078:in `block in process_route'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1076:in `catch'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1076:in `process_route'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1028:in `block in route!'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1025:in `each'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1025:in `route!'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1147:in `block in dispatch!'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1119:in `block in invoke'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1119:in `catch'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1119:in `invoke'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1142:in `dispatch!'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:956:in `block in call!'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1119:in `block in invoke'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1119:in `catch'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1119:in `invoke'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:956:in `call!'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:945:in `call'
/usr/share/gems/gems/rack-2.2.8/lib/rack/method_override.rb:24:in `call'
/usr/share/foreman-proxy/lib/proxy/log.rb:101:in `call'
/usr/share/foreman-proxy/lib/proxy/request_id_middleware.rb:11:in `call'
/usr/share/gems/gems/rack-protection-2.2.4/lib/rack/protection/xss_header.rb:18:in `call'
/usr/share/gems/gems/rack-protection-2.2.4/lib/rack/protection/path_traversal.rb:16:in `call'
/usr/share/gems/gems/rack-protection-2.2.4/lib/rack/protection/json_csrf.rb:26:in `call'
/usr/share/gems/gems/rack-protection-2.2.4/lib/rack/protection/base.rb:50:in `call'
/usr/share/gems/gems/rack-protection-2.2.4/lib/rack/protection/base.rb:50:in `call'
/usr/share/gems/gems/rack-protection-2.2.4/lib/rack/protection/frame_options.rb:31:in `call'
/usr/share/gems/gems/rack-2.2.8/lib/rack/null_logger.rb:11:in `call'
/usr/share/gems/gems/rack-2.2.8/lib/rack/head.rb:12:in `call'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/show_exceptions.rb:22:in `call'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:218:in `call'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:2004:in `call'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1564:in `block in call'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1780:in `synchronize'
/usr/share/gems/gems/sinatra-2.2.4/lib/sinatra/base.rb:1564:in `call'
/usr/share/gems/gems/rack-2.2.8/lib/rack/urlmap.rb:74:in `block in call'
/usr/share/gems/gems/rack-2.2.8/lib/rack/urlmap.rb:58:in `each'
/usr/share/gems/gems/rack-2.2.8/lib/rack/urlmap.rb:58:in `call'
/usr/share/gems/gems/rack-2.2.8/lib/rack/builder.rb:244:in `call'
/usr/share/gems/gems/rack-2.2.8/lib/rack/handler/webrick.rb:95:in `service'
/usr/share/gems/gems/webrick-1.8.1/lib/webrick/httpserver.rb:140:in `service'
/usr/share/gems/gems/webrick-1.8.1/lib/webrick/httpserver.rb:96:in `run'
/usr/share/gems/gems/webrick-1.8.1/lib/webrick/server.rb:310:in `block in start_thread'
/usr/share/gems/gems/logging-2.3.1/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'
2024-02-15T00:10:37 19e7c07a [I] Finished POST /dhcp/10.2.1.0 with 400 (47.87 ms)

I’m running Kea on the Foreman server with MySQL backend. The logs mention Postgresql which according to Foreman service status is working fine.

So what does this mean:

Unable to parse postgres subnets: received invalid response to GSSAPI negotiation: q

Still having this issue, anyone with any idea?