Host provision DHCP error

Support
,
1

Hi all,

We are seeing the below error in the proxy logs when trying to provision a host. We have recently updated our dev foreman server to 1.16. We also have a production foreman server that doesnt behave in the same way:

E, [2018-02-12T16:16:12.968429 4901eaf4] ERROR -- : Error: 404 { "Error": "AdmConDataNotFoundError: View default not found",
  "code": "Client.Ibap.Data.NotFound",
  "text": "View default not found"
}
D, [2018-02-12T16:16:12.968502 4901eaf4] DEBUG -- : Error: 404 { "Error": "AdmConDataNotFoundError: View default not found",
  "code": "Client.Ibap.Data.NotFound",
  "text": "View default not found"
} (Infoblox::Error)
/usr/share/gems/gems/infoblox-2.0.4/lib/infoblox/connection.rb:89:in `block in wrap'
/usr/share/gems/gems/infoblox-2.0.4/lib/infoblox/connection.rb:87:in `tap'
/usr/share/gems/gems/infoblox-2.0.4/lib/infoblox/connection.rb:87:in `wrap'
/usr/share/gems/gems/infoblox-2.0.4/lib/infoblox/connection.rb:16:in `get'
/usr/share/gems/gems/infoblox-2.0.4/lib/infoblox/resource.rb:110:in `find'
/usr/share/gems/gems/smart_proxy_dhcp_infoblox-0.0.10/lib/smart_proxy_dhcp_infoblox/host_ipv4_address_crud.rb:55:in `find_hosts'
/usr/share/gems/gems/smart_proxy_dhcp_infoblox-0.0.10/lib/smart_proxy_dhcp_infoblox/host_ipv4_address_crud.rb:42:in `find_record_by_mac'
/usr/share/gems/gems/smart_proxy_dhcp_infoblox-0.0.10/lib/smart_proxy_dhcp_infoblox/dhcp_infoblox_main.rb:44:in `find_record_by_mac'

We expected the host to obtain the next available DHCP IP from Infoblox but we arent sure if we even get that far as it throws an error as soon as the build is submitted.

We have the following Rubygem versions for Infoblox installed on the non-working Foreman Proxy server:

rubygem-smart_proxy_dns_infoblox-0.0.6-1.fm1_15.el7.noarch
rubygem-smart_proxy_dhcp_infoblox-0.0.10-1.fm1_16.el7.noarch

Thanks

James

2

By default infoblox dhcp provider uses “default” network view, which doesn’t appear to be available. You can configure the provider to use a non-default view by defining “network_view” setting in the provider configuration file.

1 Like
3

Actually, in Infoblox they are all lumped under Infoblox “Default” network view in the grid. We also don’t have that parameter configured on the Working host, which is running 1.15 - it appears only this Foreman 1.16 host is misbehaving. Thanks for the insight though, hopefully we can narrow this down further.

settings.d]# grep network_view *
4

Just an update, we found that 1.16 errors in proxy:

Create DHCP Settings for dev-testvm240.mydomain.net task failed with the following error: ERF12-6899 [ProxyAPI::ProxyException]: Unable to set DHCP entry ([RestClient::BadRequest]: 400 Bad Request) for proxy https://foremanproxy.mydomain.net:8443/dhcp

It also seems “suggest IP” isn’t working, in the Interfaces tab during provisioning. No error for that in either log.

Troubleshooting documentation doesn’t seem to help much, it just believes this is usually because DHCP is not enabled in settings files. We’ve confirmed this is not the case, though, and DHCP is an option available on the UI for the given foreman proxy.

Any help is appreciated, we’d love to make use of 1.16 in production.

5

Create DHCP Settings for dev-testvm240.mydomain.net task failed with the following error: ERF12-6899 [ProxyAPI::ProxyException]: Unable to set DHCP entry ([RestClient::BadRequest]: 400 Bad Request) for proxy https://foremanproxy.mydomain.net:8443/dhcp

That’s a snippet from Foreman log. Could you enable debug level of logging on the smart-proxy and post the error as it appears there?

It also seems “suggest IP” isn’t working, in the Interfaces tab during provisioning. No error for that in either log.

I can suggest to try using curl to see if you can get an ip from smart-proxy:

curl -k https://.../<network_address_here>/unused_ip

If not, check the errors in the proxy and infoblox logs. I would also suggest asking these questions in the #theforeman irc channel – I don’t think there are many infoblox users here.

6

I tried using curl to the foreman master with a specific network and I get a 404 “This page cannot be found.”

I see nothing in the debug foreman proxy logs. However, I do see the following rails error in debug foreman master logs:

2018-04-04 11:28:39 ae3cdfdc [app] [I] Started GET "/10.x.x.0/unused_ip" for 10.x.x.96 at 2018-04-04 11:28:39 +0000
2018-04-04 11:28:39 ae3cdfdc [app] [F] 
 | ActionController::RoutingError (No route matches [GET] "/10.x.x.0/unused_ip"):
 |   lib/middleware/tagged_logging.rb:18:in `call'
 | 
 |

It seems like there’s a missing route in the rails system. I guess I will give IRC a shout, thanks.

7

I should’ve been more clear when I suggested to use curl: unused_ip endpoint is served by smart-proxy not Foreman.

8

Sorry, so this is what happens when I do that on the proxy:

$ curl -k https://foremanproxy:8443/10.x.x.0/unused_ip
Requested url was not found

The debug on foreman proxy log shows the following:

D, [2018-04-11T14:09:26.868996 ] DEBUG -- : Rack::Handler::WEBrick is mounted on /.
I, [2018-04-11T14:09:26.869171 ]  INFO -- : WEBrick::HTTPServer#start: pid=7773 port=8443
D, [2018-04-11T15:38:26.364856 ] DEBUG -- : accept: 10.x.x.84:41384
D, [2018-04-11T15:38:26.368446 ] DEBUG -- : Rack::Handler::WEBrick is invoked.
I, [2018-04-11T15:38:26.384124 ]  INFO -- : 10.x.x.84 - - [11/Apr/2018:15:38:26 +0000] "GET /10.x.x.0/unused_ip HTTP/1.1" 404 27 0.0046

Any ideas?

9

Additionally this is what happens when I restart foreman proxy, it seems dhcp says it’s successfully initialized, but the URL 404’s:

I, [2018-04-11T15:41:35.535396 ]  INFO -- : going to shutdown ...
I, [2018-04-11T15:41:35.535824 ]  INFO -- : WEBrick::HTTPServer#start done.
D, [2018-04-11T15:41:35.880099 ] DEBUG -- : 'discovery' settings: 'enabled': true, 'node_port': 8443 (default), 'node_scheme': https (default)
D, [2018-04-11T15:41:35.887551 ] DEBUG -- : 'tftp' settings: 'enabled': https, 'tftproot': /var/lib/tftpboot (default)
D, [2018-04-11T15:41:35.894954 ] DEBUG -- : 'dhcp' settings: 'enabled': true, 'server': infoblox.mydomain.net, 'subnets': [] (default), 'use_provider': dhcp_infoblox
D, [2018-04-11T15:41:35.925631 ] DEBUG -- : 'bmc' settings: 'bmc_default_provider': ipmitool, 'enabled': https
D, [2018-04-11T15:41:35.928218 ] DEBUG -- : 'logs' settings: 'enabled': https
D, [2018-04-11T15:41:35.928700 ] DEBUG -- : Providers ['dhcp_infoblox'] are going to be configured for 'dhcp'
D, [2018-04-11T15:41:35.942850 ] DEBUG -- : 'dhcp_infoblox' settings: 'password': [hidden], 'range': false (default), 'record_type': host (default), 'server': infoblox.mydomain.net, 'subnets': [], 'use_provider': dhcp_infoblox, 'use_ranges': false, 'username': [hidden]
I, [2018-04-11T15:41:35.943287 ]  INFO -- : Successfully initialized 'discovery'
I, [2018-04-11T15:41:35.943359 ]  INFO -- : Successfully initialized 'foreman_proxy'
I, [2018-04-11T15:41:35.943413 ]  INFO -- : Successfully initialized 'tftp'
I, [2018-04-11T15:41:35.943498 ]  INFO -- : Successfully initialized 'dhcp_infoblox'
I, [2018-04-11T15:41:35.943551 ]  INFO -- : Successfully initialized 'dhcp'
I, [2018-04-11T15:41:35.943629 ]  INFO -- : Successfully initialized 'bmc'
D, [2018-04-11T15:41:35.943682 ] DEBUG -- : Log buffer API initialized, available capacity: 2000/1000
I, [2018-04-11T15:41:35.943724 ]  INFO -- : Successfully initialized 'logs'
I, [2018-04-11T15:41:36.033717 ]  INFO -- : WEBrick 1.3.1
I, [2018-04-11T15:41:36.033878 ]  INFO -- : ruby 2.0.0 (2015-12-16) [x86_64-linux]
D, [2018-04-11T15:41:36.034434 ] DEBUG -- : TCPServer.new(0.0.0.0, 8443)
D, [2018-04-11T15:41:36.035687 ] DEBUG -- : TCPServer.new(::, 8443)
W, [2018-04-11T15:41:36.035885 ]  WARN -- : TCPServer Error: Address already in use - bind(2)
I, [2018-04-11T15:41:36.037364 ]  INFO -- : 
Certificate:
[whole pile of SSL stuff here]

D, [2018-04-11T15:41:36.041043 ] DEBUG -- : Rack::Handler::WEBrick is mounted on /.
I, [2018-04-11T15:41:36.041269 ]  INFO -- : WEBrick::HTTPServer#start: pid=19709 port=8443
10

My apologies, the url I suggested was incorrect. The correct one is https://foremanproxy:8443/dhcp/10.x.x.0/unused_ip, please note the “dhcp” part.

11

Same issue occuring with Foreman 1.17 and Proxy 1.17:

E, [2018-05-03T12:29:37.665005 bace2873] ERROR -- : Error: 404 { "Error": "AdmConDataNotFoundError: View default not found", 
  "code": "Client.Ibap.Data.NotFound", 
  "text": "View default not found"
}
D, [2018-05-03T12:29:37.665132 bace2873] DEBUG -- : Error: 404 { "Error": "AdmConDataNotFoundError: View default not found", 
  "code": "Client.Ibap.Data.NotFound", 
  "text": "View default not found"
} (Infoblox::Error)
/usr/share/gems/gems/infoblox-2.0.4/lib/infoblox/connection.rb:89:in `block in wrap'
/usr/share/gems/gems/infoblox-2.0.4/lib/infoblox/connection.rb:87:in `tap'
/usr/share/gems/gems/infoblox-2.0.4/lib/infoblox/connection.rb:87:in `wrap'
/usr/share/gems/gems/infoblox-2.0.4/lib/infoblox/connection.rb:16:in `get'
/usr/share/gems/gems/infoblox-2.0.4/lib/infoblox/resource.rb:110:in `find'
/usr/share/gems/gems/smart_proxy_dhcp_infoblox-0.0.13/lib/smart_proxy_dhcp_infoblox/host_ipv4_address_crud.rb:18:in `all_hosts'
/usr/share/gems/gems/smart_proxy_dhcp_infoblox-0.0.13/lib/smart_proxy_dhcp_infoblox/dhcp_infoblox_main.rb:31:in `all_hosts'
/usr/share/foreman-proxy/modules/dhcp_common/server.rb:101:in `unused_ip'
/usr/share/foreman-proxy/modules/dhcp/dhcp_api.rb:34:in `block in <class:DhcpApi>'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:1611:in `call'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:1611:in `block in compile!'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:975:in `[]'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:975:in `block (3 levels) in route!'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:994:in `route_eval'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:975:in `block (2 levels) in route!'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:1015:in `block in process_route'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:1013:in `catch'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:1013:in `process_route'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:973:in `block in route!'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:972:in `each'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:972:in `route!'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:1085:in `block in dispatch!'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:1067:in `block in invoke'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:1067:in `catch'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:1067:in `invoke'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:1082:in `dispatch!'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:907:in `block in call!'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:1067:in `block in invoke'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:1067:in `catch'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:1067:in `invoke'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:907:in `call!'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:895:in `call'
/usr/share/gems/gems/rack-1.6.4/lib/rack/methodoverride.rb:22:in `call'
/usr/share/gems/gems/rack-1.6.4/lib/rack/commonlogger.rb:33:in `call'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:219:in `call'
/usr/share/foreman-proxy/lib/proxy/log.rb:109:in `call'
/usr/share/foreman-proxy/lib/proxy/request_id_middleware.rb:9:in `call'
/usr/share/gems/gems/rack-protection-1.5.3/lib/rack/protection/xss_header.rb:18:in `call'
/usr/share/gems/gems/rack-protection-1.5.3/lib/rack/protection/path_traversal.rb:16:in `call'
/usr/share/gems/gems/rack-protection-1.5.3/lib/rack/protection/json_csrf.rb:18:in `call'
/usr/share/gems/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in `call'
/usr/share/gems/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in `call'
/usr/share/gems/gems/rack-protection-1.5.3/lib/rack/protection/frame_options.rb:31:in `call'
/usr/share/gems/gems/rack-1.6.4/lib/rack/nulllogger.rb:9:in `call'
/usr/share/gems/gems/rack-1.6.4/lib/rack/head.rb:13:in `call'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/show_exceptions.rb:25:in `call'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:182:in `call'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:2013:in `call'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:1487:in `block in call'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:1787:in `synchronize'
/usr/share/gems/gems/sinatra-1.4.8/lib/sinatra/base.rb:1487:in `call'
/usr/share/gems/gems/rack-1.6.4/lib/rack/urlmap.rb:66:in `block in call'
/usr/share/gems/gems/rack-1.6.4/lib/rack/urlmap.rb:50:in `each'
/usr/share/gems/gems/rack-1.6.4/lib/rack/urlmap.rb:50:in `call'
/usr/share/gems/gems/rack-1.6.4/lib/rack/builder.rb:153:in `call'
/usr/share/gems/gems/rack-1.6.4/lib/rack/handler/webrick.rb:88:in `service'
/usr/share/ruby/webrick/httpserver.rb:138:in `service'
/usr/share/ruby/webrick/httpserver.rb:94:in `run'
/usr/share/ruby/webrick/server.rb:295:in `block in start_thread'
12

E, [2018-05-03T12:29:37.665005 bace2873] ERROR – : Error: 404 { “Error”: “AdmConDataNotFoundError: View default not found”,
“code”: “Client.Ibap.Data.NotFound”,
“text”: “View default not found”
}

A non-default dns/network view is configured. To configure smart-proxy to use non-default views, use dns_view and network_view provider settings.

13

I wasn’t sure where this setting was as I didn’t see it in smart_proxy git, I did find it in smart_proxy_dhcp_infoblox git though as a couple of undocumented settings.

So I added the following to the /etc/foreman-proxy/settings.d/dhcp_infoblox.yml
:dns_view: Internal
:network_view: default

Restarted foreman-proxy, and that was it! DHCP Infoblox plugin works perfectly now.

Thank you so much, it’s good we can finally move forward :slight_smile: