Automate IP allocation over specifying subnets in machine creation

Development
#1

Hi Guys,

We're trying to provision a machine (physical) in discovery from one subnet
(LA-PXE) to another (LA-SUBNET) using DHCP as IPAM, and we're getting an
error of "Ip does not match selected subnet" unless specifying an ip
address in advance (which is really a pain cause this is messing up our
automation).

Another workaround that could be fine is getting a free ip via API and
specify it at the hammer command, but we couldn't find a way to do that
either.

This is the debug from the hammer command:

  • hammer -d discovery provision --name mac3417ebef850b --new-name
    machinename266 --hostgroup-id=4 --environment production --puppet-ca-proxy-
    id=1 --puppet-proxy-id=23 --domain la.mydomain.com --subnet LA-SUBNET --architecture
    x86_64 --operatingsystem 'CentOS 6.7' --build yes --medium 'CentOS Vault' –
    partition-table-id 119

[ INFO 2016-08-29T17:02:53 Init] Initialization of Hammer CLI (0.7.0) has
started…

[DEBUG 2016-08-29T17:02:53 Init] Running at ruby 2.2.3-p173

[ INFO 2016-08-29T17:02:53 Init] Configuration from the file /Users/eliraz.a
/.hammer/cli_config.yml has been loaded

[ INFO 2016-08-29T17:02:53 Init] Configuration from the file /Users/eliraz.a
/.hammer/cli.modules.d/foreman.yml has been loaded

[ INFO 2016-08-29T17:02:53 Init] Configuration from the file /Users/eliraz.a
/.hammer/cli.modules.d/foreman_discovery.yml has been loaded

[DEBUG 2016-08-29T17:02:54 Connection] Registered: foreman

[DEBUG 2016-08-29T17:02:54 API] Global headers: {

    :content_type => "application/json", 

          :accept => "application/json;version=2", 

"Accept-Language" => "en"

}

[DEBUG 2016-08-29T17:02:54 API] Follow redirects: never

[ INFO 2016-08-29T17:02:54 Modules] Extension module hammer_cli_foreman (0.7
.0) loaded

[ INFO 2016-08-29T17:02:54 Modules] Extension module
hammer_cli_foreman_discovery (0.0.3) loaded

[DEBUG 2016-08-29T17:02:54 Init] Using locale 'en'

[DEBUG 2016-08-29T17:02:54 Init] 'mo' files for locale domain 'hammer-cli'
loaded from
'/Users/eliraz.a/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/hammer_cli-0.7.0/locale'

[DEBUG 2016-08-29T17:02:54 Init] 'mo' files for locale domain
'hammer-cli-foreman' loaded from
'/Users/eliraz.a/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/hammer_cli_foreman-0.7.0/locale'

[ INFO 2016-08-29T17:02:54 HammerCLI::MainCommand] Called with options: {
"option_debug"=>true}

[ INFO 2016-08-29T17:02:54 HammerCLIForemanDiscovery::DiscoveredHost] Called
with options: {}

[ INFO 2016-08-29T17:02:54 HammerCLIForemanDiscovery::DiscoveredHost::
ProvisionCommand] Called with options: {"option_partition_table_id"=>"119",
"option_build"=>true, "option_interface_list"=>[], "option_name"=>
"mac3417ebef850b", "option_environment_name"=>"production",
"option_architecture_name"=>"x86_64", "option_domain_name"=>
"la.mydomain.com", "option_operatingsystem_title"=>"CentOS 6.7",
"option_medium_name"=>"CentOS Vault", "option_subnet_name"=>"LA-SUBNET",
"option_hostgroup_id"=>4, "option_puppet_proxy_id"=>23,
"option_puppet_ca_proxy_id"=>1, "option_new_name"=>"machinename266"}

[ INFO 2016-08-29T17:02:54 API] Server: https://foreman004.mydomain.com

[ INFO 2016-08-29T17:02:54 API] GET /api/environments

[DEBUG 2016-08-29T17:02:54 API] Params: {

:search => "name = \"production\""

}

[DEBUG 2016-08-29T17:02:54 API] Headers: {

:params => { 

    :search => "name = \"production\"" 

}

}

[DEBUG 2016-08-29T17:02:55 API] Response: {

   "total" => 7, 

"subtotal" => 1, 

    "page" => 1, 

"per_page" => 30, 

  "search" => "name = \"production\"", 

    "sort" => { 

       "by" => nil, 

    "order" => nil 

}, 

 "results" => [ 

    [0] { 

        "created_at" => "2013-01-28 14:55:48 UTC", 

        "updated_at" => "2013-01-28 14:55:48 UTC", 

              "name" => "production", 

                "id" => 1 

    } 

]

}

[DEBUG 2016-08-29T17:02:55 API] Response headers: {

                  :date => "Mon, 29 Aug 2016 14:02:54 GMT", 

                :server => "Apache/2.4.6 (CentOS)", 

       :x_frame_options => "SAMEORIGIN", 

      :x_xss_protection => "1; mode=block", 

:x_content_type_options => "nosniff", 

       :foreman_version => "1.12.2", 

   :foreman_api_version => "2", 

       :apipie_checksum => "3793deca260091ff927bc62b6bb53ed7", 

         :cache_control => "max-age=0, private, must-revalidate", 

          :x_request_id => "b6843d25-19aa-4c3e-8d71-7038bbb958dc", 

             :x_runtime => "1.077152", 

          :x_powered_by => "Phusion Passenger 4.0.53", 

            :set_cookie => [ 

    [0] "_session_id=96f9a8145615909aef9b2bd9e529d8a9; path=/; secure;

HttpOnly"

], 

                  :etag => "W/\"291e70168205fc11081d837b02f551dd-gzip\""
··· , 
                :status => "200 OK", 

                  :vary => "Accept-Encoding", 

      :content_encoding => "gzip", 

        :content_length => "183", 

          :content_type => "application/json; charset=utf-8"

}

[ INFO 2016-08-29T17:02:55 API] Server: https://foreman004.mydomain.com

[ INFO 2016-08-29T17:02:55 API] GET /api/architectures

[DEBUG 2016-08-29T17:02:55 API] Params: {

:search => "name = \"x86_64\""

}

[DEBUG 2016-08-29T17:02:55 API] Headers: {

:params => { 

    :search => "name = \"x86_64\"" 

}

}

[DEBUG 2016-08-29T17:02:56 API] Response: {

   "total" => 2, 

"subtotal" => 1, 

    "page" => 1, 

"per_page" => 30, 

  "search" => "name = \"x86_64\"", 

    "sort" => { 

       "by" => nil, 

    "order" => nil 

}, 

 "results" => [ 

    [0] { 

        "created_at" => "2013-01-28 15:11:58 UTC", 

        "updated_at" => "2013-01-28 15:11:58 UTC", 

              "name" => "x86_64", 

                "id" => 1 

    } 

]

}

[DEBUG 2016-08-29T17:02:56 API] Response headers: {

                  :date => "Mon, 29 Aug 2016 14:02:55 GMT", 

                :server => "Apache/2.4.6 (CentOS)", 

       :x_frame_options => "SAMEORIGIN", 

      :x_xss_protection => "1; mode=block", 

:x_content_type_options => "nosniff", 

       :foreman_version => "1.12.2", 

   :foreman_api_version => "2", 

       :apipie_checksum => "3793deca260091ff927bc62b6bb53ed7", 

         :cache_control => "max-age=0, private, must-revalidate", 

          :x_request_id => "6eba87c1-53b8-4400-8f44-0c1cc964804e", 

             :x_runtime => "1.284867", 

          :x_powered_by => "Phusion Passenger 4.0.53", 

            :set_cookie => [ 

    [0] "_session_id=4f0a6f2bbd34660ce9a72529ad2a327e; path=/; secure;

HttpOnly"

], 

                  :etag => "W/\"463f1fc8553ecf7162cdb120026594ad-gzip\""

,

                :status => "200 OK", 

                  :vary => "Accept-Encoding", 

      :content_encoding => "gzip", 

        :content_length => "181", 

          :content_type => "application/json; charset=utf-8"

}

[ INFO 2016-08-29T17:02:56 API] Server: https://foreman004.mydomain.com

[ INFO 2016-08-29T17:02:56 API] GET /api/domains

[DEBUG 2016-08-29T17:02:56 API] Params: {

:search => "name = \"la.mydomain.com\""

}

[DEBUG 2016-08-29T17:02:56 API] Headers: {

:params => { 

    :search => "name = \"la.mydomain.com\"" 

}

}

[DEBUG 2016-08-29T17:02:58 API] Response: {

   "total" => 14, 

"subtotal" => 1, 

    "page" => 1, 

"per_page" => 30, 

  "search" => "name = \"la.mydomain.com\"", 

    "sort" => { 

       "by" => nil, 

    "order" => nil 

}, 

 "results" => [ 

    [0] { 

          "fullname" => "la.mydomain.com", 

            "dns_id" => 15, 

        "created_at" => "2013-04-30 01:07:27 UTC", 

        "updated_at" => "2016-08-29 11:20:13 UTC", 

                "id" => 4, 

              "name" => "la.mydomain.com" 

    } 

]

}

[DEBUG 2016-08-29T17:02:58 API] Response headers: {

                  :date => "Mon, 29 Aug 2016 14:02:56 GMT", 

                :server => "Apache/2.4.6 (CentOS)", 

       :x_frame_options => "SAMEORIGIN", 

      :x_xss_protection => "1; mode=block", 

:x_content_type_options => "nosniff", 

       :foreman_version => "1.12.2", 

   :foreman_api_version => "2", 

       :apipie_checksum => "3793deca260091ff927bc62b6bb53ed7", 

         :cache_control => "max-age=0, private, must-revalidate", 

          :x_request_id => "70be7db2-d3ca-47e1-9ca5-bc169b1e3ac9", 

             :x_runtime => "1.117908", 

          :x_powered_by => "Phusion Passenger 4.0.53", 

            :set_cookie => [ 

    [0] "_session_id=553da35504c37fca4207e9cd08bb55df; path=/; secure;

HttpOnly"

], 

                  :etag => "W/\"dbf10bd655ceaa5e46bbc6c860b20c40-gzip\""

,

                :status => "200 OK", 

                  :vary => "Accept-Encoding", 

      :content_encoding => "gzip", 

        :content_length => "226", 

          :content_type => "application/json; charset=utf-8"

}

[ INFO 2016-08-29T17:02:58 API] Server: https://foreman004.mydomain.com

[ INFO 2016-08-29T17:02:58 API] GET /api/operatingsystems

[DEBUG 2016-08-29T17:02:58 API] Params: {

:search => "title = \"CentOS 6.7\""

}

[DEBUG 2016-08-29T17:02:58 API] Headers: {

:params => { 

    :search => "title = \"CentOS 6.7\"" 

}

}

[DEBUG 2016-08-29T17:02:59 API] Response: {

   "total" => 27, 

"subtotal" => 1, 

    "page" => 1, 

"per_page" => 30, 

  "search" => "title = \"CentOS 6.7\"", 

    "sort" => { 

       "by" => nil, 

    "order" => nil 

}, 

 "results" => [ 

    [0] { 

          "description" => "CentOS 6.7", 

                "major" => "6", 

                "minor" => "7", 

               "family" => "Redhat", 

         "release_name" => "", 

        "password_hash" => "MD5", 

           "created_at" => "2015-08-07 14:03:26 UTC", 

           "updated_at" => "2016-01-25 13:11:09 UTC", 

                   "id" => 20, 

                 "name" => "CentOS", 

                "title" => "CentOS 6.7" 

    } 

]

}

[DEBUG 2016-08-29T17:02:59 API] Response headers: {

                  :date => "Mon, 29 Aug 2016 14:02:58 GMT", 

                :server => "Apache/2.4.6 (CentOS)", 

       :x_frame_options => "SAMEORIGIN", 

      :x_xss_protection => "1; mode=block", 

:x_content_type_options => "nosniff", 

       :foreman_version => "1.12.2", 

   :foreman_api_version => "2", 

       :apipie_checksum => "3793deca260091ff927bc62b6bb53ed7", 

         :cache_control => "max-age=0, private, must-revalidate", 

          :x_request_id => "ac7a88c3-3777-4087-b22e-2fd025800407", 

             :x_runtime => "1.066450", 

          :x_powered_by => "Phusion Passenger 4.0.53", 

            :set_cookie => [ 

    [0] "_session_id=028eb7431946b966dd5e23e2df52cb4d; path=/; secure;

HttpOnly"

], 

                  :etag => "W/\"2a275a188c682b751bc5d5e8b80eb880-gzip\""

,

                :status => "200 OK", 

                  :vary => "Accept-Encoding", 

      :content_encoding => "gzip", 

        :content_length => "270", 

          :content_type => "application/json; charset=utf-8"

}

[ INFO 2016-08-29T17:02:59 API] Server: https://foreman004.mydomain.com

[ INFO 2016-08-29T17:02:59 API] GET /api/media

[DEBUG 2016-08-29T17:02:59 API] Params: {

:search => "name = \"CentOS Vault\""

}

[DEBUG 2016-08-29T17:02:59 API] Headers: {

:params => { 

    :search => "name = \"CentOS Vault\"" 

}

}

[DEBUG 2016-08-29T17:03:00 API] Response: {

   "total" => 24, 

"subtotal" => 1, 

    "page" => 1, 

"per_page" => 30, 

  "search" => "name = \"CentOS Vault\"", 

    "sort" => { 

       "by" => nil, 

    "order" => nil 

}, 

 "results" => [ 

    [0] { 

              "path" =>

“http://vault.centos.org/$major.$minor/os/x86_64/”,

         "os_family" => "Redhat", 

        "created_at" => "2016-06-02 13:56:13 UTC", 

        "updated_at" => "2016-06-02 13:56:13 UTC", 

                "id" => 29, 

              "name" => "CentOS Vault" 

    } 

]

}

[DEBUG 2016-08-29T17:03:00 API] Response headers: {

                  :date => "Mon, 29 Aug 2016 14:02:59 GMT", 

                :server => "Apache/2.4.6 (CentOS)", 

       :x_frame_options => "SAMEORIGIN", 

      :x_xss_protection => "1; mode=block", 

:x_content_type_options => "nosniff", 

       :foreman_version => "1.12.2", 

   :foreman_api_version => "2", 

       :apipie_checksum => "3793deca260091ff927bc62b6bb53ed7", 

         :cache_control => "max-age=0, private, must-revalidate", 

          :x_request_id => "150426af-6830-4fa2-a0c3-1040d7f610b2", 

             :x_runtime => "1.201093", 

          :x_powered_by => "Phusion Passenger 4.0.53", 

            :set_cookie => [ 

    [0] "_session_id=583c09b43aff4450d8163deeb2e16d7e; path=/; secure;

HttpOnly"

], 

                  :etag => "W/\"b9252617abff69aa87d8c931e23af24a-gzip\""

,

                :status => "200 OK", 

                  :vary => "Accept-Encoding", 

      :content_encoding => "gzip", 

        :content_length => "249", 

          :content_type => "application/json; charset=utf-8"

}

[ INFO 2016-08-29T17:03:00 API] Server: https://foreman004.mydomain.com

[ INFO 2016-08-29T17:03:00 API] GET /api/subnets

[DEBUG 2016-08-29T17:03:00 API] Params: {

:search => "name = \"LA-SUBNET\""

}

[DEBUG 2016-08-29T17:03:00 API] Headers: {

:params => { 

    :search => "name = \"LA-SUBNET\"" 

}

}

[DEBUG 2016-08-29T17:03:01 API] Response: {

   "total" => 117, 

"subtotal" => 1, 

    "page" => 1, 

"per_page" => 30, 

  "search" => "name = \"LA-SUBNET\"", 

    "sort" => { 

       "by" => nil, 

    "order" => nil 

}, 

 "results" => [ 

    [0] { 

                "network" => "10.1.20.0", 

                   "cidr" => 23, 

                   "mask" => "255.255.254.0", 

               "priority" => nil, 

                 "vlanid" => "20", 

                "gateway" => "10.1.20.1", 

            "dns_primary" => "10.1.0.4", 

          "dns_secondary" => "10.1.0.5", 

                   "from" => "10.1.20.10", 

                     "to" => "10.1.21.240", 

             "created_at" => "2016-06-27 08:51:21 UTC", 

             "updated_at" => "2016-06-27 08:53:08 UTC", 

                   "ipam" => "DHCP", 

              "boot_mode" => "DHCP", 

                     "id" => 58, 

                   "name" => "LA-SUBNET", 

        "network_address" => "10.1.20.0/23", 

                   "dhcp" => { 

            "name" => "foreman.la", 

              "id" => 6, 

             "url" => "https://dhcpserver.la.mydomain.com:8443" 

        }, 

                   "tftp" => { 

            "name" => "foreman.la", 

              "id" => 6, 

             "url" => "https://dhcpserver.la.mydomain.com:8443" 

        }, 

                    "dns" => { 

            "name" => "dns201", 

              "id" => 15, 

             "url" => "https://dns201.la.mydomain.com:8443" 

        } 

    } 

]

}

[DEBUG 2016-08-29T17:03:01 API] Response headers: {

                  :date => "Mon, 29 Aug 2016 14:03:00 GMT", 

                :server => "Apache/2.4.6 (CentOS)", 

       :x_frame_options => "SAMEORIGIN", 

      :x_xss_protection => "1; mode=block", 

:x_content_type_options => "nosniff", 

       :foreman_version => "1.12.2", 

   :foreman_api_version => "2", 

       :apipie_checksum => "3793deca260091ff927bc62b6bb53ed7", 

         :cache_control => "max-age=0, private, must-revalidate", 

          :x_request_id => "1f8f7559-a4b2-42a0-a31c-d12a9e9c47de", 

             :x_runtime => "1.104068", 

          :x_powered_by => "Phusion Passenger 4.0.53", 

            :set_cookie => [ 

    [0] "_session_id=90a730e38917fb8ab7e54977f887b4d4; path=/; secure;

HttpOnly"

], 

                  :etag => "W/\"0cb36337c717c719ed9d2c540fb6e7b5-gzip\""

,

                :status => "200 OK", 

                  :vary => "Accept-Encoding", 

      :content_encoding => "gzip", 

        :content_length => "407", 

          :content_type => "application/json; charset=utf-8"

}

[ INFO 2016-08-29T17:03:01 API] Server: https://foreman004.mydomain.com

[ INFO 2016-08-29T17:03:01 API] GET /api/v2/discovered_hosts

[DEBUG 2016-08-29T17:03:01 API] Params: {

:search => "name = \"mac3417ebef850b\""

}

[DEBUG 2016-08-29T17:03:01 API] Headers: {

:params => { 

    :search => "name = \"mac3417ebef850b\"" 

}

}

[DEBUG 2016-08-29T17:03:02 API] Response: {

   "total" => 167, 

"subtotal" => 1, 

    "page" => 1, 

"per_page" => 30, 

  "search" => "name = \"mac3417ebef850b\"", 

    "sort" => { 

       "by" => nil, 

    "order" => nil 

}, 

 "results" => [ 

    [0] { 

                         "ip" => "10.1.110.25", 

                        "mac" => "34:17:eb:ef:85:0b", 

                "last_report" => "2016-08-21 09:07:54 UTC", 

                  "subnet_id" => 65, 

                "subnet_name" => "LA-PXE", 

        "hardware_model_name" => "PowerEdge FC430", 

                     "memory" => 128831, 

                 "disk_count" => 2, 

                 "disks_size" => 763109, 

                       "cpus" => 2, 

                       "name" => "
#2

Thanks Lukas! Looking forward for that patch!

BTW I also tried to work with the API of "Free IP" but couldn't managed to
authenticate properly, can you give me an example of how to use it?
that way we could use the workaround I spoke of before…

#3

Hello,

> We're trying to provision a machine (physical) in discovery from one subnet
> (LA-PXE) to another (LA-SUBNET) using DHCP as IPAM, and we're getting an
> error of "Ip does not match selected subnet" unless specifying an ip
> address in advance (which is really a pain cause this is messing up our
> automation).
>
> Another workaround that could be fine is getting a free ip via API and
> specify it at the hammer command, but we couldn't find a way to do that
> either.

at the moment, discovery keeps the provisioning interface IP filled in
the form, so when you provision it with different subnet, it would error
out.

We are tracking a ticket to change this behavior to delete the IP
address and call "Free IP" API during provisioning:

http://projects.theforeman.org/issues/16143

The same applies to provisioning via API.

··· -- Later, Lukas #lzap Zapletal
#4

> BTW I also tried to work with the API of "Free IP" but couldn't managed to
> authenticate properly, can you give me an example of how to use it?
> that way we could use the workaround I spoke of before…

The proxy API directly? That's not possible, unless you use HTTP client
certificate. Sign it with your puppet ca and use it if you want.

··· -- Later, Lukas #lzap Zapletal
#5

Well, it is possible - but you have to make some changes to the proxy :slight_smile:

The proxy, by default, sets:

:trusted_hosts:

This means 2 things - one, API calls need to be wrapped with a certificate
signed by the proxy's CA, and 2, the cert's name has to be in the
trusted_hosts.

So, you have three options, from most to least-secure:

  1. If only a few machines will need to call the free_ip endpoint, you can
    generate new certs from the same CA, give them to the clients, and add them
    to trusted_hosts (i.e what Lukas said)
  2. disable trusted_hosts entirely, at which point any valid SSL request can
    access the API
  3. Allow non-ssl requests to that proxy module

Hope that helps!
Greg

··· On 31 August 2016 at 11:45, Lukas Zapletal wrote:

BTW I also tried to work with the API of “Free IP” but couldn’t managed
to
authenticate properly, can you give me an example of how to use it?
that way we could use the workaround I spoke of before…

The proxy API directly? That’s not possible, unless you use HTTP client
certificate. Sign it with your puppet ca and use it if you want.

#6

… you can also get a free ip via Forman's API, without calling the proxy directly. Do you need any help doing that? I'm just not sure if support was added in 1.12 or 1.13. :slight_smile:

··· On 31.08.2016, at 12:45, Lukas Zapletal wrote:

BTW I also tried to work with the API of “Free IP” but couldn’t managed to
authenticate properly, can you give me an example of how to use it?
that way we could use the workaround I spoke of before…

The proxy API directly? That’s not possible, unless you use HTTP client
certificate. Sign it with your puppet ca and use it if you want.