VMware Template Provisioning with Static IP

eroji · August 29, 2018, 12:08am

I have created a VMware template image for provisioning hosts in Foreman using UserData. I have cloud-init installed and configured on the template image and the following userdata provisioning template. It appears to be able to provision the machine correctly up to the point of configuring the network information of the NIC. No matter what I tried, it always came up with DHCP.

The template image is CentOS 7.5 minimal install, with puppet-agent and cloud-init (configured to point to Foreman). Foreman is running version 1.18.1 and VMware vSphere is 6.7.

identity:
  LinuxPrep:
    domain: <%= @host.domain %>
    hostName: <%= @host.shortname %>
    hwClockUTC: true
    timeZone: America/Los_Angeles
  
globalIPSettings:
  dnsSuffixList: [<%= @host.domain %>]
  <%- @host.interfaces.each do |interface| -%>
  <%- next unless interface.subnet -%>
  dnsServerList: [<%= interface.subnet.dns_primary %>, <%= interface.subnet.dns_secondary %>]
  <%- end -%>

nicSettingMap:
<%- @host.interfaces.each do |interface| -%>
<%- next unless interface.subnet -%>
  - adapter:
      dnsDomain: <%= interface.domain %>
      dnsServerList: [<%= interface.subnet.dns_primary %>, <%= interface.subnet.dns_secondary %>]
      gateway: [<%= interface.subnet.gateway %>]
      ip: <%= interface.ip %>
      subnetMask: <%= interface.subnet.mask %>
<%- end -%>

lzap · August 29, 2018, 7:47am

Here is my statement from year ago (Bug #8601: use cloudinit like user data in vsphere compute ressource - Foreman), unless @iNecas has some new findings I guess this is an accurate description:

VMWare has actually special cloud-init limited scope support, it
does not pass the cloud-init into the image as-is, but rather converts
it into individual actions referred as Custom Spec in VMWare docs.
Only some fields are converted, the rest is unused. It does not accept
“call home” wget/curl call therefore VMWare VMs initialized with
cloud-init stays in build mode forever. This is known limitation.

github.com

fog/fog-vsphere/blob/master/lib/fog/vsphere/requests/compute/cloudinit_to_customspec.rb

module Fog
  module Compute
    class Vsphere
      class Real
        def cloudinit_to_customspec(user_data)
          raise ArgumentError, "user_data can't be nil" if user_data.nil?
          custom_spec = { 'customization_spec' => {} }
          user_data = YAML.safe_load(user_data)
          # https://pubs.vmware.com/vsphere-55/index.jsp#com.vmware.wssdk.apiref.doc/vim.vm.customization.Specification.html
          # encryptionKey expects an array
          # globalIPSettings expects a hash, REQUIRED
          # identity expects an hash, REQUIRED
          # nicSettingMap expects an array
          # options expects an hash

          custom_spec['encryptionKey']    = user_data['encryptionKey'] if user_data.key?('encryptionKey')
          custom_spec['globalIPSettings'] = user_data['globalIPSettings'] if user_data.key?('globalIPSettings')
          custom_spec['identity']         = user_data['identity'] if user_data.key?('identity')
          custom_spec['identity']         = { 'Sysprep' => { 'guiRunOnce' => { 'commandList' => user_data['runcmd'] } } } if user_data.key?('runcmd') && !user_data.key?('identity')
          custom_spec['nicSettingMap']    = user_data['nicSettingMap'] if user_data.key?('nicSettingMap')

This file has been truncated. show original

https://pubs.vmware.com/vsphere-55/index.jsp#com.vmware.wssdk.apiref.doc/vim.vm.customization.Specification.html

iNecas · August 29, 2018, 9:24am

I think it’s valid description: there is no full integration of cloud-init available in vmware as far as we know, and the custom Vsphere’s mechanism is required.

TimoGoebel · August 29, 2018, 10:59am

https://github.com/theforeman/foreman_userdata might offer a workaround for you. It allows you to run the real cloud init after vmware has done the customisation magic.

eroji · August 29, 2018, 6:26pm

Thank you for the clarification.

eroji · August 29, 2018, 6:36pm

I actually found this guide before and tried to follow it. It’s where I got the UserData template from. However, I am not quite sure where to put the other 2 templates. They appear to be cloud-init templates. Are they supposed to be created in Foreman and linked up to the OS or do they actually live as config on the VMware image template?

TimoGoebel · August 30, 2018, 7:53am

The former. The cloud-init templates are supposed to be set up in Foreman and linked to the OS. Do you mind clarifying the guide? It’s always hard to think of every pitfall when actually know what’s needed. Either send a pull request or post the updated version in this thread.

I’ll try to add the templates to the plugin so that they are seeded when you install them.

lzap · August 30, 2018, 11:20am

Time if you are interested, I can help pushing this into core. This is very useful. The only issue is the new controller, I think this belongs to the unattended controller. It’s just another kind, no magic if I am not mistaken.

TimoGoebel · August 30, 2018, 11:23am

I’m currently working on improving some small bits, but sure, let’s move this to core once we have improved unattended controller.

lzap · August 30, 2018, 11:42am

Oh yeah, that controller. The controller

tommaso.paoletto · September 5, 2018, 1:14pm

Hi there,
thank you for the clarifications.
I came across the same situation of @eroji in the exact way:

Provisioning a new host from image cloning a CentOS 7.5 template on VMware
Entering manually the IP of the new cloned VM in Foreman UI
Using the custom “user_data” Template
Following Foreman manual chapter 5.2.9

The same process can also be done using a user_data template. To configure image/template-based provisioning without SSH, make the following adjustments for the former procedure:

*** Enable user_data checking the proper checkbox in the compute resource definition**
*** Associate a user_data template to the host, using cloud-init syntax**
*** Note that the images dont need cloudinit installed, as the cloudinit is converted under the hood to a CustomisationSpec object that VMware can process**

The template is correctly cloned, but there is no customization about hostname and network.

Could you please help me debugging the call from Foreman to VMWare APIs to check what is going wrong?

edergillian · November 7, 2018, 5:44pm

Hello, I tried to use foreman_userdata, but I don’t think I got it right; when I try to create the host, it says “globalIPSettings are required when using Customization Spec”, although I can clearly see that this is defined in the userdata openvm-tools.

Also when configuring the host to be created, under the Operating System tab, when I click on the “Resolve” button for Provisioning Templates, it gives me “Sorry but no templates were configured.”, when there are 2 templates associated with the OS: userdata openvm-tools as provisioning and cloud-init default as user_data.

Can someone help´me with that? Thanks in advance!

awslovato · March 27, 2019, 8:29pm

Experiencing this right now as well.

edergillian · March 28, 2019, 1:26am

I managed to solve my problem by running these rake commands to update the foreman database after installing the plugin (run as foreman user):

foreman-rake db:migrate
foreman-rake db:seed

After that, the templates were correctly installed and I could proceed and create a VM from a template!

Stefan_Lasiewski · March 29, 2019, 8:40pm

@edergillian, I’m running into the same error. Are you saying that you ran those rake commands after installing the VMware plugin, or are you referring to a different plugin.

edergillian · March 29, 2019, 10:12pm

I’m talking about the foreman_userdata plugin!

Stefan_Lasiewski · March 29, 2019, 10:37pm

Ah, I see. Did you happen to install this via Github, and if so mind sharing what you did? I’m using Ubuntu, and I’m not sure there’s a package available for my OS.

edergillian · March 29, 2019, 11:03pm

I installed it using the ruby gem, and following the instructions on the Plugins manual:

https://theforeman.org/plugins/#2.3AdvancedInstallationfromGems

Remember to do it using the foreman user and to run the rake commands after installation.