VMware provisioned hosts never register

Problem: Any VMware VM provisioned from an image template will be created and re-IP’d just fine but then nothing else happens. Build status will say “Pending Installation” until the token expires. I’m able to run the registration manually after the fact with no issues.

Expected outcome: Hosts should register to foreman after the fact without manual intervention

Foreman and Proxy versions: 3.4.1

Foreman and Proxy plugin versions:
foreman-tasks 7.0.0
foreman-ansible 9.0.1
foreman_remote_execution 8.0.0
katello 4.6.0

Distribution and version: Rocky Linux 8.6 (both for the foreman server and the hosts I’m provisioning)

Other relevant data:
My VMware templates do not have any cloud-init set up (not sure if this is required). Don’t see anything in production.log even after enabling debug logging.

Update: One thing I noticed since I posted the first post was that the smart proxy wasn’t associated to any one of my sites. I’ve addressed this but it still doesn’t appear to fix the issue.

Update 2: Added cloud-init to my image but now the userdata template no longer IPs the VM nor does it set the hostname.

Update 3: OK I’ve managed to figure this out. So cloud-init does need to be installed but there are some modifications that need to be done to get this to work. I’ve listed everything below:

Cloud-init cannot be the one modifying the network configuration on the VM, therefore as per this doc I’ve created the file /etc/cloud/cloud.cfg.d/80_disable_network.cfg with the following contents:

network:
  config: disabled

In case someone wants to know how to do this via Ansible like I am doing, you can do something like this in your playbook:

- name: "Cloud-init required config for disabling network config"
  ansible.builtin.copy:
    src: ../files/80_disable_network.cfg
    dest: /etc/cloud/cloud.cfg.d/80_disable_network.cfg

The next thing that needed to be done was to ensure that the Perl based customization engine was being used by disabling Guest OS Customization with cloud-init. For this we need to edit the file /etc/cloud/cloud.cfg and remove the following line:

disable_vmware_customization: false

You can do this via an Ansible playbook as such:

- name: "Revert guest customizations to use perl based customization engine"
  ansible.builtin.lineinfile:
    path: /etc/cloud/cloud.cfg
    regex: 'disable_vmware_customization'
    state: absent

The rest pretty much comes from this doc

Afterwards I was able to provision and enroll a new VM successfully